Engineering Notebook: Aaron Van De Brook - AVanDeBrook/citation-graph GitHub Wiki
Engineering Notebook: Aaron Van De Brook
4/27
Made a PR to merge and merged the edit paper info branch into the main branch.
4/25
Fixed a bug in the edit page where paper citations were not always showing up.
4/21
Made a few style changes to make to reference page look a bit better with the text boxes
4/20
Added button to update paper info in the reference info page
Added keys (or tried) to the form tables so that it can be parsed on the back end
Problems with key errors, not sure why. I think it's because there are two forms on the web page:
It is because there are no input fields in the form for the server to parse
Need to change some of the html (mess with that tonight)
Changed table entries to textareas; also had to give the form an id since there is more than one form on the page
New problem: return template rendered with new info; need to change the logic of the /papers/new endpoint:
Need to parse information, store in database
Template needs to be changed to populate fields from database entries instead of directly from the parsers
Database schema needs to be changed a bit
4/15
Added database functionalities into the reference info page
Still need to mess with the parsers to speed up the response times
Need to add some kind visual response to the edit page so the user knows they're are editing the page and that it can be edited
4/13
Looking at Corrina's pull request
Tried to get some work done search function and database integration
3/29
Added manifest and setup files so that the project is easier to setup/install and run
Setup so that running pip install -e . automatically installs all project dependencies (note: these files need to be updated to reflect any new dependencies added as the project grows)
pip list | grep -i citation-graph (Linux) and pip list | find "citation-graph" (Windows, I think; I haven't tested it)
Updated readme to reflect changes and to better explain how to set up the project, dependencies, and how to run
PLEASE REVIEW MY PULL REQUEST, IT'S BEEN THERE SINCE THURSDAY
3/25
Made some small changes (line breaks between input fields) to the input screen to make it more readable
Removed plaintext server secret key from initialization file and moved all user configurable fields into a config.py file which is ignored by git
Demo
3/23
Set up server to convert papers into C, and run doxygen on them
Current problem: when reading the doxygen files I get the error codec cant decode byte 0x89 in position 0: invalid start byte
3/22
Integrated web pages (from Luis and Sam) into the server
Set up endpoints to handle new papers and searches from users
Wrote a template to return with paper/bib data once the server processes the data
New paper endpoint handles POST requests from the browser, saves the files entered, processes them, and returns the data back to the user in a readable/edittable format
Last steps: integrate doxygen, add a way to access doxygen web pages, integrate database (might be pushed off to next sprint), add search functionality
Also, ran into a weird bug in the bibtex parser when testing the forms; when the bibtexparser library was parsing the document it ran into errors because it could not decode the byte 0x92 in utf-8. This is a 'smart quote' from windows, which doesn't exist in utf-8
Solved by switching the encoding/decoding scheme from utf-8 to cp1252 (in retrospect, using dos2unix might have worked too, but using a different encoding scheme is probably a better idea)
3/18
Fixed an issue in the Latex parser where it wasn't saving all citations
Refactored server to make static resources easier to use (all contained in the static folder now, use something like {{ url_for('static', filename='<file_or_path>') }} in templates to link the pages together easily
Set up blueprint to use for the request and response form pages
Any more progress for the server is at a roadblock until the parser(s), webpages, and (possibly) database are complete or closer to a point where they can be tested with other modules
Database will be useful for ensuring we're not having the parsers do extra work when they don't need to, improves efficiency and performance
Database is really easy to implement and integrate with flask, but need info on which to use because the syntax and initialization processes for all of them are different
3/12
Started refactoring server and setting up Jinja templates for form pages (input paper, paper info like references, authors, publication date, etc.)
3/11
Refactor endpoint function to use @after_this_request function to (hopefully) make the responses faster or at the very least easier to read
Searched for fonts and messed around with linking external fonts in HTML documents as well as scripts
3/8
Did some testing to make sure stylesheets linked correctly, turns out endpoints need to be set up for everything
Set up endpoints and error handling for stylesheets, scripts, images, and favicon(s)
Some browsers (Firefox) won't accept server responses if specific HTTP packet header properties are not set, so for all CSS documents the 'Content-Type' property is set to 'text/css' and for JS documents it is set to 'application/javascript'. These are also defined by standards which are referenced in the MDN documentation
Should probably be setting response headers with the @after_this_request decorator, so this will be kept in mind as I refactor the code
3/4
Got a basic server started with flask, serves placeholder pages at the moment, but these can be easily replaced/added when the front-end starts to come together
The index (home page), and 404 page are the only endpoints working at the moment
Assembled a list resources and documentation for HTML, CSS, Python, and Flask (will update these more as we incorporate more things)
3/2
Created and assigned tasks for sprint 2, started looking into flask as a means of setting up a back-end server for the system
2/22
Finished the LaTeX parser (with the exception of the index terms). Grabs the abstract, citations, bibliographies, etc. from a LaTeX paper.
Skipped flask and opted for just opening the generated doxygen html from a python script.
2/18
Refining/creating code to grab important sections from the paper (have the abstract working so far, working on citations/contributions next).
Started on manually generated graphs. Using example papers first to keep things simple and then expanding from there.
Going to start messing with flask this weekend so we can have some semblance of a prototype for the demo.
2/16
Finally created a system design (for the back-end at least). Still need to come up with ideas for the front end design.
Thinking of using Flask for ease of use between front and back ends
Created/cleaned up some code to give us a starting point on the parser modules
2/10
Figured out some details for the LaTeX parser, starting writing some code to find important sections (e.g. Abstract, Citations, etc.)
Need to convert to classes for modularity when using it in a 'back-end'
Once the parsers are separated into classes, will starting working on creating the C-like functions for Doxygen to process, then front-end framework
2/2 - 2/4
Got BibTeX parser 'working', wrote a few functions to grab paper information and convert them into C-like functions
Uploaded some (or all) of the files that Liu emailed us