You pretty much only need to look at my @approute methods in the server.py file.
Integration
I want the front-end to send stuff to the db: I left comments for how to change the method so that it pulls data from the front-end instead of using hardcoded data.
I want the db to send stuff to the front-end: See comment on line 106
I want to see something other than index.html: I left comments for that, index.html was just used as a placeholder
I want to use existing pages/links instead of /exampleXXX: Take what I do inside an examples and move it into whatever non-example @approute method you think can use that database action.
Other helpful things
In my code, there are two ways to run a query:
By calling helper method query_db, which returns a result (like when you SELECT from the paper table)
By calling helper method execute_db, which does not return a result but does check to see if your change to the db failed (like when you UPDATE or INSERT INTO the database)
I'm happy to help, just ask
4/8/2021
Working on modifying values on the database (SQL UPDATE)
Need to hook up the db functionality to the real front-end pages
Right now it is going through the server, but to example pages instead of the real ones (so I could test easier)
Go to /exAddPaper (or /exGetPapers or /exDatabaseUsage) and that's what runs the database queries
Note that all of my example pages just render the homepage html, then the query runs on the background and prints to the console. Aka, if you want to see if it's working, don't look at the browser, look at the console
4/6/2021
Got queries to the database working!
Get all papers (SQL SELECT)
Add a paper to the database (SQL INSERT)
4/1/2021
Reviewed Aaron's PR that's been around since Thursday SORRY AARON
Used Aaron's new instructions to install dependencies
Pulled it into my branch
Made small edits to README.md, like the grep command is UNIX only
Made small edits to run.bat, .bat files do not use # as a comment so it was printing an error, use @REM instead
Continued work on database/writing
So far, I've run the initialization of the db and resolved errors that appeared there
Database exists with good schema!
Using examples from online, I have all the building blocks. Next I need to run and test each one to work out kinks like:
Do they need moved inside of create_app?
Did I assume correctly what type each query will return?
3/25/2021
Created sprint 2 demo presentation slides
Met with team to go over presentation
3/24/2021
Checkout out bug Luis found
He said the parsers failed to get the citations written to the c file
I explained that the latex files that don't follow the naming convention LastnameYear won't work because the name won't match with the all.bib
From chat: ""
He said it still doesn't work
Turns out he was using a paper called liuLi2008.tex, and the all.bib entry for that paper was LiuLi2008. The capitalization is off.
Decided to make a fix to the code to ignore capitalization. Going to make a PR for that.
Reviewed and approved PRs
3/23/2021
Reviewed and approved PRs
3/?/2021
Continued work on the SQL
Decided that the list of citaitons for a paper could not be held in a column but needed its own table. The paper id (ie. Liu2008) is used to connect the entries.
Decided not to delete Sultan's excess columns in case they come in useful later. Commented them out for now.
3/?/2021
Started revision on the SQL script
Created tables that have a column for every paper attribute that we get from the bibtex and latex parsers.
Needs revision on the data types.
Sultan's tables were helpful to see syntax/data types.
3/21/2021
Received SQL schema script from Sultan
Took a quick look; like Sultan already acknowledged it has a lot more than what we need that will need to be taken out, including some complicated looking stuff that I assume Sultan got from examples he found online
3/18/2021
Finished merging parsers story card
Found out the latexparser didn't work on all example tex files given, Aaron fixed it, I gotta test it more
Assumed the id of the latexpapaer is going to be the filename (for example Liu2008.tex has an id of Liu2008) and used that to match the latex info to one of the bibtex entries. Fixed tehe problem from 3/17/2021.
Currently the writer handels one bibtex file and one latex file.
3/17/2021
Made minor adjustements to latexparser
If something isn't found, set it to UNAVAILABLE like how bibtexparser does
Exception for this^ is the citation list, which should be left empty. We don't want the cwriter to write method calls to UNAVAILABLE(); if there are no citations
Now can use latexparser in cwriter class
Considering replacing all of the get methods with one single one that returns all of the attributes at once
Currently, the writing happens by looping through the bibtex entries and adding in latex info in that loop. I need to add a condition that it only writes the latex info if the latex paper matches the current bibtex entry id. Without that, it writes the one latex file's info over and over.
Once that^ is done, the c file will have one function for each of the million paper names in the all.bib but only one of those functions will be fully filled out using a .tex file. I need to make the latexparser and cwriter handle many .tex files at once. I'm considering making it take in a list of .tex files instead of just one. The question is: does it need to take in multiple .bib files too? Or no bc there's only one all.bib?
I'm not just doing these things yet because I want to talk to the group first. Gotta consider how the user will input stuff and how we plan to store/rewrite our data. Gotta see what Sultan came up with for the database.
3/16/2021
Finished moving the BibTex parser to the cwriter class. A success!
Required renaming bibtex-paser and latex-parser to remove the hyphens, hypens are no good when you're attempting to import them
Still have to use LaTex parser in the writer class, need to look into how it works more
3/11/2021
Work on BibTex parser
Handled 8 more attributes
Initially only ID, title, year, and author were handled
I found the rest by looking at the all.bib, I think I got them all but it wasn't a foolproof plan, might've missed one somewhere
Since now there's 12 attributes, I didn't want a method for each, I abstracted what I could
Also, I modified it to use hardcoded strings as little as possible, constants when I had to
It's good practice. Otherwise it's prone to errors, more opportunities for typos or missing something, or strings that should be the same might not match up. If you make it a constant at the top of the class, you can modify it it one place instead of multiple (and it's easier to find).
Seperated out a Writer class
Previously, the BibTex parser was the only thing writing
Soon, output from the LaTex parser will join output from the BibTex parser, so I seperated the writing out of the BibTex and into it's own file (Writer.py) so that it'll be a place for the two parsers to merge and write.
Worked on Test Plan
Wrote the first 14 test cases (covering SRS requirements up to UHSR1)
Updated req to test case tracability matrix for those 14
3/9/2021
Wrote short guide for 3/4/2021 entry
Starting work on adding to parsers
3/4/2021
Met with team to give an introduction to HTML and CSS
Install live server extension on VSCode
How: click extensions on left bar, search for live server, install, click "go live" on bottom bar to open HTML
Why: you don't have to reload your browser to see changes made to the HTML
HTML is what it is, CSS is what it looks like, JavaScript is what it does
HTML
"what it is" the content itself, not any styling
In VScode, in your HTML file, type ! and hit enter and it'll give you the header
Tags
start tag looks like <tag>, end tag looks like </tag>
attributes (is that what they're called? idk) can go in the start tag
like <h1 id="my-heading-id" class="my-heading-class">This is my heading</h1>
<h1> headings, goes up to h5
<ul> unordered list contains <li> list items
<ol> ordered list gives you numbers instead of bullet points
<input> is a text field
<div> doesn't show up, it's just used to group things. Aaron said to avoid them and use more descriptive ones like <header> or <section>
Stacks top to bottom, CSS is needed to make it go any other way (what it looks like)
CSS
"what it looks like" used to style the HTML content
I attempted to run doxygen commands in a command line but it didn't work at first
You can run doxygen commands, but only if you're in the directory C:\Program Files\doxygen\bin where doxygen.exe is located
Open command prompt by typing cmd in the windows search bar
Move to the directory by typing command cd C:\Program Files\doxygen\bin
Check if you can use doxygen by typing command doxygen -v (will print version number if you're good)
To use doxygen from any directory, you need to add it to your system path
Open environment variables by typing environment variables in the windows search bar
Click Environment Variables... > path (under user variables) > edit > new
type C:\Program Files\doxygen\bin
Then I attempted to run a command through python and it didn't work at first
To do so, you need to add it to your system variables, not just your user variables
Same thing as before, but select path unser system variables
Run doxygen in a .py script in our project
import os
os.system("command")
commands needed:
doxygen -g filename
Creates a config
If you don't put a filename, it's auto Doxyfile
doxygen filename
Runs config file and generates HTML
You have to edit the config to set the input file (INPUT = ) and where you want your output (OUTPUT_LOCATION or something like that)
Luis knows more about the config file than me, ask him
Worked with Luis to show him all of this^ turns out he knew most of it
Manually created a C file in the form we might want, ran Doxygen to see HTML, discussed with group
2/22/2021
Left the new LaTeXML parser idea alone, sounds like Aaron figured his out
Started a new branch for BibTex to C parser story
Pulled Aaron's branch in
Fixed folder structure to match what Liu wanted (common, graph, paper)
Reviewed and all of the existing code to see where we're at
Got stuck on what the C functions we're building from the BibTex are supposed to look like. Decided it would be best to manually make a C file and use Doxygen to generate HTML I can see, then model the BibTex to C code after that.
Learning how to run Doxygen commands in a script in our project. This is encroaching on Luis' story, so I'll probably work on that with him.
2/22/2021
Aaron said he was having trouble wiht the LaTex parsing, may have to consier a new parser
Looked up alternatives to the LaTex parser Aaron was using, found LaTeXML
Used LaTeXML to covert reduced papers from LaTex to HTML, XML to see if the information would be usable
HTML, to the human eye, looked more usable
Looked up ways to use the tags in HTML or XML to get the data inside. Example: tag
Found an HTML parser that does this, but needs some hanlding
I'm aware that XML files are often used for configuration so there has to be an easy way to get the values out of that, I think I might just be missing some understanding of
2/18/2021
Added to Initial Design wiki.
Added notes from an old meeting with Liu.
Created design flow chart.
Used draw.io, downloaded as an image, uploaded with Git to wiki folder, added link to picutre to Initial Design wiki so it'll show.
Worked on SDD
Reviewed and made minor edits to Luis' contribution to the SDD
Worked on unfinished sections of the SDD
2/16/2021
Created Use Case Diagram for project.
Used draw.io, downloaded as an image, uploaded with Git to wiki folder, added link to picutre to SRS wiki so it'll show.
Wrote use case descriptions for SRS.
Made some other minor changes to the SRS.
Converted SRS from wiki to a document for David.
Every requirement table needed extra spaces removed/columns adjusted.
Downloaded as a PDF and uploaded with Git to the deliverables folder.
Assigned tasks, I'm doing BibTex to C.
Talked with Aaron about what is currently in Git.
2/11/2021
Met with team to finish SRS.
2/9/2021
Met with team to work on SRS.
2/8/2021
Worked on SRS.
Modified SRS template on wiki to include Akbas' instructions on how to fill sections.
Added 21 requirements to the SRS.
Started a few sections.
2/3/2021
Created the first 12 story cards on the project board for documentation tasks
Created template wiki pages for the SRS and SDD.
2/2/2021
Created and wrote wiki page for initial design ideas, based on meeting notes from 1/28/2020
Met with Liu and team, updated page^
Edited readme.md and wiki homepage to include application background
2/1/2021
Started GitHub wiki
Created enginerring notebook pages for team members
Transfered over whatever was useful from our old Wiki
Created home page
Created sidebar
1/28/2021
Met with team in person to discuss initial design
First meeting with new memebr, Sultan
Took rough notes, will later be captured in a wiki page
1/27/2021
Attempted to meet, rescheduled for 1/28/2020
1/26/2021
Met with Liu to discuss proposal
Clarified we're not automating using the web: user manaully inputs LaTex paper and manually enters additional info
1/25/2021
Met with team to discuss project
Determined it would be near impossible to automate retreiving info from the web
Brainstormed other features to add to citation tool
Wrote proposal to email Liu
1/21/2021
Team met with Liu to discuss project
Liu proposed citation graph tool, a tree graph showing research papers' citations referencing other research papers