Log (2019 01 03) - Gandhie/AICS-Project GitHub Wiki

Note: the research question mentioned in the previous log was discussed by Amelie and Simon over Skype and the results of this can be found on the wiki-page "estimating_variation".

What did I do?

I corresponded with Mehdi throughout the earlier parts of the day, discussing the next steps and what I could do today. We ended up agreeing that I should start with making representations of the bounding box data for targets and landmarks in the images, for the WHERE part of the research question.

For this, we're currently going with the same approach as the VisKE paper (F. Sadeghi et al., 2015), that is, 11-dimensional vectors ({dx, dy, ov, ov1, ov2, h1, w1, h2, w2, a1, a2}).

I created a cell of code in the Python Notebook in the repo that should get these vectors for different image descriptions and putting them into lists for each spatial relation we're currently working with (over, under, above, below). The lists are the values and the spatial relations are the keys in the dictionary that contains it all after this cell has been run.

Unfortunately, I ran into an issue when running the cell - where the data rate seems to be exceeded. I have contacted Mehdi about this after doing some googling for solutions myself and not managing to solve it. Once this issue is solved/bypassed, I should be able to see if my vector creation is working as intended.

What is left to do?

After the creation of the 11D vectors works, the next step would probably be to start looking into the comparing methods with cosine, standard deviation etc.. At this point, I will need to confer with Mehdi or Simon again to make sure that I'm understanding what I am doing correctly and doing it right.

After that, i.e. once the WHERE part is essentially done, it will be time to move on to the WHAT part - which will (I think?) require access to the pretrained models that we have discussed previously, to get visual features.

And lastly, once WHERE and WHAT are both done and I have comparisons and graphs etc. (i.e. results) for both (and thus also some conclusions about the results), it will be time to write the report and prepare the presentation for the 17th.

Author

Amelie