Public Interactive Graph of Art History

You can right now [by Feb] explore all of art history, and all the code and data that makes it possible.

This talk could go very deep in many directions, but I'm only here to introduce you to something to explore. To do that, I'm going to take you through our journey from, "exhibit of JPGs," to, "graph all-the-things for everyone."

Art history as a playground of ideas, without years of study. All the undiscovered nuances like Degas', "horses are for men, and flowers are for girls," or which countries and cultures made the most cat pictures.

• You (everyone) can do this simply by clicking on images of artists to their cultures to their most relevant subjects to their most common genres.
• You can contribute better algorithms to the engine serving these results.
• You can clone the whole thing and do it all your own way. Soon, you can add your own artwork [On Roadmap, date TBD].

Outline

1. Project concept: museum exhibit for digital media.
2. But how!? Software engineering magic... which could do so much more!
3. With museum archives and graphs, we can do all of art history.
4. Make that graph easy to read and get into a user-interface.
5. Make an interface that is as simple as, "touch the pretty thing."
6. You. Yes you. Here's where you can find it, and play with it.
7. Thanks, attributions, and contributors.

1. Concept

Our project started with a single vague goal: create a touch-interactive exhibit meant to bring digital media into a museum. We needed to design and build a system to organize and deliver images, videos, 3d assets, panoramas, in order to make that exhibit possible. This system became Steven Cutchin's enterprise, and the graph database from it, my adventure.

2. Engineering Magic

Steven's archival system, later named OpenPipe, had to store all the images and their data, and make that data useful. He imagined a system which would run multiple targeted algorithms over likely dirty images and data. Some would stitch the panoramas together, others would clean dirty meta-data. Boise State University wound up developing dozens of such compute packages for AWS Lambda. Those algorithms wouldn't always be reliable, and would require curation. Thus OpenPipe also needed an interface for curators to fill missing data, make corrections, and create specialized collections of these assets.

For any particular creator, what are all their works? Just their 3d models? Similar 3d models? Who are those artists? OpenPipe could organize and clean all of that, which meant it could handle any image archive with meta-data. We quickly realized that such a system is capable of being far more than an image archive. We can load museum archives into it just as easily, like the MET's openArchive... or eventually every digital archive of every museum.

3. Graph Database

Data alone was half the battle, how do we use all this useful data?

The best way to represent all these relationships was with a graph database, and Neo4j. Each image is loaded with meta-data including many potential topics: artists, tags, cultures, times, genres, mediums, etc. Pull these topics out of the artwork data, and make each topic a node in a graph. With Neo4j we could find all the artists or tags for any culture, order by most relevant (shared relationships). Eventually we will use path embedding to show guests the artists or works which bridge cultures and ideas.

Look at an artist, and see all the cultures, tags, works, mediums, and on and on... with an artwork representation for the most meaningful of each, relative to that artist.

4. Java Plugin and JSON API

I wanted to take that graph, and provide structured results for any window into the graph that might be interesting to any user. However, that structure had to be simple to keep the heavy-lifting out of the UI. Building it as a plugin for Neo4j made the whole thing as portable, and distributed, as Neo4j itself.

5. Design magic, and 1000 hours of head-scratching.

• [TBD, I need to get this from the app devs.]

6. Go play with it now.

• OpenPipe
  • Github
  • API
  • Curation UI [Public available date TBD]
  • Add your own art [Public available date TBD]
• GraphDB API
  • Github
  • API
• Touch-interactive
  • Github [In discussion]
  • World Museum Exhibit
  • Windows Installer? [In discussion]
• Web interactive [date TBD]