I like representing graphs on paper, it’s like a map of my data

A visual map of my data.

Computers can’t read maps good

But apparently, the neural networks and I have different taste in how we like information. I like drawings of graphs. Neural networks want them all spelled out.

So, at 11pm on a Thursday night, I write my first ADG (my first line of Scala), and early Friday morning, I write my 60th ADG (and my thousandth line of Scala). Whilst listening to

Loud music, Scala, and a notebook filled with maps of imaginary data.

Instead of a visual map, I wrote “directions”. Defining each node by its label and its content. Defining each edge connecting children to parents, and making data siblings. And defining the ADG itself, as the mix of both.

What started as:

Ended up like:

and

Test, hope, fix, rinse, repeat

When we have the input files, and the output ADGs, all that’s left is to write the test: a program that takes the input files, sends it through the transformer and checks that the output ADG is what we expected.

And, of course, hope!

The true story of startup life

This isn’t a story about Abstract Data Graphs.

This isn’t a story about working hard late into the night and early into the morning to make sure that all of your tests are ready, so that you can be comfortable that your product is ready for release.

This isn’t a story of how to bring a little bit of order into a world with too many data formats.

This is a little glimpse at what startup life is really like.

There are no “non-technical” people in a technical startup. There is no-one that is not a salesperson. There is no one that is not going to have to drag his or her friends in to the office to test the product.

In a startup, no one is allowed to stay in their box. Everyone has to do everything, and no one is above any work. And that’s awesome.

Marginalia

Thanks to Tim Delisle and Nicolas Joseph for teaching me everything I know about ADGs and data structures. Learn more about Datalogue or sign up for a demo.

1. (I remember (when I was reading through the book) that I would (before hitting compile which would take ages) count (by hand) the (extremely nested (not kidding, extremely nested)) parentheses, to make sure there were as many “(s” as “)s”). 
   By the way, this is a list of lists, and would make an interesting ADG.
2. Take, for example, key/value based data formats like JSON, tabular formats like CSV, relational databases, anything you can think of.
3. The Abstract Data Graph is quite funky, as it is incomparably expressive, and quite efficient. Instead of displaying the whole database in a single graph, each “view” of the database (e.g. a row in a table, or an object in JSON) becomes one ADG, the group of which represents all of the information in the database.
4. If you want to read a bit more about this, check out Our Story.
5. This might be too much to get into exhaustively here, but, it is enough to understand that: tables are difficult because they don’t clearly separate data values from data labels; JSON is difficult because of its flexibility.
6. E.g. {"array": ["0", 0]}.