There are ways to do relational stuff in mongo in a way that is basically sane, if error prone, i.e. without relational integrity. But man we are doing it all wrong.

Another favorite of mine is that there is a data type A that contains a list of Bs. Bs are not unique - many many As may have the same B.

But they went ahead and embedded the full B object into the A. So if anyone changes a B you have to search and replace them all. **** me.

Oh and we also are using an ORM, so it's not like we are taking advantage of the ad hoc structure of mongo very much. All the down sides, none of the benefits

Yes but mongo is cool.

For stuff it's good at, I really like it. For other stuff, I don't.

A real problem with it is, you can create a new db or collection at will. But making a new mysql/postgres/etc table usually involves coordinating with someone at your company and making a ****ing migration plan and all this ****. So people are like, **** it, if I use mongo, I can do whatever I want! No migrations! No DML!

And so tons of **** gets rolled into mongo where EVERYONE agrees it should be in a RDBMS.

Mongoose is awesome and does have schemas and works amazingly with passport (node auth) so um use that.

The problem is if you have non-node clients which need to access mongo (at least according to the startup guy).

Theres a lot of times where nesting stuff is awesome, and its probably not a good reason at all, but I like having 10 collections vs 30 tables for representing the same data.

Also async db libraries for mongo! Which admittedly are more frustrating than they are worth

I've had to deal with SQL DBs that were massively over-normalized like that. Not fun. Like zip code would get its own table and address would have a reference to the primary key of the zip code.

How many people at the startup?

Congrats, totally awesome.

10-12, all remote, mostly in AUS. ChrisV - might be something. Supposedly they've got 18 months of seed money and are closing in on series A.

What kind of work is it?

Development. It's a startup. The stack is RethinkDB/node/react.

I seem to recall it's mostly a job doing nothing, which is perfect for a 2nd full time job, especially when the first job is also doing nothing

It's getting better. I have stuff to do on the second job.

Sounds so much like my first few databases. Wow, they were terrible.

Congrats on the dual income.

RethinkDB looks interesting, anyone used it? Looks a bit Meteor-ish.

Yeah I've seen things you wouldn't believe, c-beams, attack ships etc.

One of my favorites was a DBA who insisted that every column in a table begin with the name of the table, that way, in every query you could tell what table a column came from.

At my last job, again, it was really hard to modify existing tables, but easy to make new ones. So we had like 9 tables that held "user" settings, i.e. a 1-to-1 mapping to the main user table. This is of course propped up by the ORM, the average user of the ORM models doesn't even know this is happening. Some of these tables were 2 columns: user_id + whatever new setting.

I interviewed someone once, and asked him to make a little program that would track books - basically it would have like author, title, publish date, isbn, etc. He made a structure where the 1st row in the table was title, the 2nd was author, the 3rd was publish date, etc. To add a new book, he used "alter table add column", i.e. each book was a new column. He was, apparently, way ahead of his time.

I've seen the "new data, new column" pattern before. It's advanced ****, not to be tried at home.

We're using it because they like the real-time push aspect (basically you can watch attributes/data and push to the client when those change, so you don't have to poll). I'm not 100% sure why we need that but there is a lot I still don't know.

Right now we are stuck on this problem. So if anyone has any ideas please chime in. From the dev working on it:

Followed by this:

And this:

Here is a similar issue but they don't think it's the same as ours: https://github.com/rethinkdb/rethinkdb/issues/4036

Coming from Postgres, my initial thoughts:

https://www.rethinkdb.com/docs/memory-usage/

In simple terms, large documents require more blocks. Sounds like you are reading from several blocks.

There is a simple, non-mathematical way of thinking about indexes. They take up some space > 0Mb. This is going to be some N indexes in each block, depending on the size of each index. Larger indexes are going to take more memory space, but with limits:

https://www.rethinkdb.com/docs/optimization/

Also, if he is attempting parallel queries, the order of the query seems to matter:

My take, without seeing the data and query, is that the documents may be too large, and that is causing large indexes to fire, which isn't going to help the query run any faster (one indexes are so large, you are effectively doing sequential scans anyways, plus using up a bunch of memory that could be causing read blocking issues, and it appears the memory is shared across blocks, so more work). Also, there is no guarantee that an index is even firing in the first place. It could be index size or query issues, possibly a combination of both.

First question I would ask is: "are you sure the indexes are firing?" If so, are they firing over the entire query?

To add to above, when I read or hear "I don't know why that query is slow, the table has indexes," I translate that to "I only shot my foot with a bazooka, I don't know why my toes are missing."

Awesome, congratulations suzzer! Do you know if they may still be looking for people 4-8 months from now? I may be looking for a part-time arrangement like this at some point, depending on how things go at my current job. It seems like remote team + significant time difference would make things quite a bit easier when combined with a full-time job.

Yeah they might be if they get their series A, which they seem fairly confident about. Do you have any react or rethinkDB experience?

DaveT I sent our RethinkDB guy your response and got two kind of cryptic responses from him about it. I think he's a bit lost.

Great. No experience with RethinkDB but I'm at a React shop so I should have a decent amount of experience by the time I'm looking to do contract work.