It's been awhile but I believe the setup is that usually you have a cold standby that is synchronized with the master (I think with Oracle it was done by tailing a transaction log). If the master goes down you switch over to the other guy.

Edit: I guess it's actually a hot standby.

databases at scale are a hard problem. shoe you might enjoy reading about the CAP Theorem.

at my work we use mysql with replication via binary logs (plus a harem of intermediate masters, read-only replicas for development, etc.).

Ive been trying to get into NoSQL but consistency and dealing with concurrency in general seems like a nightmare with replication. Not sure yet how non transactional storage has picked up so much steam

I got snubbed at work last week because it turned out an OSS package we're using had a bug in it that changed the input data unexpectedly and incorrectly and I thought that was pretty scary. Equated it to a calculator giving wrong answers. I was told every calculator in existence gives wrong answers and all software has bugs and patted on the head and given a lolly. Did I overreact?

Why wouldn't you use an enum? It's basically a class designed for stuff like this.

tyler_cracker,

Yeah I read about the CAP theorem in the past, just never in depth.

It's really a non-issue for me in either case. I just find it funny how everyone says that postgres/mysql is dead but sites like disqus pump out 8 billion page views a month and postgres is their primary db. They even use django for most of their web app stack.

I guess vertical scaling can go a looooooooooong ways.

Who is "everybody?"

PostgreSQL has vertical partitioning, horizontal sharding, JSON / XML, many options for full text search via the built-in t_vector and other add-on libraries. The size of the ecosystem, the tools, and the ability to customize it is quite astounding really.

I guess I may be wrong or misunderstanding the situation, but from what I've been gathering (and reading from some books on NoSQL concepts), NoSQL isn't about the tools you are using, but the techniques you are using. This would imply that you can use Mongo as a full ACID transactional database** and use MySQL as a scaling beast. Of course, you would prefer to use tools that are easier, but it is entirely possible many people are confusing the tools with the concepts.

** okay, probably not, but the point being that there are SQL ideas propagating in many "NoSQL" databases..?

Everyone was pretty strong. I just recall multiple Netflix engineers saying Cassandra is the best thing ever and it's clearly the way forward.

I tend to trust their word but at the same time you can't ignore the success of other large sites using postgres. I think the only difference is those postgres sites aren't on a typical cloud host. They just have crazy big servers that are self hosted. Netflix leans heavily on ec2 and a bunch of other aws features.

Thats certainly the complaint of most SQL DBs and the reason NoSQL has picked up any steam. Itd be interesting to see how NoSQL architecture unfolded after a few decades of being the norm the way relational DBs have. Right now though it seems like a lot of it is pretty complex and messy.

I think quoting a few people who use a NoSQL database as "everyone" is a bit of a stretch though.

Because I forgot enums existed. It really is exactly what I need, thanks.

This discussion caused me to find the Netflix Techblog. Interesting stuff: http://techblog.netflix.com/

Wow.

I totally get it. For their needs, relational databases must be dead. I think they used to use Oracle, so they must know what they are talking about, but for the average use-case, and for most uses, relational databases are plenty powerful. Even Google migrated from MySQL to MariaDB in lieu of other options.

Among startups NoSQL databases are pretty common and I'd guess they're more common than postgres/mysql in more tech-talented shops (meaning places not just focused on a basic consumer app).

Horizontal scaling is nice not just because of the rare situations where vertical scaling isn't actually possible but also because it gives you better redundancy, cheaper hardware costs, generally less ops overhead, and a bunch of other benefits I'm probably forgetting.

And in the end, we use Mongo not because of the horizontal scaling features, but mainly because ****s just really easy. JSON is really flexible and easy to work with. Migration is easier. Releases are easier. And in general day to day development is easier.

Its pretty rare that I find myself trying to do something with Mongo where I want to switch back to Oracle.

I'm surprised some Euro company doesn't just pony up the 20k as a PR move. NSA...yada...yada...20k here you go. Call us again next year. I mean I know what idiotic stuff tech companies spend 20k on without even blinking...surely this would be massively more +EV.

Seriously though, you should very much care for what happens to OpenBSD. They are essentially the testbed for new and interesting (security related) things. Mostly because they don't mind braking backwards compatibility.
And their man pages are amazing (at least compared to the Linux ones I know).
And I'm fully behind the no-blob policy.

[insert Theo rant video here where he (rightfully) points out how horrible Linux and FreeBSD are at implementing security mitigations...I belive his words were "Miscrosoft does it better than those two"]

Re: noSQL...pretty sure the hipster pendulum has swung back from ZOMG-best-thing-ever to always-knew-SQL-is-king. They obviously all have their use cases. I kind of wish object databases like ZODB would be more common. Was pretty fun to work with (ignoring the indexing via mysql for speed reasons :P). Oh well hipster pendulum has swung away from OOP anyways.

How is the hipster pendulum measured?

With some archaic steam-punk device that will fit on the back of a single speed.

There are 10,000 watchers and 10 doers. When 8 of the 10 doers start getting burned by limitations, the 10,000 watchers form an opinion. Whether that opinion is well-informed or not is beside the point: the opinions are written and the influence is there.

Does everyone hate MongoDB? gives a nice overview of the many issues people have been facing with the system. He collects many articles talking about how people have been burned by Mongo and gives point-by-point rebuttals and cautions to each of them.

It's stuff like this that would give me pause for using MongoDB. I can't think of a single useful database I've built that a limit like this would be okay, and I certainly cannot tolerate data-loss no matter how small it is. I mean, isn't that the point of storing data?

There's a ton of interesting articles on hnsearch: https://www.hnsearch.com/search#requ...ongodb&start=0 . Note the clearly negative trend.

Many of these people are much smarter than I am and have switched back to relational or switched to something else. They are a ton smarter than I am and I value their opinion. Personally, relational is something I am familiar with and until I "outgrow" it, I see no reason to move off of it. I've joined over 1M rows on Postgres on a single quad-core Win machine and it works incredibly fast.

Not meant to be a burn. I find the data storage and manipulation endlessly fascinating. I just think that many people under-rate the power of relational databases, either due to ignorance or pie-in-the-sky "needs" that probably don't exist for most use-cases. Plus thinking that you wouldn't need an equivalent to a database expert for NoSQL sounds patently absurd to me, and this is probably where many of the issues stem from.

With that said, I certainly plan to learn much more about the basics of NoSQL. I have that 7 databases in 7 weeks and have been reading it over. Very interesting stuff indeed.

It's not - and this is coming from someone that has worked for years with both Oracle and Mongo. (And no, I'm not saying every company with a SQL database needs a DBA and every company with Mongo doesn't need one).

I'm also not claiming that relational databases don't have a place. They do for lots of things. But they're also crappy at a lot of things (just like NoSQL databases).

First, this issue is more of a usability/documentation problem. That doesn't completely excuse it by any means, but its not like Mongo doesn't support safe writes.

Second, there are a ton of cases where you can tolerate tiny amounts of data-loss (and we're really talking tiny when using unsafe writes). We store all sorts of status/progress/log type messages. If we miss one, it really doesn't matter. We have extra safety in the cases of the important messages like a job being done (and not because of Mongo, but because when you rely on various AWS services you have to be super robust) but if we lose an intermediate message - it really doesn't matter.

Even in something like a consumer website, if you lose a user rating of an item - does it really matter? Will that hurt your user experience that much?

Edit: Also want to add that at least a couple of the issues mentioned in that blog have been fixed/addressed in more recent releases. And yes, the fact that NoSQL options aren't as mature as traditional DBs is certainly a fair complaint about them.

Weighted average hacker news articles

I certainly wasn't implying that were suggesting either opinion.

I probably over-stated that opinion. At the minimal, I would think that someone that has some decent experience and sense of schema-design would mitigate many of the issues they are running into. Even if it is schema-less, there are still relationships that must be understood, if any of that makes any sense. I'm just trying to figure out why people run into so many problems, of course, using my very small viewpoint of the world.

Yeah, was wondering about that claim.

I was reading a bit of the documentation on Mongo yesterday. I have to say that it is quite good.

A review? Nothing hurt.

A financial transaction or customer sales history? Zero loss please. Oddly, the system they use at work does lose this information, so it's not like this is an issue limited to NoSQL. The problem is a poorly designed system despite being relational, so it would appear that NoSQL should be able to handle this well, assuming decent system design?

Ding!

... and hacker news also include "everyone."

People run into problems with everything. It's often the people that's the problem and not necessarily the tool.

I actually like the approach of thinking of Mongo as still having a schema. It's just enforced at the client level instead of the database level. I can't remember where I first heard this but it makes a lot of sense.

As above the biggest problem is the person and not the tool.

There are a few places we need to rely on our database to do things like enforce uniqueness constraints - in those cases we make sure to use a safe write since the performance impact is much less important than the guarantee of properly handling a uniqueness violation.

But definitely, if you find the majority of your use cases are related to things like that then Mongo probably isn't the best tool for the job.

Dave, I also think this quote from the blog you posted is particularly apt:

Hype Cycle: https://en.wikipedia.org/wiki/Hype_cycle

My only real issue with NoSQL so far is that I go in with a relational mindset, and a relational set of knowledge, so I have 0 clue how to orient any of the data. Its like trying to learn how to walk on your hands after using your feet the whole time.

Probably a bad analogy, but Im really enjoying trying to get into it, I just think people are getting way ahead of themselves on its importance so far.

The documentation is pretty good. The design part is pretty decent as well. This only one page of several, but it doesn't appear to be a quantum leap from relational:

http://docs.mongodb.org/manual/core/data-model-design/

I especially like how they consider many-to-many relations.

Interesting stuff.

Yeah, I like how the article is well-balanced.

Thanks for the thoughts. All very interesting.

Think I'll give Mongo a shot, perhaps with that poker bot when I get some time to work on it again. It also looks a lot like Korma.