To answer your other question, no, I don't program for a living. I've had contractor jobs and that required me to take my laptop in to work, so I never had the joy of working on larger screens. I'd probably be lost and shrink everything down.

At home I use 3x4k displays and a 1440p display and at work I have 3x1080p displays and a 1600p display. Programming on a laptop is an awful experience I do my best to avoid.

Pics of home setup?

I got a rejection letter that was actually well-written and wasn't superficial and flaky. I'd have to say, it is the best rejection letter I ever read.

paragraph one: this is how many people we interviewed the week before last. Thanks for being willing to talk to us on Skype, etc.

paragraph two: I'd like to share, as promised, some of the insight into the interview process.

more paragraphs about exactly what we were looking for. It is super specific about the employee-finding goals and is written such that you'd have to agree that you didn't make the grade. Some items they weren't sure if they could find ended up being found in spades, so they shifted the weight of their decision towards those preferences.

Finally, some stuff about the company goals and how the decision to employ someone new would impact those goals.

Overall, I'm wildly impressed by this one. Someone actually put some thought into the letter, and was able to describe in lucid detail about their decision. At no point would I want to write back and ask for more information. I'll keep this one on file for future reference.

Does anyone have an opinion on modeling classes based on partial select queries, meaning I want to only grab a user email address and his weight from 2 diff tables. I could do a join select * and populate a full user object and a full user weight object, or I could select the needed fields, and populate just those fields leaving the rest blank. Or 3. I could make a new class just for one query.

Number one is slowest, number 2 has unexpected results, i.e. A non null able field is null on the object modeling the table, and 3 seems like a Maintenence nightmare. Any ideas?

Before you get too far into it, what is the *actual performance comparison* between populating the full object, vs just the few fields you want? That is, does it actually matter? Sometimes I get caught up in these optimizations to find out it actually makes no difference.

If it actually matters, I would lean toward using one "user" or whatever class, that has all fields defined, with each field having a clearly null default value (as in, it's possible to tell the difference between 'the field was not selected' compared to 'the field's value is blank or 0 or the equivalent'

In my recent experience, modern databases are so fast that the difference between selection 10 field and 3 is really just not significant.

The 3rd option is a no-go, imo. If you really think that your use cases are limited
1) the full object
2) a specific sub-object (like username + email)
then you could consider having class that is just username+email, and having the User class contain that class, and then a raft of other fields.

Do you see what I mean? Like...
UserThing has the fields username and email
User has fields of UserThing, address, age, sex, birthday, blah blah

You could do this via inheritance or composition, either way is probably fine.

We originally did something like that, the hibernate Java type method where say UserWeight is a composed field on the user object but that kind of breaks the abstraction that a user represents a database user entity

Right now the current solution is we have separate "join" objects that map the hierarchy of related tables, like one would be a UserHierarchy and just contain a User object and a User weight object. But when more than 2 tables are connected you end up with an object holding 5 objects and sometimes it only populates 2, sometimes 4 and sometimes not all the fields of the objects either.

Fwiw this is an incredibly early iteration and I would love to do the hibernate model and select * everywhere till it becomes an issue but my partner disagrees strongly with pretty much everything

If you're really concerned about that abstraction, then, imo, it's all or nothing

I have to say, I have seen more projects that are limited by their languages data model, than they are by their database structure. Your database has the ability for you to fetch only pieces of rows. Therefore, imo, your data structure should support the notion that some circumstances only need some of the data.

But please, tell me first that you have verified that fetching the whole user record is actually a bottleneck? because in my experience it really isn't. In which case you're optimizing for no reason.

I have made a complete 180 when it comes to database stuff. When I started programming database backed stuff, the database query was the blocker. That just doesn't seem to be the case any more. My current job does so much stuff that is against SQL "best practice" but basically it just doesn't matter. Computers are fast, networks still aren't that fast, so the bottleneck is the network to the consumer now. We have these huge pipes to incredibly performant databases, but the same network perf from our data centers to the clients that we had 10 years ago (for the most part)

The dominance of mobile really underscores this. In your datacenter you can almost do whatever you want because the bottleneck is pushing those bytes to the mobile app.

We're too early to know or even care about any bottle necks. I contend it's all seeing the forest before the trees or whatever it is, but I'm also ultra obsessive about conventions of code, i.e. An entity class represents the entity. So combined, we make up a pair of weirdos who just argue about dumb **** instead of writing code

this might be a silly question, but are you making your own ORM? for example Django (python) implements .only which pulls a few fields rather than SELECT *

https://docs.djangoproject.com/en/1....uerysets/#only

Bringing the actual data into the app isn't the problem, it's deciding how to model it on the server


For example if we select user name but not id from the table, in the code it could be expected that an id is never null on a user retrieved from the db because it's a primary key, and we could end up screwing some stuff up due to our partial select

Stop obsessing over details, get the job done, and expect to redo it in 9 months when you figure out what you actually need.

The number of people I know who accurately figure out exactly what they need before they get going converges to zero.

I completely agree, although it's hard in practice. However, what about things like maintaining no business logic in controllers or db access objects? Should I care if we violate that? A lot of times it seems like we're doing things that are easy to write then impossible to maintain when we touch other code and that's mostly my concern right now because it drastically effects dev time. Should I not give any kind of **** and just hack?

The more business logic your ORM models need, the more tied to them you are. I don't have an easy answer for you, but I can tell you, I don't know of a single person who designed an ORM system and loved it one year later. So the best thing you can do is to make it easy for 1-year-from-now you. Think about that guy. What **** is he going to have to dig himself out of?

My personal philisophy has been that every system has ugly parts. Certify that those parts work, and *bury them*. Re-evaluate when you need to use those parts in some other system.

Example: my company's product is heavily embedded in a Django ORM system. Increasingly people are abandoning the Django part and developing "microservices" that are, well, who knows. Whatever language you want. Whatever database you want. Whatever whatever you want. The happiest people in this new world order are the ones who don't have a ****load of "business logic", whatever the **** that is, in the original django ORM.

Make your database so that your system doesn't require a ****load of logic.

Said another way, a piece of wisdom that was once said to me (god, I think this was Philip Greenspun or maybe even older): if your database model is good, the code flows out of it, no worries. If your database model is bad, you are ****ed.

PJ,

Just skimming over your original question and the discussion with Rusty, the real problem is a deeper architectural one with the ActiveRecord pattern (or antipattern). Basically, as your application grows, you are going to end up with fat (or god-object like) models whose methods and properties mix up completely unrelated use-cases, and that's going to be trouble. In addition, your database details are leaking into your application objects. That database is supposed to be an implementation detail that your application doesn't know about, but what you describe is a re-implementation of your database tables and all their relationships within your ORM objects.

Really fixing this is hard, and probably not what you should do right now, but I just wanted to point it out in case you wanted to delve deeper. Look into the Repository and DataMapper patterns. This article discusses related themes as well: http://solnic.eu/2015/09/18/ditch-your-orm.html

I'm totally lost here. Why are people afraid of databases? This whole conversation sounds insane.

I feel like, if you have a good design, the rest is pretty simple. Fred Brooks quote:

Show me your flowcharts and conceal your tables, and I shall continue to be mystified. Show me your tables, and I won’t usually need your flowcharts; they’ll be obvious.

But I'm probably just blabbering and super biased.

dave do you use ORM or just write raw SQL?

Databases are great. And I've always loved the FB quote. But it doesn't mean your application is the same thing as your database.

1. A database is not a good place for your business rules. Even with advanced integrity constraints, triggers, etc, they are not a good fit for expressing that logic, compared to something like ruby or python.

2. A database usually serves multiple applications, or at least different but overlapping use-cases within an application. Each of these applications or use-cases is going to want its data in the format that's best for it. If your application has to work directly with ORM objects that mirror the database, you're forcing your database structure on an application that really wants a different structure.

For very simple, single-purpose applications, you won't feel any of this pain, because there won't be a mismatch between the database structure and the way your application wants to use the data.

So it's not that databases are bad, but really that, conceptually, you want each of your applications (really, each of your use-cases) to be served by it's own mini database -- a data structure custom made for it. But in large, real-world applications, you want a single physical database, mainly because that data is shared for many purposes. So to have your cake and eat it too, you have to solve this mapping problem between the database and your application code.

I'm not really sure I follow here. It sounds like he's suggesting making a object for any use case that holds the data it needs. This sounds incredibly unwieldy and bloated with objects, so I just must be misunderstanding.

I can sort of see how applying fields directly to table columns is "tying" to the database, but at the same time how I fill those fields is abstracted into my repositories so I can realistically fill the same fields anyway I want. So why is this bad exactly?

I generally write raw SQL, but of course Psycopg2 gives a few extras. I've used ORMs, but I didn't walk away with the impression they solved much. I didn't feel like they built good expressive database designs and the queries are questionable to say the least. I cry foul on object-relational impedance.

I agree with this in part, but I also disagree.

1- There is no blanket truism for each RDMS. For example, Postgres gives you insert / update / delete CTEs (mySQL has no analog to CTEs at all), which does wonders for any back and forth between the db and the system. This is, of course, one use case, but there are many other examples that can be used.

2- Data integrity is the sole purpose of a database, and ignoring this fact is not using a database correctly. Code and databases have very specific purpose, and when you say that a database isn't good for this or that, I'm not entirely sure what the point because I don't see any connection between a database and the code.

In databases, saying something twice doesn't make it more true. If you want to use an ORM to make specific calls or whatever, I'm not going to argue for or against. I often wonder how these things go so sideways, then I realize that much if it is that there is no understanding of the underlying data and very little understanding of the database system you are using, and that is, IMO, very dangerous.

At the end of the day, you are getting a list of tuples back from the database query. How you get said data isn't really important.

Who came off as afraid of databases? I just want to keep business logic in the application so I always know what the data im retrieving will look like.


As for orms, I have access to raw sql and the result set with my app, I just don't know where to put it all in the app!

I think we're talking past in each other. Sorry about that, but this stuff is so abstract it's to be expected. Maybe an example will help. If you need a rule that says customers who have spent a lifetime total of $1000, not including any discounts (your VIPs), get a special discount that entitles them to 50% off any purchase on their birthday, but not to exceed $50 in total discount, that rule shouldn't live in your database. Even though, at the end of the day, that *is* about data integrity: it's just expressing a bunch of relations between numbers in the users table, the user_details table, and the orders table (or some such). RDBs, even sophisticated ones, do not have the expressive power needed to model complex business applications naturally.


EDIT: perhaps even more importantly, such rules are subject to very different rates of change than, say, rules about the allowed length of a first name. so you don't want them encoded in the database, otherwise when you change them you will break the integrity of existing data.


I was arguing against ORMs (at least, in the ubiquitous incarnation using the AR pattern), so I don't know what to say here....