This is true regardless of whether you learn from books or from a professor at a University. One of the keys to learning (at least for me) is to identify multiple credible sources to compare and contrast.

Yeah, one of the advantages of a compiler is that it's a great little teacher.

The problem with writing things out is that, and it's been studied and replicated many times, the person writing things down is likely to believe what they wrote is true, up to the point that they will dispute any evidence that shows they are wrong, and in some cases, even if they knew what they wrote down was wrong before writing it.

That's the trap: one can easily write a teaching article that is wrong from top to bottom, especially if there is no solid fundamentals. The writer can fill in the blanks with whatever makes sense for the article they are writing.

A few years back. I wrote some article on some topic I couldn't find any information on, and created a solution. Over the next year, it got a few hits each day. I got quite a few "thank you" emails.

I went back and re-read that article, and holy cow, it is a steaming pile of trash. The concept was entirely wrong, and even worse, it opened up massive security holes. Not a single person ever once wrote in informing me of these flaws. I deleted that article.

In sum, I wrote what I truly thought was correct at that time, and I didn't have the base knowledge to understand that I was writing something very wrong. I was able to look at it critically at a later point, but not on the day I wrote it.

For me, self-learning is more a function of accepting that I will be wrong half the time, and right the other half. While cross-pollinating resources is a decent idea in many ways, the fact is, we are all bound to succumb to something that is either wrong or out of context (or confirmation bias). The ideas of "fundamentals" should increase and stack on each other with every new learning experience, where fundamentals could be well out of reach for the person you were 6 months prior.

That doesn't sound surprising at all. Pick your favorite college and look at the graduation requirements; meeting the minimum in the major could probably be done in a year [1] if those were the only courses taken and [2] ignoring the existence of prerequisites/sequential learning. (I would guess most graduates take quite a few more courses, electives of interest, etc.) That's probably true for most majors in a typical liberal arts course of study in the US.

Yeah the college timeline is that they force you to take other courses which cover the "basics" of a bit of English, science, Math, history for any major. Then the courses before the actual specialized courses as pre-reqs. That's why a masters programs just give you those specialized courses without the fluff before it. Obviously it might come easier for those who went from business major to business school, but for those who went from English major to MBA it isn't going to really set you back.

Dave, things like SO have a feedback mechanism so you can learn of what you're writing is useful, correct, people agree with it, etc. That's the useful part.

And yes, there are some wrong answers and negative feedback (as in the wrong things are reinforced as correct). But overall it would definitely be a good and useful source of external feedback.

Sure but a Masters degree is usually going to take you 2 years because you can't take 5 or 6 subject matter classes at a time and expect to really learn them, and the prereq tree is usually at least 3 levels deep, plus not every class it taught every semester.

Also I think that a fairly large percentage of non-major classes add something valuable to a person, even stuff you might think of as fluff. I took a lot more math than was required for my degree - I double majored for 3 or 4 semesters before I finally decided I didn't want to spend 5 years in college. There are a lot of things that come easily for me that don't for a lot of people.

I may have told this anecdote before, it's one I really like. I was taking a semiconductor theory class and we had this assignment where they gave you a function that represented the density of electrons in a semiconductor as a function along it's length, and asked for the total number of electrons. Easy, I think, it's just the definite integral of the function from x_0 to x_1.

Except I spent hours trying to integrate the function and failed miserably. Just totally ****ing couldn't do it. I went to the prof and asked him. He drew the function on the board, drew some rectangles and triangles that approximated it, and said, you just add these up. I was dumbfounded. No math professor would have permitted that, but in the context of engineering it made a great deal of sense.

Which is a long winded way of saying, math taught me things engineering did not, and engineering taught me things math did not. And hell, I learned a fair bit about art, history, literature, psychology/sociology and a fair bit about biology, chemistry and physics. Nothing too deep, but enough to know what's out there and to get an idea of what I don't know.

Thanks for the responses. I'm sure it can be difficult for setup, etc. to be a pain when you're not used to tooling and the language. That being said, these kind of mirror some other stuff I've read.

I'm not super pumped about the job anyway, but it'd be nice to at least have an offer before moving to Seattle.

Are you talking about a job customizing SalesForce or a job working for SalesForce? I have a few friends who work for them and haven't heard anything horrible about it.

It'd be customizing SF in a company that does a lot of SF consultancy type work (B.lue W.olf)

They were interested in having me interview for a position as a developer, consultant, or something else and I declined them.

My dad did SF consultancy work for a bit and it was horrible. He much prefers the mothership.

Thanks for adding another data point.

Would you say you declined because of your dad's experience or was there some-thing or things that swayed you? Feel free to answer via PM.

I'm definitely conflicted because my lizard brain is like "safety and survival is paramount", i.e. "it would be great to have a job lined up BEFORE moving/driving 2600 miles." It seems like they have solid policies on working from home and open vacation, but compensation seems to be a little bit on the low side of things. And I'm not really sure how interested I am in learning a proprietary set of frameworks.

On the other hand, I do have an open invite for an in-person interview with a really cool eCommerce company once I get to Seattle. This is probably my ideal company tbh. Much smaller dev team and it sounds like they have a solid process in place for their dev team. They book plane tickets and lodging for the entire team (+ family) to go somewhere warm for the month of February. I know they've been to Belize, Costa Rica, and Mexico for sure. Not sure about total compensation since there aren't many devs, that information isn't out there yet from what I've seen.

In addition to that, I've also got a new recruiter looking at positions for me and he was hoping to have something lined up by the time I get to Seattle. This recruiter is from the same company that got my last position and they've done a great job up to this point. Though I was working with a different recruiter in a different market, so MMMV.

Like writing a Stack Overflow question, I may have figured out my best option after writing that all out.

If you want more detail I could go into it via PM gladly, but basically I worked at an open source web agency for 3.5 years doing custom implementations for clients. While my role was head of sales/marketing, I still was involved in many projects that went sideways and it was never fun. The biggest challenges were in dealing with personalities and the technical challenges were mostly due to self-inflicted harm on account of client legacy practices/code. Add in about 80% of the people on these projects between the client/you/other vendors being business people who want to solve problems by having meetings to discuss "pie in the sky" requirements for weeks on end... I'm ranting now.

Basically I don't wanna do billable client work.

No more detail needed. That's totally understandable.

Once you are tagged as an "enterprise" programmer, it can be very difficult to get into other areas if that arena isn't your main interest, no matter how complicated or difficult your background is.

If you are looking to work somewhere else 2 years from now, the Ruby / Pyhon / Node companies aren't going to take much risk on someone who spent those years as a SaleForce developer. I doubt most would even know what languages are used, but you'd have no recent or long-term experience with the current language flavors.

One data-point: http://blog.triplebyte.com/who-y-com...companies-want

You'll also have to deal with recruiters...

maybe 2 years. but yea basically. unless you're a self taught genius though, those courses would be extremely difficult if not impossible without the prereqs knowledge. like i imagine an upper division architecture type class would be impossible without a lot of those prereqs.

I'm really in need of a book or web resource that is intended for designing databases and queries for high throughput / high volume data with mysql. I am running into severe problems, and also learning some surprising things.

For example: the fastest way to sort query results is to use an index, obviously. The worse methods include writing blocks to disk and sorting on disk. In between is doing the same sort, but in memory.

mysql will only use an index for sorting if it's an index on the first table in the query. This is a huge ****ing nuisance, and imo, a ****ed up query optimizer. Consider

It should be clear that this is the same query if I select from table a and join with table b as vice versa (I mean, logically it is, clearly, the way it approaches the 2 queries is different)

If I run the query this way, it might take, say, 10 seconds. If I reorder it so table b is first, 10ms. That is to say, 1000 times faster.

group bys are inherent sorts, also, so be careful there. I suspect that outside of some particular optimizations, mysql will use ONE index per table per query so if you need one index for your where clause and a different for your orderby, I'm not sure it'll use the index for the order by. I'm still looking into this.

I suspect that I need to look into better settings for sort caching - like maybe its doing these sorts on disk when it could use RAM, or maybe RAM isn't possible but it's doing them on a slow disk when it could be using the large solid state disk we have.

Anyway I am kind of out of my league here. Tables have tens of millions of rows, query where clauses can easily return 100s of thousands or even millions of rows. Everything is fine when the where clause can narrow it down to a few hundred rows but often that's impossible.

BTW this index choosing to sort tables kind of means that proper normalization ****s you because you can't order by stuff in the tables you join with unless you want to wait, or want to dynamically rewrite all your queries so that the right table goes first, making your own hillbilly query optimizer.

Is it too late to switch dbs? If you have to run a marathon to get it running in mysql but only have to do a couple of wind sprints to get it to work in postgres or something else seems like time to switch.

I should have insisted on postgres in the first place, but we were already using mysql (lightly) when I started. And with the boss I have now, switching is probably not going to be an option. I am about 95% done migrating the last of some mongo stuff into mysql and he's already very impatient about this, wants to know what the "value" of doing it is.

I am going to try to find some time to do a few pretty radical things, I think. The way our data is normalized makes a lot of sense for certain kinds of queries, and certainly for compact data storage and for writing data. In fact I kind of optimized for writes when really I think probably reads are our real bottleneck.

But anyway, it might make sense to take a data warehousing approach and make some kind of read-only table that is in exactly the form I want for this set of queries, optimize the **** out of the query, and update the warehouse table on a regular interval, or maybe even keep it up to date via triggers. It would, I think, be way better to get search results in a timely manner than to get everything strictly Up To Date.

And I think I could probably manage to get that warehouse table updated once an hour, maybe more. Think of it like a giant cache of the whole ****in thing.

A few things to remember about MySQL:

1- It wasn't built to be a database system from the ground up. Many things that are "database" isn't actually a part of MySQL. It really is the original NoSQL and probably should be used as such in many cases.

2- MySQL has had a memory leak bug for the past 10 years. It hasn't been fixed and MariaDB inherited it. Take whatever memory you would need in PostgreSQL and double it.

3- MySQL has multiple engines available, and you can use those engines interchangeable in a single database: https://dev.mysql.com/doc/refman/5.7...e-engines.html

4- The my.conf file isn't always where you think it is.

Man if true, that is some terrible query execution planning.

We don't leak at all. And even if we did, we have a shiiiiitload of free ram. I have checked about a million times, I have allocated 3/4 of the RAM (I think we have 32Gb) to caching, but it never even comes close to filling up. Of course, our whole database is like 6Gb so I guess it's not THAT surprising. Anyway, least of my troubles.

It's my impression that only innodb is worth anything. Is that wrong?


I'm using RDS, I know where my conf is

So I actually went ahead and deployed the query I have so far to our CI environment, which has an (old, but large) copy of the data in the prod database, and it's actually doing admirably there. So it's possible that some of the problems I was having were from the anemic dev database. I tried a half dozen variations and everything was under 500ms, which is acceptable. In the dev database I get more like 10s, which is not.

The point isn't really if you are flooding out your memory, but if you are using more memory than needed. Sounds like you have enough dedicated RAM, so probably not much to worry about, but I'd be sure to check the memory when joining a large table (or two) with indices.

It depends. MyISAM is optimized for speed, but you sacrifice FKs, transactions, table-locking, and other things. I'm pretty sure MyISAM is the original engine, where InnoDB was added later.

I'm aware that you can mix MyISAM tables and InnoDB tables in the same database, depending on the tables you are using, etc. I'd consider building the warehousing tables to by MyISAM. I'd research that more, but here's a quick self-confirmation: https://lists.mysql.com/mysql/109403

Sounds like you know where it is on prod, but not dev. Of course, RDS is likely to have saner defaults....

At a prior contract, the sysadmin argued with me night and day about why AWS (RDS?) was slower than my laptop. I thought "my computer is older than dirt and has a blown hard drive, therefore AWS needs to be tuned," should have been enough argument, but nope, I lost.

this was a huge headache in my database class and something I never completely understood and we were expected to

That is some horse ****. I mean, sure, there exist RDS images that are ****tier than even an old laptop but they're not for real prod use. Like they have micro instances that are a fraction of a CPU, and use virtual CPU banking, tiny RAM, etc. But any reasonably sized RDS instance should blow your laptop away, otherwise, you should probably host your database on your laptop (which is so absurd that it should be obvious that you need to fix your RDS instance)

I like AWS but I am occaisonally annoyed that their server levels are kind of package deals. Like, I couldn't get more disk storage without also getting more RAM, which, ok, more RAM is fine, but I'm paying for it and not using/needing it.

We really don't leak RAM though. I can't get into AWS because I don't have my stupid 2fa thing with me but our server has been up for months and memory usage is a straight horizontal line