To be fair, I don't know how things are since the financial crisis - I'm talking about this as a way to illustrate how things could look like if there was actually a massive imbalance in supply and demand.

At one point, I personally had about 10 years of experience programming and 3 years of experience in finance (mostly as a programmer no less) and the job market was such that I was about twice as valuable for a non-programming finance job (doing something which I had basically never done) as I was for a non-finance programming job (doing something that I had been doing excellently for years and years). I have not taken a single finance or economics course. And tech doesn't pay significantly better now than it did then, once you adjust for inflation and everything.

CB, you did it, you won through pure exhaustion. I mean, you've made a whole bunch of incorrect assumptions/statements in your posts and not actually cited anything (sorry if I've missed something), but I can't keep up with this.

Top-tier software engineers are absolutely comparable to attorneys at top law firms and physicians in terms of talent. In the US at least, number of years of education beyond bachelor's is not that important or strongly correlated with high pay. Outside of medicine (or maybe even inclusive of medicine now due to Caribbean schools, DO programs and such), simply getting a postgraduate degree is much less important than getting the right one from the right school. Consider relatively dismal career prospects for law school graduates outside of top schools.

This isn't an argument anyone is making - the general thrust of my argument is that software engineers as a group are paid quite fairly relative to what it takes to become one. And that it's easy to see the software engineering pay as too low or too high, depending on the perspective. I'll add another point I made earlier that if you want to see compensation that's more out of control, you need to look at industries where supply and demand are more arbitrarily constrained.

You're making a fairly extraordinary argument here - that somehow software engineers as a group, despite not being organized in any sense and being completely at the mercy of market forces and not having any sort of protection from the law or having otherwise created artificial barriers to entry, were able to bend the basic laws of economics in their favor to an extraordinary degree. If it's so easy to become a software engineer relative to how well it pays, more people will become software engineers until that stops being the case. This is generally going to be true at every level of competence.

That you're missing something is obvious enough that I'm not even trying to prove the case. I'm just using a bunch of anecdotes here like everyone else does, not as a way to argue my case but to share some facts about the world that may hopefully justify why you feel that way even though that's not the case. Kind of like how professional athletes making 20MM a year may seem outrageous, until you learn about the thousands and thousands who don't make it, how profitable professional sports are for the owners and cable networks and how the laws of economics suggest that the owners will maximize revenue regardless of how much the athletes are paid. That's what I'm getting at here because a lot of people seem stuck at the glib outrage stage.

lol. CB, I love that you just can't even conceive of how it can happen. Your faith in your econ 101 knowledge is unshakeable! And now I get, after thousands of words from you, that you're not even trying. I mean, come on. Do you really expect me to believe that?

And of course, one final non-sequitor about professional athletes.

Thanks for that last post, I was starting to feel bad and considering going one more round. It's nice to know I made the right decision.

I understand that you mean this as an insult but there are a few major things that are missed if you apply a straightforward econ 101-style model to this situation:

1) Price stickiness or inability of market actors to adjust instantaneously to accept new true prices
2) Information asymmetry
3) Barrier to entry
4) Imperfect competition
5) Market size and liquidity
6) Price control and other regulatory issues

I've addressed all 6. 2-6 in the past day or so and 1 in two parts over time - if true, it implies that software engineers are currently significantly underpaid, which I don't think is what you're arguing, not to mention that the sheer size and flexibility of the market in question implies less volatility in true prices compared to other markets. While it's impossible to exhaustively address all possible exceptions, what makes your point especially disingenuous is that I've specifically thought to address these first, you know in those thousands of words that you deride, to note that the market for software engineering talent is uniquely resilient, before I went on to talk about the facts about this world without which one may develop a warped perspective. The way in which we develop feelings about how much skills should be worth and how much someone ought to be paid relatively to another person often has nothing to do with how markets work but are largely based on very personal values and durable psychological anchors we internalize over time.

And if there was something specific about the market for software engineering talent that is causing some kind of market failure of epic proportions, you would've already mentioned it, instead of going on about how your company's inability to find anyone good is evidence of anything. I'd add that my understanding of your glib rejection of my points in regards to finance, etc, where you seemed confident that those guys must bring some valuable skills to the table if they are paid lots of money despite being completely unfamiliar with any facts, was that when it comes to other industries and other job roles, you're pretty sure that econ-101 works. What else was that based on?

Actually this may have been a little unfair since you may have missed this post, which was made prior to prior to my first response this time around:

@candybar - making economic arguments about whether or not a labor shortage exists is like totally appropriate. If we're not debating an economic phenomena then what are we debating when we discuss the possible existence of a labor shortage ? FWIW economic arguments to me exist on three levels:

1. "Hand waving" where people debate and cite reasons that support their views. The weakest form and if two opposing sides remain unconvinced then in my view it is best left at we agree to disagree. You have made an excellent argument with lots of ideas to consider.

2. Lots of data that support an argument. Again, FWIW, the link that daveT provided regarding employer preferences is the type of thing I am referring to. It is only one piece of data and I wouldn't consider it conclusive but it is information I put a lot of value on.

3. Economic modeling where there is transparency in how the model was constructed. The kind of economic modeling the CBO and the Fed do for instance.

Candybar, those aren't the only 6 factors that would affect a skills shortage. I've mentioned a few in the past, but quite frankly you're one of those people that are SO confident your view of the world is perfect that you skip over them. Feel free to go back over my posts but I'm not responding anymore because when I do you ignore 90% of my post and then make a bunch of false/incorrect statements, and then we start all over again.

It's extra funny that you want to claim you made these details arguments after you already said:

Of course they're appropriate.

I've addressed other factors as well but it's up to you to make a coherent economic argument. I've repeated myself plenty of times and no I haven't seen anything from you that addresses why the existing market incentives aren't working. I'm not just saying it either - I scanned back to the point where I wrote this:

And no, I don't see you making any economic argument in your favor. Things like the tech industry being willing to hire socially awkward people and those without on-paper credentials, from an economic perspective, is a sign of lack of barriers inherent in the market, which supports my point. Shortage is more likely in industries where this isn't possible - for example, you could more easily have a shortage of doctors because the credentials required to be a doctor are supply-constrained and do not respond to market incentives.

Oh really.

Exactly, I quickly explained what the situation is, didn't think the case really need any more support, at which point I moved onto a narrative explanation about why it is that things may seem that way, etc. Those conjugations mean something.

Agreed but I don't feel anyone's actually making an economic argument on the other side - I haven't seen any arguments that rise above, say, random sports fans being outraged at how much the athletes get paid, or how older single men/women talking about there's a shortage of quality men/women/etc. All I'm seeing is this visceral feeling that the salaries being thrown around are too high and the skill levels of people getting certain salaries, too low and rationalization that follows half-coherently from that visceral feeling. I totally understand why it may feel that way - and the converse is true, if you're a job-seeker, I think it's understandable for you to feel that there's a shortage of quality jobs - which is why I've pivoted to trying to explain that perception, instead of dealing with the economic side of the argument.

Think about all these people contributing to open source without getting paid in return and how if you post some stupid interview question online, 100 people rush to provide their own solutions. If you're not thinking through the situation economically and paying attention to the structure of the market and instead approach the situation more like jjshabado, with random anecdotes that kind of appear to be on your side, it's not a stretch to say we have a skills surplus.

Edit:

At least from jj's perspective, I think, this is really the core sentiment here around which the entire skills shortage argument revolves.

I basically agree but I think one contention is though the supply is being artificially restricted from not enough H1-B visas being granted. You wrote this in another post.

Quote:

Another thought on skills shortage: the market for software engineering talent is probably the deepest, most liquid labor market that's ever existed in human history thanks to the sheer size of the talent pool, ease of remote work and outsourcing, considerable willingness of many in the industry to relocate, even to another country, global conventions dominating over local, low cost and general availability of training and tools, lack of licensing or other regulatory barrier to entry and most importantly, fungibility of software engineering talent.

I assume your referring to the global market of developer talent, not just the USA. If companies are restricted from the global market of developers due to immigration law then clearly shortages could happen. I think this is pretty much the essence of jj's argument, highly restrictive immigration law can hurt a country economically.

Not going to multi-quote, but some random thoughts from all of this:

First off, barriers to entry aren't a bad thing, per se. If a doctor doesn't know what he's doing, people die; if a lawyer doesn't know, people die on an electric chair; if a civil engineer doesn't know what they are doing, buildings tip over and bridges fall into the river; a nuclear engineer doesn't know what they are doing, well, Chernobyl happens every year.

It is also false that restricted supply causes inflated wages. Most lawyers earn little more than McDonald's wages. A civil engineer has to go to school for 4 years, go through a program with a very high failure rate, then bust take a battery of tests to become Jr, and after 10 years, must take more tests to become full civil engineers. When a civil engineer in SV is about to retire, he may be earning about 100k / year, but only if he is lucky.

A software engineer can work in the industry for 20 years and still have no clue how to secure a device, which is, in case people are missing what has been happening lately, getting more and more dangerous. They have multiple chances and many of these mistakes and oversights are arguably causing people to die. There is a reason I don't use the word "software engineer" and I'm starting to believe that certain fields of programming should have some barriers of entry. People's identity and lives are at stakes now more than ever before, but we are getting more and more IOT devices and Theranos.

It is possible that Google / FB / Amazon / etc are no longer sponsoring H1-Bs:

https://www.quora.com/Why-did-Amazon...lished?share=1

****

When I was asking about what specific skills are missing, I was referring to the experiences I've had. I guess I could try two more things.

I can't count how many interviews I've been in where, after we discuss the job, I've asked "so, what are you looking for and how do I fit in with this?"

The answers are, quite often, some roundabout way of saying "we don't really know." This doesn't bother me much since pretty much every job I've had was basically "we don't know what we are dong, dave can figure it out," but it betrays a lack of thought and a lack of knowing what needs must be filled. I don't think my experiences are unique.

I also think that it is a good exercise to write out what you are looking for and compose (not send) a rejection letter to someone, and figure out how to do it while making it obvious to the applicant that they definitely didn't make the cut.

Not the same field, but I've been looking for a singer for quite while. I was once asked by someone why I said "no" to her, and I wrote out what I was looking for, what I was able to compromise on, etc. I did send this one and her response was "thank you for showing me your perspective, and you know. what? You are right, I'm definitely not what you are looking for because I don't want to be X, etc." Of course, this is an area where everyone has thick skins and doesn't really care, but at the same time, the process of dong it really helped me figure out what is important to me.

You can't cast a spell if you don't give it a name and you don't know what it is supposed to do. I do appreciate JJ's response to it, but I don't think that really applies to me. I know more than many people, and I know less than many people. I have a decent grasp of CS fundamentals (sort of wanes away against the hours I've spent doing other things like looking for work) and I'm pretty sure I have the ability to grow, as every project I've taken up has became more and more complex, and I can certainly go harder. I mean, how many people have taught themselves enough calculus to understand a FFTs and signals? On the other hand, I totally admit I have holes in my knowledge and I'm not ashamed to say that.

The joy of open source - redux-form broken with a minor release.
https://github.com/erikras/redux-form/issues/2255
https://github.com/erikras/redux-form/issues/2265

I never understand why the default behavior from npm install -save is to add the carat so that any minor version is acceptable. Causes so much headache when devs are on different versions. The bugs don't happen too often, but when they do they can be nasty to figure out.

I spent the better part of a day trying to debug this thing, so another dev wouldn't be stuck, while in a meeting with the startup CEO and founder. I'm sure they were annoyed with me that I was paying zero attention to everything else they were talking about.

https://yarnpkg.com/en/docs/yarn-lock

use dis

I don't do FE stuff but apparently a minor rev of angular 1 the other day hosed us good. Like 1.X.8 to 1.X.9 had some non-backwards-compatible changes. That is some bull****.

Removing the carat in the checked in package.json has the same effect.

Few months ago upgraded from socketio 1.3 to 1.4 and found that the server side references to the sockets themselves (found in the lovely "io. sockets. sockets property") was changing from an array to an object? Isn't that the definition of a breaking change?

Man I am in such a ****ing state.

About 5 or 6 hours ago I got paged because our web performance fell off a cliff. Like from 100ms avg response to 60s. mongo databases (3 beefy clustered machines) pegged at 100%. Alarms ringing everywhere, every metric just terrible.

The mongo logs show that a few specific queries are taking forever - 50-200 seconds. These are queries that are literally done at a rate of like 20/s all day every day. Nothing changed today that should make those queries terrible.

So we try basically everything. We drain the web connections to let the mongo collections settle, which they do. We add the web traffic back in and they peg again.

So I'm like, ****, these web requests really are the problem. And I look at the code and the queries and the databases and all that. I finally decide, **** it, let me add an index for this query. And it instantly fixes everything.

So hard to fathom. This was a mongo collection, maybe 400k rows, each row very tiny. We add maybe 1000 rows/day.

I guess it crossed some threshold where it couldn't all be held in RAM or something, just randomly today. That's just generally the last thing I think of.

Good job fixing the problem!

Should have listened to my warning on the previous page. Arguing with CB is like arguing with a creationist.

Interesting. I'd like to see an official source because I think this could easily be confusion about not sponsoring a visa *right now* (because the next lottery is months away) and not sponsoring them at all. Although I could also see that its what someone mentioned in the comments and that they're only sponsoring them for people with multiple shots at it.

That being said, I suspect for people they're hiring that don't need to be promised a very specific role they'd be willing to give the H1-B process a shot. They don't have a lot to lose.


We have our internal system that we use for tracking candidates and interviews and feedback there is usually much more honest than what we'll send to the candidate. I definitely agree that its good to think about this. And as a company grows its also good to have someone looking at these comments to ensure the reasons being written down are actual reasons not to hire someone.

Dave, this is still focusing on your technical skills/knowledge. I genuinely don't think that's your problem in at least some (and possibly many) interviews.

Those numbers are pretty sad for a so-called "big data" software. I"m also curious why there was no index on this query in the first place.

A bit OT to this but still topical, I lost count how many times I've had this in a phone call or interview:

"We never had a database person on our team, but we ran into this situation that took us [3 days ] 2 weeks] to solve. Now, we don't expect you to figure out the answer, but what would be your thinking process on..."

The honest answer is drop the database and rebuild, go back to bed, and look at the logs in the morning, but no one ever wants to hear that one. This question is a serious pet peeve of many database people.

They also mention that they stopped about a year ago. I did look for a better source to no avail, so you may be correct on your thoughts. I'm pretty ignorant of this stuff.

Interesting. I bet that is pure gold to read, but ****, it sounds risky as well.

You already suggested I'm a social moron, unable to read people. I'm not asking for further extrapolation on this because I'm guessing you are suggesting I'm some sort of weirdo. I've always been unwilling to get into these debates with people who don't know me, nor is anything of that sort productive to read.

Also, this popped up on HN: https://news.ycombinator.com/item?id=13163367

I'm not the OP in that, but what an interesting coincidence.

Hah, its not that exciting. Honestly though, if the people you have interviewing are writing stuff that is 'risky' you've got bigger problems.

Hah, I don't think this either. Almost everything is a spectrum. I think you need to work on some people skills. And I suspect its exasperated in the stressful situation fo an interview. I don't think you're a social moron or any sort of weirdo. This first sentence is a good example of what I'm talking about.