I'm 2 years into a software engineer's career, and I think a lot about this question. When I started CS as a sophomore in college I thought it was ridiculously easy--getting a program to print y after given conditions x and z was simple. However, as I work in the industry I've realized what separates "programmers" from elite software engineers is the ability to build a system that is maintainable and works.

This seems important because the entry into programming is so easy and accessible that it's hard to see how steep the learning curve is. So, given the assumption that software development is very hard, what is it that makes it so hard?

Probably the fact that the amount of technologies out there is pretty big, so when building something entirely new it is important to have experience and foresight when choosing the technologies and designing the core architecture.

Also for the non-elite engineers... with OOP you can build some pretty big cluster**** if you are not able to use abstraction properly and you can do so without recognizing this immediately.
It only starts hurting in the late stages... in the beginning it might feel like there's nothing wrong with having put these methods into all these classes and interfaces and having inheritance for common functions, it feels like "wow we really built something great here" using OOP.

Like anything else it mostly comes down to experience. When I had been programming for a few years i thought I knew as much as the more experienced guys but 20 yeqrs on it is very clear that I did not. With experience comes exposure to more situations and technologies which you can consider when designing something new.

Everyone loves new technology but I would strongly recommend working in a mainframe environment for the first few years or at least working for a large company. Mainframes have a 40 year headstart on design and development principals and the structured way mainframe people go about massive and complex fasks is an incredibly useful skillset to have.

You also have to genuinly love working with computers. It is hard to be great at anything without some level of passion for it. I still love it 20 years in.

Cam

Would you please elaborate on using abstraction properly?

it's hard because building software is an intractable problem. you are only iterating a solution to a problem that can't even be completely specified.

also, there's so much outside of making the initial program and designing it well:

maintenance and debugging
code rot (constantly upgrading libraries, operating systems, etc, or else missing out on fixes)
leaky abstractions (if your framework has a bug in it and the maintainers don't care to fix it, can you?)
non-functional requirements (security, high availability / robustness, performance, recover-ability, etc..)
automated testing
configuration management and infrastructure as code
communication (both with non-nerds and nerds)
keeping up with the new stuff

understanding basic **** about the ubiquitous tech out there (dns, https, ssh, public/private key encryption, etc etc) seems not too much to ask but a surprising amount of programmers don't know these.

most programmers are bad. i only trust autodidacts for the most part.

What about the bussiness domain? I keep hearing that good programmers are not those that produce godly code, but the ones who understand the business they are developing for and how development works.

There's people who do software only as a job. After work, they never think about it.

Then there's people who do software because they love it, or at least like it. Enough to learn a new technology from scratch on their own because they find it interesting.

Of course experience matters... the first type of person who does it for their job can still be an extremely good developer after many years.

Also, a lot of people knock algorithmic knowledge and theory. It does not come up that much in the workplace, but when you do need, it can be a lifesaver for a project. Additionally, theory teaches you how to think about a problem. The very best programmer combines their theoretical knowledge, with their experience, and passionate for learning new technologies.

How do you define an autodidact? It's pretty hard to be completely self taught for programming... if someone is using StackOverflow for example, it's not self taught knowledge.

going by this definition, no one is self taught, unless you cut your internet connection and invent your own operating system and programming language like the templeos guy.

you can be self taught while using stack overflow. stack overflow won't teach you how to program. the questions are limited in scope. it has lots of good answers, but it has lots of bad ones as well. you need the wisdom to tell the difference, and the wisdom to know lots of other things that can't be googled.

Yea true, that's why I was wondering what you meant... cause that term "autodidact" seems to apply to people from pre-internet times: https://en.wikipedia.org/wiki/List_of_autodidacts

Do you just mean somebody who did not go to University?

Using your definition, an autodidact never went to school and never read a book, which are both false assertions contradicted by the link you posted.

No autodidact is illiterate; no autodidact works as an island. There's no difference between an autodidact in post- or pre- internet times. In some ways, self-learning without the internet is easier because the signal to noise ratio is much lower. It is hard for authors to bull**** their way through a 1,000pg textbook. Spewing garbage on a blog or YouTube video is relatively simple. These days, an autodidact has to spend a lot of time thinking about proper resources before taking any dives.

This is what I've discovered after years of using internets - to be a good programmer, you have to be good at things that I'm good at. Also, it's not at all important to be good at things I'm not good at and it may even be harmful because being good at things I'm not good at makes you pay attention to the wrong things, which I define as things I'm not good at. Also it's really helpful if you use the technologies I use. Using technologies that I don't use is a sign that you lack taste/morals/intellect and is a sure way to throw your career potential away. I don't remember if I'm self-taught or learned in school, so it may seem that it's not important but it is. Whatever category I fall into, has all the great programmers and being in the wrong category is a sure sign of a doomed career. Anyone who disagrees is a terrible programmer, uses gotos and eats callback spaghetti for breakfast.

what make's software development hard, at least for me, is thinking how you should implement a software that is used by 10 persons or 100k and still work the same way (I work in web development so i know what it takes) and providing your users a good experience with your software.

I would think that making UX decisions for software being used by 100k people are not made by just one developer.

Where do you see me ever giving a definition? I was asking what it is, and still wondering why an "auto-didact" is better. It's a term I have never heard before... so I really have no idea what it is when referring to programming.

I meant using Stack Overflow in the sense of posting questions, not reading answers.

Nope... I keep my developers away from the business for the most part. This is why you hire good technical BSAs

How does that work? BSAs prepare long detailed requirements and engineers implement them without knowing much about the logic?

These are not mutually exclusive. It's hard to make good technical decisions without understanding business needs. Design, even technical design, is a series of trade-offs and properly navigating through them requires business insight. It doesn't do a lot of good to produce reams of beautiful code that don't solve business problems - that's a liability, not an asset. At the same time, understanding business needs doesn't make you a good programmer by itself - you still have to be able to code and be able to make sense of business needs in the context of software development.

Having this kind of process is generally unattractive for good developers. The good ones aren't going to want to work in a situation where they are in the dark in regards to actual business needs and are simply told implement the architecture and design as laid out by architects who don't code.

This is a pretty good talk by Steve McConnell somewhat related to this topic:

GTAC 2011: Closing Keynote - Secrets of World Class Software Organizations

The whole talk is worth watching but I really like his answer to the question about what to do with developers who aren't interested in business (from 49:22 to 52:45)

^^ definitely, that guy speaks the truth.

As an end-user of companies who have a fear of developers interacting with humans, I can tell you that each one of those products are utterly unusable, and it would take a lot of arcane mathematics to convince me that any of those products don't end up causing the company to lose money in the long-run.

For starters, don't rely on SO as a source of knowledge. The actual difficult questions never get answered. Example... you ask a Jquery question and you get 20 answers with 15 minutes. You ask a question about shell scripts and you will be lucky to get even one answer. I asked a question about a shell script that populated a mongodb db. l only ever got one response that didn't end up helping. On SO, you will also get a shocking amount of people that literally don't read the question you ask. Half of the responders in the questions I posted didn't even attempt to answer my original question.

StackOverflow is a really good resource for searching questions. But a really horrible resource for actually asking the questions.