I've gotten pretty much all of my jobs based on #2, even though I'm pretty sure I would have no problem demonstrating #1 and #3. My last two jobs I haven't even had to interview for and I've been promoted 4 times in my first 5 years out of college, mostly because I have specific domain knowledge that is getting harder and harder to find.

I'm planning on getting out of storage 3 years from now, so this interview/hiring discussion is super interesting to me. I would have probably winged interviews at that time thinking I could rely on #1, when I can pretty much guarantee a pass of most technical interviews if I do leetcode problems for a few months before starting to look (#3).

Good discussion everyone.

Speaking of jobs I’ve got myself in quite the conundrum now.

I accepted a Principal Engineer position back home in MN at the company I used to work for but haven’t actually moved and started yet.

Today my director (boss’s boss) pulls me into his office out of the blue and offers me a promotion and a team lead position.

#firstworldproblems but I have no idea what to do. Making this level is basically unheard of with only 5 years experience but I don’t want to burn any bridges. My wife already moved and started a job 3 months ago — she would have to quit and move back, and I would have to renege on new job. We also are closing on a house in MN 1/30 we would have to back out of.

I told him it would take a lot of money for me to take it so we will see what kind of official offer he can come up with. I should probably ask for some sort of retention bonus to cover expenses from moving my wife back and getting out of the home purchase too?

It seems like the best opportunities in life always come at the worst times and when you aren’t looking for them.

Is remote an option or something you’d consider?

I asked them a few months ago to let me work remote or that I was planning on leaving and finding a job back home and they said no. It's not really a culture of letting people work remote except for the occasional day here and there.

It’s possible others disagree, but I feel like coupling your official commitment back in MN with all those external factors makes this an easy decision to go to MN.

Also you sound like a bad ass everyone loves. Time for you to start contributing here more!

I've looked for things I can contribute on but I'd be the first to admit I don't know the first thing about WebDev which is what mostly seems to be discussed here. I'm an Embedded Firmware Engineer working on / designing data center and enterprise SSDs.

Talk about how great rust is!

Gotcha. To be honest, you also don’t want to do a remote management position (or really any remote position) at a company that doesn’t have a culture of supporting it. It’s just very frustrating and really hard to succeed.

npm package-lock.json seems to have solved some problems and opened some gnarly ones.

The aws-amplify library package.json points to a bunch of subdependencies with ^ in front of them.

So we're using an older version of aws-amplify. But if we delete our package-lock.json (which our lead front end dev always wants to do for some reason) - it installs much newer versions of all the dependencies and the app is completely fried. Obviously that shouldn't happen with just minor and patch upgrades - but it does.

So we can never just wipe out our package-lock.json until we have time to go through a painful massive upgrade. In the old days this problem would have been instantly obvious (with users complaining) on any fresh install. Now package-lock.json can apparently mask issues when the lib has been embedded in a working app for a while.

https://unstuckpolitics.com/t/programming/192

It's interesting to me that you see this as a negative. I think it's probably the best possible way of doing dependency management. But I'm fairly sure the intent is that you literally never just wipe out the lock file for an existing project.

The idea is that if there are upgrade issues for a given dependency, then those exist regardless of when you choose to upgrade. But it's much nicer to actually choose when to upgrade then to have the package manager just randomly break your stuff periodically.

I have only used NPM prior to the lock system, and several years ago we switched to yarn which has a lock file and some other features around deterministic dependency resolution that NPM used to not have, but maybe now does? At least for us issues around dependencies changing with no warning was a pain in the ass with NPM that has entirely gone away with yarn. And sure, we have to manage upgrade paths and tech debt still, but it seems quite valuable to be able to have some control over the process. It's also much easier with the lock file to understand what's going on.

I get that it can be a feature. But it scares me to be this far out of date. If AWS decides to change something so Cognito isn't compatible with our old version of amplify - it's gonna get ugly. I'd rather risk a little break in our install every now and then.

It just seems weird to me that all the dependencies and transitive dependencies with ^ and ~ get frozen at the time of first install on some dev's computer (unless you suppress and ignore package-lock). What's the point of even having ^ and ~ then?

Btw at my old job we killed all ^s in our direct dependencies for exactly this reason. We'd get weird errors on the build or a new dev's box, but not a dev that installed it a while back. Which is what package-lock solves. But it seems like mixing it with ^ and ~ is a bad idea.

I wonder how amplify works and if they check in their package-lock.json. If they do they aren't even testing new versions of the dependencies. Seems like they couldn't. So they still have to worry about the problem of their dependencies having ^ and ~ in package.json - and basically just fix any weird issues that come up with a fresh install. Like the old days.

I even think there might be weird issues with the npm cache. I was getting npm audit errors for an old version of lodash that react-scripts used. react-scripts had ^4.11.7 or something in it's package.json - which was one patch too low. I tried deleting node_modules but the error was still there. Then when I emptied the cache it went away (I think - lots of other things were happening). So it looks like npm says "Oh, 4.11.7? I have that in the cache - here let me save you a hit to the server." But if it's not in the cache then it downloads 4.12.4 or whatever the latest was. Maybe.

What language? That sounds fun.

Related-ish, coworker and friend from my last job is now working on hardware devices as well (though not as low-level as I imagine SSDs probably are). He says they're on C++98 or 03, I guess I don't know which one but they both probably blow equally compared to C++11 so w/e. It drives him a little crazy and as a C++ evangelist I can't imagine how frustrated I'd be having to work in such an ancient language with people actively resisting a better choice - I guess part of the reason is that they have to worry about supporting legacy devices, but also it seems like their developer team is just kinda resistant to change? Like, one anecdote he related is that a guy looked up lambda syntax and said "look how confusing this is". I just...what do you say to that???

C++? Why so modern?

At my last job I was assigned a proof of concept to do, I didn't really think it was going anywhere so I did it in perl as it would be the fastest way to get it done. I got it done and working in perl.

We end up getting several customers that want it, and my group was not going to maintain it so my boss told me one Friday it needed to be in Python by next Friday. So I bust my ass along with everyone on my team and we have 19 of the 20 scripts converted. The following Friday I am laid off. I literally had a day of work left to finish the last script and a day or two of testing and it would be producing data for the customers.

Fast forward a few months and I hear they still haven't gotten it working in python. Talk to a coworker a month ago and they threw the python code over the wall to the ops group, they didn't like it and are rewriting it in C. Yes C. Not C++ or some other variant, C. I've been gone 9 months and it's still not working with no completion date on the horizon.

The project consisted of taking in a large amount of data and parsing through it for the good stuff. The vast majority of the processing time was moving the data and unzipping it. The second biggest thing was comparing it to what's in a database. No idea why they are doing it in C. One month of salary of one coder is more money than a years worth of the difference in the AWS bill. The customers are willing to pay $500k-$1M a year for the data, they are still waiting for it.

Lol companies are so so dumb sometimes.

A combination of C, C++ 14 with a lot of libraries / stuff stripped out, and some ARM Assembly.

Pretty much what KatoKrazy posted. Then there is the fact that the current devs may have a learning curve which of course came be overcome. Leadership (including leads) being unfamiliar with revisions from C++ 11 and later are going to be reluctant. Existing code bases in C++ 03 are another factor in why there is resistance. As KatoKrazy points out there are portions of the revisions from C++ 11 and later that just don’t fit in with embedded type development. Dynamic memory allocation (although you can use custom allocators for STL) being one thing. The concurrency model probably too. Leadership may not see the added benefits as a result. I have experienced what your friend is experiencing btw.

Windows and Linux system level APIs are C language interfaces more or less. Applications making Windows and/or system level calls would be my guess as to why.

It would have to be an absurd amount of money for you renege and make your wife quit. I just don’t see it happening. Good luck and please let us know what happens.

I can understand reluctance to use the STL of course - when I worked on video games, we used C++ without any of the STL - but like, std::vector dynamically allocates in older versions of C++ too. Looking strictly at language features, it's just better, including for things that are especially relevant to embedded environments like native move operators (C++11) or guaranteed copy elision (C++17).

I kinda wonder if the video game company I worked at is still on old school C++. When I left in 2014 I didn't even really know C++11 was a thing, so I feel like there's a good chance they are, lol.

I'm self taught and will admit to knowing very little about system level calls.
The program I wrote in perl ran on ubuntu on aws and did the following:

pull data from data supplier s3 bucket, in the form of gzip'd single line json file.
unzip file
parse lines of json
look for keywords,
if keywords match, grab other data
compare data to our database
if match found in our database put into "good" file, make entry in database
zip up good file
put good file in our s3 bucket

The reason they had abandoned it a few years before I got there is there was so much data no one figured out a way to process it faster than we got it in. We got 100 Million rows a night and it took them a weeks to process.

When I started my script took 2 weeks to process one day. By using the ssd's on aws, threading, optimizing databases, optimizing sql and threading the hell out of the process I got it down to 4 hours. I also set it up to run on as many servers as you wanted to and farm the work out to them as well which made it as fast as you wanted it to be. The fact that aws had just released 96 cpu boxes and I could run 50 threads at once was a big help too. And yes, before I was fired I was looking into lambda to do this even faster.

I'm guessing they never figured out some of that stuff in python, as the last script that I didn't finish was the one that controlled kicking off threads/servers and parsing out the work.

How much faster would a script running C be than a python one for parsing json and looking for keywords? I know the perl script was way faster than python, but I can't see the savings being enough for a rewrite that's going to take months.

The total aws bill for a month using my perl scripts was going to be <$2000. I set it up with aurora which would shut down when not running, and the 96 cpu boxes would fire up only when needed for processing and then shut down when done so only one dinky little box would run 24/7.

At my new job (new-ish, I guess, almost been there a year now) we have a big, organization-wide effort to migrate services from a Python implementation to C++. The resource savings/performance improvements have been enormous - like, 10x. So I'm inclined to think, especially given that we have really good infrastructure support to make sure all our platforms and languages run as well as possible, that C would be much more performant than Python.

That said:

This sounds very reasonable! We're doing our project because the scale we're working with is millions of dollars. Spending months of engineer time to rewrite a $2k/mo script is like, lol, come on, what are you doing.

Agreed.

It’s looking like even if it is an absurd amount of money that my wife isn’t on board, it’s going to have to be an obscene amount of money which almost certainly won’t happen.

I'm a big fan of the hybrid approach - figure out the worst performing parts of your system and replace it with C++.

Another big piece of low hanging fruit is that there are lots of python or perl or whatever libraries, but many of them are swiss army knives and all you want is a pair of tweezers. Re-implementing library code to do exactly and only what you want is often a big improvement. You usually trade off some flexibility though (as you do when you move to C++ or even just making some C/C++ libraries that integrate with your scripting language)

Like one that kind of surprised me was that I was able to get a huge speedup by taking a few endpoints and not converting them to json with the python json library (well, it's pretty slow tbf). The data was simple and the format established, so instead of using a general purpose json encoder, I just did static string formatting. It cut the encoding time by, like 99%. That encoding time was probably 20% of our total CPU usage for the whole site, so it made a big difference.

(I dont know why python's json encoder is so slow - the JS one is at least 4 or 5x faster. There are lots of 'aftermarket' json libraries that are faster than the base version though)

What's funny about that Rusty, I was looking for a comparison of C/perl/python json parsing speeds and found a perl json library that is written in C and is 10x faster than the library I was using. Could have got that 4 hours down to 1 hour. LOL rewriting the whole thing in C.