Open Side Menu Go to the Top
Register
** UnhandledExceptionEventHandler :: OFFICIAL LC / CHATTER THREAD ** ** UnhandledExceptionEventHandler :: OFFICIAL LC / CHATTER THREAD **

04-16-2016 , 01:32 PM
I'm curious, without ORMs how do you make an app (standard JS frontend, API backend) in a timely matter?

As the scope of your app grows, isn't most of your time spent writing SQL instead of features?

How do you avoid rolling your own ORM if you want basic customization such as field filtering API serializers? (api/object/id?fields=title,body,etc)
** UnhandledExceptionEventHandler :: OFFICIAL LC / CHATTER THREAD ** Quote
04-16-2016 , 02:10 PM
Quote:
Originally Posted by gaming_mouse
i'm writing an API in node against a postgres backend.

i've posted about this before, and have done some research on various ORMs like sequelize, and query builders like knex. now i'm feeling like, **** it, i should just use node-postgres and write actual SQL which a) I have lots of experience with and b) is likely a better and more stable language than whatever SQL-adapter language any of these libraries has conconcocted.

What am I losing? The ability to swap out another database with no changes to my code? Yet a) that prospect is vanishingly unlikely to happen with the production app so b) the only real advantage is the ability to swap out to an in-memory db for testing. But I can always run a local postgres for testing which should still be fast, so advantage of b) isn't as great as it would seem.

Still, I feel like it's kind of a dirty thing to do, probably mainly because it's not done that much, which is ofc a terrible reason. Thoughts?
Totally agree with this from my experience. If you know your specific requirements well including how they might evolve, and aren't some monster site with sophisticated caching needs, it seems always cleaner to just roll your own. Then after a while, as patterns emerge, you can look for or write a lightweight helper to abstract the repetitive stuff.

I've worked in web development since the late 90s and have never once heard a requirement to swap out DBs w/o touching the rest of the stack. DBs are very very sticky. Usually if you swap them out you're also revamping the whole stack and site - so very unlikely your old DB-access business logic would even be relevant. The only time I can think of that you would need this capability - is if you're writing middleware for sale or distribution to a lot of clients and being database agnostic is a requirement.

We pay Oracle $27M per year in ransomware and would desperately like to get off of it. Not gonna happen any time soon. It will be the absolute last thing we swap out and by far the most onerous/risky.

Last edited by suzzer99; 04-16-2016 at 02:15 PM.
** UnhandledExceptionEventHandler :: OFFICIAL LC / CHATTER THREAD ** Quote
04-16-2016 , 03:34 PM
Quote:
Originally Posted by RogerKwok
I'm curious, without ORMs how do you make an app (standard JS frontend, API backend) in a timely matter?

As the scope of your app grows, isn't most of your time spent writing SQL instead of features?

How do you avoid rolling your own ORM if you want basic customization such as field filtering API serializers? (api/object/id?fields=title,body,etc)
I never used an ORM before a year or 2 ago, and I've been working in database backed applications for about 20 years. I don't find them useful. Writing SQL is very easy once you've done it a little and I do not in general spend a lot of time writing it.

In terms of frontend/backend dev time, I am a backend dev and by far the holdups are on the front end side. I'm not being critical here, just saying that I can deliver backend API functions way faster than they can be integrated into the FE. So much so that I am learning more FE stuff so that the project can be more balanced.
** UnhandledExceptionEventHandler :: OFFICIAL LC / CHATTER THREAD ** Quote
04-16-2016 , 05:18 PM
Can you talk a bit more about your role and the type of app(s) you work on?

I'm still kinda new to the industry and basically the 2 companies I've been with, there's only fullstack engineers. Do you guys do continuous integration?
** UnhandledExceptionEventHandler :: OFFICIAL LC / CHATTER THREAD ** Quote
04-16-2016 , 05:26 PM
Quote:
Originally Posted by RogerKwok
Can you talk a bit more about your role and the type of app(s) you work on?

I'm still kinda new to the industry and basically the 2 companies I've been with, there's only fullstack engineers. Do you guys do continuous integration?
Yeah, we do continuous integration. Both the FE and BE release multiple times/day. The app we're currently working on is mostly used in-house which simplifies things. There are 2 BE engineers and 3 FE (which can be increased to 3 BE and 4 FE if we need it).

Most of the work we do is tightly synchronized, i.e. we need a new feature, the BE and FE will work on it together at the same time, then release it for testing, then release it to production.

We also have customer facing stuff, but most of the work we've done in the last 6 months has been, in essence, to automate the creation of everything customer facing.

(I work for MapMyFitness, which is owned by UnderArmour now, in their client services group)
** UnhandledExceptionEventHandler :: OFFICIAL LC / CHATTER THREAD ** Quote
04-16-2016 , 05:27 PM
Quote:
Originally Posted by RogerKwok
Can you talk a bit more about your role and the type of app(s) you work on?

I'm still kinda new to the industry and basically the 2 companies I've been with, there's only fullstack engineers. Do you guys do continuous integration?
Just curious - full stack meaning DB schema design to CSS? I've worked at one place like that but it seems far from the norm.
** UnhandledExceptionEventHandler :: OFFICIAL LC / CHATTER THREAD ** Quote
04-16-2016 , 05:30 PM
Quote:
Originally Posted by suzzer99
Just curious - full stack meaning SQL to CSS? I've worked at one place like that but it seems far from the norm.
I've never seen it personally, and never really heard of it, unless the backend and frontend used the same tech, i.e. node.js or something similar. However, I am working on my JS/React so I guess I might be a full stack developer again.

When I started programming, that was a looooot simpler.
** UnhandledExceptionEventHandler :: OFFICIAL LC / CHATTER THREAD ** Quote
04-16-2016 , 05:30 PM
Quote:
Originally Posted by RustyBrooks
Yeah, we do continuous integration. Both the FE and BE release multiple times/day. The app we're currently working on is mostly used in-house which simplifies things. There are 2 BE engineers and 3 FE (which can be increased to 3 BE and 4 FE if we need it).

Most of the work we do is tightly synchronized, i.e. we need a new feature, the BE and FE will work on it together at the same time, then release it for testing, then release it to production.

We also have customer facing stuff, but most of the work we've done in the last 6 months has been, in essence, to automate the creation of everything customer facing.

(I work for MapMyFitness, which is owned by UnderArmour now, in their client services group)
Must be nice to work at a place that's committed to CI/CD beyond slideware*.

*Meaning as long as my bosses can claim CI/CD on the Powerpoints, they aren't interested in pushing it any further.
** UnhandledExceptionEventHandler :: OFFICIAL LC / CHATTER THREAD ** Quote
04-16-2016 , 05:33 PM
We moved to real CI about 2 years ago, same time I started. I'm not going to say it's been without hiccups, but, once your ticket has passed tests and been peer reviewed, it goes to production. On some projects, getting the "+2" on a peer review automatically deploys it right then. On others you have to also press another button (mostly the difference is, depending on the deploy methodology, some are easier to roll back than others, and/or take much longer than others, so they are in need of a "are you sure" method)

Prior to that we released 1x/day, and to get your ticket released, you just had to link it to the release ticket in JIRA.
** UnhandledExceptionEventHandler :: OFFICIAL LC / CHATTER THREAD ** Quote
04-16-2016 , 05:36 PM
That's awesome. There are a bunch of reasons why I don't think we could ever be true CI/CD - mainly that we connect to about a dozen back-end systems over which we have zero control, and those systems connect to hundreds more. So creating intelligent test data that lives in those systems is out of the question.

I'm not sure it can be done at a monster site like ours in a big corporation with a ton of functionality - w/o the infinite resources and top-down commitment of a google or Facebook.

But we could definitely be pushing forward a lot harder than we are. I've sent about a dozen emails to management and leads about how we're not taking testing seriously, it needs to be mandatory not optional, and how we're not evaluating our testing at all to see if it's effective or we're getting the most bang for our buck. Crickets every time. They just want the checkbox checked.
** UnhandledExceptionEventHandler :: OFFICIAL LC / CHATTER THREAD ** Quote
04-16-2016 , 05:51 PM
Quote:
Originally Posted by adios
FWIW the reasons you received are complete BS and not stating this just to make you feel better. Don't beat youself up but yeah sure hone your interviewing skills.
Yup, definitely could use some improvements there.

Quote:
Originally Posted by daveT
Wow, I'm glad you got some feedback, even if it is something everyone would disagree with.
Yeah I was surprised to hear any feedback and the recruiter seemed reasonably sincere in wanting me to reapply in a year or so, though I guess that would be true of any non-terrible candidate from their perspective.

Quote:
Originally Posted by candybar
one near-rejection where they were about to make an offer but a last-minute corporate level decision just made it unlikely they can move forward with any hires;
Got another email from them later in the evening saying that I did so well that they are still trying to figure out a way to get me onboard despite the near-hiring freeze. Interesting though it's probably not a great idea to go into that kind of situation.
** UnhandledExceptionEventHandler :: OFFICIAL LC / CHATTER THREAD ** Quote
04-16-2016 , 06:03 PM
Quote:
Originally Posted by muttiah
Hey candybar, what are you looking for (fee free to PM me). We are always hiring senior people
Hey, I'm looking for a senior-level position or above in NYC at a substantially derisked company with at least 30+ engineers, where I'd spend at least 25% of my time on hands-on development. I'm well-rounded and can learn most things pretty quickly so I don't care too much about the technology stack or which part of the stack as long as they are comfortable with my skills and experience. Beyond that I guess my comp requirements are kind of on high side.
** UnhandledExceptionEventHandler :: OFFICIAL LC / CHATTER THREAD ** Quote
04-16-2016 , 06:06 PM
Thanks Rusty for sharing your experience. I'm very curious how your backend is structured. I took a look at your site and 'Developer/API' links to Under Armour, who seems to have REST for web, but SDK for mobile. How do you fit into that?

In your opinion, if you need to add more features and need to hire 50 more engineers, do you think separating FE and BE is still possible?

Suzzer, we do CSS down to DB schema, but we use Django. The company started with PHP and wrote an in-house mapper, but it didn't scale well past 30+ engineers or so.
** UnhandledExceptionEventHandler :: OFFICIAL LC / CHATTER THREAD ** Quote
04-16-2016 , 06:14 PM
Quote:
Originally Posted by gaming_mouse
i've posted about this before, and have done some research on various ORMs like sequelize, and query builders like knex. now i'm feeling like, **** it, i should just use node-postgres and write actual SQL which a) I have lots of experience with and b) is likely a better and more stable language than whatever SQL-adapter language any of these libraries has conconcocted.

What am I losing? The ability to swap out another database with no changes to my code? Yet a) that prospect is vanishingly unlikely to happen with the production app so b) the only real advantage is the ability to swap out to an in-memory db for testing. But I can always run a local postgres for testing which should still be fast, so advantage of b) isn't as great as it would seem.
I don't mind lightweight database wrapper "ORM" that lets you do simple things simply and lets you write whatever SQL you want for complex queries but I've never found much use for full-blown ORM and complex query builders and such. I guess they make SQL injection less likely.
** UnhandledExceptionEventHandler :: OFFICIAL LC / CHATTER THREAD ** Quote
04-16-2016 , 06:25 PM
^^^
** UnhandledExceptionEventHandler :: OFFICIAL LC / CHATTER THREAD ** Quote
04-16-2016 , 07:30 PM
Quote:
Originally Posted by RogerKwok
Thanks Rusty for sharing your experience. I'm very curious how your backend is structured. I took a look at your site and 'Developer/API' links to Under Armour, who seems to have REST for web, but SDK for mobile. How do you fit into that?
I honestly don't like our API at all. I don't really know what the SDK provides, mobile just uses the same API endpoints as web, maybe it wraps it into native objects or something, I dunno. I am sure you'll be shocked to find that we don't really use our own API much.

Quote:
In your opinion, if you need to add more features and need to hire 50 more engineers, do you think separating FE and BE is still possible?
If we had a project that needed 50 more engineers, I am quite confident it would never be completed.

Most groups at our company are between 5 and 10 people, and that'll include FE, BE, possible a mobile dev, a project manager and/or a team lead. Adding people is scalable as long as they're working on things that can be broken down into relatively independent teams. Any attempt I've ever seen to scale engineering groups past about that size have been pretty awful.

But anyway, I'm not sure what it is about the FE/BE division that you think makes it unscalable or untenable. I add a backend feature, I deploy it to our test environment, the FE deploys his FE part, and we try it out. We've usually agreed on the all the details before either of us started. 90% of the time it goes off without a hitch, the rest of the time, there is some misunderstanding about something which gets cleared up in an hour or so.

I think possibly our group is much better than avg, even at our company. Certainly, we shocked a lot of people with the quality/scope of this project and everyone who's seen it has really liked it. A few people have quietly said "I wish the rest of the site was this good". But that's a rant for some other day.
** UnhandledExceptionEventHandler :: OFFICIAL LC / CHATTER THREAD ** Quote
04-17-2016 , 11:09 AM
Quote:
Originally Posted by candybar
I don't mind lightweight database wrapper "ORM" that lets you do simple things simply and lets you write whatever SQL you want for complex queries but I've never found much use for full-blown ORM and complex query builders and such. I guess they make SQL injection less likely.
well node-postgres has prepared statements, so it protects against injections. but i'm still writing raw sql in that case -- it's just that inputs are left as variables in the strings, and node-postgres i believe makes sure they are kosher before executing.

but yeah, it sounds like i'm basically getting a thumbs up from you, suzzer, and rusty, so i'm feeling better about my plan.
** UnhandledExceptionEventHandler :: OFFICIAL LC / CHATTER THREAD ** Quote
04-17-2016 , 01:25 PM
Can someone explain to me, in plain English, why it is so hard to store properly hashed passwords and why they feel that sending your password in plain-text is okay to do?

This is a college, which is simply mind-blowing, but not as bad as the site that bragged about securing the global digital world.
** UnhandledExceptionEventHandler :: OFFICIAL LC / CHATTER THREAD ** Quote
04-17-2016 , 03:32 PM
Quote:
Originally Posted by candybar
Hey, I'm looking for a senior-level position or above in NYC at a substantially derisked company with at least 30+ engineers, where I'd spend at least 25% of my time on hands-on development. I'm well-rounded and can learn most things pretty quickly so I don't care too much about the technology stack or which part of the stack as long as they are comfortable with my skills and experience. Beyond that I guess my comp requirements are kind of on high side.
We fit the criteria except it's San Francisco.
** UnhandledExceptionEventHandler :: OFFICIAL LC / CHATTER THREAD ** Quote
04-17-2016 , 03:33 PM
Quote:
Originally Posted by daveT
Can someone explain to me, in plain English, why it is so hard to store properly hashed passwords and why they feel that sending your password in plain-text is okay to do?

This is a college, which is simply mind-blowing, but not as bad as the site that bragged about securing the global digital world.
Yeah it is totally clown pants, especially since secure encryption of passwords has been part of the unix OS for many many decades. Which is to say, every unix admin in the world should understand the basics because they're confronted with it literally all the time.
** UnhandledExceptionEventHandler :: OFFICIAL LC / CHATTER THREAD ** Quote
04-17-2016 , 03:46 PM
Rusty it seems like you have a great group! I guess I was overestimating the amount of time needed to sync between FE and BE, it seems you guys have that under control.

When we were doing our migration from PHP to Django, the API changes were bothersome to coordinate with mobile, but I guess that was a one time thing.

Can you talk more about your BE work? I feel like with Django we spend maybe 10-20% of our time there, since the libraries are just amazing. What kind of problems take up most of your time?
** UnhandledExceptionEventHandler :: OFFICIAL LC / CHATTER THREAD ** Quote
04-17-2016 , 09:43 PM
Quote:
Originally Posted by muttiah
We fit the criteria except it's San Francisco.
Yeah that's a bit of a problem for me. It's quite unbelievable how heavily concentrated great tech jobs are in SF/SV - NYC is probably in the second tier along with Seattle and Boston and the gap is larger than it seemed before I started looking. But relocation is not even close to being an option for me at the moment.
** UnhandledExceptionEventHandler :: OFFICIAL LC / CHATTER THREAD ** Quote
04-17-2016 , 09:50 PM
You think NYC, Seattle and Boston have more tech jobs than LA?
** UnhandledExceptionEventHandler :: OFFICIAL LC / CHATTER THREAD ** Quote
04-17-2016 , 11:11 PM
Quote:
Originally Posted by suzzer99
You think NYC, Seattle and Boston have more tech jobs than LA?
I didn't mean to exclude LA per se since I really don't know where Austin, LA, Boulder, etc fit in but compared to LA I do think NYC/Seattle/Boston have a higher concentration of companies that consider technological innovation an important competitive advantage, better funding infrastructure for tech startups and somewhat higher compensation for software engineers. For example:

https://mattermark.com/where-to-get-...ilicon-valley/

LA could very well surpass Boston and Seattle over the next decade or so as top tech companies expand their presence and we see more convergence between the tech economy and the broader economy though.
** UnhandledExceptionEventHandler :: OFFICIAL LC / CHATTER THREAD ** Quote
04-18-2016 , 12:06 AM
Hey Kansas City made the list. I could go back there and live like a king. A very bored king.
** UnhandledExceptionEventHandler :: OFFICIAL LC / CHATTER THREAD ** Quote

      
m