Don't forget to check out the source code of the article.

?

<mild brag>

Got into Insight data engineering program. Start in June 2 and super pumped. Seems like a really good program for job placement. I will be in the first data engineering class (previous ones were data science PhDs). 5% acceptance rate. It was application, coding challenge (1 week), and video interview.

If you're interested just apply guys. I kind of applied on a whim and did not expect to get in

http://insightdataengineering.com/

Sorry, didn't realize that a site had reposted this, look at the source of the original post on the guy's blog:

http://stilldrinking.org/

Congrats dude, if you don't mind sharing, what's your background like?

Win7 at home and at work. I've found Chrome to be amply fast, but unstable once on a site with javascript.

I also use other utilities to make my life more pleasant.

I agree with this part, but looks like we don't have a choice anymore. I still like ff on Linux more.

Mild? That looks hard as stones to get into. Nice work.

anyone here ever run recursive chmod on /* instead of ./* accidentally?

;_;

apparently I just ruined my installation and have to reinstall. fml. maybe its time to learn vagrant.

Vagrant is about to get a whole lot better too. With 1.6 you can spin up docker containers instead of virtual machines.

http://www.vagrantup.com/blog/featur...ironments.html

Going to be awesome for quickly setting up staging environments to test deploys.

So I did some number crunching today..

I can quit my job, and start going back to school Fall 2014 for second major in CS, and expect to graduate Spring 2017(Maybe Winter 2017) if I get lucky and do summer classes.

This would leave me graduating at age 28 with ~40k in student loans.

Another option is to delay going to school so I can work & save $$ which leaves me graduating in Winter 2018 with ~20k in student loans at age 29.

Thoughts?

Ah probably stupid idea. My friend talked me out of it. School will teach me way more than what I need if I wanted to get into Mobile Development. I will make much more use of my time self-studying and supplementing it with online courses that help me reach my goal.

Amazing article indeed!

My fav quote for now, but the whole thing is full f suitable candidates:

yeah that describes my life pretty accurately

Count me in as someone who would like to know more about your background and current job situation. I am looking to go into a data science boot camp and didn't know Insight offered a non-Ph.D. program. I just started a thread about my possible attendance to Zipfian Academy. Did you apply there or even heard of it?

6 weeks is a very short amount of time and with my current skill set I don't think it'd be enough to give me the skills to secure the right position for the risk. The free tuition sounds really nice. Although, is it free only if you are hired through their recruitment services?

need some help to connect Postgres and AHK... PM plz

Did anyone see this explanation of partitioning in PostgreSQL?

https://github.com/fiksu/partitioned..._EXPLAINED.txt

There is so much awesome in the database, but then something like this comes along and brings it all crashing down.

can you elaborate? i read it and found it to be a good explanation, but i wasn't clear at one point i would need to worry about this stuff, if ever. the practical implications were missing...

The example is just fine if you have 5 companies, but what happens when you have 1,000 companies or say 100 companies with 30 year-old records?

The caveats section talks about a few of the issues: http://www.postgresql.org/docs/9.3/i...IONING-CAVEATS

As for practical. You are correct, there isn't much in the way of practical outside of using larger stores like PostGIS or something. I guess this is one of those areas where if you need scale, you will hopefully migrate to something else, but still, the implementation is gross no matter which way you chop it.

Thanks. No problem. I have bachelors in computer engineering and math. Played poker for 5 years. Returned to do masters in math few years ago. Recently 2 years experience writing supply chain software (first developer job).

We can take it offline if you're interested in the interview process specifics.

The data engineering program (non PhD) is new, and I will be in the first cohort. I did research other programs but this one seemed the best fit for my background.

The program is targeted at experienced developers. That doesn't mean you need 10 years experience. I think they are looking for hungry developers with existing ability to code and wanting to learn new technologies. The tuition is free regardless of employment. But you do sign a contract that agrees to interview with their partner companies. If I'm interested in companies not involved in the program I have to go through them instead of job seeking on my own. You're also required to mentor at least 5 days (in the following year) for future students. Mentoring is about 2 hours 5 times.

Can you briefly summarize what a data engineer is and what technologies you would expect to be working with when you get out of the program?

It's the #1 most desired job according to the last ranking I saw...before that I hadn't even heard of the job description. I'm assuming they find interesting ways to crunch and report data and learn from existing data (trend analysis, predictions)

So my guess would be algorithm and stats heavy, lots of Bayes, lots of AI stuff, SVMs, probably some econometrics etc.

I thought big data was dying already?

Generally it is building infrastructure for the data pipeline. That is, setting up a distributed system that extracts data from multiple sources (e.g. logs, messages), cleanses the data, maps to SQL and NoSQL systems, and finally presents an API and visualization tools to the end user (data scientists, other web services etc).

The standard stack is Hadoop and it's related infrastructure (Hive, HBase, Pig, Cassandra to name a few). Java and Python are the most widely supported languages (Hadoop is written in Java).

I suppose if companies are willing to purge their enormous volumes of valuable data

The backend stuff seems like a different job/skill than the frontend visualization stuff. I feel like visualizations (stuff like what this guy does) would be more analogous to UX / Graphic Design (in the traditional web dev metaphor) whereas the backend stuff would be, well, like backend stuff.

Does the program teach all of it?