This is certainly reasonable - and if I were using Heroku I'd be planning to get off of it at some point to start to realize those savings. But there are lots of things that companies can do to save/make money. And it's silly to say that learning to do your sys admin stuff right from the start is definitely a priority over all of those other things.

Changing the topic - does anyone know of good free poker datasets? I don't really care what it is so long as its sized somewhere between 10MB and 10GB and somewhat related to poker.

I run ******** and we datamine around 8 gigs a day. What do you need it for? PM me if you want or post on here. We like helping out developers / researchers.

I want to play around with a Pig/Hadoop tool we're building and since I like poker it seemed like a poker dataset would be more interesting for me to play with then most of the other free datasets online.

perfect

Is this like a generalizable version of that scrensaver thingy that use to be popular where your computer would analyze data looking for ET communication signals when you weren't using it?

I'm not sure what you mean by "this". Do you mean hadoop?

you really think there's no correlation between the skillsets of a developer and a sysadmin? There's common ground in the skills themselves, and also in the fact that people who are interested in one are more likely to have some affinity for the other (more than, say, your average truck driver).

Plus the more you understand about the whole setup, the better. Driving the delivery truck probably won't help you write better code, but being familiar with the system on which it's going to run definitely will.

All that said, I'm sure Heroku does make economic sense for a lot of companies. But you can't pretend one skill has absolutely nothing to do with the other.

shabby,

by "this" he means "your project with poker data". by "screensaver thingy" he means "seti@home".

hth,
tyler

no, i meant "seti@home" and "hadoop or any of its subprojects"

i'm not saying there is no correlation, but neither skill implies the other logically, and the absence of either skill does not imply the absence of the other.

the vast majority of people who know as much as i do about programming and web development know way more about server admin. it's definitely a big knowledge gap for me and lately i've been really feeling the consequences. so i agree with you in that sense.

on the other hand, many server admin types, while they might know a bit of programming, are really not good at it imo, and make all sorts of sloppy mistakes.

Hadoop is basically an open source framework for supporting MapReduce jobs on distributed systems. With it you can pretty easily write some Java code to process a huge chunk of data on a distributed system of N nodes.

Pig is a layer on top of Hadoop and offers a simple programming language for writing data processing jobs which it then compiles into map reduce jobs that get run on Hadoop.

It's not really similar to seti@home - except for the fact that they're all dealing with distributed computations.

could you use it to do parallel computing tasks like solving big poker abstractions, or does that require too much node to node communication?

I am really interested in this myself and I am really interested in also doing some stuff with Mahout on top of that. Talk to me via PM or skype 'kaderbelbina' and we'll figure out if we could work on something like this together (I supply data / we both play around w/ it and have access to code / environments etc).

I also have about 5 computers doing nothing right now for a test cluster :-D

It would most likely help, but it depends a lot on the specific problem.

If you're talking about something that takes in hundreds of thousands (or millions, or billions) of hand histories and boils that information down to something small (like a ranking of biggest money winners to biggest money losers) - then it would definitely help. That's basically the ideal situation for a MapReduce problem.

If you're talking about something that is much more related to pure computation (like simulating a hand match-up billions of times), the answer is a lot less clear. Hadoop would still probably help because it abstracts the problem of dealing with a cluster of computers - but like you said you might be paying a big price in terms of inter-node communication. It's surprising though how often a problem can be broken up in a way that works really well with Hadoop.

Thanks for helping me out!

I'm hoping to take a look at these tomorrow and maybe I'll start a thread where I give some examples of what I'm doing.

*disclaimer* I am not a sysadmin. I'm a programmer by nature but also have a fairly good set of sysadmin skills

You are naively simplifying the task of an admin. Setting up a server properly isn't about popping an installation cd in the computer and then double clicking some .exe for the application server, then simply dragging and dropping your deployment to some directory.

On the flip side, do you think an administrator can google for 2-3hrs and start writing enterprise level applications?

If your company can't afford it, fine...that's your perogative to save money. but when

• server gets hacked
• server stops responding because disk is full and you didn't partition properly

See how many hours of googling it takes your team to fix it.

I have enough experience with it that I'm not just randomly typing empty statements. I wouldn't call myself a full blown sys admin either but I've installed multiple distros of Linux, setup multiple web servers (not using Apache/PHP), actively develop in Linux and feel comfy enough that my inability to know every last detail about Linux isn't holding me back.

Epic documentary some of you will love:
http://watchdocumentary.com/watch/co...d562b3a03.html

Follows netscape at their peak, going open source (Mozilla) and then selling to AOL. Had some classic lines in it: "AOL is for idiots" and had me lolling a lot from the endearing nature of how computers were only 10 years ago.

A*

Wow epic. Great quote about 11:50 in:

'This is a monk like existence. There are very few women in these societies. These are male societies. These are secret societies. They function very much like the Masons or some street gang.'

I loved the dude they couldn't find to fix bugs who had a bazillion cans of coke on his desk. And the scenes of families playing in the house with the man in the dark room in the background programming with a headset on. So great!

The coder from thousands of miles away doing open source contributions made me lol, when asked why he was doing it he just said "because the code is really bad and messy!" lololol what a great documentary

Haha yeah. That guy just sat around refactoring their code for free, what a hero. Absolutely awesome doc. Gonna make everyone watch it here at our startup.

Another one of my favorite quotes was 'Its almost like an addiciton, an adreneline rush, of going for perfection and then, as you see the results, you get the feedback to push harder'.

The irony was I've been up since 5am coding and watched that documentary at around 730am.

HTML5 physics
http://www.scirra.com/labs/physics/

I don't ever write many .bat commands for windows often, but can someone tell me if it's possible to:

- Connect to FTP
- Rename _app_offline.htm to app_offline.htm in the remote folder
- Upload by FTP a directory I specify ignoring folders/files I specify to ignore
- Delete app_offline.htm

In one set of commands?

Also, is it possible to publish a website into a local folder (usually done via VS2010) with a bat command thus saving me another click?

This would be super useful if it was possible but I don't know if it is.

it's bound to be possible. no idea how on windows though