** UnhandledExceptionEventHandler :: OFFICIAL LC / CHATTER THREAD ** - Page 190 - Computer Technical Help

Two Plus Two Forums Other Topics Computer and Technical Help

** UnhandledExceptionEventHandler :: OFFICIAL LC / CHATTER THREAD **

Post Reply Subscribe

...

Page 190 of 1603

First

90 140 170 180 185 186 187 188 189 190 191 192 193 194 195 200 210 240 290 390 690 1190

Last

Page 190 of 1603

First

90 140 170 180 185 186 187 188 189 190 191 192 193 194 195 200 210 240 290 390 690 1190

Last

07-26-2012 , 06:18 PM

#4726

Gullanian

Carpal \'Tunnel

Join Date: Dec 2006 Posts: 14,014

That structure to me just seems really difficult to work with but maybe that's just because I'm conditioned to relational.

I find this bit interesting:

Code:

  votes : 5,
  voters : [ "jane", "joe", "spencer", "phyllis", "li" ],

Quote

$25m Guaranteed WPM on CoinPoker

Join the action now

Daily Rewards • Splash Pots • CoinRaces

07-26-2012 , 06:19 PM

#4727

Gullanian

Carpal \'Tunnel

Join Date: Dec 2006 Posts: 14,014

Quote:

Originally Posted by jjshabado

db.posts.distinct("tags")

Edit: So you'd query the documents and not create a separate collection. If you're expecting a large result set I think you need to use a map-reduce job.

Wouldn't this sort of thing scale horribly?

Quote

07-26-2012 , 06:23 PM

#4728

Shoe Lace

Pooh-Bah

Join Date: Sep 2004 Posts: 3,654

Quote:

Originally Posted by Neko

So if you wanted to query for all Tags, would you have to look through every single blog post to find out what tags are defined? Or do you store a collection of your tags elsewhere?

edit: or are those tag names, keys to retrieve information about a tag (like a foreign key)?

This is where it gets fun. It depends on your use case.

If you think you will need to query tags or comments on their own without the blog content then you'll likely want to separate them into their own collection (sort of like a table).

Now you're trading performance for flexibility because you're running multiple queries instead of 1.

You could also do both at once and depending on what area of your page is being read, you could choose to pick the most efficient one. Now you're trading disk space for performance (reasonable IMO on a very high concurrency page).

You could also start caching stuff at the application level so your multiple queries are just stored in redis and now reading them is insta. This trades performance for code complexity/maintenance/worrying about cache bs.

I think a lot of people really under estimate what a good host is capable of serving. If you step away from $5/month shared hosting, you can actually serve a lot of requests per second.

TBH a blog post is kind of a bad example because really it's highly connected data. I would use a graph DB for a bloggy type of site.

Quote

07-26-2012 , 06:24 PM

#4729

jjshabado

Carpal Tunnel

Join Date: Jul 2006 Posts: 22,732

Quote:

Originally Posted by Gullanian

Wouldn't this sort of thing scale horribly?

It's web scale.

It depends a lot on your use cases. If you're doing tons of different analytics then mongo might not be the best solution. If you're doing a standard blog server - then the benefits easily outweigh the costs.

I haven't done much map-reduce querying, but I suspect they've got the performance covered.

Also, if you're doing a web blog server the structure of the document in the data is probably going to match the structure of the document on the client - which is pretty freaking awesome.

Quote

07-26-2012 , 06:45 PM

#4730

anononon

Guest

Posts: n/a

Mongo configuration and monitoring is the biggest pain in the ass, though.

Quote

07-26-2012 , 06:49 PM

#4731

jjshabado

Carpal Tunnel

Join Date: Jul 2006 Posts: 22,732

Quote:

Originally Posted by kyleb

Mongo configuration and monitoring is the biggest pain in the ass, though.

I can't remember any major problems, but we moved to https://mongolab.com/home awhile ago so we don't have to worry about much.

Quote

07-26-2012 , 06:53 PM

#4732

Shoe Lace

Pooh-Bah

Join Date: Sep 2004 Posts: 3,654

I never had any problems either. I didn't have to configure anything other than telling it which directory the db should live in.

Quote

07-26-2012 , 06:56 PM

#4733

Shoe Lace

Pooh-Bah

Join Date: Sep 2004 Posts: 3,654

CouchDB also has free cloud hosting.

I set one up for 2+2:
http://2p2.iriscouch.com/_utils

Feel free to do whatever you want, it's free. I figured this would save people the 5 seconds it took to sign up (no need to verify your e-mail).

Ignore the _users and _replicator databases. Those are internally used by couch.

Quote

07-26-2012 , 07:04 PM

#4734

gaming_mouse

Carpal \'Tunnel

Join Date: Oct 2004 Posts: 13,786

for those who've used both mongo and couch, what would you recommend learning? no specific project in mind, so i guess ease/fun/interestingness of use would be a big factor

Quote

07-26-2012 , 07:12 PM

#4735

Shoe Lace

Pooh-Bah

Join Date: Sep 2004 Posts: 3,654

There's Riak too. IMO Mongo is better for general purpose, that is why it's so popular. You give up small edges for general usage but the things you give up kind of don't matter because chances are your web app is never going to be the biggest thing ever.

Throwing a few hundred million documents in it and having a couple hundred thousand users is fine as long as you budgeted your hardware correctly and your full web stack is setup properly. If you grow past this point then congrats, you can afford to hire the best guys in the industry and let them worry about everything.

Quote

07-26-2012 , 07:39 PM

#4736

kerowo

lolcat

Join Date: Nov 2005 Posts: 37,183

Quote:

Originally Posted by kerowo

Still waiting for my app credit for picking up the new MBP.... damit!

Apple support says I probably won't get it until Saturday. I don't know if I want to wait that long to save 20 bucks...

Quote

07-26-2012 , 07:42 PM

#4737

LA_Price

adept

Join Date: Feb 2004 Posts: 1,119

@Shoe Lace: and anyone else that's interested. I put sample companies in the database from the Crunchbase API so you can see what the data looks like. I'll put some sample MapReduce funcitions in javascript in their too.

Here's the script in python to upload(although you need a list of what are called permalinks). These function as sort of the 'strong keys' of the database

Code:

# save all the json objects to couchdb
server = couchdb.Server('http://2p2.iriscouch.com')
db = server['companies']

# point program at links file and pick out company permalinks

permalinks = [line.strip() for line in open('C:\Documents and Settings\Luke\Desktop\Programming\Python\py web programming\crunchbase_test.txt', 'r')]

# loop over the list fetching all json objects. Then save to couchdb
for name in permalinks:
    h = urllib.urlopen('http://api.crunchbase.com/v/1/company/%s.js' % (name))
    try:
        data = json.loads(h.read())
    except:
        continue
    if not 'name' in data:
        continue
    db.save(data)

Quote

07-26-2012 , 07:42 PM

#4738

jjshabado

Carpal Tunnel

Join Date: Jul 2006 Posts: 22,732

Quote:

Originally Posted by Shoe Lace

+1.

The main reason we used Mongo is because it makes development fast. Our team has a lot of experience with traditional databases but if you're building something where you're adding/deleting tables/fields regularly then Mongo is awesome. It's quite possible that we'll get to a point where we'll need to change databases - but if we make it to that point its probably a good thing overall.

Quote

07-26-2012 , 08:02 PM

#4739

clowntable

Carpal \'Tunnel

Join Date: Jun 2006 Posts: 45,557

To add a very subjective fact to the debate...the main downside of using a noSQL DB would be that Postgres is really ****ing awesome

I really don't get why companies even bother with Oracle or MSSQL or the like (well I kinda get it stuff like lol-SAP only run on those and there's this and that but really comeonson).

Quote

07-26-2012 , 08:16 PM

#4740

anononon

Guest

Posts: n/a

MS SQL makes me want to vomit

Last edited by anononon; 07-26-2012 at 08:16 PM. Reason: oracle only slightly less so

Quote

07-26-2012 , 08:34 PM

#4741

Shoe Lace

Pooh-Bah

Join Date: Sep 2004 Posts: 3,654

Quote:

Originally Posted by LA_Price

Cool man, thanks for filling it with sample data. Also for people viewing this, in couch's web based admin panel you can view the design document here:

http://2p2.iriscouch.com/_utils/docu...design/company

Then expand each view. That will show you the map and reduce (reduce isn't present atm) code.

Then go back to the main document list:
http://2p2.iriscouch.com/_utils/data...nies/_all_docs
Select one of the views from the drop down on the top right'ish. The 2 bottom most options.

Alternatively the following links will show you the results of running the map on each view. I just directly linked them:

View: company_tags_descripton
http://2p2.iriscouch.com/_utils/data...ags_descripton

View: location_if_ipo
http://2p2.iriscouch.com/_utils/data...ocation_if_ipo

Quote

07-26-2012 , 08:53 PM

#4742

LA_Price

adept

Join Date: Feb 2004 Posts: 1,119

Here's a map reduce view of categories on the attribute founded year using the built in _stats function

http://2p2.iriscouch.com/_utils/data...dedyear__stats

you just have to hit the reduce check box in the top right

Quote

07-27-2012 , 09:17 AM

#4743

clowntable

Carpal \'Tunnel

Join Date: Jun 2006 Posts: 45,557

Feel free to ramble/provide ideas regarding the design of hand histories for a poker server, including DB stuff:
http://forumserver.twoplustwo.com/19...s-etc-1227234/

it's mostly a thread to keep track of my own thoughts but any input will be gladly appreciated

Quote

07-27-2012 , 10:56 AM

#4744

Chips Ahoy

help me help you

Join Date: Apr 2007 Posts: 28,869

Maybe a mod could break out this database discussion to a thread? Seems like good stuff.

Are there mods in this forum?

Quote

07-27-2012 , 11:00 AM

#4745

MrWooster

veteran

Join Date: Mar 2007 Posts: 2,984

10gen offer a free MongoDB monitoring service at https://mms.10gen.com

Quote

07-27-2012 , 11:49 AM

#4746

LA_Price

adept

Join Date: Feb 2004 Posts: 1,119

Couchdb has admin party(which is basically just let anyone do anything at the start). What does Mongo have?

Quote

07-27-2012 , 03:48 PM

#4747

Shoe Lace

Pooh-Bah

Join Date: Sep 2004 Posts: 3,654

Quote:

Originally Posted by LA_Price

Couchdb has admin party(which is basically just let anyone do anything at the start). What does Mongo have?

Mongo has native drivers for whatever language you're working with (well, the popular ones at least). You could however build your own http api on top of one if you wanted. There's likely already open source projects in most languages where this was done.

The main reference guide for drivers:
http://www.mongodb.org/display/DOCS/Drivers

Quote

07-27-2012 , 04:20 PM

#4748

anononon

Guest

Posts: n/a

If anyone wants to take a swipe at this, that'd be awesome:

http://stackoverflow.com/questions/1...5-3-threadsafe

I can't connect to MSSQL (not my choice, trust me, I hate MSSQL) from PHP's libraries from Mac/Linux/Windows/whatever. My first choice is obv to use sqlsrv_connect() under Linux, but that is basically a ****ing pipedream. I will settle for using it in Windows, which should be easy, but hasn't been.

Goddamn I hate Microsoft products, particularly MSSQL. Also PHP but that is another debate entirely.

Quote

07-27-2012 , 05:03 PM

#4749

daveT

S.A.G.E. Master

Join Date: Jun 2005 Posts: 23,955

Quote:

Originally Posted by MrWooster

Just initiated my first Facebook ad campaign. Anyone had any experience with FB ads? I have set it up to be very specifically targeted and the advert points to the Facebook page of my site. Basically doing a minimum spend to see what happens and try and asses the worth based on the results.

If you're doing a banner-type ad, you're doing really good if you get 1 click per 1,000 impressions. That's actually common advertising data for web interfaces: usually 1/10% click-through is your "expectation" with almost every web portal that has that metric available. So, extrapolate your b/e and go from there.

That data-point made little sense to me until I read something along the lines of this: "Think about it a TV commercial you seen recently. Easy, isn't it? Now think about the last web banner you seen, or any web banner you've seen in the past week. Can't do it, can you?"

Edit to ad: I need to be specific that a banner ad is usually, "My Company Logo and Why We Rule," which shouldn't be confused with a product ad, which is "Click Here to Buy this Brand New POS."

Another Edit: There's a reason that banner ads are so low cost-per-impression. It would behoove you to remember you get what you pay for, and if one is 8c per impression compared to 4c per impression, buy the 8c per impression.

Quote

07-27-2012 , 07:25 PM

#4750

Neko

Pooh-Bah

Join Date: Jul 2005 Posts: 4,000

Quote:

Originally Posted by kyleb

I'm more familiar with Postgres but I've been using SQL Server 2008 with python/pyodbc for a medium sized Django app at work and it's been working out great so far. Seems to be pretty decent software IMO.

Anyways, I know next to nothing about PHP but have you tried PHP's odbc_connect? It should work fine from Linux too I think.

edit: Also, I could have sworn you were defending PHP recently?