Two Plus Two Publishing LLC Two Plus Two Publishing LLC
 

Go Back   Two Plus Two Poker Forums > >

Notices

Computer Technical Help Post your questions about computer hardware and software and configuring same here.

Reply
 
Thread Tools Display Modes
Old 04-27-2017, 04:05 AM   #1
IsaacAsimov
old hand
 
IsaacAsimov's Avatar
 
Join Date: Oct 2006
Posts: 1,395
RAMDisk Inquiry

Hi guys. I was just wondering if anybody could give me some information on how compatible RAMDisks are with HM2 database setups. From what I know, a RAMDisk should drastically increase performance of database access times (based on the idea that it is so much faster than any currently possible SSD setup), however based on what I've read, it seems like that may not be the case and I also don't personally know anybody who runs his database off of a RAMDisk, even though there are plenty of people with large amounts of RAM nowadays.
What am I missing here? Is the PSQL database already caching so much in RAM, that it wouldn't make much of a difference either way?
For some background info, I typically work with not overly large databases. 10GB or less I would say typically.
IsaacAsimov is offline   Reply With Quote
Old 04-27-2017, 08:58 AM   #2
Noodle Wazlib
just about tolerable
 
Noodle Wazlib's Avatar
 
Join Date: Nov 2015
Location: Drowning in robot chocolate
Posts: 9,203
Re: RAMDisk Inquiry

Are you talking about nvram? Kind with m2 or whatever connections?
Noodle Wazlib is offline   Reply With Quote
Old 04-27-2017, 09:07 AM   #3
IsaacAsimov
old hand
 
IsaacAsimov's Avatar
 
Join Date: Oct 2006
Posts: 1,395
Re: RAMDisk Inquiry

No, I didn't know what NVRAM was until just now. I was thinking about DRAM, creating a RAMDisk using part of the RAM I have installed on my computer.
IsaacAsimov is offline   Reply With Quote
Old 04-27-2017, 01:03 PM   #4
headtrauma
centurion
 
Join Date: Jan 2007
Location: Redmond, WA
Posts: 178
Re: RAMDisk Inquiry

I don't know that it would be worth the effort/risk vs just configuring SQL to use a large amount of available RAM.
What is your use case here and what problem are you trying to solve? Are you reading from static DBs to do analysis, or are you using this as an OLTP actively importing hands?
If you were to go down this road, you would need at least a UPS for the PC to avoid dropping the DB on power loss. I would also have some concerns about potential data corruption from random flipped bits on non-ECC RAM (assuming this is running on a fairly standard desktop, not a pro-grade workstation or server).
headtrauma is offline   Reply With Quote
Old 04-27-2017, 10:41 PM   #5
IsaacAsimov
old hand
 
IsaacAsimov's Avatar
 
Join Date: Oct 2006
Posts: 1,395
Re: RAMDisk Inquiry

Quote:
Originally Posted by headtrauma View Post
I don't know that it would be worth the effort/risk vs just configuring SQL to use a large amount of available RAM.
What is your use case here and what problem are you trying to solve? Are you reading from static DBs to do analysis, or are you using this as an OLTP actively importing hands?
If you were to go down this road, you would need at least a UPS for the PC to avoid dropping the DB on power loss. I would also have some concerns about potential data corruption from random flipped bits on non-ECC RAM (assuming this is running on a fairly standard desktop, not a pro-grade workstation or server).
Mostly reading from static DBs for analysis, but I'm assuming the performance would also affect a HUD in-game, as it does involve active hand-import, especially if using Notecaddy.
I would be using a UPS, yes, and AM4 architecture, which does support ECC RAM.
IsaacAsimov is offline   Reply With Quote
Old 04-28-2017, 02:00 AM   #6
headtrauma
centurion
 
Join Date: Jan 2007
Location: Redmond, WA
Posts: 178
Re: RAMDisk Inquiry

You can give it a go, but from a quick search it seems that getting high performance out of HM2 and side tools/POSTGRESQL is difficult.
The guy in the thread below tried a RAM disk and NVME storage without resolving his issues with a large DB.
http://forums.holdemmanager.com/note...ml#post2462358
headtrauma is offline   Reply With Quote
Old 04-28-2017, 03:52 AM   #7
IsaacAsimov
old hand
 
IsaacAsimov's Avatar
 
Join Date: Oct 2006
Posts: 1,395
Re: RAMDisk Inquiry

Quote:
Originally Posted by headtrauma View Post
You can give it a go, but from a quick search it seems that getting high performance out of HM2 and side tools/POSTGRESQL is difficult.
The guy in the thread below tried a RAM disk and NVME storage without resolving his issues with a large DB.
http://forums.holdemmanager.com/note...ml#post2462358
Thanks for that link, it is very informational. I must have missed it during my initial research. OP does seem to think that for more HUD performance, I should be using a CPU with a high IPC per core, which would currenlty not favour using the AM4 platform (perhaps Ryzen 5 series will shine here).

For static database analysis, you are saying that I can simply play around with the PSQL settings to use more RAM correct? Also based on what OP said in that thread, having a very fast SSD will not affect performance much in that case (I guess higher performing RAM would potentially, but there aren't large performance differences with RAM for this use case I believe. Dual channel DDR4-RAM should be sufficient.).
IsaacAsimov is offline   Reply With Quote
Old 04-28-2017, 01:28 PM   #8
headtrauma
centurion
 
Join Date: Jan 2007
Location: Redmond, WA
Posts: 178
Re: RAMDisk Inquiry

Ryzen IPC should be fine. I'm looking to build a Ryzen system as soon as they finally bring out ITX boards. 5 series does not look to be any better or worse on single threaded performance. What I've read so far is that the fab process seems to hit a hard power wall at 3.9-4.1 GHz, so the 5 series parts are unlikely to clock much if any higher than the 7 series. Throwing 8 physical cores at a box used for SQL and other tasks is a good thing.
Assuming you have at least 32GB RAM, I think you should just look at tuning your PSQL settings to let it use a large amount of RAM and see where you are at. Having the DB on flash should be helpful for baseline performance. If you have 16GB or less, I would up your RAM to 32GB or 64GB. The ECC support on AM4 at this point comes with a giant *. As far as I know, none of the boards or CPUs has been through server/workstation class validation with ECC modules. With sufficient RAM assigned to SQL, it should leave tables cached in RAM while you're doing analysis work.
I'm not sure what the default HM2 install does as far as SQL backups, and I apologize for any language here that ends up needing translated from MS SQL to POSTGRESQL.
You should be sure to have an agent scheduled daily backup run (to a secondary drive, HDD is fine for BAK drive) with restore headers only validation. Running an index optimization/rebuild task during idle time should improve query perf as well (not sure if "AutoVacuum" covers this or not). Running the db in full recovery mode with log backups every 15 minutes gives you point in time recovery and can improve DB health.
headtrauma is offline   Reply With Quote
Old 04-28-2017, 10:07 PM   #9
IsaacAsimov
old hand
 
IsaacAsimov's Avatar
 
Join Date: Oct 2006
Posts: 1,395
Re: RAMDisk Inquiry

Quote:
Originally Posted by headtrauma View Post
Ryzen IPC should be fine. I'm looking to build a Ryzen system as soon as they finally bring out ITX boards. 5 series does not look to be any better or worse on single threaded performance. What I've read so far is that the fab process seems to hit a hard power wall at 3.9-4.1 GHz, so the 5 series parts are unlikely to clock much if any higher than the 7 series. Throwing 8 physical cores at a box used for SQL and other tasks is a good thing.
Assuming you have at least 32GB RAM, I think you should just look at tuning your PSQL settings to let it use a large amount of RAM and see where you are at. Having the DB on flash should be helpful for baseline performance. If you have 16GB or less, I would up your RAM to 32GB or 64GB. The ECC support on AM4 at this point comes with a giant *. As far as I know, none of the boards or CPUs has been through server/workstation class validation with ECC modules. With sufficient RAM assigned to SQL, it should leave tables cached in RAM while you're doing analysis work.
I'm not sure what the default HM2 install does as far as SQL backups, and I apologize for any language here that ends up needing translated from MS SQL to POSTGRESQL.
You should be sure to have an agent scheduled daily backup run (to a secondary drive, HDD is fine for BAK drive) with restore headers only validation. Running an index optimization/rebuild task during idle time should improve query perf as well (not sure if "AutoVacuum" covers this or not). Running the db in full recovery mode with log backups every 15 minutes gives you point in time recovery and can improve DB health.
ITX means Mini-ITX right? I would be interested in that too potentially, though I'm unsure how well that would work with overclocking given size restrictions for cooling purposes. If you do such a build at some point, please share it with us. I was surprised not to see any Ryzen builds here.

I heard somebody mention putting the DB on flash, but I'm not sure I quite understand that. Do you simply mean putting the DB on a flash drive, as in a USB-stick?

I actually found out yesterday that there is a new solver for PLO called Monker Solver. This might make a Ryzen build unattractive for the interim, since Ryzen doesn't support more than 64GB of RAM currently I believe. Using a virtual machine or Xeon build seems a lot more attractive now, heavily leaning towards the former.
IsaacAsimov is offline   Reply With Quote
Old 04-29-2017, 04:25 AM   #10
headtrauma
centurion
 
Join Date: Jan 2007
Location: Redmond, WA
Posts: 178
Re: RAMDisk Inquiry

First, by DB on flash, I just meant storing the actual DB files, logs, etc on an SSD. If you have an NVME SSD, all on one drive is probably your best bet. If you are running SATA, you may pick up a bit of speed having tempDB on its own SSD, but we're fairly deep in the weeds here. If you're getting poor performance on a 10GB DB with 2-3 connections on SSD with 32+GB of RAM and 8 cores, I blame the devs for any lack of performance their, assuming good baseline SQL config. I haven't played online for a long time though, so this is adapting just enough to be dangerous SQL knowledge from my day job.

With the right case, ITX shouldn't be a problem for overclocking, assuming high quality boards are made and not just bottom dollar junk. I go to LANs 3-4x a year and don't anticipate going back to a cross-fire or SLI setup (my current rig started with 2x7970s, went to a 980 for VR, currently on a 1080 because of charity raffle run-good).
I'm mainly looking at the case below which has room for a 2x120 closed loop liquid CPU cooler. Bear in mind that current Ryzen products are going to top out at 3.8-4.1 GHz. I don't have a problem with AMD leaving little OC headroom on the table, they're playing from behind and need to bring as much as they can stock.
http://www.caselabs-store.com/bullet...-customizable/
A VM (assuming cloud based) with 128GB seem like it would be pretty pricey. IIRC Azure SKUs with than much RAM run a few kilobucks a month, which would add up even running part time. I'm not familiar with Google or AWS pricing for similar.
headtrauma is offline   Reply With Quote
Old 04-30-2017, 12:37 AM   #11
IsaacAsimov
old hand
 
IsaacAsimov's Avatar
 
Join Date: Oct 2006
Posts: 1,395
Re: RAMDisk Inquiry

Thanks for elaborating.

I briefly tried Google Cloud to get a rough estimate for a VM with about 120 GB RAM (32 cores) and SSD running Windows.

https://gyazo.com/99ae56d1cdfad9f2e7e95536f9998311

Assuming I didn't mess something up, it does come out to about $170 per month running 2 hours per day, six days a week, so that is fairly pricey, and I agree that I could probably build a machine suited for my task at around $2000 that obviously adds additional value.
IsaacAsimov is offline   Reply With Quote
Old 04-30-2017, 03:11 AM   #12
headtrauma
centurion
 
Join Date: Jan 2007
Location: Redmond, WA
Posts: 178
Re: RAMDisk Inquiry

The SKU I had in my head for Azure (overkill for this task, we use them more for general purpose servers and also run Windows which you probably wouldn't need here) was the DS15_V2 (similar to below but with premium storage). It looks like pricing ends up in a similar ballpark.
D15 v2
20
140.00 GiB
1,000 GB
~$1,807.92/mo(always running) or $2.43/hr
https://azure.microsoft.com/en-us/pr...hines/windows/
headtrauma is offline   Reply With Quote
Old 04-30-2017, 04:25 AM   #13
IsaacAsimov
old hand
 
IsaacAsimov's Avatar
 
Join Date: Oct 2006
Posts: 1,395
Re: RAMDisk Inquiry

Thanks, yea that sounds like a good choice for the task at hand and is cheaper than the option I had selected, as I didn't selected an optimized for memory instance.
It's a little hard to say how much I would be running the machine, for simplicity I could assume 10hr/wk which would be reasonable given my current work schedule. So approximately 100$ per month. If you compared that to building a similar machine and replacing it every 3 years, you would probably break even on cost, working with very rough numbers here, so that makes the virtual machine a little bit more interesting, given the added flexibility (I do travel quite a bit).
Not easy to evaluate exactly what is best for me, but this isn't a bad option and doesn't require a lot of initial investment.

Why do you say I probably wouldn't need Windows here? Monkerware only runs on Windows or MacOs. Are you saying perhaps that I can easily get this to run on Linux somehow?
IsaacAsimov is offline   Reply With Quote
Old 04-30-2017, 08:18 AM   #14
headtrauma
centurion
 
Join Date: Jan 2007
Location: Redmond, WA
Posts: 178
Re: RAMDisk Inquiry

Ah, I see. I just erroneously assumed the tool ran on Linux as well.
Depending on usage and your other computer needs, a hybrid option may work for you. Build your self a 32GB Ryzen system for general awesomeness and small/medium simulations, then fire up the cloud instance for serious crunch time. I'm not sure how much cloud work you've done before, but Azure SKUs I work with have a fast but transient D: volume. It's faster than the OS disk, but loses its contents on reboot. This may be of use if Monkerware or your other tools can make use of a scratch drive.

A used single or dual socket 2011 workstation may be an option to consider as well.
As an example, here's a Z820 for $920 with 2x 8 core older gen Xeons. You can put 128-256GB of memory in this box and add disks, if the processor speed is enough to keep up with your workflow. You can spec it with 128GB for $1299 or go to 256GB for ~$2400
https://www.newegg.com/Product/Produ...2NS-0006-0NPR8

Depending on your router setup and security preferences, you could set up remote access for offsite use. Of course, the other side of the coin vs cloud here is giving up flexibility to upsize or downsize more or less on demand (Azure is pretty flexible that way within families, i.e. D1-15V2) as well as space, noise and electricity for having on prem.
headtrauma is offline   Reply With Quote
Old 04-30-2017, 11:23 PM   #15
IsaacAsimov
old hand
 
IsaacAsimov's Avatar
 
Join Date: Oct 2006
Posts: 1,395
Re: RAMDisk Inquiry

As far as I know, it's just Odds Oracle that runs on Linux, because it was written using Ruby on Rails. I'll ask the author of Monker Ware though, if perhaps he can support Linux, since I don't know much about his coding process and maybe this is something he would be willing to support. It does save a lot of money not to have to use the Windows license.

I think the way using Monker Ware with a high-powered virtual machine would work, would be to solve for a large game tree and then save that game tree. So persistent storage is probably not necessary, as this solution could presumably be viewed at a later point in time on a less powerful workstation.

Your hybrid solution sounds decent too or perhaps even better, since it would give me more frequent access to a more powerful machine at not so great a cost.
I've thought about the used dual Xeon workstation configurations, and while I do think they are good value for the money, I think that for actual playing, higher single core performance is still of benefit as it supposedly improves pop-up time in the HM HUD, which can make a significant difference when multi-tabling. In that sense, running a Ryzen build with OC and then a cloud machine for the occasional tasks that require a lot of RAM seems more appealing.
IsaacAsimov is offline   Reply With Quote

Reply
      

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off


Forum Jump


All times are GMT -4. The time now is 11:06 AM.


Powered by vBulletin®
Copyright ©2000 - 2017, Jelsoft Enterprises Ltd.
Search Engine Optimisation provided by DragonByte SEO v2.0.33 (Pro) - vBulletin Mods & Addons Copyright © 2017 DragonByte Technologies Ltd.
Copyright ę 2008-2010, Two Plus Two Interactive
 
 
Poker Players - Streaming Live Online