Tag Archives: Folding@home

How to Make a Folding@Home Space Heater (and why would you want to?)

My normal posts on this site are all about how to do as much science as possible with Folding@Home, for the least amount of power. This is because I think disease research, while a noble and essential cause, shouldn’t be done without respecting the environment.

With that said, I think there is a use case for a power-hungry, inefficient Folding@Home computer. Namely, as a space heater for those in colder climates.

The logic is this: Running Folding@Home, or any other piece of software, makes your computer do work. Electricity flows through the circuits, flipping tiny silicon switches, and producing heat in the process. Ultimately all of the energy that flows into your computer comes back out as heat (well, a small amount comes out as light, or electromagnetic radiation, or noise, but all of those can and do get converted back into heat as they strike things in the room).

Have you ever noticed how running your gaming computer with the door to your room closed makes your feet nice and toasty in the winter? It’s the same idea. Here, one of my high-performance rigs (dual NVidia 980 Ti GPUs) is silently humming away, putting off about 500 watts of pleasant heat. My son is investigating:

My Folding@Home Space Heater Experiment

Folding@Home uses CPUs and GPUs to run molecular dynamic models to help research understand and fight diseases. You get the most points per day (PPD) by using cutting-edge hardware, but the Folding@Home Consortium and Stanford University openly encourage everyone to run the software on whatever they happen to have.

With this in mind, I started thinking about all the old hardware that is out there…CPUs and graphics cards that are destined for landfills because they are no longer fast enough to do any useful gaming or decode 4K video. People describe this type of hardware as “bricks” or “space heaters”–useful for nothing other than wasting power.

That gave me an idea…

It didn’t take me long to find a sweet deal on an nForce 680i-based system on eBay for $60 shipped (EVGA board with Nvidia n680i chipset, supporting three full-length PCI-E X16 slots). I swapped out the Core 2 Duo that this machine came with for a Core 2 Quad, and purchased four Fermi-based Nvidia graphics cards, plus a used 1300 Watt Seasonic 80+ Gold power supply. All of this was amazingly cheap. The beautiful Antec case was worth the $60 cost of the parts that came with it alone. Because I knew lots of power would be critical here, I spent most of the money on a high-end power supply (also used on eBay). Later on, I found that I needed to also upgrade the cooling (read: cut a hole in the side panel and strap on some more fans).

  • Antec Mid-Tower Case + Corsair 520 Watt PSU, EVGA 680i motherboard, Core 2 Duo CPU, 4 GB Ram, CD Drives, and 4 Fans = $60
  • 2x EVGA Nvidia GeForce GTX 480 graphics cards: $40
  • 1 x EVGA NVidia GeForce GTX 580 Graphics Card: $50
  • 1 x EVGA NVidia GeForce GeForce GTX 460 Graphics Card: $20
  • 1 x PCI-E X1 to X16 Riser: $10
  • 1 x Core 2 Quad Q6600 CPU (Socket 775) – $6
  • 1 x Seasonic 1300 Watt 80+ Gold Modular Power Supply: $90
  • 2 x Noctua 120 MM fans + custom aluminum bracket (for modifying side panel): $60
  • 1 x Arctic Cooling Freezer Tower Cooler – $10
  • 1 x Western Digital Black 640GB HDD – $10

Total Cost (Estimated): $356

This is the cost before I sold some of the parts I didn’t need (Core 2 Duo, Corsair PSU, etc).

Here is a shot of the final build. It took a bit of tweaking to get it to this point.

F@H_Space_Heater_Quad_GPUs

Used Parts Disclaimer!

Note that when dealing with used parts on eBay, it’s always good to do some basic service. For the GPUs in this build, I took them apart, cleaned them, applied fresh thermal paste (Arctic MX-4), and re-assembled. It was good that I did…these cards were pretty gross, and the decade-old thermal paste was dried on from years of use.

 

I mean, come on now, look at the dust cake on the second GTX 480! Clean your graphics cards, random eBay people!

GTX 480 Dust

Here’s how the 3 + 1 GPUs are set up. The two GTX 480s and the GTX 580 are on the mobo in the X16 slots. I remotely mounted the GTX 460 in the drive bay. I used blower-style (slot exhaust) cards on purpose here, because they exhaust 100% of the hot air outside the case. Open-fan style cards would have overheated instantly in this setup.

To keep costs down, I just used Ubuntu Linux as the operating system. I configured the machine for 4-slot GPU folding using proprietary Nvidia drivers. Although I ultimately control all of my remote Linux machines with TeamViewer, it is helpful to have a portable monitor and combo wireless keyboard/mouse for initial configuration and testing. In the shot below (of an earlier config), I learned a lot just trying the get the machine stable with 3 cards.

Space_Heater_Early_Config_Initial_Fireup_small

Initial Testing on the Space Heater (3 GPUs installed). This test showed me that I needed better CPU cooling (hence I chucked that stock Intel cooler)

I also did some thermal testing along the way to make sure things weren’t getting too hot. It turns out this testing was a bit misleading, because the system was running a lot cooler with the side panel off than with it on.

Some Thermal Camera Images During Initial Burn-In (3 GPUs, stock CPU cooler):

Now that’s some heat coming out of this beast! Thankfully, the upgraded 14-gauge power plug and my watt meter aren’t at risk of melting, although they are pretty warm.

Once I had the machine up and running with all four GPUs the final configuration, I found that it produced about 55-95K PPD on average (based on the work unit), with the following breakdown

  • GTX 460: 10-20K PPD
  • GTX 480: 20-30K PPD each
  • GTX 580: 25-45 K PPD

Power consumption, as measured at the wall, ranged from 900 to 1000 watts with all 4 GPUs engaged. By turning different GPUs on and off, I could get varying levels of power (about 200 Watts idle. I typically ran it with one 580 and one 480 folding, for an average power consumption of about 600 watts).

Space_Heater_Power_Consumption

After running the machine for a while, my room was nice and toasty, as expected!

One thing that I should mention was the effect of the two additional intake fans that I mounted in the side panel. Originally I did not have these, and the top graphics card in the stack was hitting 97 degrees C according to the onboard monitoring! After modding this custom side-intake into the case (found a nice fan bracket on Amazon, and put my dremel tool to good use), the temps went down quite a lot. I used fan grilles on the inside of the fans to keep internal cables out of them, and mesh filters on the outside to match the intake filters on the rest of the case.

 

The top card stays under 85 degrees C (with the fan at 50%). The middle card stays under 80 degrees C, and the bottom card runs at 60 degrees C. The GTX 460 mounted in the drive bay never goes over 60 degrees C, but it’s a less powerful card and is mounted on the other side of the case.

Here’s some more pictures of the modded side panel, along with a little cooling diagram I threw together:

PPD, Wattage, and Efficiency Comparison

I debated about putting these plots in here, because the point of this machine was not primarily to make points (pun intended), or to be efficient from a PPD/Watt perspective. The point of this machine was to replace the 1500 watt space heater I use in the winter to keep a room warm.

As you can see, the scientific production (PPD) on this machine, even with 4 GPUs, is not all that impressive in 2020, since the GPUs being used are ten years old. Similarly, the efficiency (PPD/Watt) is terrible. There’s no surprise there, since it averages just under 1000 watts of power consumption at the wall!

Conclusion

It is totally possible to build a (relatively) inexpensive desktop computer out of old, used parts to use as a space heater. If the primary goal is to make heat, then this might not be a bad idea (although at $350, it still costs way more than a $20 heater from Walmart). The obvious benefit is that this sort of space heater is actually doing something useful besides keeping you warm (in this case, helping scientists learn more about diseases thanks to Folding@Home).

Other benefits that I found were the remote control (TeamViewer), which lets me use my cellphone to turn GPUs on and off to vary the heat output. Also, I think running this machine for extended durations in its medium-high setting (700 watts or so) is much healthier for the electrical wiring in my house vs. the constant cycling on and off of a traditional 1500 watt space heater.

From an environmental standpoint, you can do much worse than using electric heat. In my case, electric space heaters make a lot of sense, especially at night. I can shut off the entire heating zone (my house only has two zones) to the upstairs and just keep the bedroom warm. This drastically reduces my fossil fuel usage (good old New England, where home heating oil is the primary method of keeping warm in the winter). Since my house has an 8.23 KW solar panel array on the roof, a lot of my electricity comes directly from the sun, making this electric heat solution even greener.

Parting Thoughts:

I would not recommend running a machine like this during the warmer months. If warm air is not wanted, all the waste heat from this machine will do nothing but rack up your power bill for relatively little science being done. If you want to run an efficient summer-time F@H rig that uses low power (so as to not fight your AC) , check out my article on the GTX 1660 and 1650.

In a future article, I plan to show how I actually saved on heating costs by running Folding@Home space heaters all last winter (with a total of seven Folding@Home desktops placed strategically throughout my house, so that I hardly had to burn any oil).

 

AMD Ryzen 9 3950X Folding@Home Review: Part 1: PPD vs # of Threads

Welcome back everyone. Over the last month, I’ve been experimenting with my new Folding@Home benchmark machine to see how effectively AMD’s flagship Ryzen processor (Ryzen 9 3950X) can fight diseases such as COVID-19, Cancer, and Alzheimer’s. I’ve been running Folding@Home, a charitable distributed computing project, which provides scientists with valuable computing resources to study diseases and learn how to combat them.

This blog is typically focused on energy efficiency, where I try to show how to do the most science for the least amount of power consumption possible. In this post, I’m stepping away from that (at least for now) in order to understand something much simpler: how does the Folding@Home CPU client scale with # of processor threads?

I’d previously investigated Folding@Home performance and efficiency vs. # of CPU cores on an old Intel Q6600. I’ve also done a few CPU articles on AMD’s venerable Phenom II X6 1000T and my previous processor, the AMD FX-8320e. These CPU articles were few and far-between however, as I typically focus on using graphics cards (GPUs). The reason is twofold. Historically, graphics cards have produced many more points per day (PPD) for a given amount of power, thanks to their massively parallel architecture, which is well-suited for running single precision molecular dynamics problems such as those used by Folding@Home. Also, graphics cards are much easier to swap out, so it was relatively easy to make a large database of GPU performance and efficiency.

Still, CPU folding is just as important, because there are certain classes of problems that can only be efficiently computed on the CPU. Folding@Home, while originally a project that ran exclusively on CPUs, obtains the bulk of its computational power from GPU donors these days. However, the CPU folders sill play a key part, running work units that cannot be solved on GPUs, thus providing a complete picture of the molecular dynamics.

In my last article, I highlighted the need for me to build a new benchmark machine for testing out GPUs, since my old rig would soon become a bottleneck and slow the GPUs down (thus potentially affecting any comparison plots I make). Now that this Ryzen-based 16-core monster of a desktop is complete, I figured I’d revisit CPU folding once more to see just what a modern enthusiast-class processor like the $749 Ryzen 9 3950X is capable of. For this first part of a multi-part review, I am simply looking at the preliminary results from running Folding@Home on the CPU. Instead of running with the default thread settings, I manually set up the client, examining just how performance results scale from the 1 to 32 available threads on the Ryzen 9 3950x.

Test Setup

Testing was performed in Windows 10 Home, using the latest Folding@Home client (7.6.13). Points Per Day were estimated from the client window for each setting of # of CPU threads. These instantaneous estimates have a lot of variability, so future testing will investigate the effect of averaging (running multiple tests at each setting) on the results.

Benchmark Machine Hardware:

Case Raidmax Sagitta (2006)
Power Supply Seasonic Prime 750 Titanium
Fresh Air 2 x 120 mm Enermax Front Intake
Rear Exhaust 1 x 120 mm Scythe Gentile Typhoon
Side Exhaust 1 x 80 mm Noctua
Top Exhaust 1 x 120 mm (Seasonic PSU)
CPU Cooler Noctua NH-D15 SE AM4
Thermal Paste Arctic MX-4
CPU AMD Ryzen 9 3950X 16 Core 32 Thread (105W TDP)
Motherboard ASUS Prime X570-P Socket AM4
Memory 32 GB (4 x 8 GB) Corsair Vengeance LPX DDR4 3600 MHz
GPU Zotac Nvidia GeForce 1650
OS Drive Samsung 970 Evo Plus 512 GB NVME SSD
Storage #1 Samsung 860 Evo 2 TB SSD
Storage #2 Western Digital Blue 256 GB NVME SSD (for Linux)
Optical Samsung SH-B123L Blu-Ray Drive
OS Windows 10 Home, Ubuntu Linux (on 2nd NVME)

Processor Settings:

The AMD Ryzen 9 3950x is a beast. With 16 cores and 32 threads, it has a nominal power consumption of 105 watts, but can easily double that when overclocked. With the factory Core Performance Boost (CPB) enabled, the processor will routinely draw 150+ watts when loaded due to the individual cores turboing as high as 4.7 GHz, up from the 3.5 GHz base clock. Under heavy multi-threaded work loads, the processor supports an all-core overclock of up to 4.3 GHz, assuming sufficient cooling and motherboard power delivery.

This automatic core turbo behavior is problematic for creating a plot of folding at home performance (PPD) vs # of threads, since for lightly threaded loads, the processor will scale up individual cores to much higher speeds. In order to make an apples to apples comparison, I disabled CPB, so that all CPU cores run at the base speed of 3.5 GHz when loaded. In future testing, I will perform this study with CPB on in order to see the effect of the factory automatic overclocking.

A note about Cores vs. Threads

Like many Intel processors with Hyper-Threading, AMD supports running multiple code execution strings (known as threads) on one CPU core. The Simultaneous Multi-Threading (SMT) on the Ryzen 9 3950x is simply AMD’s term for the same thing: a doubling of certain parts within each processor core (or sometimes the virtualization of multiple threads within one CPU core) to allow multiple thread execution (two threads per core, in this case). The historical problem with both Hyper-Threading and SMT is that it does not actually double a CPU core’s capacity to perform complex floating point mathematics, since there is only one FPU per CPU core. SMT and Hyperthreading work best when there is one large job hogging a core, and the smaller job can execute in the remaining part of the core as a second thread. Two equally intensive threads can end up competing for resuorses within a core, making the SMT-enabled processor actually slower. For example: https://www.techspot.com/review/1882-ryzen-9-smt-on-vs-off/

For the purposes of this article, I left SMT on in order to make the coolest plot possible (1-32 threads!). However, I suspect that SMT might actually hurt Folding@Home performance, for the reasons mentioned above. Thus in future testing, I will also try disabling this to see the effect.

Preliminary Results: PPD vs # Threads on Ryzen 9 3950x

So, to summarize the caveats, this test was performed once under each test condition (# of threads), so there are 32 data points for 32 threads. SMT was on (so Folding@Home can run two threads on one CPU core). CPB was off (all cores set to 3.5 GHz).

The figure below shows the results. As you can see, there is a general trend of increasing performance with # of threads, up to around the halfway point. Then, the trend appears to get messy, although by the end of the plot, it is clear that the higher thread counts realize a higher PPD.

Ryzen 9 3950X PPD vs Thread Count 1

Observations

It is clear that, at least initially, adding threads to the solution makes a fairly linear improvement in points per day. Eventually, however, the CPU cores are likely becoming saturated, and more of the work is being executed in via SMT. Due to the significant work unit variability in Folding@Home (as much as 10-20% between molecules), these results should be taken with a grain of salt. I am currently re-running all of these tests, so that I can show a plot of average PPD vs. # of Threads. I am also logging power using my watt meter, so that we can make wall power consumption and efficiency plots.

Conclusions

Seeing a processor produce nearly half a million points per day in Folding@Home was insane! My previous testing with old 4, 6, and 8-core processors was lucky to show numbers over 20K PPD. In general, allowing Folding@Home to use more processor threads increases performance, but there is significant additional work needed to verify a statistical trend. Stay tuned for Part II (averaging).

P.S.

Man, that’s a lot of cores! You’d better be scared, COVID-19…I’m coming for you!

Cores!

So Many Cores!

New Folding@Home Benchmark Machine: It’s RYZEN TIME!

Folding@Home, the distributed computing project that fights diseases such as COVID-19 and cancer, has hit an all-time high in popularity. I’m stunned to find that my blog is now getting more views every day than it did every month last year. With that said, this is a perfect opportunity to reach out and see if all the new donors are interested in tuning their computers for efficiency, to save a little on power, lighten the burden on your wallet, and hopefully produce nearly the same amount of science. If this sounds interesting to you, let me know in the comments below!

In my last post, I noted that the latest generation of graphics cards are starting to push the limits of what my primary GPU Folding@Home benchmark rig can do. That computer is based on an 11-year-old chipset (AMD 880), and only supports PCI-Express 2.0. In order for me to keep testing modern fast graphics cards in Windows 10, I wanted to make sure that PCI-Express slot bandwidth wasn’t going to artificially bottleneck me.

So, without further ado, let me present the new, re-built Folding@Home rig, SAGITTA:

Sagitta Desktop

I’ve (re)created a monster!

This build leverages the Raidmax Sagitta case that I’ve had since 2006. This machine has hosted multiple builds (Pentium D 805, Core 2 Duo e8600, Core 2 Quad Q6600, Phenom II X6 1100T, and the most recent FX-8320e Bulldozer). There have been too many graphics cards to count, but the latest one (Nvidia GTX 1650 by Zotac) was carried over for some continuity testing. The case fans and power supply (initially) were also the same since the previous FX build (they aren’t the same ones from back in 2006…those got loud and died long ago). I also kept my Blu-Ray drive and 3.5 inch card reader. That’s where the similarities end. Here is a specs comparison:

Sagitta Rebuild Benchmark Machine Specs

  • Note I ended up updating the power supply to the one shown in the table. More on that below…

System Power Consumption

Initially, the power consumption at idle of the new Ryzen 9 build, measured with my P3 Kill A Watt Meter, was 86 watts. The power consumption while running GPU Folding was 170 watts (and the all-core CPU folding was over 250 watts, but that’s another article entirely).

Using the same Nvidia GeForce GTX 1650 graphics card, these idle and GPU folding power numbers were unfortunately higher than the old benchmark machine, which came in at 70 watts idle and 145 watts load. This is likely due to the overkill hardware that I put into the new rig (X570 motherboards alone are known to draw twice the power of a more normal board). The system’s power consumption difference of 25 watts while folding was especially problematic for my efficiency testing, since new plots compared to graphics cards tested on the old benchmark machine would not be comparable.

To solve this, I could either:

A: Use a 25 watt offset to scale the new GPU F@H efficiency plots

B: Do nothing and just have less accurate efficiency comparisons to previous tests

C: Reduce the power consumption of the new build so that it matches the old one

This being a blog about energy efficiency, I decided to go with Option C, since that’s the one that actually helps the environment. Lets see if we can trim the fat off of this beast of a computer!

Efficiency Boost #1: Power Supply Upgrade

The first thing I tried was to upgrade the power supply. As noted here, the power supply’s efficiency rating is a great place to start when building an energy efficient machine. My old Seasonic X-650 is a very good power supply, and caries an 80+ Gold rating. Still, things have come a long way, and switching to an 80+ Titanium PSU can gain a few efficiency percentage points, especially at low loads.

80+ Table

80+ Efficiency Table

With that 3-5% efficiency boost in mind, I picked up a new Seasonic 750 Watt Prime 80+ Titanium modular power supply. At $200, this PSU isn’t cheap, but it provides a noticeable efficiency improvement at both idle and load. Other nice features were the additional 100 watts of capacity, and the fact that it supported my new motherboard’s dual pin (8 + 4) CPU aux power connection. That extra 4-pin isn’t required to make the X570 board work, but it does allow for more overclocking headroom.

Disclaimer: Before we get into it, I should note that these power readings are “eyeball” readings, taken by glancing at the watt meter and trying to judge the average usage. The actual number jumps around a bit (even at idle) as the computer executes various background tasks. I’d say the measurement precision on any eyeball watt meter readings is +/- 5 watts, so take the below with a grain of salt. These are very small efficiency improvements that are difficult to measure, and your mileage may vary. 

After upgrading the power supply, idle power dropped an impressive 10 watts, from 86 watts to 76. This is an awesome 11% efficiency improvement. This might be due to the new 80+ Titanium power supply having an efficiency target at very low loads (90% efficiency at 10% load), whereas the old 80+ Gold spec did not have a low load efficiency requirement. Thus, even though I used a large 750 watt power supply, the machine can still remain relatively efficient at idle.

Under moderate load (GPU folding), the new 80+ titanium PSU provided a 4% efficiency improvement, dropping the power consumption from 170 watts to 163. This is more in line with expectations.

Efficiency Boost #2: Processor Underclock / Undervolt

Thanks to video gaming mentality, enthusiast-grade desktop processors and motherboards are tuned out of the box for performance. We’re talking about blistering fast, competition-crushing benchmark scores. For most computing tasks (such as running Folding@Home on a graphics card), this aggressive CPU behavior is wasting electricity while offering no discernible performance benefit. Despite what my kid’s shirt says, we need to reel these power hungry CPUs in for maximum GPU folding efficiency.

Never Slow Down

Kai Says: Never Slow Down

One way to improve processor efficiency is to reduce the clock rate and associated voltage. I’d previously investigated this here. It takes exponentially more voltage to support high frequencies, so just by dropping the clock rate by 100 MHz or so, you can lower the voltage a bunch and save on power.

With the advent of processors that up-clock and up-volt themselves (as well as going in the other direction), manual tuning can be a bit more difficult. It’s far easier to first try the automatic settings, to see if some efficiency can be gained.

But wait, this is a GPU folding benchmark rig? Why does the CPU’s frequency and power settings matter?

For GPU folding with an Nvidia graphics card, one CPU core is fully loaded per GPU slot in order to “feed” the card. This is because Nvidia’s implementation of open CL support using a polling (checking) method. In order to keep the graphics card chugging along, the CPU constantly checks on the GPU to see if it needs any data. This polling loop is not efficient and burns unnecessary power. You can read more about it here: https://foldingforum.org/viewtopic.php?f=80&t=34023. In contrast, AMD’s method (interrupts) is a much more graceful implementation that doesn’t lock up a CPU core.

The constant polling loop drives modern gaming-oriented processors to clock up their cores unnecessarily. For the most part, the GPU does not need work at every waking moment. To save power, we can turn down the frequency, so that the CPU is not constantly knocking on the GPU’s metaphorical door.

To do this, I disabled AMD’s Core Performance Boost (CPB) in the AMD Overclocking section of the BIOS (same thing as Intel’s Turbo Boost). This caps the processor speed at the base maximum clock rate (3.5 GHz for the Ryzen 9 3950x), and also eliminates any high voltage values required to support the boost clocks.

Success! GPU folding total system power consumption is now much lower. With less superfluous power draw from the CPU, the wattage is much more comparable to the old Bulldozer rig.

Ryzen 9 3950x Power Reduction Table

It is interesting that idle power consumption came down as well. That wasn’t expected. When the computer isn’t doing anything, the CPU cores should be down-clocked / slept out. Perhaps my machine was doing something in the background during the earlier tests, thus throwing the results off. More investigation is needed.

GPU Benchmark Consistency Check

I fired up GPU folding on the Nvidia GeForce GTX 1650, a card that I have performance data for from my previous benchmark desktop. After monitoring it for a week, the Folding@Home Points Per Day performance was so similar to the previous results that I ended up using the same value (310K PPD) as the official estimate for the 1650’s production. This shows that the old benchmark rig was not a bottleneck for a budget card like the GeForce GTX 1650.

Using the updated system power consumption of nominally 140 watts (vs 145 watts of the previous benchmark machine), the efficiency plots (PPD/Watt) come out very nearly the same. I typically consider power measurements of + / – 5 watts to be within the measurement accuracy of my eyeball on the watt meter anyway, due to normal variations as the system runs. The good news is that even with this variation, it doesn’t change the conclusion of the figure (in terms of graphics card efficiency ranking).

GTX 1650 Efficiency on Ryzen 9

* Benchmark performed on updated Ryzen 9 build

Conclusion

I have a new 16-core beast of a benchmark machine. This computer wasn’t built exclusively for efficiency, but after a few tweaks, I was able to improve energy efficiency at low CPU loads (such as Windows Idle + GPU Folding).

For most of the graphics cards I have tested so far, the massive upgrade in system hardware will not likely affect performance or efficiency results. Very fast cards, such as the 1080 Ti, might benefit from the new benchmark rig’s faster hardware, especially that PCI-Express 4.0 x16 graphics card slot. Most importantly, future tests of blistering fast graphics cards (2080 Ti, 3080 Ti, etc) will probably not be limited by the benchmark machine’s background hardware.

Oh, I can also now encode my backup copies of my blu-ray movies at 40 fps in H.265 in Handbrake (old speed was 6.5 fps on the FX-8320e). That’s a nice bonus too.

Efficiency Note (for GPU Folding@Home Users)

Disabling the automatic processor frequency and voltage scaling (Turbo Boost / Core Performance Boost) didn’t have any effect on the PPD being generated by the graphics card. This makes sense; even relatively slow 2.0 GHz CPU cores are still fast enough to feed most GPUs, and my modern Ryzen 9 at 3.5 GHz is no bottleneck for feeding the 1650. By disabling CPB, I shaved 23 watts off of the system’s power consumption for literally no performance impact while running GPU folding. This is a 16 percent boost in PPD/Watt efficiency, for free!

This also dropped CPU temps from 70 degrees C to 55, and resulted in a lower CPU cooler fan speed / quieter machine. This should promote longevity of the hardware, and reduce how much my computer fights my air conditioning in the summer, thus having a compounding positive effect on my monthly electric bill.

Future Articles

  • Re-Test the 1080 Ti to see if a fast graphics card makes better use of the faster PCI-Express bus on the AM4 build
  • Investigate CPU folding efficiency on the Ryzen 9 3950x

 

Shout out to the helpers…Kai and Sam

Folding@Home on Turing (NVidia GTX 1660 Super and GTX 1650 Combined Review)

Hey everyone. Sorry for the long delay (I have been working on another writing project, more on that later…). Recently I got a pair of new graphics cards based on Nvidia’s new Turing architecture. This has been advertised as being more efficient than the outgoing Pascal architecture, and is the basis of the popular RTX series Geforce cards (2060, 2070, 2080, etc). It’s time to see how well they do some charitable computing, running the now world-famous disease research distributed computing project Folding@Home.

Since those RTX cards with their ray-tracing cores (which does nothing for Folding) are so expensive, I opted to start testing with two lower-end models: the GeForce GTX 1660 Super and the GeForce GTX 1650.

 

These are really tiny cards, and should be perfect for some low-power consumption summertime folding. Also, today is the first time I’ve tested anything from Zotac (the 1650). The 1660 super is from EVGA.

GPU Specifications

Here’s a quick table I threw together comparing these latest Turing-based GTX 16xx series cards to the older Pascal lineup.

Turing GPU Specs

It should be immediately apparent that these are very low power cards. The GTX 1650 has a design power of only 75 watts, and doesn’t even need a supplemental PCI-Express power cable. The GTX 1660 Super also has a very low power rating at 125 Watts. Due to their small size and power requirements, these cards are good options for small form factor PCs with non-gaming oriented power supplies.

Test Setup

Testing was done in Windows 10 using Folding@Home Client version 7.5.1. The Nvidia Graphics Card driver version was 445.87. All power measurements were made at the wall (measuring total system power consumption) with my trusty P3 Kill-A-Watt Power Meter. Performance numbers in terms of Points Per Day (PPD) were estimated from the client during individual work units. This is a departure from my normal PPD metric (averaging the time-history results reported by Folding@Home’s servers), but was necessary due to the recent lack of work units caused by the surge in F@H users due to COVID-19.

Note: This will likely be the last test I do with my aging AMD FX-8320e based desktop, since the motherboard only supports PCI Express 2.0. That is not a problem for the cards tested here, but Folding@Home on very fast modern cards (such as the GTX 2080 Ti) shows a modest slowdown if the cards are limited by PCI Express 2.0 x16 (around 10%). Thus, in the next article, expect to see a new benchmark machine!

System Specs:

  • CPU: AMD FX-8320e
  • Mainboard : Gigabyte GA-880GMA-USB3
  • GPU: EVGA 1080 Ti (Reference Design)
  • Ram: 16 GB DDR3L (low voltage)
  • Power Supply: Seasonic X-650 80+ Gold
  • Drives: 1x SSD, 2 x 7200 RPM HDDs, Blu-Ray Burner
  • Fans: 1x CPU, 2 x 120 mm intake, 1 x 120 mm exhaust, 1 x 80 mm exhaust
  • OS: Win10 64 bit

Goal of the Testing

For those of you who have been following along, you know that the point of this blog is to determine not only which hardware configurations can fight the most cancer (or coronavirus), but to determine how to do the most science with the least amount of electrical power. This is important. Just because we have all these diseases (and computers to combat them with) doesn’t mean we should kill the planet by sucking down untold gigawatts of electricity.

To that end, I will be reporting the following:

Net Worth of Science Performed: Points Per Day (PPD)

System Power Consumption (Watts)

Folding Efficiency (PPD/Watt)

As a side-note, I used MSI afterburner to reduce the GPU Power Limit of the GTX 1660 Super and GTX 1650 to the minimum allowed by the driver / board vendor (in this case, 56% for the 1660 and 50% for the 1650). This is because my previous testing, plus the results of various people in the Folding@Home forums and all over, have shown that by reducing the power cap on the card, you can get an efficiency boost. Let’s see if that holds true for the Turing architecture!

Performance

The following plots show the two new Turing architecture cards relative to everything else I have tested. As can be seen, these little cards punch well above their weight class, with the GTX 1660 Super and GTX 1650 giving the 1070 Ti and 1060 a run for their money. Also, the power throttling applied to the cards did reduce raw PPD, but not by too much.

Nvidia GTX 1650 and 1660 performance

Power Draw

This is the plot where I was most impressed. In the summer, any Folding@Home I do directly competes with the air conditioning. Running big graphics cards, like the 1080 Ti, causes not only my power bill to go crazy due to my computer, but also due to the increased air conditioning required.

Thus, for people in hot climates, extra consideration should be given to the overall power consumption of your Folding@Home computer. With the GTX 1660 running in reduced power mode, I was able to get a total system power consumption of just over 150 watts while still making over 500K PPD! That’s not half bad. On the super low power end, I was able to beat the GTX 1050’s power consumption level…getting my beastly FX-8320e 8-core rig to draw 125 watts total while folding was quite a feat. The best thing was that it still made almost 300K PPD, which is well above last generations small cards.

Nvidia GTX 1650 and 1660 Power Consumption

Efficiency

This is my favorite part. How do these low-power Turing cards do on the efficiency scale? This is simply looking at how many PPD you can get per watt of power draw at the wall.

Nvidia GTX 1650 and 1660 Efficiency

And…wow! Just wow. For about $220 new, you can pick up a GTX 1660 Super and be just as efficient than the previous generation’s top card (GTX 1080 Ti), which still goes for $400-500 used on eBay. Sure the 1660 Super won’t be as good of a gaming card, and it  makes only about 2/3’s the PPD as the 1080 Ti, but on an energy efficiency metric it holds its own.

The GTX 1650 did pretty good as well, coming in somewhere towards the middle of the pack. It is still much more efficient than the similar market segment cards of the previous generation (GTX 1050), but it is overall hampered by not being able to return work units as quickly to the scientists, who prioritize fast work with bonus points (Quick Return Bonus).

Conclusion

NVIDIA’s entry-level Turing architecture graphics cards perform very well in Folding@Home, both from a performance and an efficiency standpoint. They offer significant gains relative to legacy cards, and can be a good option for a budget Folding@Home build.

Join My Team!

Interested in fighting COVID-19, Cancer, Alzheimer’s, Parkinson’s, and many other diseases with your computer? Please consider downloading Folding@Home and joining Team Nuclear Wessels (54345). See my tutorial here.

How to Run Folding@Home on a Graphics Card in Windows 10

(A Folding at Home Unofficial Configuration Guide for GPU, Multi-GPU, and CPU/GPU Folding)

Folding@Home is a distributed computing project for fighting diseases. If you’re reading this post, they you are probably looking for some help getting Folding@Home running on your graphics card. GPU folding, when configured properly, is one of the best way to do tons of science efficiently. I hope this Folding@Home GPU Guide helps you start kicking butt against cancer and other diseases. So, let’s get started.

Note: for people who already have the Folding@Home client up and running and you want to switch from CPU folding to GPU folding, skip right to Step 3. Please note that if you are changing your hardware configuration on a machine that is already folding, it is courteous to let the existing work units finish by using the “finish” option on the client prior to re-arranging hardware. This keeps work units from being lost.

Step 0: System Requirements

Yes, we’re starting at zero, because computer indexing starts here too. Plus, before you even try this, you need the right stuff in the box.

Operating System

While Folding@Home supports many operating systems, this guide is aimed at Windows users. I’ll be using Windows 10, but the steps are the same for Windows 7.

Overall Computer

CPU
Give Me Efficiency or Give Me an Empty CPU Socket!

You do need to think about what goes in this socket, even if you’re GPU folding

Even though this is a guide about graphics card folding, the rest of your computer needs to be up to snuff to keep the card fed. Ideally, you want one dedicated CPU core for your overall Windows environment, plus one CPU core for each graphics card you want to run F@H on. So, for a 1-GPU computer, having two CPU cores available is optimal. A dual-GPU computer should have a 3 cores available, a three-GPU computer should have four cores available, etc. In terms of clock rate, almost all modern processors with clock rates above 2.0 GHz will work. Remember, we aren’t doing CPU folding here; the CPU just needs to be fast enough to keep the GPU fed.

Motherboard
Circuit City

Circuit City

Motherboards don’t matter too much, except that you should have a full-width PCI-Express x16 slot for each graphics card you want to fold on. When you get into really fast, new graphics cards like the RTX 2080,  a PCI-E 3.0 x16 slot will ensure the data flows fast enough to the card. PCI-Express 2.0 bandwidth will work with these ultra-fast cards, but there will be a slight bottleneck. Note I have never seen any bottlenecks with my GTX 1080 Ti on PCI-Express 2.0 x16 in Windows, but when adding a second card (using an x1 riser), I did see a slowdown on my Gigabyte 880-series socket AM3 board.

Memory

You should also aim to have 8 GB of ram (16 ideally), just because Windows tends to be a resource hog. Some people can fold just fine with 4 GB, but for this guide I am assuming you want to be able to use the machine as well. Memory channel configuration and speed doesn’t matter very much for Folding@Home, especially on GPUs.

Hard Drives

Any old hard drive with 60 GB or so of free space will do. The F@H client takes up almost no space. The 60 GB of free space is really just what you need for Windows 10 to not run really bad, regardless of what the machine is being used for.

Internet Connection

Almost anything works, as long as it doesn’t drop out.

Power Supply
PC P&C PSU

PC Power & Cooling SILENCER PSU

This is a critical and often overlooked component in the world of computational computing. I’ve written many articles on power supplies, so feel free to browse through my site to learn more. In short, make sure your system has enough PSU wattage to drive the video card, based on the video card’s recommendation. You’ll also need to make sure your power supply has the correct auxiliary power cables (PCI-Express 6-pin and/ or 8-pin) to supply enough current to cards requiring supplemental power.

For multiple cards, you’ll need more nameplate PSU wattage. Power supplies should be 80+ Bronze certified or better to help deliver power efficiently, because no one likes wasting money on misused electricity (and this hurts the environment). Also, you should try and stick with major manufacturers, such as (but not limited to) Corsair, Antec, Seasonic, Cooler Master, PC Power & Cooling, Thermaltake, etc.

Here are some common computer configurations and a reasonable power supply wattage to drive them:

  • 1 x Low-End GPU –> (GTX 1050, RX560, etc) –> 380 Watt PSU
  • 1 x Mid-Range GPU (GTX 1060, RX570, etc) –> 450 Watt PSU
  • 1 x High-End GPU (GTX 1080, Vega64, etc) –> 550 Watt PSU
  • 2 x Mid-Range GPUs  or 3 x Low-End GPUs–> 600 Watt PSU
  • 2 x High-End GPUs or 3 x Mid-Range GPUs –> 800 Watt PSU
  • 3 x High-End GPUs or  4 x Mid-Range GPUs–> 1000 Watt PSU
  • 4 x High-End GPUs (you’re crazy!) –> 1200+ Watt PSU

Saving the Planet Tip: Any PSU supplying an active load of 600 Watts or more should be 80+ Gold certified or better. This will minimize waste heat due to efficiency losses, which really start to add up for high power-draw computers.

Cooling

This is another overlooked requirement. Any computer doing 24/7 computations on a graphics card is going to get pretty toasty. Thankfully, most modern CPU cases come with enough space and fans to deal with this. You’ll want at least 1 dedicated 120 MM exhaust fan (not including the PSU fan) and one 120 MM intake fan to keep the air flowing. If you have dual graphics cards, having an intake fan right on the side panel blowing on the cards is one of the best way to keep a hot pocket of air from forming between the cards. Consider reference-style video cards (centrifugal 2-slot blower coolers) for multi-card setups to help dump the heat, since open-fan cards tend to just drown in their own heat if there isn’t enough airflow. I also recommend aftermarket coolers on CPUs, since your processor will be actively spooled up and feeding your graphics card. Yet, CPU cooling doesn’t need to be overkill.

Icy Opteron 4184

NOCTUA OVERKILL!

Graphics Card
Graphics Card Showdown: EVGA Nvidia Geforce GTX 1050 TI vs. Gigabyte AMD Radeon HD 7970 GHz Edition

Graphics Cards: You’ll Need One

First off, you should actually have a discrete graphics card. While F@H might run on some onboard / APU graphics solutions, the performance won’t be worth it, and you might as well just run CPU folding.

Folding@Home works on many discrete graphics cards that support OpenCL, but not all cards are supported. AMD RADEON HD 5xxx cards and Nvidia GeForce 4xx cards and newer are currently supported, but that can always change. See the project’s system requirements for a complete list. I personally recommend using Nvidia 9xxx series cards or AMD RX 5xx cards or newer, since these are more efficient than older hardware. My review of the GeForce 1080 Ti has some plots on efficiency and performance that might be helpful if you are selecting a card specifically for folding. Make sure you have the latest drivers for your card from either AMD or Nvidia.

Step 1: System Prep

Before even downloading Folding@Home, you should do a few basic things just to make sure the system is going to be stable for heavy computations. On the software side, this means updating drivers, making sure virus definitions and Windows updates are up to date, etc. On the hardware side, I recommend fully air-canning the dust out of your machine to optimize cooling. If the computer is older and the GPU you plan to use has been installed for a while, it’s worth taking the graphics card out and hitting it with some compressed air from all angles to clean out the heat sinks.

Step 2: Download and Install V7 Client

The Folding@Home V7 client can be found here:

The current client version is 7.5.1. Go ahead and install it. For this part, it’s basically just following the prompts. F@H’s default Windows install guide works well enough, and you can read that here. All of this can be configured later within the client (and this will be required for GPU folding). So, I’m linking to the standard install guide instead of regurgitating the steps, because I’m lazy I want this to be done identically to how Stanford * the F@H Consortium recommends it be done. If you don’t want to fold anonymously, select the “Set up an identity” button. You’ll want to pick a user name and enter a team number if you have one you’d like to join.

For example, if you wanted to join our team, you’d enter number 54345 in the team number field to join team Nuclear Wessels!

A note about Passkeys: you want one of these if you want to get lots of points and compete on the F@H leaderboards. Passkeys are a secure key that makes sure your points are your own (i.e. no one is using your username to generate points elsewhere). You need to have a Passkey if you want to be eligible for the Quick Return Bonus (more points given to users who do science quickly). You become eligible for the bonus once you have successfully completed ten work units and you have a valid passkey. You can get a Passkey here (but you don’t have to do this right away. Just like configuring your GPUs, it can be done later).

Step 3: Configure the Client for GPU Folding

FAH_Molecule

Now we are going to edit settings within the Advanced Control section of the Folding@Home client. To get here, look at your Windows task bar (next to the clock). Once F@H is installed, there should be a little molecule there. Right-click that bad boy and select “Advanced Control” to open the local client window.

Right-Click FAH

This opens up the client view. Here is what mine currently looks like (with GPU slots configured). Depending on how you got here, you might or might not have a team name and user identity displayed, and you might or might not have CPU folding enabled.

F@H Control V7

Go ahead and click the “Configure” button in the top-left of the window. Go to the “Identity” tab first.

Identity

Here, you can change any of the user info and team name info you entered when you installed the V7 client. You can also enter a Passkey if you have one (for those sweet, sweet Quick Return Bonus Points!).

Pitch: I’d be honored if you joined team # 54345 (Nuclear Wessels). We are currently doing everything we can to fight the COVID-19 coronavirus.

Nuclear Wessels Meme

Next, go one tab over to “Slots”. Here, you can see what devices Folding@Home plans to use (either CPU or GPU). For my setup, I have removed all CPU slots and added two GPU slots (one slot for my 980 Ti and one for my 1080 Ti). If you originally started folding on the CPU and want to switch to GPU folding, you can delete your CPU slot here and add GPU slot(s) for your graphics card(s).

Note: If you want to do mixed hardware folding (CPU + GPU), I will talk about that in Step 4.

Slots

The slot configuration window opens up when you add or edit a slot. Here are the options.

Slot Config Selecting the GPU button and leaving all the index settings at -1 is a good place to start. Nine times out of ten, the client will properly detect graphics cards this way. For my computer, adding two GPU slots with settings like this resulted in it properly detecting and folding on my installed GTX 980 Ti and GTX 1080 Ti cards.

In rare cases, the client might get confused. This happens in systems with onboard graphics (such as with AMD APUs). What happens is you are trying to fold on your discrete graphics card, and instead the F@H client is running the GPU slot on the APU. When this happens, I’ve found the easiest thing to do is reboot the computer, go into the BIOS, and disable the APU graphics from there, so that the client can’t even see the APU. Thus, the GPU slot with a -1 index defaults to the discrete graphics card.

Alternatively, you can use the gpu-index, opencl-index, and cuda-index boxes to try and get the slot to run on the correct graphics card. This is a trial and error process that is beyond the scope of this guide (leave me a comment if you need help with this, or ask someone in the Folding@Home Forums).

Advanced Slot Options

The Extra Slot Options (expert only) box on the bottom can sometimes help you eek a bit more performance out of the GPU slots. However, your mileage may vary. You can add or remove slot options with the + and – buttons on the bottom-right.

The settings I tend to add are these:

Advanced Options

Here, client-type advanced lets me get “late stage beta” work units, which might be a bit more unstable than normal work units, yet this helps the Folding@Home Consortium get new projects tested sooner. Max-Packet-Size Big (other options are “normal” and “small”) lets me download large molecules that will push the system a bit harder (more VRAM needed, more internet bandwidth, etc). Pause-on-start (value of “true” or “false”) tells the system to pause the folding slot when the computer boots (instead of automatically folding as soon as the machine is on). This is nice for when I want to kick folding off manually. Set this to “false” or leave it blank if you want the computer to fold automatically after a restart.

For a detailed list of these slot options, see the config guide here. Note: some of this is out of date.

Step 4 (Optional): Configure a CPU Slot as well

If you have CPU cores to spare, you can add a CPU folding slot in addition to the GPU slots. I recommend leaving 1 CPU core free for Windows background tasks (unless you are making a dedicated folding rig and don’t mind it being a bit slow to use). You should also keep 1 CPU core free for feeding each GPU that you have in your system. So, for my 8-core AMD FX-8320e with my two graphics cards, I could do something like this:

Total CPU Cores: 8

Cores needed for Windows: 1

Cores Needed for GPU Slots: 2 (one for each GPU)

Cores Remaining: = 8-1-2 = 5

So, theoretically, I can set my CPU folding slot to use 5 CPU cores. Now, an interesting fact is that in multi-core computing, prime numbers like 3, 5, and 7 do not work so well. Folding at home also doesn’t do well with high prime numbers, or multiples thereof (such as 14 threads, which is a multiple of prime number 7). It has to do with how all the data threads are stitched together.

For example, you get similar performance folding with 4 CPU cores as with 5 (4 is a nice base 2 number that computers like). In my case, for a non-dedicated folding rig, I set up a CPU slot with 4 CPU cores enabled, leaving two cores to handle whatever else the computer is doing and 2 cores to feed the graphics cards. Incidentally, if this were a guide about just setting up CPU folding, I would leave this box at “-1”.

4 CPU Core Config

Now, just hit the OK button and then save the slot configuration.

Save Slot Config

Step 5: Observe Slots Descriptions in the Client

Now, I can see that I have three slots (two GPU and one CPU) listed in the client window.

Ready Slots

Here, you should see that the CPU slot is using the number of threads you told it to use (4, in my case), and that the graphics cards are correctly identified. This all looks good.

Step 5: Watch it run!

Once you have your slots configured, you should be able to sit back and watch your computer fight disease with everything it’s got. One last thing: A helpful tool for graphics card monitoring is something like MSI Afterburner, or AMD’s built-in tool Wattman. It’s good to use these to make sure your card has enough thermal headroom to perform (keep it under 80 degrees C if you can!). If your card is thermally throttling, you’ll see an impact to folding@home PPD. I find that setting custom fan curves, or just setting the fan to run a bit faster than it normally would, is often enough to eliminate this.

Troubleshooting

The V7 client installer does the best job at detecting your specific graphics hardware during initial software installation. If you added a new graphics card that is not recognized, you should do a clean re-install of the V7 client. Write down your Name, Team Number, and Passkey, uninstall the client completely (including data), reinstall, and see if the new card is detected.

Some new graphics cards are also not immediately supported upon release. For example, the Radeon 5700 XT is only recently gaining support with advanced beta work units, but work is progressing to get this card fully supported (as of 3/2020). You can read up on which cards are supported and which aren’t yet on the GPU Whitelist Thread.

Leave me a comment if…

Did this guide help you? Did I miss something? Let me know how I can help and make this better by leaving a comment. Thanks!

-Chris

Addendum: Helpful Links to Other Tutorials

HFM.net – A remote monitoring program for F@H Clients

HFM.net monitoring tutorial (Youtube) – Video Tutorial by Frax1006

Teamviewer Guide – A remote desktop solution to let you log into folding machines and monitor / configure them. This is an excellent write-up by Pyroball.

Official F@H Advanced User Custom Installation Guide

Official F@H Configuration Guide

Overclocker’s Club F@H Guide

 

Folding@Home Review: NVIDIA GeForce GTX 1080 Ti

Released in March 2017, Nvidia’s GeForce GTX 1080 Ti was the top-tier card of the Pascal line-up. This is the graphics card that super-nerds and gamers drooled over. With an MSRP of $699 for the base model, board partners such as EVGA, Asus, Gigabyte, MSI, and Zotac (among others) all quickly jumped on board (pun intended) with custom designs costing well over the MSRP, as well as their own takes on the reference design.

GTX 1080 Ti Reference EVGA

EVGA GeForce GTX 1080 Ti – Reference

Three years later, with the release of the RTX 2080 Ti, the 1080 Ti still holds its own, and still commands well over $400 on the used market. These are beastly cards, capable of running most games with max settings in 4K resolutions.

But, how does it fold?

Folding@Home

Folding at home is a distributed computing project originally developed by Stanford University, where everyday users can lend their PC’s computational horsepower to help disease researchers understand and fight things like cancer, Alzheimer’s, and most recently the COVID-19 Coronavirus. User’s computers solve molecular dynamics problems in the background, which help the Folding@Home Consortium understand how proteins “misfold” to cause disease. For computer nerds, this is an awesome way to give (money–>electricity–>computer work–>fighting disease).

Folding at home (or F@H) can be run on both CPUs and GPUs. CPUs provide a good baseline of performance, and certain molecular simulations can only be done here. However, GPUs, with their massively parallel shader cores, can do certain types of single-precision math much faster than CPUs. GPUs provide the majority of the computational performance of F@H.

Geforce GTX 1080 Ti Specs

The 1080 Ti is at the top of Nvidia’s lineup of their 10-series cards.

1080 Ti Specs

With 3584 CUDA Cores, the 1080 Ti is an absolute beast. In benchmarks, it holds its own against the much newer RTX cards, besting even the RTX 2080 and matching the RTX 2080 Super. Only the RTX 2080 Ti is decidedly faster.

Folding@Home Testing

Testing is performed in my old but trusty benchmark machine, running Windows 10 Pro and using Stanford’s V7 Client. The Nvidia graphics driver version was 441.87. Power consumption measurements are taken on the system-level using a P3 Watt Meter at the wall.

System Specs:

  • CPU: AMD FX-8320e
  • Mainboard : Gigabyte GA-880GMA-USB3
  • GPU: EVGA 1080 Ti (Reference Design)
  • Ram: 16 GB DDR3L (low voltage)
  • Power Supply: Seasonic X-650 80+ Gold
  • Drives: 1x SSD, 2 x 7200 RPM HDDs, Blu-Ray Burner
  • Fans: 1x CPU, 2 x 120 mm intake, 1 x 120 mm exhaust, 1 x 80 mm exhaust
  • OS: Win10 64 bit

I did extensive testing of the 1080 Ti over many weeks. Folding@Home rewards donors with “Points” for their contributions, based on how much science is done and how quickly it is returned. A typical performance metric is “Points per Day” (PPD). Here, I have averaged my Points Per Day results out over many work units to provide a consistent number. Note that any given work unit can produce more or less PPD than the average, with variation of 10% being very common. For example, here are five screen shots of the client, showing five different instantaneous PPD values for the 1080 Ti.

 

GTX 1080 Ti Folding@Home Performance

The following plot shows just how fast the 1080 Ti is compared to other graphics cards I have tested. As you can see, with nearly 1.1 Million PPD, this card does a lot of science.

1080 Ti Folding Performance

GTX 1080 Ti Power Consumption

With a board power rating of 250 Watts, this is a power hungry graphics card. Thus, it isn’t surprising to see that power consumption is at the top of the pack.

1080 Ti Folding Power

GTX 1080 Ti Efficiency

Power consumption alone isn’t the whole story. Being a blog about doing the most work possible for the least amount of power, I am all about finding Folding@Home hardware that is highly efficient. Here, efficiency is defined as Performance Out / Power In. So, for F@H, it is PPD/Watt. The best F@H hardware is gear that maximizes disease research (performance) done per watt of power consumed.

Here’s the efficiency plot.

1080 Ti Folding Efficiency

Conclusion

The Geforce GTX 1080 Ti is the fastest and most efficient graphics card that I’ve tested so far for Stanford’s Folding@Home distributed computing project. With a raw performance of nearly 1.1 Million PPD in windows and an efficiency of almost 3500 PPD/Watt, this card is a good choice for doing science effectively.

Stay tuned to see how Nvidia’s latest Turing architecture stacks up.

GTX 460 Graphics Card Review: Is Folding on Ancient Hardware Worth It?

Recently, I picked up an old Core 2 duo build on Ebay for $25 + shipping. It was missing some pieces (Graphics card, drives, etc), but it was a good deal, especially for the all-metal Antec P182 case and included Corsair PSU + Antec 3-speed case fans. So, I figured what the heck, let’s see if this vintage rig can fold!

Antec 775 Purchase

To complement this old Socket 775 build, I picked up a well loved EVGA GeForce GTX 460 on eBay for a grand total of $26.85. It should be noted that this generation of Nvidia graphics cards (based on the Fermi architecture from back in 2010) is the oldest GPU hardware that is still supported by Stanford. It will be interesting to see how much science one of these old cards can do.

GTX 460 Purchase

I supplied a dusty Western Digital 640 Black Hard Drive that I had kicking around, along with a TP Link USB wireless adapter (about $7 on Amazon). The Operating System was free (go Linux!). So, for under $100 I had this setup:

  • Case: Antec P182 Steel ATX
  • PSU: Corsair HX 520
  • Processor: Intel Core2duo E8300
  • Motherboard: EVGA nForce 680i SLI
  • Ram: 2 x 2 GB DDR2 6400 (800 MHz)
  • HDD: Western Digital Black 640GB
  • GPU: EVGA GeForce GTX 460
  • Operating System: Ubuntu Linux 18.04
  • Folding@Home Client: V7

I fired up folding, and after some fiddling I got it running nice and stable. The first thing I noticed was that the power draw was higher than I had expected. Measured at the wall, this vintage folding rig was consuming a whopping 220 Watts! That’s a good deal more than the 185 watts that my main computer draws when folding on a modern GTX 1060. Some of this is due to differences in hardware configuration between the two boxes, but one thing to note is that the older GTX 460 has a TDP of 160 watts, whereas the GTX 1060 has a TDP of only 120 Watts.

Here’s a quick comparison of the GTX 460 vs the GTX 1060. At the time of their release, both of these cards were Nvidia’s baseline GTX model, offering serious gaming performance for a better price than the more aggressive GTX -70 and -80-series variants. I threw a GTX 1080 into the table for good measure.

GTX 460 Spec Comparison

GTX 460 Specification Comparison

The key takeaways here are that six years later, the equivalent graphics card to the GTX 460 was over three and a half times faster while using forty watts less power.

Power Consumption

I typically don’t report power consumption directly, because I’m more interested in optimizing efficiency (doing more work for less power). However, in this case, there is an interesting point to be made by looking at the wattage numbers directly. Namely, the GTX 460 (a mid-range card) uses almost as much power as a modern high-end GTX 1080, and uses seriously more power than the modern GTX 1060 mid-range card. Note: these power consumption numbers must be taken with a grain of salt, because the GTX 460 was installed in a different host system (the Core2 Duo rig) as the other cards, but the resutls are still telling. This is also consistent with the advertised TDP of the GTX 460, which is 40 watts higher than the GTX 1060.

GTX 460 Power Consumption (Wall)

Total System Power Consumption

Folding@Home Results

Folding on the old GTX 460 produced a rough average of 20,000 points per day, with the normal +/- 10% variation in production seen between work units. Back in 2006 when I was making a few hundred PPD on an old Athlon 64 X2 CPU, this would have been a huge amount of points! Nowadays, this is not so impressive. As I mentioned before, the power consumption at the wall for this system was 220 Watts. This yields an efficiency of 20,000 PPD / 220 Watts = 90 PPD/Watt.

Based off the relative performance, one would think the six-year newer GTX 1060 would produce somewhere between 3 and 4 times as many PPD as the older 460 card. This would mean roughly 60-80K PPD. However, my GTX 1060 frequently produces over 300K PPD. This is due to Stanford’s Quick Return Bonus, which essentially rewards donors for doing science quickly. You can read more about this incentive-based points system at Stanford’s website. The gist is, the faster you return a work unit to the scientists, the sooner they can get to developing cures for diseases. Thus, they award you more points for fast work. As the performance plot below shows, this quick return bonus really adds up, so that someone doing 3-4 times more (GTX 1060 vs. GTX 460 linear benchmark performance) results in 15 times more F@H performance.

GTX 460 Performance and Efficiency

Old vs. New Graphics Card Comparison: Folding@Home Efficiency and PPD

This being a blog about energy-conscious computing, I’d be remiss if I didn’t point out just how inefficient the ancient GTX 460 is compared to the newer cards. Due to the relatively high power consumption for a midrange card, the GTX 460 is eighteen times less efficient than the GTX 1060, and a whopping thirty three times less efficient than the GTX 1080.

Conclusion

Stanford eventually drops support for old hardware (anyone remember PS3 folding?), and it might not be long before they do the same for Fermi-based GPUs. Compared with relatively modern GPUs, the GTX 460 just doesn’t stack up in 2020. Now that the 10-series cards are almost four years old, you can often get GTX 1060s for less than $200 on eBay, so if you can afford to build a folding rig around one of these cards, it will be 18 times more efficient and make 15 times more points.

Still, I only paid about $100 total to build this vintage folding@home rig for this experiment. One could argue that putting old hardware to use like this keeps it out of landfills and still does some good work. Additionally, if you ignore bonus points and look at pure science done, the GTX 460 is “only” about 4 times slower than its modern equivalent.

Ultimately, for the sake of the environment, I can’t recommend folding on graphics cards that are many years out of date, unless you plan on using the machine as a space heater to offset heating costs in the winter. More on that later…

Addendum

Since doing the initial testing and outline for this article, I picked up a GTX 480 and a few GTX 980 Ti cards. Here are some updated plots showing these cards added to the mix. The GTX 480 was tested in the Core2 build, and the GTX 980 Ti in my standard benchmark rig (AMD FX-based Socket AM3 system).

Various GPU Power Consumption

GTX 980 and 480 Performance

GTX 980 and 480 Efficiency

I think the conclusion holds: even though the GTX 480 is slightly faster and more efficient than it’s little brother, it is still leaps and bounds worse than the more modern cards. The 980 Ti, being a top-tier card from a few generations back, holds its own nicely, and is almost as efficient as a GTX 1060. I’d say that the 980 Ti is still a relatively efficient card to use in 2020 if you can get one for cheap enough.

Power Supply Efficiency: Let’s Save Some Money

A while ago, I wrote a pair of articles on why it’s important to consider the energy efficiency of your computer’s power supply. Those articles showed how maximizing the efficiency of your Power Supply Unit (PSU) can actually save you money, since less electricity is wasted as heat with efficient power supplies.

Efficient Power Supplies: Part 1

Energy Efficient Power Supplies: Part 2

In this article, I’m putting this into practice, because the PSU in my Ubuntu folding box (Codenamed “Voyager”) is on the fritz.

This PSU is a basic Seasonic S12 III, which is a surprisingly bad power supply for such a good company as Seasonic. For one, it uses a group regulated design, which is inherently less efficient than the more modern DC-DC units. Also, the S12 is prone to coil whine (mine makes tons of noise even when the power supply is off). Finally, in my case, the computer puts a bunch of feedback onto the electrical circuits in my house, causing my LED lights to flicker when I’m running Folding@Home. That’s no good at all! Shame on you, Seasonic, shame!

Don’t believe me on how bad this PSU is? Read reviews here:

https://www.newegg.com/seasonic-s12iii-bronze-series-ssr-500gb3-500w/p/N82E16817151226

Now, I love Seasonic in general. They are one of the leading PSU manufactures, and I use their high-end units in all of my machines. So, to replace the S12iii, I picked up one of their midrange PSU’s in the Focus line…specifically, the Focus Gold 450. I got a sweet deal on eBay (got a used one for about $40, MSRP new on the SSR-450FM is $80).

SSR-450M Ebay Purchase Price

Here they are side by side. One immediate advantage of the new Focus PSU is that it is semi-modular, which will help me with some cable clutter.

Seasonic PSU Comparison: Focus Gold 450W (left) vs S12iii 500W (right)

Seasonic PSU Comparison: Focus Gold 450W (left) vs S12iii 500W (right)

Inspecting the specification labels also shows a few differences…namely the Focus is a bit less powerful (three less amps on the +12v rail), which isn’t a big deal for Voyager, since it is only running a single GeForce 1070 Ti card (180 Watt TDP) and an AMD A10-7700K (95 Watt TDP). Another point worth noting is the efficiency…whereas the S12iii is certified to the 80+ Bronze standard, the new Focus unit is certified as 80+ Gold.

 

 

 

 

Now this is where things get interesting. Voyager has a theoretical power draw of about 300 Watts max (180 Watts for the video card, 95 for the CPU, and about 25 Watts for the motherboard, ram, and drives combined). This is right around the 60% capacity rating of these power supplies. Here is the efficiency scorecard for the various 80+ certifications:

80+ Table

80+ Efficiency Table

As you can see, there is about a 5% improvement in efficiency going from 80+ bronze to 80+ gold. For a 300 watt machine, that would equate to 15 watts of difference between the Focus and the S12iii PSU’s. By upgrading to the Focus, I should more effectively turn the 120V AC power from my wall into 12V DC to run my computer, resulting in less total power draw from the wall (and less waste heat into my room).

I tested it out, using Stanford’s Folding@Home distributed computing project of course! Might as well cure some cancer, you know!

The Test

To do this test, I first let Voyager pull down a set of work units from Stanford’s server (GPU + CPU folding slots enabled). When the computer was in the middle of number crunching, I took a look at the instantaneous power consumption as measured by my watt meter:

Voyager_Old_PSU_Peak

80+ Bronze PSU: 259.1 Watts @ Full Load

260 Watts is about the max I ever see Voyager draw in practice, since Folding@Home never fully loads the hardware (typically it can hit the GFX card for about 90% capacity). So, this result made perfect sense. Next, I shut the machine down with the work units half-finished and swapped out the 80+ Bronze S12iii for the 80+ Gold Focus unit. I turned the machine back on and let it get right back to doing science.

Here is the updated power consumption number with the more efficient power supply.

Voyager_New_PSU_Peak

80+ Gold PSU Power Consumption @ 100% Load

As you can see, the 80+ Gold Rated power supply shaved 11.8 watts off the top. This is about 4.5% of the old PSU unit’s previous draw, and it is about 4.8% of the new PSU unit’s power draw. So, it is very close to the advertised 5% efficiency improvement one would expect per the 80+ specifications. Conclusion: I’m saving electricity and the planet! Yay! 

As a side note, all the weird coil whine and light flickering issues I was having with the S12iii went away when I switched to Seasonic’s better Focus PSU.

But, Was It Worth It?

Now, as an environmentalist, I would say that this type of power savings is of course worth it, because it’s that much less energy wasted and that much less pollution. But, we are really talking about just a few watts (albeit on a machine that is trying to cure cancer 24/7 for years on end).

To get a better understanding of the financial implications of my $40 upgrade, I did a quick calc in Excel, using Connecticut’s average price of electricity as provided by Eversource ($0.18 per KWH).

Voyager PSU Efficiency Upgrade Calc

Voyager PSU Efficiency Upgrade Calc

Performing this calculation is fairly straightforward. Basically, it’s just taking the difference in wattage between the two power supply units and turning that into energy by multiplying it by one year’s worth of run time (Energy = Power * Time). Then, I multiply that out by the cost of energy to get a yearly cost savings of about $20 bucks. That’s not bad! Basically, I could pay for my PSU upgrade in two years if I run the machine constantly.

Things get better if I sell the old PSU. Getting $20 for a Seasonic anything should be easly (ignoring the moral dilemma of sticking someone with a shitty power supply that whines and makes their lights flicker). Then, I’d recoup my investment in a year, all while saving the planet!

So, from my perspective as someone who runs the computer 24/7, this power supply efficiency upgrade makes a lot of sense. It might not make as much sense for people whose computers are off for most of the day, or for computers that just sit around idle, because then it would take a lot longer to recover the costs.

P.S. Now when I pop the side panel off Voyager, I am reminded to focus…

Voyager New PSU

AMD Radeon RX 580 8GB Folding@Home Review

Hello again.

Today, I’ll be reviewing the AMD Radeon RX 580 graphics card in terms of its computational performance and power efficiency for Stanford University’s Folding@Home project. For those that don’t know, Folding@Home lets users donate their computer’s computations to support disease research. This consumes electrical power, and the point of my blog is to look at how much scientific work (Points Per Day or PPD) can be computed for the least amount of electrical power consumption. Why? Because in trying to save ourselves from things like cancer, we shouldn’t needlessly pollute the Earth. Also, electricity is expensive!

The Card

AMD released the RX 580 in April 2017 with an MSRP of $229. This is an updated card based on the Polaris architecture. I previously reviewed the RX 480 (also Polaris) here, for those interested. I picked up my MSI-flavored RX 580 in 2019 on eBay for about $120, which is a pretty nice depreciated value. Those who have been following along know that I prefer to buy used video cards that are 2-3 years old, because of the significant initial cost savings, and the fact that I can often sell them for the same as I paid after running Folding for a while.

RX_580

MSI Radeon RX 580

I ran into an interesting problem installing this card, in that at 11 inches long, it was about a half inch too long for my old Raidmax Sagitta gaming case. The solution was to take the fan shroud off, since it was the part that was sticking out ever so slightly. This involved an annoying amount of disassembly, since the fans actually needed to be removed from the heat sink for the plastic shroud to come off. Reattaching the fans was a pain (you need a teeny screw driver that can fit between the fan blade gaps to get the screws by the hub).

RX_580_noShroud

RX 580 with Fan Shroud Removed. Look at those heat pipes! This card has a 185 Watt TDP (Board Power Rating). 

RX_580_Installed

RX 580 Installed (note the masking tape used to keep the little side LED light plate off of the fan)

RX_580_tightFit

Now That’s a Tight Fit (the PCI Express Power Plug on the video card is right up against the case’s hard drive bays)

The Test Setup

Testing was done on my rather aged, yet still able, AMD FX-based system using Stanford’s Folding@Home V7 client. Since this is an AMD graphics card, I made sure to switch the video card mode to “compute” within the driver panel. This optimizes things for Folding@home’s workload (as opposed to games).

Test Setup Specs

  • Case: Raidmax Sagitta
  • CPU: AMD FX-8320e
  • Mainboard : Gigabyte GA-880GMA-USB3
  • GPU: MSI Radeon RX 580 8GB
  • Ram: 16 GB DDR3L (low voltage)
  • Power Supply: Seasonic X-650 80+ Gold
  • Drives: 1x SSD, 2 x 7200 RPM HDDs, Blu-Ray Burner
  • Fans: 1x CPU, 2 x 120 mm intake, 1 x 120 mm exhaust, 1 x 80 mm exhaust
  • OS: Win10 64 bit
  • Video Card Driver Version: 19.10.1

 

Performance and Power

I ran the RX 580 through its paces for about a week in order to get a good feel for a variety of work units. In general, the card produced as high as 425,000 points per day (PPD), as reported by Stanford’s servers. The average was closer to 375K PPD, so I used that number as my final value for uninterrupted folding. Note that during my testing, I occasionally used the machine for other tasks, so you can see the drops in production on those days.

RX 580 Client

Example of Client View – RX 580

RX580 History

RX 580 Performance – About 375K PPD

I measured total system power consumption at the wall using my P3 Watt Meter. The system averaged about 250 watts. That’s on the higher end of power consumption, but then again this is a big card.

Comparison Plots

RX 580 Performance

AMD Radeon RX 580 Folding@Home Performance Comparison

RX 580 Efficiency

AMD Radeon RX 580 Folding@Home Efficiency Comparison

Conclusion

For $120 used on eBay, I was pretty happy with the RX 580’s performance. When it was released, it was directly competing with Nvidia’s GTX 1060. All the gaming reviews I read showed that Team Red was indeed able to beat Team Green, with the RX 580 scoring 5-10% faster than the 1060 in most games. The same is true for Folding@Home performance.

However, that is not the end of the story. Where the Nvidia GTX 1060 has a 120 Watt TDP (Thermal Dissipated Power), AMD’s RX 580 needs 185 Watts. It is a hungry card, and that shows up in the efficiency plots, which take the raw PPD (performance) and divide out the power consumption in watts I am measuring at the wall. Here, the RX 580 falls a bit short, although it is still a healthy improvement over the previous generation RX 480.

Thus, if you care about CO2 emissions and the cost of your folding habits on your wallet, I am forced to recommend the GTX 1060 over the RX 580, especially because you can get one used on eBay for about the same price. However, if you can get a good deal on an RX 580 (say, for $80 or less), it would be a good investment until more efficient cards show up on the used market.

Folding@Home: Nvidia GTX 1080 Review Part 3: Memory Speed

In the last article, I investigated how the power limit setting on an Nvidia Geforce GTX 1080 graphics card could affect the card’s performance and efficiency for doing charitable disease research in the Folding@Home distributed computing project. The conclusion was that a power limit of 60% offers only a slight reduction in raw performance (Points Per Day), but a large boost in energy efficiency (PPD/Watt). Two articles ago, I looked at the effect of GPU core clock. In this article, I’m experimenting with a different variable. Namely, the memory clock rate.

The effect of memory clock rate on video games is well defined. Gamers looking for the highest frame rates typically overclock both their graphics GPU and Memory speeds, and see benefits from both. For computation projects like Stanford University’s Folding@Home, the results aren’t as clear. I’ve seen arguments made both ways in the hardware forums. The intent of this article is to simply add another data point, albeit with a bit more scientific rigor.

The Test

To conduct this experiment, I ran the Folding@Home V7 GPU client for a minimum of 3 days continuously on my Windows 10 test computer. Folding@Home points per day (PPD) numbers were taken from Stanford’s Servers via the helpful team at https://folding.extremeoverclocking.com.  I measured total system power consumption at the wall with my P3 Kill A Watt meter. I used the meter’s KWH function to capture the total energy consumed, and divided out by the time the computer was on in order to get an average wattage value (thus eliminating a lot of variability). The test computer specs are as follows:

Test Setup Specs

  • Case: Raidmax Sagitta
  • CPU: AMD FX-8320e
  • Mainboard : Gigabyte GA-880GMA-USB3
  • GPU: Asus GeForce 1080 Turbo
  • Ram: 16 GB DDR3L (low voltage)
  • Power Supply: Seasonic X-650 80+ Gold
  • Drives: 1x SSD, 2 x 7200 RPM HDDs, Blu-Ray Burner
  • Fans: 1x CPU, 2 x 120 mm intake, 1 x 120 mm exhaust, 1 x 80 mm exhaust
  • OS: Win10 64 bit
  • Video Card Driver Version: 372.90

I ran this test with the memory clock rate at the stock clock for the P2 power state (4500 MHz), along with the gaming clock rate of 5000 MHz and a reduced clock rate of 4000 MHz. This gives me three data points of comparison. I left the GPU core clock at +175 MHz (the optimum setting from my first article on the 1080 GTX) and the power limit at 100%, to ensure I had headroom to move the memory clock without affecting the core clock. I verified I wasn’t hitting the power limit in MSI Afterburner.

*Update. Some people may ask why I didn’t go beyond the standard P0 gaming memory clock rate of 5000 MHz (same thing as 10,000 MHz double data rate, which is the card’s advertised memory clock). Basically, I didn’t want to get into the territory where the GDDR5’s error checking comes into play. If you push the memory too hard, there can be errors in the computation but work units can still complete (unlike a GPU core overclock, where work units will fail due to errors). The reason is the built-in error checking on the card memory, which corrects errors as they come up but results in reduced performance. By staying away from 5000+ MHz territory on the memory, I can ensure the relationship between performance and memory clock rate is not affected by memory error correction.

1080 Memory Boost Example

Memory Overclocking Performed in MSI Afterburner

Tabular Results

I put together a table of results in order to show how the averaging was done, and the # of work units backing up my +500 MHz and -500 MHz data points. Having a bunch of work units is key, because there is significant variability in PPD and power consumption numbers between work units. Note that the performance and efficiency numbers for the baseline memory speed (+0 MHz, aka 4500 MHz) come from my extended testing baseline for the 1080 and have even more sample points.

Geforce 1080 PPD Production - Ram Study

Nvidia GTX 1080 Folding@Home Production History: Data shows increased performance with a higher memory speed

Graphic Results

The following graphs show the PPD, Power Consumption, and Efficiency curves as a function of graphics card memory speed. Since I had three points of data, I was able to do a simple three-point-curve linear trendline fit. The R-squared value of the trendline shows how well the data points represent a linear relationship (higher is better, with 1 being ideal). Note that for the power consumption, the card seems to have used more power with a lower memory clock rate than the baseline memory clock. I am not sure why this is…however, the difference is so small that it is likely due to work unit variability or background tasks running on the computer. One could even argue that all of the power consumption results are suspect, since the changes are so small (on the order of 5-10 watts between data points).

Geforce 1080 Performance vs Ram Speed

Geforce 1080 Power vs Ram Speed

Geforce 1080 Efficiency vs Ram Speed

Conclusion

Increasing the memory speed of the Nvidia Geforce GTX 1080 results in a modest increase in PPD and efficiency, and arguably a slight increase in power consumption. The difference between the fastest (+500 MHz) and slowest (-500 MHz) data points I tested are:

PPD: +81K PPD (11.5%)

Power: +9.36 Watts (3.8%)

Efficiency: +212.8 PPD/Watt (7.4%)

Keep in mind that these are for a massive difference in ram speed (5000 MHz vs 4000 MHz).

Another way to look at these results is that underclocking the graphics card ram in hopes of improving efficiency doesn’t work (you’ll actually lose efficiency). I expect this trend will hold true for the rest of the Nvidia Pascal series of cards (GTX 10xx), although so far my testing of this has been limited to this one card, so your mileage may vary. Please post any insights if you have them.