At BrandYourself, we are diving deep in to Machine Learning to enhance some of our upcoming features. The issue I have with the current ML battlefield is that cloud based GPU instances are still relatively new and very expensive. When you start adding high availability requirements, as well as potential concurrent load, it gets prohibitively expensive quickly.
The Solution
I set out with the goal to build my own 1u low power server with an NVIDIA GPU. I found a good low power server candidate from Supermicro, the 5018D-FN4T 8 core D-1541 Intel® Xeon® processor. It has a 200W power supply and PCI-E 3.0 x16. It's not the cheapest platform option, but it has great performance: m.2 for storage and a full x16 PCI-E 3.0 slot.
For the GPU it took a lot of looking at photos to determine a card that could be easily modified to fit in a 1u chassis and still maintain an adequate cooling solution. With the launch of the GTX 1050Ti from NVIDIA, I was excited to see MSI launch a product that seemed like it may be my perfect fit. The GEFORCE® GTX 1050 TI 4G OC is a bus powered (75W) 4G 10 series GPU with a stock cooler that has the heatsink fins running end to end on the card.
4G of ram means we can run 4 workers on the card (our inference model footprint for image processing right now is ~850Mb) and still have a small buffer incase something grows.
The 75W power consumption over the PCI bus means no extra power cables required and a lower expected running temperature.
The stock heatsink on the card already has end to end fins, so we have the ability to place a 1u high RPM fan at the end of the card and maintain a decent level of airflow over the heatsink.
Making it fit
WARNING: This WILL void your GPU's warranty...
As you can see, the card is already pretty small and the DVI port is in the first row. This means that we can easily cut the bracket and not make any physical modifications to the card itself (such as desoldering or brute force cutting of the DVI port).
Disassembling is very straight forward.
The 4 heatsink screws on the back of the card need to be removed.
Carefully disconnect the fan power connector and split apart. Avoid disturbing the heat transfer compound.
There are 4 screws holding the fan shroud to the heatsink. Remove those.
Place the heatsink back on (keeping the same orientation as before) and fasten in place with the original screws (the ones with the springs).
Next, locate and remove the screws holding the mounting plate.
As you can see, this card is very slim.
With a pair of pliers, carefully straighten the board mounting support so that the plate will sit flush on a surface.
Apply some masking tape and draw your cutting line. Single slot plates are typically 18mm wide.
While using appropriate safety equipment, cut along your line using a bandsaw (a hacksaw would probably work, too).
File off the sharp edges.
Hooray, we have a single slot mounting plate!
Bend back the board mounting support.
Now we have a low profile GPU, without heavy modification or destruction of the PCB/components!
As you can see, it mounts nicely inside this 1u chassis, and a 1u high RPM fan can be placed behind it. I mount the fan in place with some high strength double sided tape.
The GPU runs perfectly on ubuntu with the latest NVIDIA linux drivers and latest CUDA and CUDNN libraries. If you have any questions, drop me a comment below or tweet at me on twitter @dchote.
