It often means using a graphics card to process graphics, or other highly parallel tasks. the graphics card has a different processor architecture than a traditional CPU.
The GPU (graphics card) excels at performing on operation (such as multiplying by a number) on a huge number of data points. Imagine an excel spreadsheet, multiplying 10,000 different values in column A with 10,000 different values in column B, to give the result in column C. A GPU could do this task very quickly, (theoretically) performing all of the multiplications at the same time (taking only a few clock cycles).
In the same scenario, a traditional CPU would have to do each multiplication one at a time, because it's hardware structure focuses on processing each instruction it receives quickly. The GPU isn't always faster, however it is more specialized for this type of task.
Hardware acceleration, is often used to describe a process in software that uses the GPU to perform tasks when it is able to perform operations faster than the CPU can. Depending on the tasks and the way the software is written, leveraging the GPU can be 1000's of times faster.