Debunking C# vs C++ Performance

January 03, 2009blog, c-sharp, code, cpp, optimization

If you were on reddit today, you probably saw this article, damning C#’s performance as being ten times worse than C++’s. Holy shit balls, batman!

Running his C# code, here are the results I got:

Original C# Code

Array Size SortTest SortTestT SortTestTC SortIndirect
1024 10.7162 2.3441 3.8781 1.1366
2048 22.9509 4.3889 8.4408 1.8714
4096 49.3709 8.4452 17.3883 3.7319
8192 103.5701 18.5369 38.1285 8.0310
16384 220.9323 39.6958 80.9258 18.5821
32768 469.5507 84.5129 172.2964 41.2126
65536 1016.2149 188.6718 380.3507 93.2924
131072 2156.4188 399.7299 791.6437 210.9526
262144 4616.3540 847.9829 1692.9814 467.6020
524288 9732.4311 1793.9729 3545.2089 1038.2164

Pretty slow! So I took a look at the code. The first thing that would catch the eye of any C# programmer is this:

unsafe struct Data
{
    public int key;
    public fixed char data[128];
}

That’s the data structure he’s sorting. An unsafe struct with a fixed array? I had to look up fixed to even know what that means. Now, I understand that he’s trying to make an apples/apples comparison and keep the data structure as close to the C++ one as possible, but I think that’s missing the point. If you’re going to compare two languages, using their built-in typical sort functions, shouldn’t you use their typical data structures too? Here’s what how a regular C# developer would define Data:

class Data
{
    public int key;
    public char[] data;

    public Data() { data = new char[128]; }
}

No unmanaged code, no structs (which are rarely used in C#). Just a regular class with an array. Here’s the results:

Modified to Typical C# Code

Array Size SortTest SortTestT SortTestTC SortIndirect
1024 0.3605 0.3626 0.4150 0.5918
2048 0.7651 0.7446 0.8749 0.5021
4096 1.6434 1.6094 1.9468 1.2030
8192 3.6497 3.5216 4.1014 2.3926
16384 7.9555 8.0842 9.3324 5.4752
32768 21.1833 19.1183 23.1170 15.1998
65536 54.6938 53.4892 72.3932 34.6554
131072 122.5008 114.1937 141.3504 75.9064
262144 279.8014 262.5908 343.4204 160.8344
524288 598.5605 577.7487 759.4405 359.7824

Let’s compare the last lines of each:

Data Type SortTest SortTestT SortTestTC SortIndirect
struct/fixed 9732.4311 1793.9729 3545.2089 1038.2164
class 598.5605 577.7487 759.4405 359.7824
how much faster 16.259x 3.105x 4.668x 2.885x

Um, slightly different? In his original post, he states that the indirect sorting is twice as fast in C++ than in C#. I can’t do a direct comparison since I didn’t run the C++ code, but since my change to the C# made it run 2.885 times faster than his C# code, it stands to reason that the C# and C++ performance are neck and neck, if not a bit faster in C#.

Apples to Oranges to Avocados

If you’re rooting for the C++ side, you’re probably thinking, “No fair! The C# one didn’t have to move the whole array around in memory!” Well, yeah, it didn’t: because that’s how C# programmers use the language. Since it’s safe to rely on the garbage collector to handle deallocations, C# programmers don’t spend effort avoiding using “dangerous” pointers (i.e. reference types). This is simply how the language is used. To me, the fairest comparison is one that preserves both the procedures (which he did by using the built-in sorts) and the data structures (which he did not do) used by each language.

The Code

Aside from the Data change above, I cleaned up some of the copy and paste in his code. Here’s what I used:

using System;
using System.Collections;
using System.Collections.Generic;
using System.Text;
using System.Runtime.InteropServices;

namespace CachePressureCS
{
    // normal c# type
    class Data
    {
        public int key;
        public char[] data;

        public Data() { data = new char[128]; }
    }

    class DataComparer : IComparer
    {
        int IComparer.Compare(Object x, Object y)
        {
            return ((Data)x).key - ((Data)y).key;
        }
    }

    class DataComparerT : IComparer<Data>
    {
        public int Compare(Data x, Data y)
        {
            return x.key - y.key;
        }
    }

    class Timer
    {
        [DllImport("Kernel32.dll")]
        private static extern bool QueryPerformanceCounter(out long counter);

        [DllImport("Kernel32.dll")]
        private static extern bool QueryPerformanceFrequency(out long frequency);

        public Timer()
        {
            mStart = mEnd = 0;
            QueryPerformanceFrequency(out mFrequency);
        }

        public void Start() { QueryPerformanceCounter(out mStart); }

        public void End() { QueryPerformanceCounter(out mEnd); }

        public double Time
        {
            get { return 1000.0 * (double)(mEnd - mStart) / (double)mFrequency; }
        }

        long mFrequency;
        long mStart;
        long mEnd;
    }

    class Program
    {
        static int CompareData(Data x, Data y)
        {
            return x.key - y.key;
        }

        static Data[] MakeData(int size)
        {
            Random rng = new Random(0);
            Data[] data = new Data[size];

            for (int i = 0; i < data.Length; i++)
            {
                data[i] = new Data();
                data[i].key = rng.Next();
            }

            return data;
        }

        static double Test(int size, Action<Data[]> sort)
        {
            Timer time = new Timer();
            Data[] data = MakeData(size);

            time.Start();
            sort(data);
            time.End();

            return time.Time;
        }

        static double SortTest(int size)
        {
            return Test(size, data => Array.Sort(data, new DataComparer()));
        }

        static double SortTestT(int size)
        {
            return Test(size, data => Array.Sort<Data>(data, new DataComparerT()));
        }

        static double SortTestTC(int size)
        {
            return Test(size, data => Array.Sort<Data>(data, CompareData));
        }

        static double SortTestC(int size)
        {
            return Test(size, data => Array.Sort(data, CompareData));
        }

        static double SortTestIndirect(int size)
        {
            Random rng = new Random(0);
            Timer time = new Timer();
            Data[] data = new Data[size];
            int[] indirect = new int[size];

            for (int i = 0; i < data.Length; i++)
            {
                data[i] = new Data();
                data[i].key = rng.Next();
                indirect[i] = data[i].key;
            }

            time.Start();
            Array.Sort<int, Data>(indirect, data);
            time.End();

            return time.Time;
        }

        private static void Time(Func<int, double> fn, int size)
        {
            double time = 0;
            for (int j = 0; j < 10; j++)
            {
                time += fn(size);
            }
            time /= 10.0;

            Console.Write("{0,14:F4}", time);
        }

        static void Main(string[] args)
        {
            Console.WriteLine("    size      SortTest     SortTestT    SortTestTC  SortIndirect");
            Console.WriteLine("-------- ------------- ------------- ------------- -------------");

            for (int i = 0; i < 10; i++)
            {
                int size = 1024 << i;

                Console.Write("{0,8}", size);
                Time(SortTest, size);
                Time(SortTestT, size);
                Time(SortTestTC, size);
                Time(SortTestIndirect, size);

                Console.WriteLine();
            }

            Console.ReadKey();
        }
    }
}