Debunking C# vs C++ Performance

↩ ↪

January 03, 2009 c-sharp code cpp optimization

If you were on reddit today, you probably saw this article, damning C#’s performance as being ten times worse than C++’s. Holy shit balls, batman!

Running his C# code, here are the results I got:

Original C# code

Array Size	SortTest	SortTestT	SortTestTC	SortIndirect
1024	10.7162	2.3441	3.8781	1.1366
2048	22.9509	4.3889	8.4408	1.8714
4096	49.3709	8.4452	17.3883	3.7319
8192	103.5701	18.5369	38.1285	8.0310
16384	220.9323	39.6958	80.9258	18.5821
32768	469.5507	84.5129	172.2964	41.2126
65536	1016.2149	188.6718	380.3507	93.2924
131072	2156.4188	399.7299	791.6437	210.9526
262144	4616.3540	847.9829	1692.9814	467.6020
524288	9732.4311	1793.9729	3545.2089	1038.2164

Pretty slow! So I took a look at the code. The first thing that would catch the eye of any C# programmer is this:

unsafe struct Data
{
    public int key;
    public fixed char data[128];
}

That’s the data structure he’s sorting. An unsafe struct with a fixed array? I had to look up fixed to even know what that means. Now, I understand that he’s trying to make an apples/apples comparison and keep the data structure as close to the C++ one as possible, but I think that’s missing the point. If you’re going to compare two languages, using their built-in typical sort functions, shouldn’t you use their typical data structures too? Here’s how a regular C# developer would define Data:

class Data
{
    public int key;
    public char[] data;

    public Data() { data = new char[128]; }
}

No unmanaged code, no structs (which are rarely used in C#). Just a regular class with an array. Here’s the results:

Modified to typical C# code

Array Size	SortTest	SortTestT	SortTestTC	SortIndirect
1024	0.3605	0.3626	0.4150	0.5918
2048	0.7651	0.7446	0.8749	0.5021
4096	1.6434	1.6094	1.9468	1.2030
8192	3.6497	3.5216	4.1014	2.3926
16384	7.9555	8.0842	9.3324	5.4752
32768	21.1833	19.1183	23.1170	15.1998
65536	54.6938	53.4892	72.3932	34.6554
131072	122.5008	114.1937	141.3504	75.9064
262144	279.8014	262.5908	343.4204	160.8344
524288	598.5605	577.7487	759.4405	359.7824

Let’s compare the last lines of each:

Data Type	SortTest	SortTestT	SortTestTC	SortIndirect
struct/fixed	9732.4311	1793.9729	3545.2089	1038.2164
class	598.5605	577.7487	759.4405	359.7824
how much faster	16.259x	3.105x	4.668x	2.885x

Um, slightly different? In his original post, he states that the indirect sorting is twice as fast in C++ than in C#. I can’t do a direct comparison since I didn’t run the C++ code, but since my change to the C# made it run 2.885 times faster than his C# code, it stands to reason that the C# and C++ performance are neck and neck, if not a bit faster in C#.

Apples to oranges to avocados

If you’re rooting for the C++ side, you’re probably thinking, “No fair! The C# one didn’t have to move the whole array around in memory!” Well, yeah, it didn’t: because that’s how C# programmers use the language. Since it’s safe to rely on the garbage collector to handle deallocations, C# programmers don’t spend effort avoiding using “dangerous” pointers (i.e. reference types). This is simply how the language is used. To me, the fairest comparison is one that preserves both the procedures (which he did by using the built-in sorts) and the data structures (which he did not do) used by each language.

The code

Aside from the Data change above, I cleaned up some of the copy and paste in his code. Here’s what I used:

using System;
using System.Collections;
using System.Collections.Generic;
using System.Text;
using System.Runtime.InteropServices;

namespace CachePressureCS
{
    // normal c# type
    class Data
    {
        public int key;
        public char[] data;

        public Data() { data = new char[128]; }
    }

    class DataComparer : IComparer
    {
        int IComparer.Compare(Object x, Object y)
        {
            return ((Data)x).key - ((Data)y).key;
        }
    }

    class DataComparerT : IComparer<Data>
    {
        public int Compare(Data x, Data y)
        {
            return x.key - y.key;
        }
    }

    class Timer
    {
        [DllImport("Kernel32.dll")]
        private static extern bool QueryPerformanceCounter(
            out long counter);

        [DllImport("Kernel32.dll")]
        private static extern bool QueryPerformanceFrequency(
            out long frequency);

        public Timer()
        {
            mStart = mEnd = 0;
            QueryPerformanceFrequency(out mFrequency);
        }

        public void Start() { QueryPerformanceCounter(out mStart); }

        public void End() { QueryPerformanceCounter(out mEnd); }

        public double Time
        {
            get {
                return 1000.0 * (double)(mEnd - mStart) /
                    (double)mFrequency;
            }
        }

        long mFrequency;
        long mStart;
        long mEnd;
    }

    class Program
    {
        static int CompareData(Data x, Data y)
        {
            return x.key - y.key;
        }

        static Data[] MakeData(int size)
        {
            Random rng = new Random(0);
            Data[] data = new Data[size];

            for (int i = 0; i < data.Length; i++)
            {
                data[i] = new Data();
                data[i].key = rng.Next();
            }

            return data;
        }

        static double Test(int size, Action<Data[]> sort)
        {
            Timer time = new Timer();
            Data[] data = MakeData(size);

            time.Start();
            sort(data);
            time.End();

            return time.Time;
        }

        static double SortTest(int size)
        {
            return Test(size, data =>
                Array.Sort(data, new DataComparer()));
        }

        static double SortTestT(int size)
        {
            return Test(size, data =>
                Array.Sort<Data>(data, new DataComparerT()));
        }

        static double SortTestTC(int size)
        {
            return Test(size, data =>
                Array.Sort<Data>(data, CompareData));
        }

        static double SortTestC(int size)
        {
            return Test(size, data =>
                Array.Sort(data, CompareData));
        }

        static double SortTestIndirect(int size)
        {
            Random rng = new Random(0);
            Timer time = new Timer();
            Data[] data = new Data[size];
            int[] indirect = new int[size];

            for (int i = 0; i < data.Length; i++)
            {
                data[i] = new Data();
                data[i].key = rng.Next();
                indirect[i] = data[i].key;
            }

            time.Start();
            Array.Sort<int, Data>(indirect, data);
            time.End();

            return time.Time;
        }

        private static void Time(Func<int, double> fn, int size)
        {
            double time = 0;
            for (int j = 0; j < 10; j++)
            {
                time += fn(size);
            }
            time /= 10.0;

            Console.Write("{0,14:F4}", time);
        }

        static void Main(string[] args)
        {
            Console.WriteLine("    size      SortTest     SortTestT" +
                "    SortTestTC  SortIndirect");
            Console.WriteLine("-------- ------------- " +
                "------------- ------------- -------------");

            for (int i = 0; i < 10; i++)
            {
                int size = 1024 << i;

                Console.Write("{0,8}", size);
                Time(SortTest, size);
                Time(SortTestT, size);
                Time(SortTestTC, size);
                Time(SortTestIndirect, size);

                Console.WriteLine();
            }

            Console.ReadKey();
        }
    }
}