Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

My question is - why does Postgres need its own scheduler? Shouldn't that be the job of the OS? Is it a legacy thing or just something to squeeze out a tiny bit of extra performance?


For similar reasons to why DB engines often implement their own caching and disable OS disk caching; a combination of more information about what needs to be done, and a very competitive marketplace.


Indeed, there are lessons here, especially from people like PHK and projects like Varnish. Understanding the underlying system is critical for these sorts of projects. For 99% of users the OS will do a much better job of handling your resources than you ever will, but there will always remain a segment where additional knowledge about how those resources are going to be used can yield significant performance gains.


PostgreSQL's spinlock implementation is very fast and scalable and not all supported platforms have good implementations of mutexes. The Linux futexes might be fast enough to replace PostgreSQL's spinlock implementation, but even that is not known yet.


Here—have a microbenchmark!

Here are two really simple mutexes: a wrapper around the pthread functions, and a no-thought-required spinlock:

    class Spinlock {
    private:
        bool state;
    public:
        Spinlock() { state = false; }
        void lock() {
            while(!__sync_bool_compare_and_swap(&state, false, true));
        }
        void unlock() {
            state = false;
            __sync_synchronize();
        }
    };
    
    class PthreadMutex {
    private:
        pthread_mutex_t mutex;
    public:
        PthreadMutex() {
            pthread_mutex_init(&mutex, NULL);
        }
        void lock() {
            pthread_mutex_lock(&mutex);
        }
        void unlock() {
            pthread_mutex_unlock(&mutex);
        }
        ~PthreadMutex() {
            pthread_mutex_destroy(&mutex);
        }
    };
I used these to protect a not-optimized-at-all dispatch queue built on top of std::deque (single reader, single writer, reader busy-waits until work is available, work is an empty function). I did this test on Mac OS X 10.6.8 (with Apple s gcc 4.2.1 build).

A run using PthreadMutex:

    0,5944405
    1,1627173
    2,135598
    3,363801
    4,3360217
    5,5773374
    6,5727638
    7,5953505
    8,4567648
    9,5106300
Using Spinlock:

    0,154817
    1,187507
    2,145171
    3,102454
    4,180604
    5,155360
    6,128448
    7,82418
    8,49538
    9,39560
All times are in microseconds to run 1,000,000 iterations of the test. Only time to enqueue is measured.

I've seen similar results with a very similar test on FreeBSD, though I don't have a box to retest on at the moment.

I can only conclude it's not at all unreasonable for PostgreSQL to use its own spinlocks.


It will have more information about what is trying to be achieved than can be given to the kernel with the available (across all platforms where possible) API. More knowledge/more specialised means it can have greater optimisation.


PostgreSQL does not have its own scheduler. Here's a comment at LWN that tries to correct the article a bit: http://lwn.net/Articles/518405/


Everyone does this. Even SQL Server, which runs on only Windows and is made by that very same company, had to have Fibres invented in the kernel so it can do its own user mode scheduling.

http://msdn.microsoft.com/en-us/library/aa175385(v=sql.80).a...




Consider applying for YC's Fall 2026 batch! Applications are open till July 27.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: