rGADEM soon to support Grand Central Dispatch

By on April 17, 2010 in News, Research, Software

So rGADEM and GCDI just wanted to post an update regarding our development of rGADEM. As I said in a previous post, we have been working very hard to improve the computing efficiency of our ChIP-seq pipeline. After PICS, that supports parallel computing via snowfall, rGADEM will soon support Grand Central Dispatch (GCD) on Snow Leopard. As you can see on the teaser on the right, rGADEM is using 16 threads on my Mac Pro. In comparison to snowfall, GCD does not need any  input configuration. GCD will automatically decide how many threads to use at run time based on what resources are available. On a preliminary test run, I was able to run rGADEM in less 30 minutes on 4000 ChIP-Seq sequences. This is compared to more than 4 hours using the serial version.

Pretty cool, no? I will add another post as soon as its officially available on BioC. Stay tuned.

About Raphael Gottardo

I am now an Associate Member within the Vaccine and Infectious Disease Division at the FHCRC and PI of the rglab.
View all posts by Raphael Gottardo →

, ,

4 Responses to rGADEM soon to support Grand Central Dispatch

  1. S Kong June 25, 2010 at 12:11 pm #

    Dear Raphael,

    I am wondering whether GCD implementation is already integrated into rGADEM package.

    Thanks,

    Reply
    • Raphael Gottardo June 25, 2010 at 4:57 pm #

      Dear Sekwon,

      Yes it’s already in Bioconductor! It’s working very well and I encourage you to check it out. We are also working on an openmp implementation that will work under linux and hopefully windows. In the meantime, if you have a mac running Snow Leopard, you can simply download the package and use it!
      Please make sure you are running R 2.11 and the latest version of rGADEM.

      Enjoy,

      Raphael

      Reply
  2. S Kong July 13, 2010 at 1:35 pm #

    Hi, Raphael

    It works great! Fantanstic!
    A quick question:
    I changed
    #define MAX_NUM_SEQ
    to 60000 in defines.h to analyze more sequences.

    The error I got was

    *** Start C Programm ***
    tmp calloc failed!

    Do I need to change any other memory allocation parameter in different files too?

    Thanks much!
    -Sek Won

    Reply
    • Raphael Gottardo July 14, 2010 at 6:12 am #

      Hi Sek,

      There is a hard coded number of sequences in the C code. By default this is set to MAX_NUM_SEQ=60000. How many sequences do you have?
      Do you have more than 60000? Although we could change that upper limit I would encourage you to use fewer sequences.
      It might take quite a long time to run if you have more than 10000 sequences. With the openMP implementation you should be able to increase that number but for the time being I would encourage you to use 5000-10000 sequences to begin with.

      Hope this helps,

      Raphael

      Reply

Leave a Reply