Building an arm-cross-gnu-toolchain can be somewhat painful. I'm posting here how I could achieve this after reading lots of documents over the web.
This will be helping me keeping up-to-date with binutils and gcc versions when building arm kernels 
Requirements:
- binutils - http://www.gnu.org/software/binutils/
- GCC - http://gcc.gnu.org/
- Newlib - http://sourceware.org/newlib/
To achieve this we must start by building binutils.
Building binutils
Get the most recent version of it, unpack it and cd to that directory. Now follow these steps:
$ mkdir objdir-arm $ cd objdir-arm $ ../configure --target=arm-none-linux-gnueabi --prefix=/usr/local --program-prefix=arm-linux- --with-newlib $ make # make install
Building GCC
Get the most recent version of it, unpack it and cd to that directory. Now follow these steps:
$ mkdir objdir-arm $ cd objdir-arm $ ../configure --target=arm-none-linux-gnueabi --with-newlib --without-headers --with-gnu-as --with-gnu-ld \ --enable-languages=c --disable-nls --disable-libssp --prefix=/usr/local --program-prefix=arm-linux- \ --disable-shared $ make # make install
Building newlib
Get the most recent version of it, unpack it and cd to that directory. Now follow these steps:
$ mkdir objdir-arm $ cd objdir-arm $ ../configure --target=arm-none-linux-gnueabi --prefix=/usr/local --with-gnu-as --with-gnu-ld --program-prefix=arm-linux- $ make # make install