Their Open Source so the code should be available for you to download. If you are working on a windows machine then you can use XAMPP to setup a local enviroment on your machine for Joomla!
When I toyed with it a while back that is what I did and it worked pretty nicely. I believe Joomla! even has a walkthru on how to get started.
Personally I have worked on multiple CMS projects and they are not something you want to start from scratch. They take lots of time to develop and its best just to use the free versions out their and customize them or write plugin to them to do what you want.
Another important tip to save you some frustration. Make sure u keep an original of the file before you start going crazy editing code. That way if you really botch it you can at least get things back to where you started.
Good luck!