(You are Anonymous)

Project Directory Layout with CGI::Application

The file system layout of a project is important, because an intuitive system will be intuitive to navigate around, and will scale well as the project grows.

Especially a standard early is also beneficial so that future projects will be consistent, and also because changes may become more difficult later.

CGI::Application developers tend to agree on the principles of how they layout their projects, while differing on specific naming choices. The general principles that are most often seen applied are:

  1. Organize first by file type
  2. Within each file type, organize by sub-projects, if any.

Here's a thorough example from David Kaufman.

David Kaufman's example layout

I learned the way I do development environments from Jesse [Erlbaum, the CGI::Application creator] during my stint at Vanguard, have used this methodology pretty religiously since, and it's served me well. My development environments are always under svn revision control, so one requirement of this system is a strong separation between what is, and what's not, under revision control. Of course, perl modules and templates are always in svn, while system generated data files, file uploads, and logs are not, but even these non-svn files have homes in the project's directory structure.

I lay out each project under a PROJECT_ROOT directory that contains no files itself, just the subdirectories, with "project_name" being the svn repository name, as well:

project_name/
       htdocs/
       modules/
       templates/
       devdocs/
       conf/
       bin/
       externals/
       logs/

The "externals" and "logs" subdirectories are where transient data files and apache logs live, but only the directories themselves exist in svn, no files in these directories are version-controlled.

To keep svn from complaining that all the (e.g. log) files in such directories are "new" (and listing all the file names with along with a question mark when you do an "svn stat" command) set the "svn:ignore" property on the directory with the value , like this:

svn propset svn:ingore '*' logs

bin/

Is where I put miscellaneous command-line utilities such as backup scripts, cron jobs and tools for developers to use on a new checkout to say, set permissions on files and initially generate any local conf files.

htdocs/

is obviously the apache document root of the site - I don't use cgi-bin directories, as such, outside of the document root, preferring instead to let my instance scripts live in place right alongside the static html pages and images etc. of the site's publicly accessible files. some clients, and other places that i've worked, have insisted on using a cgi-bin outside of the doc-root, and only allowing executable scripts to be invoked from there (using apache's Script Alias directive), which is fine and works well also. It just makes the list of subdirectories in the project_root one entry longer :-)

modules/

Is of course where the perl modules reside, and the

templates/

Directory contains the html-template files (more below)

devdocs/

Is just that, development docs, where the database schema and a log of database structure changes (an "executable" log of SQL DDL statements) is maintained, along with installation notes, and sometimes even tarballs of other software needed to be installed in the project root to setup new dev environments or production sites. Having the database structure and its changes, add-hoc installation notes and original versions of installed modules here, under svn has saved me much grief.

conf/

Contains the local environment's configuration files, in particular an apache.conf file (or a relevant subset thereof, such as this site's <Virtual Host> block). Jesse often made this file an html template, especially in situations where there were multiple development environments, a staging server and a production site, each an svn checkout and updated directly from svn. In this case, one would run a script in bin/ after an svn checkout or update if necessary, to regenerate a local conf file from the template.

For a simpler setup I'll often keep a vhost.conf-sample (or .htaccess-sample) in svn and maintain the corresponding "live" conf file for it in each environment by hand.

but whether the site conf needs to be maintained from a template or by hand, it keeps all the local installation-specific information in one place, so that the virtual hostname, and Apache environment variables defined below, are all that have to be tailored so that the software will have what it needs to locate perl modules, templates, database connections to use, without having to touch the code. In the virtual host conf (or even an .htaccess file) I will typically have:

# for perl to find my local modules
SetEnv PERL5LIB /path/to/project_name/modules

# for C::A and H::T to find my templates
SetEnv HTML_TEMPLATE_ROOT /path/to/project_name/templates

# for scripts to use as the base dir for data files, uploads, etc.
SetEnv EXTERNALS /path/to/project_name/externals

# for DBI to find a database connection
# DBI_DSN and/or DBI_USER may also be defined here

This way, perl scripts can just say:

use MyProject::User::Manager;

And since PER5LIB points to my environment's own modules directory, apache has already configured my scripts to resolve the relative location of:

MyProject/User/Manager.pm
   as
/path/to/project_name/modules/MyProject/User/Manager.pm

And scripts don't have to be modified to work when checked out of svn into new development environments. You just have to tailor the conf files to the particular installation environment, in the same place you might configure the site as mod_perl vs. CGI-exec mode, or turn on and off debugging flags and other locally tweakable variables and settings.

For the templates (and to answer your actual question), if I have, for instance, these 3 C::A modules, in my modules directory:

<pre>

modules/MyProject/User/Manager.pm
modules/MyProject/User/Profile.pm
modules/MyProject/Login.pm

</pre>

for their associated templates, I like to have the a similar directory structure mirrored under templates, but unlike perl modules there's no need for a top-level My Project directory, since al the templates in here are specific to my project (and there's scant chance I'll need a place to put 3rd party "CPAN templates", the way I'll often need a place to install CPAN perl modules :-)) so I end up with a templates subdirectory structure like:

templates/User/Manager/list_users.tmpl
templates/User/Manager/add_user.tmpl
templates/User/Manager/edit_user.tmpl
templates/User/Profile/view_profile.tmpl
templates/User/Profile/edit_profile.tmpl
templates/User/Login/login_form.tmpl
templates/User/Login/password_reminder.tmpl

I like grouping similar apps, such as a User Management application (that is available only to admins), with a similar user profile app, even though they have different security modules, because if I ad a new user field, or restructure the users database these are the apps that would tend to need maintenance.

Notice also that each perl Module Name is represented in templates as a directory. That's because each CGI::Application, having multiple run-modes, tends to have several corresponding template files. And I don't like to search through long lists of template filenames to find the ones that apply to the module I'm working on. also, as you can see I tend to name my templates as $verb_$noun.tmpl (edit_user, view_profile, etc.) so if you throw a bunch of these, that belong to different apps, together into a single directory, they tend to sort badly, with all the add_this and add_that's together at the top, and their corresponding view_this and view_that waaaay down at the bottom. anyway it works for me because I also tend to lean towards modules with one "subject" (user) and no more than 3-6 major types of "operations" such the archetypical create, read, update & delete. I find that the 3-6 operations typically involve 2 run-modes per, and one template per, though a couple more of either always tend to creep in. this tendency to have more modules with fewer major "operations" per module is just my preference.

It seems that many folks tend to use a single CGI-App to house all the code for an entire site, from authentication and authorization, to user management to <whetever-else-the-site-does>. I think my leanings toward more modules, in more directories, is driven by my desire to be able to reuse a thing (like a user management script) by just scooping up its perl module and templates subdirectory from one site and plunk it down in another and not have to spend time trying to tease the relevant run-modes out of a larger monolithic .pm file :-)

Now, thanks to apache taking care of setting the HTML_TMPL_ROOT environment variable, my C::A modules can simply say:

load_tmpl('User/Manager/list_users.tmpl')

And not care what the top-level absolute pathname was, in order to find it under this environment's templates directory with no code changes (after the initial configuration of the environment).

although i've never needed to make use of them, both PERL5LIB and (I believe) HTML_TMPL_ROOT even support a colon separated list, as a means of specifying a list of absolute paths to search, if you should find a situation where you need an archy even hier than this one :) such as several sites that share common headers and menus templates but house different apps (I guess – as I said, even I never seem to need that many nested directories...)

Considering Dreamweaver

Using Dreamweaver adds to manage your site design brings another consider. Dreamweaver expects html files that it affects the design of to be within the web root.

Dreamweaver stores its own templates in /Templates, inside the web root. At Summersault where Mark Stosberg works, the standard is to put the HTML::Template files also in /Templates and protect the directory with .htaccess, since neither Dreamweaver nor the templating engine need to access the files there through the web.


Originally compiled by Steve Comrie based on Mailing List discussions between September 04 & September 07, 2003. It was significantly refactored by Mark Stosberg.