---
author: marco
categories:
- Howto
date: "2006-02-21T09:49:31Z"
title: Recovering from IMAP crash
---

One morning I connect to my IMAP server and it tells me there is an I/O error. Terrified, I connect using SSH and find out the hard drive is full of backup files. This in turn seems to have caught Cyrus in the worst possible moment, and I struggle to restart it. Whenever I do, I get weird error messages like this:  
<span style="font-family: courier new,courier,monospace;">Feb 21 04:35:15 invent lmtpunix\[9573\]: DBERROR db4: PANIC: fatal region error detected; run recovery</span>  
<span style="font-family: courier new,courier,monospace;">Feb 21 04:35:15 invent lmtpunix\[9573\]: DBERROR: critical database situation</span>  
Of course, I have not the slightest idea what to do. At least there is a hint to a recovery tool, no?

# What To Look At

First things first: the Cyrus documentation is not as good as it should be, but not as bad as it could. Fortunately, a lot of people seem to have had similar problems, and I could garner a lot of information.

There are four places that store information relevant to this process:

1. <span style="font-weight: bold;">/var/spool/imap</span> the sacred location of the mail store (your email)
2. <span style="font-weight: bold;">/usr/lib/cyrus</span> where the tools for cyrus are located
3. <span style="font-weight: bold;">/etc</span> with the configuration files
4. <span style="font-weight: bold;">/var/lib/imap</span> the location of the corrupted db files

I had no idea what was corrupt, but got good vibes running ‘file’ on most of the db files in the last directory. Only the mailboxes.db file and the db directory seemed out of whack. That’s good, you can rebuild both easily (if you know how).

# First Things First

Back up the directories listed above, at the very least the /var/spool/imap and /var/lib/imap directories! You are going to do some really invasive surgery around here, and you want to be sure you don’t lose anything permanently!

# The Script

There is a trick – you have to run a script that does most of the work for you. It is listed under "More…". Copy and paste it into an editor, save it under a convenient name, chmod +x it, and run. It should do all its work without much complaining. I got it from a Google source, that had a bunch of things wrong. After fixing paths and adding the right secret sauce, it worked just fine.

To run the script, first shut down the IMAP server, then execute. Once the script is done (which took about 10 minutes on my machine), you can restart the server.

Ensure you read the output of /var/log/mail and /var/log/messages to ensure everything is fine! The first few times, I had gotten it wrong, and the log files quickly grew again into making my system run out of space.

  <span style="font-family: courier new,courier,monospace;">\#!/bin/sh</span>  
<span style="font-family: courier new,courier,monospace;">\#</span>  
  
<span style="font-family: courier new,courier,monospace;"> </span><span style="font-family: courier new,courier,monospace;">\# backup existing mailboxes.db</span>  
<span style="font-family: courier new,courier,monospace;">mv /var/lib/imap/mailboxes.db /var/lib/imap/mailboxes.db.$$</span>  
<span style="font-family: courier new,courier,monospace;">find /var/spool/imap/user -maxdepth 1 -mindepth 1 | \\</span>  
<span style="font-family: courier new,courier,monospace;"> while read i; do</span>  
<span style="font-family: courier new,courier,monospace;"> export name=$(basename $i)</span>  
<span style="font-family: courier new,courier,monospace;"> echo "user.$name default $i lrswipcda cyrus lrswipcda" &gt;&gt; /tmp/newmboxlist.txt</span>  
<span style="font-family: courier new,courier,monospace;"> done</span>  
<span style="font-family: courier new,courier,monospace;"> </span>  
<span style="font-family: courier new,courier,monospace;">\# we have everyone’s base directory in /tmp/newmboxlist.txt, now we</span>  
<span style="font-family: courier new,courier,monospace;">\# import the new list</span>  
<span style="font-family: courier new,courier,monospace;">/usr/lib/cyrus/bin/ctl_mboxlist -u &lt;/tmp/newmboxlist.txt</span>  
  
<span style="font-family: courier new,courier,monospace;">\# and reconstruct to get subfolders into the list</span>  
<span style="font-family: courier new,courier,monospace;">find /var/spool/imap/user -maxdepth 1 -mindepth 1 | \\</span>  
<span style="font-family: courier new,courier,monospace;"> while read i; do</span>  
<span style="font-family: courier new,courier,monospace;"> export name=$(basename $i)</span>  
<span style="font-family: courier new,courier,monospace;"> /usr/lib/cyrus/bin/reconstruct -rf user.$name</span>  
<span style="font-family: courier new,courier,monospace;"> done</span>