Needed to add the clearQueue flag to jQuery's stop() method

At the end of the backup, wait a bit for dovecot and postfix to finish restarting.

Hopefully fixes #381.

if you make an API call with a user-specific API key (e.g. from control panel) but your account no longer exists on the system, there was an unhandled error

see 1039a08b

move the email address syntax validation for users and aliases into my new email_validator library (https://github.com/JoshData/python-email-validator)

see https://github.com/owncloud/core/issues/14976

We will need to update when ownCloud makes this better with MOD_X_ACCEL_REDIRECT_PREFIX.

See https://discourse.mailinabox.email/t/owncloud-can-not-read-uploaded-data/428.

stricter validation of the domain parts of email addresses: only letters, numbers, and hyphens, and the TLD ends with a letter

jquery 1.11.1, bootstrap 3.3.0, better accessibility, see https://github.com/JoshData/html5-stub

add a 'backup --verify' command to run duplicity's verify command to check that the backup files are OK

Merge branch 'dhpiggott-gpg-encrypt-backups'

* use the AES256 cipher, be explicit that only the first line of secret_key.txt is used, and sanity check that the passphrase is long enough
* change overship of the encrypted files to the user-data user
* simplify variable names in management/backup.py
* although I appreciate long comments I am trimming the commentary about the backup migration
* revise the control panel template to not refer to the old unencrypted files
* add CHANGELOG entry

[Josh merged some subsequent commits:]

* Guard via idempotency against termination between migration operations
* Final corrections and tweaks
* Pass passphrase through to all duplicity calls

Empirical evidence (a failed cron job) shows that cleanup requires the
passphrase (so it presumably needs to decrypt metadata), and though
remove-older-than has been working fine without it, it won't do any harm
to set it in case that changes or there are any special cases.

* Add back the archive-dir override but locate it at STORAGE_ROOT/backup/cache

see https://discourse.mailinabox.email/t/allow-spamassassin-to-scan-emails-larger-than-250kb/391

prevent accidental domain control validation hijacking by limiting use of admin@ etc. addresses in users/aliases

I changed my mind. In 1bf8f199 I allowed Unicode domain names to go into the database. I thought that was nice because it's what the user *means*. But it's not how the web works. Web and DNS were working, but mail wasn't. Postfix (as shipped with Ubuntu 14.04 without support for SMTPUTF8) exists in an ASCII-only world. When it goes to the users/aliases table, it queries in ASCII (IDNA) only and had no hope of delivering mail if the domain was in full Unicode in the database. I was thinking ahead to SMTPUTF8, where we *could* put Unicode in the database (though that would prevent IDNA-encoded addressing from being deliverable) not realizing it isn't well supported yet anyway.

It's IDNA that goes on the wire in most places anyway (SMTP without SMTPUTF8 (and therefore how Postfix queries our users/aliases tables), DNS zone files, nginx config, CSR 'CN' field, X509 Common Name and Subject Alternative Names fields), so we should really be talking in terms of IDNA (i.e. ASCII).

This partially reverts commit 1bf8f199, where I added a lot of Unicode=>IDNA conversions when writing configuration files. Instead I'm doing Unicode=>IDNA before email addresses get into the users/aliases table. Now we assume the database uses IDNA-encoded ASCII domain names. When adding/removing aliases, addresses are converted to ASCII (w/ IDNA). User accounts must be ASCII-only anyway because of Dovecot's auth limitations, so we don't do any IDNA conversion (don't want to change the user's login info behind their back!). The aliases control panel page converts domains back to Unicode for display to be nice. The status checks converts the domains to Unicode just for the output headings.

A migration is added to convert existing aliases with Unicode domains into IDNA. Any custom DNS or web settings with Unicode may need to be changed.

Future support for SMTPUTF8 will probably need to add columns in the users/aliases table so that it lists both IDNA and Unicode forms.

fix(read_password): regex check for spaces, quotes

* Add pop3s to the ufw firewall rules.
* Updated some comments.
* Updated CHANGELOG.

Merge branch 'master' of https://github.com/pichak/mailinabox

UIDL assigns a unique string to each email. This allows emails to
be left on the server after a client downloads them.

CHANGELOG
=========

v0.08 (April 1, 2015)
---------------------

Mail:

* The Roundcube vacation_sieve plugin by @arodier is now installed to make it easier to set vacation auto-reply messages from within Roundcube.
* Authentication-Results headers for DMARC, added in v0.07, were mistakenly added for outbound mail --- that's now removed.
* The Trash folder is now created automatically for new mail accounts, addressing a Roundcube error.

DNS:

* Custom DNS TXT records were not always working and they can now override the default SPF, DKIM, and DMARC records.

System:

* ownCloud updated to version 8.0.2.
* Brute-force SSH and IMAP login attempts are now prevented by properly configuring fail2ban.
* Status checks are run each night and any changes from night to night are emailed to the box administrator (the first user account).

Control panel:

* The new check that system services are running mistakenly checked that the Dovecot Managesieve service is publicly accessible. Although the service binds to the public network interface we don't open the port in ufw. On some machines it seems that ufw blocks the connection from the status checks (which seems correct) and on some machines (mine) it doesn't, which is why I didn't notice the problem.
* The current backup chain will now try to predict how many days until it is deleted (always at least 3 days after the next full backup).
* The list of aliases that forward to a user are removed from the Mail Users page because when there are many alises it is slow and times-out.
* Some status check errors are turned into warnings, especially those that might not apply if External DNS is used.

GET requests might be cached. Definitely happens on Internet Explorer. Makes it look like the user is getting unauthorized access.

See https://discourse.mailinabox.email/t/fresh-install-can-login-to-webmail-but-not-admin/394/4.

* Passwords must be at least four characters. So we need to check them
here to ensure that first user creation works during initial setup
* Change quotes to match rest of code

fixes #323
fixes #324

set the SPF record after custom DNS records so that the SPF record doesn't prevent all custom TXT records from coming in

Change read_password() logic to catch short passwords

Currently read_password does not verify password length. But further down the chain, passwords are checked to make sure they are longer than four characters.

If during initial setup, the user enters a password that is shorter than four characters, this will not be caught here, but when the script actually calls management/mailconfig.py to add the user, it will fail without a chance to correct the short password.

The setup script will then continue without an inital user being created and this will confuse users.

status checks: turn missing DNSSEC into a warning instead of an error; omit an error about missing TLSA if DNSSEC isn't in use; if DNSSEC is in use, make a missing TLSA record a warning instead of an error

turn some nameserver status check errors into warnings if the domain resolves correctly since the user might be using External DNS, closes #330

The contacts and calendar apps are now maintained outside of ownCloud core, so we now pull them in from github tags and must enable them explicitly.

see a8669197