see #375

=====

May 8, 2015

Mail:

* Spam checking is now performed on messages larger than the previous limit of 64KB.
* POP3S is now enabled (port 995).
* Roundcube is updated to version 1.1.1.
* Minor security improvements (more mail headers with user agent info are anonymized; crypto settings were tightened).

ownCloud:

* Downloading files you uploaded to ownCloud broke because of a change in ownCloud 8.

DNS:

* Internationalized Domain Names (IDNs) should now work in email. If you had custom DNS or custom web settings for internationalized domains, check that they are still working.
* It is now possible to set multiple TXT and other types of records on the same domain in the control panel.
* The custom DNS API was completely rewritten to support setting multiple records of the same type on a domain. Any existing client code using the DNS API will have to be rewritten. (Existing code will just get 404s back.)
* On some systems the `nsd` service failed to start if network inferfaces were not ready.

System / Control Panel:

* In order to guard against misconfiguration that can lead to domain control validation hijacking, email addresses that begin with admin, administrator, postmaster, hostmaster, and webmaster can no longer be used for (new) mail user accounts, and aliases for these addresses may direct mail only to the box's administrator(s).
* Backups now use duplicity's built-in gpg symmetric AES256 encryption rather than my home-brewed encryption. Old backups will be incorporated inside the first backup after this update but then deleted from disk (i.e. your backups from the previous few days will be backed up).
* There was a race condition between backups and the new nightly status checks.
* The control panel would sometimes lock up with an unnecessary loading indicator.
* You can no longer delete your own account from the control panel.

Setup:

* All Mail-in-a-Box release tags are now signed on github, instructions for verifying the signature are added to the README, and the integrity of some packages downloaded during setup is now verified against a SHA1 hash stored in the tag itself.
* Bugs in first user account creation were fixed.

require TLS on SMTP submission (port 587) to prevent accidental client misconfiguration, although this has no other practical consequences since without TLS clients couldn't authenticate anyway

Even though SMTP (on port 25) is typically opportunistic and a MitM attack can't be prevented, we may as well only offer ciphers that provide some level of security. If a client is so old or misconfigured that it doesn't support newer ciphers, it should hopefully fall back to a non-TLS connection.

Postfix's default was basically anything goes (anonymous and 40-bit ciphers!). Google's MTA's only offer ciphers at 112 bits at greater, and this change approximates that with Postfix's "medium" setting.

Fixes #371

This disallows aNULL and other bad ciphers in the Postfix submission server.

I missed an option in 45e93f7d recommended by the blog post I was reading.

Fixes #389.

https://discourse.mailinabox.email/t/nsd-service-not-started-at-startup-dns-not-working/449

Prior to nsd 4.0.1-1ubuntu0.1, we had to create the nsd user before installing the nsd package.

This was our issue #25 (see 4e6037c0, c7e1e29d) and I reported it upstream at https://bugs.launchpad.net/ubuntu/+source/nsd/+bug/1311886. The new package was published by Ubuntu on 2015-01-15 so this work-around is no longer needed.

permit the first user account to be a domain control validation address because a) it will necessarily be an admin and b) the user doesn't know the rules yet

I don't have a mail client that sets Mime-Version with a user agent string so I couldn't really test.

e.g. multiple TXT records

fixes #333

re-do the custom DNS get/set routines so it is possible to store more than one record for a qname-rtype pair, like multiple TXT records

Better while debugging.

use the font-awesome .fa-spinner.fa-pulse classes for the AJAX loading indicator, rather than the static glyphicon-time icon

Needed to add the clearQueue flag to jQuery's stop() method

At the end of the backup, wait a bit for dovecot and postfix to finish restarting.

Hopefully fixes #381.

if you make an API call with a user-specific API key (e.g. from control panel) but your account no longer exists on the system, there was an unhandled error

see 1039a08b

move the email address syntax validation for users and aliases into my new email_validator library (https://github.com/JoshData/python-email-validator)

see https://github.com/owncloud/core/issues/14976

We will need to update when ownCloud makes this better with MOD_X_ACCEL_REDIRECT_PREFIX.

See https://discourse.mailinabox.email/t/owncloud-can-not-read-uploaded-data/428.

stricter validation of the domain parts of email addresses: only letters, numbers, and hyphens, and the TLD ends with a letter

jquery 1.11.1, bootstrap 3.3.0, better accessibility, see https://github.com/JoshData/html5-stub

add a 'backup --verify' command to run duplicity's verify command to check that the backup files are OK

Merge branch 'dhpiggott-gpg-encrypt-backups'

* use the AES256 cipher, be explicit that only the first line of secret_key.txt is used, and sanity check that the passphrase is long enough
* change overship of the encrypted files to the user-data user
* simplify variable names in management/backup.py
* although I appreciate long comments I am trimming the commentary about the backup migration
* revise the control panel template to not refer to the old unencrypted files
* add CHANGELOG entry

[Josh merged some subsequent commits:]

* Guard via idempotency against termination between migration operations
* Final corrections and tweaks
* Pass passphrase through to all duplicity calls

Empirical evidence (a failed cron job) shows that cleanup requires the
passphrase (so it presumably needs to decrypt metadata), and though
remove-older-than has been working fine without it, it won't do any harm
to set it in case that changes or there are any special cases.

* Add back the archive-dir override but locate it at STORAGE_ROOT/backup/cache

see https://discourse.mailinabox.email/t/allow-spamassassin-to-scan-emails-larger-than-250kb/391

prevent accidental domain control validation hijacking by limiting use of admin@ etc. addresses in users/aliases

I changed my mind. In 1bf8f199 I allowed Unicode domain names to go into the database. I thought that was nice because it's what the user *means*. But it's not how the web works. Web and DNS were working, but mail wasn't. Postfix (as shipped with Ubuntu 14.04 without support for SMTPUTF8) exists in an ASCII-only world. When it goes to the users/aliases table, it queries in ASCII (IDNA) only and had no hope of delivering mail if the domain was in full Unicode in the database. I was thinking ahead to SMTPUTF8, where we *could* put Unicode in the database (though that would prevent IDNA-encoded addressing from being deliverable) not realizing it isn't well supported yet anyway.

It's IDNA that goes on the wire in most places anyway (SMTP without SMTPUTF8 (and therefore how Postfix queries our users/aliases tables), DNS zone files, nginx config, CSR 'CN' field, X509 Common Name and Subject Alternative Names fields), so we should really be talking in terms of IDNA (i.e. ASCII).

This partially reverts commit 1bf8f199, where I added a lot of Unicode=>IDNA conversions when writing configuration files. Instead I'm doing Unicode=>IDNA before email addresses get into the users/aliases table. Now we assume the database uses IDNA-encoded ASCII domain names. When adding/removing aliases, addresses are converted to ASCII (w/ IDNA). User accounts must be ASCII-only anyway because of Dovecot's auth limitations, so we don't do any IDNA conversion (don't want to change the user's login info behind their back!). The aliases control panel page converts domains back to Unicode for display to be nice. The status checks converts the domains to Unicode just for the output headings.

A migration is added to convert existing aliases with Unicode domains into IDNA. Any custom DNS or web settings with Unicode may need to be changed.

Future support for SMTPUTF8 will probably need to add columns in the users/aliases table so that it lists both IDNA and Unicode forms.