Using the spamassassin URIBL check on a primary nameserver

Usually on a primary nameserver for a number of domains, you shouldn’t be using the local nameserver for your DNS lookups. The reason for this is that if a domain is transferred away from the nameserver without your knowledge, any local lookups for that domain will continue to give the old results and this could end up sending email to the wrong server or worse.

One option is to set /etc/resolv.conf to point to some public nameservers such as Google’s 8.8.8.8 and 8.8.4.4, Cloudflare’s 1.1.1.1 or IBM’s 9.9.9.9. This works fine until you want to use the same server to provide email services using spamassassin. At this point, spamassassin will use /etc/resolv.conf to determine which nameservers to query for things like URIBL (the URI block list at uribl.com). Unfortunately if you are using the same nameservers as a lot of other people, the IP that queries the block list will be blacklisted due to query volume and you’ll end up with an error like the following:

ADMINISTRATOR NOTICE: The query to URIBL was blocked.
See http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block
for more information.

Running spamassassin and processing emails will result in a URIBL_BLOCKED rule showing in the headers with the other rules that were triggered.

A simple workaround that doesn’t involve too much of a change is to install a lightweight resolver, bind it to an internal IP such as 127.0.0.2 and set this as your default nameserver in /etc/resolv.conf

The following instructions apply to CentOS 7 but something similar should work for other Linux distributions. First install the PowerDNS Recursor. It doesn’t matter what your existing nameserver software is, I have BIND running but it could easily be something else.

[root@server ~]# yum install pdns-recursor

Now edit the configuration file /etc/pdns-recursor/recursor.conf to change the IP address that it binds to. Simply find the part of the file for the option local-address, uncomment it and change to read:

################
# local-address IP addresses to listen on, separated by spaces or commas. Also accepts ports.
#
local-address=127.0.0.2

Next create a new loopback alias for the IP 127.0.0.2 by creating the file /etc/sysconfig/network-scripts/ifcfg-lo:0 with the following content:

DEVICE=lo:0
IPADDR=127.0.0.2
NETMASK=255.255.255.255
ONBOOT=yes
NAME=dnsloopback
BOOTPROTO=static

Bring the interface up and then start the PowerDNS Recursor daemon.

[root@server ~]# ifup lo:0
[root@server ~]# systemctl enable pdns-recursor
[root@server ~]# systemctl start pdns-recursor

If it’s configured correctly, you should be able to query 127.0.0.2 with the following command:

[root@server ~]# host -t TXT test.uribl.com.multi.uribl.com 127.0.0.2
Using domain server:
Name: 127.0.0.2
Address: 127.0.0.2#53
Aliases:

test.uribl.com.multi.uribl.com descriptive text "permanent testpoint"

If this works, the final step is to change your local resolver to use 127.0.0.2 instead of whatever it’s currently set to. Any queries for domains hosted locally will go to the PowerDNS Recursor which will then query the root nameservers and then the original nameserver software on your local server. If the domain has been moved away but you still have the zone locally, the PowerDNS Recursor running on 127.0.0.2 will simply query the new nameserver instead of your local one.

Edit /etc/resolv.conf – comment out any existing nameserver lines and add one that reads:

nameserver 127.0.0.2

It’s possible that you might need to restart spamassassin for it to take effect. This can be done using:

[root@server ~]# systemctl restart spamassassin

SMF 2.0 on PHP 7.2

I recently changed my server from CentOS 6 to CentOS 7 and in the process decided that to avoid having to upgrade everything later, installing PHP 7.2 before migrating the sites would be a better option than staying with the default PHP 5.4.

This was a good idea – mostly. It forced me to upgrade a couple of mediawiki installs that hadn’t been touched in a while and those were migrated ok. When it came to migrating the Shartak Forum I was starting to think I should have gone for a slightly older version of PHP because SMF 1.1 (Simple Machines Forum) doesn’t work on such a recent version of PHP. It wasn’t just a minor issue, functions such as mysql_* that had been removed in PHP 7 were all over the place!

Time to take the plunge and upgrade to SMF 2.0 which had been stable for several years but there had never been a reason to upgrade before now. Looking at the large amount of spam user accounts and the upgrade process documentation, I decided it would be easier to start from scratch instead of trying to upgrade SMF 1.1 (with some custom tweaks) to 2.0.

I installed the SMF 2.0 code, configured the theme, found that only one of the old tweaks was really still needed and put it live. Then I checked the error logs in the SMF admin panel and found there were thousands of errors after just a few hours of it being live!

Function create_function() is deprecated

It turned out that SMF 2.0.15 only supports up to PHP 7.1 as PHP 7.2 is deprecating create_function() and this is used by the BBCode parser.

After some searching, I came across a message on the SMF forum that suggested adding an exception to the error handler to prevent it logging the deprecation warnings. The suggestion of comparing the version using the builtin PHP method version_compare() was taken into account and here is my version of the solution.

--- Sources/Errors.orig.php 2018-05-23 14:21:57.933367060 +0100
+++ Sources/Errors.php 2018-05-23 13:47:45.711567251 +0100
@@ -203,6 +203,10 @@
 {
 global $settings, $modSettings, $db_show_debug;

+ // Disable PHP 7.2 "Function create_function() is deprecated" errors from filling the forum error logs
+ if (defined('E_DEPRECATED') && $error_level == E_DEPRECATED && (version_compare(phpversion(), '7.2') >= 0) && strpos($error_string, 'Function create_function() is deprecated') !== false)
+ return;
+
 // Ignore errors if we're ignoring them or they are strict notices from PHP 5 (which cannot be solved without breaking PHP 4.)
 if (error_reporting() == 0 || (defined('E_STRICT') && $error_level == E_STRICT && (empty($modSettings['enableErrorLogging']) || $modSettings['enableErrorLogging'] != 2)))
 return;

It’s not a perfect fix but it’ll do until SMF 2.1 is released. I hope this helps someone else searching for the same issue.

Installing DBD::mysql on Mavericks

3 years ago, I posted about getting DBD::mysql to build on OSX 10.6 – it seems the same thing works on Mavericks (10.9) as well!

#   Failed test 'use DBD::mysql;'
#   at t/00base.t line 18.
#     Tried to use 'DBD::mysql'.
#     Error:  Can't load '/Users/simon/.cpan/build/DBD-mysql-4.027-wbgMWL/blib/arch/auto/DBD/mysql/mysql.bundle' for module DBD::mysql: dlopen(/Users/simon/.cpan/build/DBD-mysql-4.027-wbgMWL/blib/arch/auto/DBD/mysql/mysql.bundle, 2): Library not loaded: libmysqlclient.18.dylib
#   Referenced from: /Users/simon/.cpan/build/DBD-mysql-4.027-wbgMWL/blib/arch/auto/DBD/mysql/mysql.bundle
#   Reason: image not found at /System/Library/Perl/5.16/darwin-thread-multi-2level/DynaLoader.pm line 194.
#  at (eval 7) line 2.

The version of MySQL has changed slightly (and hence the path used) but everything else is the same.

$ sudo install_name_tool -id /usr/local/mysql-5.5.30-osx10.6-x86_64/lib/libmysqlclient.18.dylib /usr/local/mysql-5.5.30-osx10.6-x86_64/lib/libmysqlclient.18.dylib

$ otool -D `mdfind libmysqlclient.18.dylib`

cpan2rpm for CentOS 6.5

I’ve been attempting to build all the dependencies for Dancer on CentOS 6.5 and things just kept going wrong. Several of the modules complained about needing perl >= 5.006 even though CentOS 6.5 comes with Perl 5.10.

[mockbuild@build6 ~]$ cpan2rpm Test::Simple

-- cpan2rpm - Ver: 2.028 --
Upgrade check
Fetch: HTTP

-- module: Test::Simple --
Found: Test-Simple-1.001002.tar.gz
At: http://search.cpan.org//CPAN/authors/id/R/RJ/RJBS
Retrieving URL
Metadata retrieval
Tarball extraction: [/home/mockbuild/rpmbuild/SOURCES/Test-Simple-1.001002.tar.gz]
Unable to build module, the following dependencies have failed:
  perl >= 5.006
Stopped at /usr/local/bin/cpan2rpm line 491.

Despite there not being a newer version of cpan2rpm than 2.028 obviously available, there is a development version 2.028_02 which makes it compatible with Perl 5.10.  You have to download it directly from CPAN at http://search.cpan.org/CPAN/authors/id/B/BB/BBB/cpan2rpm-2.028_02.tar.gz and then use cpan2rpm to build it!

[mockbuild@build6 ~]$ cpan2rpm --no-prfx --no-sign ./cpan2rpm-2.028_02.tar.gz

Once it’s built the new cpan2rpm rpm, install it using yum localinstall which will remove the old version at the same time. This new version is able to build the packages for more up to date versions of Test::Simple and URI.

Update: Test::Simple builds but doesn’t pass all the tests, URI is ok.

Extracting ebooks from Apple iBooks for Mavericks

I decided to remove a lot of the read ebooks from my library but wanted to back them up first, especially the ones that weren’t purchased via iTunes. iBooks on Mavericks had “imported” them all from iTunes and as a result I couldn’t just right click on them in iTunes and use Show in Finder.

First step was to locate them – turns out they’re in ~/Library/Containers/com.apple.BKAgentService/Data/Documents/iBooks/Books/

Next problem was that all the files had been renamed from something human readable into a large string of hex characters! The Books.plist file contained a lot of metadata, including the original filename and the current full path of the ebook. I wasn’t about to go renaming them manually so wrote a quick Perl script using Data::Plist::BinaryReader to extract the data structures and print the values for ‘BkDisplayName’ and ‘path’ from each book.

Then I found that although some of them were still .epub files, quite a few had been unpacked into directories containing the epub contents but not with the OEBPS subdirectory.

I wasn’t about to go updating the script to repack all the epubs and just on a whim thought I’d try dragging an ebook out of iBooks onto the desktop.. it worked! The one I had tried was one of the unpacked ebooks but on the desktop was a zipped up epub file. I wish I’d tried dragging them in the first place as it would have saved a lot of hassle!

To back them up, I ended up simply selecting all the ebooks and dragging them to a separate folder. Now I can remove all the read ebooks from within iBooks without worrying about losing them.

cpan2rpm on CentOS 6.4

I’ve just been building up a new server with CentOS 6.4 and noticed that cpan2rpm wouldn’t work for some Perl modules. The error you get is as follows.

Metadata retrieval
Can't locate object method "interpolate" via package "Pod::Text" at
/usr/bin/cpan2rpm line 522.

Turns out it’s because Pod::Text changed between the version that comes with CentOS 5 (2.21) and the version with CentOS 6 (3.13) and the interpolate method disappeared.

I found a patch attached to this ticket and when applied to cpan2rpm, it works once more. https://rt.cpan.org/Public/Bug/Display.html?id=27309#txn-323456

DigitalOcean – first impressions

A couple of friends mentioned they were having a look at http://www.digitalocean.com/ as a VPS provider so I thought I’d take a quick look myself.

The signup process is painless, just an email address and a password required and you’re into a control panel that is relatively uncluttered. Just a few important options down the left side and as you click each one, the top right has a large button with what appears to be the most commonly selected option. For example, click “Billing” reveals “Manage Payments”, “Droplets” (their name for a VPS) results in a “Create” button and “Support” gives “New Ticket”.

There is plenty of space around the various options so no danger of accidentally clicking the wrong button and most of the icons have a tooltip pop up to say what they are (although it’s generally fairly obvious anyway).

There’s a well documented API available – just a couple of clicks to create an API key and you can do anything from create a new droplet to add a domain to their DNS. All the API functionality can be accessed from simple HTTP GET requests and results are JSON formatted so you could easily write a Javascript page to perform sequences of the common actions.

Support tickets get a fast response and there’s an active IRC channel (#digitalocean on Freenode) with plenty of helpful people and even some DigitalOcean staff.

Creation of a droplet takes just a couple of minutes. It says less than 55 seconds, but it took just over 2 minutes for mine – I guess there’s a certain amount of dependency on the load of the host servers. If you upload an SSH public key, it can be pre-installed on the server.

The billing is hourly and you can put some credit on your account with PayPal or add a credit card and then just create/destroy droplets as required. The control panel has a clear amount that the current month has cost you and it’s possible to create a snapshot, destroy the droplet and not incur any more costs until you restore the snapshot to a new droplet (this is apparently changing and snapshots will cost a small amount per month).

A couple of things I’ve noticed so far that aren’t so good:

They have datacentres in three locations: NYC (North Bergen, NJ), San Francisco, and Amsterdam, but only San Francisco is available for new droplets at present due to capacity issues in NYC and Amsterdam. This is supposed to be resolved soon and it’s possible to move droplets between locations by creating a snapshot and then recreating a droplet at the new location from the snapshot. I would imagine there will be limitations such as IP address changes and some downtime whilst the snapshot is made.

I created a CentOS 6.4 droplet and upgraded all the packages – when it rebooted, the kernel was still using an older one and not the one that just got installed. It turns out that you can’t boot a custom kernel, although there is a selection to choose from within the control panel (just not the most recent CentOS kernel).

Overall though, quite a nice system and not a bad price either!

DBIx::Class default column value

Having read the docs for DBIx::Class::Row new() and DBIx::Class::ResultSource add_columns() it’s hard to know whether it’s better to (a) override new() for the Result module and set a default value in the method params, or (b) use default_value in the add_columns method and set the default within the database itself.

After some investigation, (b) requires fetching the row after creating it just in case there’s a default value being added by the database. Since I already had a custom new() method to validate one of the parameters anyway, I decided to go for option (a) and add a simple check to new()

$args->{ column_name } = $default unless exists $args->{ column_name };

DBD::mysql, MySQL and OS X Lion

I downloaded and installed MySQL Community Server 5.5.15 from http://dev.mysql.com/downloads/mysql/5.5.html

Using CPAN, I then tried to install DBD::mysql on a fresh install of OS X Lion and was presented with an error saying it was unable to load the library.

#     Error:  Can’t load ‘/private/var/root/.cpan/build/DBD-mysql-4.019-H1RD6j/blib/arch/auto/DBD/mysql/mysql.bundle’ for module DBD::mysql: dlopen(/private/var/root/.cpan/build/DBD-mysql-4.019-H1RD6j/blib/arch/auto/DBD/mysql/mysql.bundle, 2): Library not loaded: libmysqlclient.18.dylib
#   Referenced from: /private/var/root/.cpan/build/DBD-mysql-4.019-H1RD6j/blib/arch/auto/DBD/mysql/mysql.bundle

Continue reading DBD::mysql, MySQL and OS X Lion