Log Rotation for MySQL using logrotate

The logrotate utility is a powerful and underrated utility used to rotate logs. It is one of Red Hat’s lesser known utilities; even so, it is available for a number of platforms, including Ubuntu.

However, its set up for MySQL is missing on Red Hat and incomplete on Ubuntu.

For Ubuntu, there is no rotation of the slow query logs. To rotate these logs, just add them to the standard Ubuntu logrotate file for MySQL – that is, /etc/logrotate.d/mysql-server. Add the logs to rotate to the beginning of the file, adding to the list of files already present there.

For Red Hat, a complete MySQL log rotation file is needed as there is none at all. The MySQL logrotation script was removed as part of a security update to Fedora Core 4 back on 17 May 2006, and later removed from Red Hat Enterprise Linux 4 Update 4. The reasoning was detailed in Bug #180639 (not available?) and Bug #182025. Since then, this missing logrotate file has been the subject of several bugs (such as Bug #547007) and also of message threads like this one from Red Hat’s rhelv5-list in July of 2007.

The response to all these queries is that the MySQL logrotate script is broken and it’s up to MySQL to fix it. However, this does not seem to take into account the new FLUSH LOGS command, and admins everywhere are creating their own scripts.

Over at Question Defense, Alex has a fabulous description of the entire process – from enabling logging through implementing log rotation. However, this process uses ~/.my.cnf to automatically log in; better is to use a file like /etc/mysql/maint.cnf the way that Debian (and Ubuntu) does it. In that case, Debian creates a special user and a password to go with it, and puts these into a file /detc/mysql/debian.cnf; here is a sample debian.cnf:

# Automatically generated for Debian scripts. DO NOT TOUCH!
[client]
host     = localhost
user     = debian-sys-maint
password = i5Px6N4SZ9UhfSWa
socket   = /var/run/mysqld/mysqld.sock
[mysql_upgrade]
host     = localhost
user     = debian-sys-maint
password = i5Px6N4SZ9UhfSWa
socket   = /var/run/mysqld/mysqld.sock
basedir  = /usr

For most purposes, only the [client] section is needed, along with the first three entries (host, user, and password). You could also specify the section as [mysqladmin] instead, which would limit the username and password to being used for mysqladmin only – which is the tool used during log rotation.

The critical command is this one:

/usr/bin/mysqladmin --defaults-file=/etc/mysql/logrotate.cnf

…where logrotate.cnf contains username and password details as described above. All the rest of the logrotate file is settings and script-bulletproofing:

# Modified Ubuntu logrotate script for MySQL server
#
# Untested under Red Hat, but should work: filenames will have to be changed
/var/log/mysql.log /var/log/mysql/mysql.log /var/log/mysql/mysql-slow.log {
        daily
        rotate 7
        missingok
        create 640 mysql adm
        compress
        sharedscripts
        postrotate
                test -x /usr/bin/mysqladmin || exit 0
                MYADMIN="/usr/bin/mysqladmin --defaults-file=/etc/mysql/logrotate.cnf"
                if [ -z "`$MYADMIN ping 2>/dev/null`" ]; then
                  if killall -q -s0 -umysql mysqld; then
                    exit 1
                  fi 
                else
                  $MYADMIN flush-logs
                fi
        endscript
}

The Death of OpenSolaris Confirmed

Recently, I posted about the future of OpenSolaris and the lack of response from Oracle.

Oracle still has no official response, and has no word on where OpenSolaris is going. However, a memo to Oracle Engineering was leaked and then posted to the OpenSolaris Discussion mailing list (osol-discuss) and was later confirmed by an Oracle employee to the mailing list.

William Yang has a nice write-up on the memo and its salient points; in short:

  • Oracle will no longer let OpenSolaris track Solaris development.
  • Solaris code will stay under the CDDL license.
  • “OpenSolaris” as a distribution will no longer be released.
  • Code will only be released after Solaris is released.

Also interesting is Oracle’s reasons for closing down OpenSolaris:

  • Not enough man-power.
  • Releases Solaris technology to competitors.
  • Prevents users from using Solaris.

Oracle has never been a popular company; most Oracle DBAs in my experience have never been happy with Oracle’s support or licensing, for example. This contrasts with Sun, which has always had a positive image.

In the area of open source, Oracle has always been a champion of closed source, in contrast with Sun which had been a positive open source champion. As a result of this, we are seeing more and more open source projects by Sun either closed down or changed into closed source: consider the closing of Project Kenai (a SourceForge-like site for open source projects), the fears over the future of MySQL, and the death of OpenSolaris.

The OpenSolaris experience under Oracle has echos in MySQL: Monty Widenius, the founder of MySQL, was quite vocal in his opposition to the Oracle purchase of Sun, and expressed his fear that MySQL would become closed source. Perhaps his experience with SAP and MaxDB had something to do with that – MaxDB had been released under the GPL through 7.6, when it was returned back into SAP and became closed source once again.

About the time that Oracle announced its purchase of Sun, Monty began the GPL-licensed version of MySQL, MariaDB which has taken hold, and the European Union mandated that MySQL shall remain dual-licensed. I wonder if MySQL’s fate would have been similar to OpenSolaris if it had not been for Monty.

It would be interesting to track the other open source projects now under Oracle’s umbrella:

  • Java (and OpenJDK), and its add-ons
  • Glassfish (J2EE)
  • MySQL
  • NetBeans
  • Lustre file system

Whither Sun Microsystems?

The recent fourth quarter reports from server manufacturers was dim, and Sun Microsystems was by far the worst (with a 35% loss compared to the same period last year). On top of this, Sun just announced in October (within their 8K filing for the SEC) intentions to lay off 3000 employees in the next 12 months. Infoworld also had a nice piece on this; according to Larry Ellison, CEO of Oracle, the company is losing $100 million each month the European Union regulators put off accepting the merger.

With the Oracle acquisition in progress, there are a lot of questions about the future viability of Sun Microsystems, and of some of its products.

I don’t think people realize just how important the Sun group of products are, and what an impact it would have if most – or even some – of the products were cancelled. Consider this list of Sun products:

Most of the most popular products were mentioned by Oracle in their Sun Acquisition FAQ (PDF), stating that they will increase money spent on each over what Sun spent. These products include: Java, Solaris, SPARC, StarOffice, NetBeans, virtualization products, Glassfish, and MySQL. Other products were not mentioned – such as Lustre, the Modular Data Center, and others.

The list above also does not list the technologies that were spearheaded by Sun – and some still are: ZFS, NFS, NIS (and NIS+), dtrace, containers, and smc.

It would be unfortunate – and materially significant – if Sun were to go under or if any of the majority of their products were to be cancelled. One can only hope this does not happen…