What’s New in Emacs 24.4

Mickey Petersen has written up an excellent round-up of the latest features
of the Emacs 24.4 release.

Read the whole article at http://www.masteringemacs.org/articles/2013/12/29/whats-new-in-emacs-24-4/.

He had also written a similar article for Emacs 24.3 (current stable version),
which can be found here.

New Settings that I am using

I have already setup the load-prefer-newer, cycle-spacing, and ns-use-srgb-colorspace options.

The new electric-pair-mode options also look interesting, and I have set these
up (I was using skeleton-pair for the auto-pairing function till now).

The new M-s . (isearch-forward-symbol-at-point) is going to be very useful. It
essentially replicates what C-s C-w does, but having a single key-stroke is a
little easier. A closely related command M-s h . (highlight-symbol-at-point)
also looks to be useful. This is especially handy with the
hi-lock-auto-select-face option.

C-x SPC (rectangle-mark-mode) finally allows replacing CUA for me.

Eshell now has even better support for visual commands; i.e., commands
which expect a terminal. See the options eshell-visual-commands,
eshell-visual-subcommands and eshell-visual-options for details.

New Interesting Modes

The new Web Browser for Emacs (eww) is great! I have already
uninstalledw3m and using eww solely right now.

The dired-hide-details-mode is also nice. It has already replaced
dired-details for me.

superword-mode looks to be useful (especially for programming Ruby, where
underscores are usually used to separate words in an identifier).

The new file notification system would be very useful, except that it does
not (yet) work in OSX (bummer).

Other Default Settings of Interest

desktop-auto-save-timeout is going to be a life-saver. The related option for
desktop-restore-frames finally does what I want.

The new electric-pair settings electric-pair-preserve-balance,
electric-pair-delete-adjacent-pairs and
electric-pair-open-newline-between-pairs do exactly what is expected.

Ruby-mode has some interesting updates, especially for indentation.


Enabling Postfix on OSX as a Relay: Revisited

Postfix on OSX: Revisited

A few years back, I had written a post on enabling the Postfix MTA as a relay server on OSX, which was quite well received. The article was originally written for OS X Lion, though it remained valid for OSX Mountain Lion, and more recently on
OSX Mavericks as well.

However, things have changed on the OS since the last two versions, which warrants a revisit of the instructions, and a refresh of some of the steps. While the old set of instructions still work, there are changes needed that will keep the configuration future-proof.

The key changes since the article was posted (in February 2012) are:

  1. The launchd configuration has changed somewhat, and a few elements of the launchctl plist file are now deprecated. The old configuration in the org.postfix.master.plist needs to be refreshed
  2. The old configuration depended on local mail-drop in the Postfix queues for the MTA daemon to be started. While this is quite correct (this is Unix, after all), there is another common scenario where the daemon is launched when a client connects to the SMTP ports (default port 25) on the
    local machine.

The Updated launchd Configuration

We would like to retain running the daemon on a mail-drop in the queue, but also enable running the MTA when a connection is made to the SMTP port (default 25). The updated org.postfix.master.plist configuration file listed below does this for us.

In addition, the redundant and deprecated plist tags have also been removed.

You can replace the existing copy with this version as a drop-in replacement at /System/Library/LaunchDaemons/ by using the following shell commands in Terminal.app (assuming you have downloaded the new version at ~/Desktop):

$ cd /System/Library/LaunchDaemons/
$ sudo cp org.postfix.master.plist org.postfix.master.plist.old
$ sudo cp ~/Desktop/org.postfix.master.plist .

The actual configuration file:

    <?xml version="1.0" encoding="UTF-8"?>
    <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
    <plist version="1.0">
    <dict>
        <key>Label</key>
        <string>org.postfix.master</string>
        <key>ProgramArguments</key>
        <array>
            <string>/usr/libexec/postfix/master</string>
            <string>-e</string>
            <string>60</string>
        </array>
        <key>Sockets</key>
        <dict>
            <key>Listeners</key>
            <dict>
                <key>SockServiceName</key>
                <string>smtp</string>
                <key>SockType</key>
                <string>stream</string>
            </dict>
        </dict>
        <key>QueueDirectories</key>
        <array>
            <string>/var/spool/postfix/maildrop</string>
        </array>
        <key>AbandonProcessGroup</key>
        <true/>
    </dict>
    </plist>

Once the file has been copied into the correct location (perhaps by using sudo), we need to reload the configuration into launchd:

$ cd /System/Library/LaunchDaemons/
$ sudo launchctl unload org.postfix.master.plist
$ sudo launchctl load -w org.postfix.master.plist

This should be sufficient to reload the new configuration.

Testing out the configuration

We can test the changes via the shell itself by trying to connect via telnet to the smtp port and trying out a few basic SMTP commands (HELO for checking a response, and QUIT to logout of the SMTP session):

$ cd ~                                  # Lets get back to $HOME directory
$ mail &lt;your_id&gt;                        # Send yourself an email to check mail-drop
$ telnet localhost smtp                 # Now lets telnet into the server
Trying ::1...
Connected to localhost.
Escape character is '^]'.
220 localhost.local ESMTP Postfix
HELO
501 Syntax: HELO hostname
QUIT
221 2.0.0 Bye
Connection closed by foreign host.

In the sample SMTP session above, the third line (starting with “220” response) indicates that the Postfix MTA is running, and is ready for accepting mail drops via the SMTP port (as opposed to the mail-drop via the mail command).


Copying the previous line to current position in Emacs

Recently, I came across a scenario where I had to quickly copy the previous line in an Emacs buffer to the current position. The usual method for doing this has been to invoke:

C-p C-a C-k C-y RET C-y

Which basically does the following:

  1. Moves to the previous line (C-p)
  2. Moves to the beginning of the previous line (C-a)
  3. Kills the line (i.e., cuts the line with C-k)
  4. Yanks the line back (i.e., restores the original line with C-y)
  5. Creates a newline (RET), and
  6. Yanks another copy of the line with the final C-y (which is what we wanted in the first place)

While this works, it is certainly not the most optimal mechanism to perform such a simple (albeit infrequent) operation.

In searching the Net, and scanning the inbuilt functions using apropos and the excellent Icicles search, I stumbled upon the:

copy-from-above-command

function, which does exactly what is needed here. This function is available in misc.el, and needs to be enabled by requiring the file to be loaded in .emacs:

(require 'misc)

Also, the there is no default key-binding for the function, and one needs to be setup:

(global-set-key (kbd "H-y") 'copy-from-above-command)

In my case, I set up the Hyper-y key-stroke for the key, which nicely mirrors the yanking keys (C-y and M-y).

This function has another nice twist, which is that it will copy the characters from the previous line only from the current column, which allows partial lines to be copied over.


OS X Mountain Lion: Need to reinstall Xcode command line tools

So I upgraded to OS X Mountain Lion yesterday. The install was pretty smooth (though had a tough time with the redeem code for the upgrade, worked on the third code that Apple sent across).

Most of the software is working fine. However, I found that my existing Xcode installation needed an upgrade (via a Xcode install from the App Store).

The problem is that any new Xcode install seems to nuke the command line development tools (think make, autoconf, etc.). The solution is pretty simple though. Once Xcode has been installed, run Xcode and then open the preferences. There is a section on downloads, which lists installing the command line tools as an option.

downloads-2012-07-27-11-51.png

Voila! Issue resolved.


Enabling postfix for outbound relay via Gmail on OS X Lion

Update on Dec 21, 2013: I have posted an update to the launchd setup for postfix. You should still read through this post, as most of the setup remains common to both posts.

The background

Mac OSX comes with the postfix MTA, which is a fully featured SMTP server. Under normal circumstances, there is usually no need to enable or configure this software, as most email access is usually done via GUI clients such as the Mail.app – which uses the POP/IMAP and SMTP settings to connect with the email service provider.

However, there are certain circumstances in which having a local SMTP server is very useful, such as:

  1. Allowing the batch logs and output from the cron daemon or other scripts to be sent via Internet email (this is otherwise delivered locally)
  2. Testing email based code; which requires a local sendmail like SMTP server to be present

For such use cases, the postfix server is ideal, as it provides all the features needed (and much more), and is also a nice drop-in replacement for the sendmail program.

While postfix can be used as a full-fledged SMTP server that connects directly to the mail-servers on the Internet, for the use cases above, it is usually better to redirect (i.e., relay) the emails via an authenticated and known server (such as Gmail), as this helps avoid a lot of constraints around open-relays, which are mostly blocked these days to prevent email spam.

Note that configuration of postfix does require dropping down to the command-line, and fiddling with system files. While not complicated, it is definitely not for faint of the heart (though much easier than configuring sendmail).

What you need to know (pre-requisites)

Some of the basic pre-requisites are:

  1. Understanding of the shell prompt and the Terminal.app program
  2. Usage of the sudo program (all the configuration files are owned by root, and hence usage of sudo is essential)
  3. Usage of any command line editor such as vim, Emacs, nano, or any other editor of your choice, that can be invoked with super-user rights (usually via sudo)
  4. A basic understanding of the Apple launchd service manager
  5. The configuration files
  6. A Gmail email ID (actually, any SMTP server credentials will do)

While this article will go step-by-step with the configuration process, knowledge of the above will allow a deeper understanding of the “why” for the changes done.

In the steps below. the $ character before any command represents the shell prompt. Also, I will assume usage of the vim editor in the steps below.

The configuration Files

The configuration files that will be changed are:

 
Name Location Purpose
org.postfix.master.plist /System/Library/LaunchDaemons launchd Configuration for postfix
main.cf /etc/postfix The main postfix configuration
aliases /etc/postfix Local recipient aliases
generic /etc/postfix Sender aliases (for external mail)
passwd /etc/postfix/sasl Relay host authentication

Note that the “/etc/postfix/sasl” directory might not exist, in which case, we will need to create it from the shell prompt:

$ sudo mkdir /etc/postfix/sasl

Step 1: Update the launchd configuration

Update Dec 21, 2013 : While this setup still works, you might want to also see an alternate configuration of postfix’s launchd setup, which I have documented in a follow-up article. The new configuration also allows postfix to be launched when network activity happens on the local SMTP port 25.

The org.postfix.master.plist file located at /System/Library/LaunchDaemons/ is used to start or stop the postfix program on demand, as and when any email is submitted to the mail system for processing. The basic Apple setup is fine, but may need a little tweaking (in my case, the file had a couple of tags which prevented postfix from being started.)

We need to edit the file (as a super user) to match the following content:

$ sudo vim /System/Library/LaunchDaemons/org.postfix.master.plist
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple Computer//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
        <key>Label</key>
        <string>org.postfix.master</string>
        <key>Program</key>
        <string>/usr/libexec/postfix/master</string>
        <key>ProgramArguments</key>
        <array>
                <string>master</string>
                <string>-e</string>
                <string>60</string>
        </array>
        <key>QueueDirectories</key>
        <array>
                <string>/var/spool/postfix/maildrop</string>
        </array>
        <key>AbandonProcessGroup</key>
        <true/>
        <key>OnDemand</key>
        <true/>
</dict>
</plist>

Step 2: Edit the /etc/postfix/main.cf file

The next step is to edit the main configuration file for postfix. Do make a backup of the current file before editing.

$ cd /etc/postfix
$ sudo cp main.cf main.cf.orig
$ sudo vim main.cf

Note that the main.cf file is a pretty large one, and has a lot of commented out sections, which should be left as is. Please add the following lines at end of the file.

# Set the relayhost to the Gmail Server.  Replace with your SMTP server as needed
relayhost = [smtp.gmail.com]:587
# Postfix 2.2 uses the generic(5) address mapping to replace local fantasy email
# addresses by valid Internet addresses. This mapping happens ONLY when mail
# leaves the machine; not when you send mail between users on the same machine.
smtp_generic_maps = hash:/etc/postfix/generic

# These settings (along with the relayhost setting above) will make
# postfix relay all outbound non-local email via Gmail using an
# authenticated TLS/SASL session.
smtp_tls_loglevel=1
smtp_tls_security_level=encrypt
smtp_sasl_auth_enable=yes
smtp_sasl_password_maps=hash:/etc/postfix/sasl/passwd
smtp_sasl_security_options = noanonymous

Step 3: Edit the /etc/postfix/aliases file

We need to make a minor edit here, to allow mails sent to the root ID to your local user mailbox.

$ cd /etc/postfix
$ whoami                # This will provide your local user name
$ sudo cp aliases aliases.orig
$ sudo vim aliases
$ sudo newaliases

Find the line in the file which is:

\#root:              you

and replace the “you” with the username provided by the whoami command above.  Also, remove the “#” from beginning of the line.

Remember to run the newaliases command (the last command above), or else changes will not take effect!

Step 4: Edit the /etc/postfix/generic file

This file maps the local user address (usually of the form yourid@machine.local) to a valid Internet email address you would like to use when sending mails to the outside world. In our case, it would basically map your Unix user name to the Gmail ID.

$ cd /etc/postfix
$ whoami                # This will provide your local user name
$ hostname              # This will provide your machine name
$ sudo cp generic generic.orig
$ sudo vim generic
$ sudo postmap generic

In the file, add the following lines at the end of the file (replacing the <username> with the output of the whoami command, and <machinename> with output of the hostname command):

# Translate my primary email address to the Gmail address
# This is ONLY for the outbound email, and does not apply to
# local email.
<yourusername>@<machinename>  <your gmail ID, e.g. user@gmail.com>
@<machinename>                <your gmail ID, e.g. user@gmail.com>

Remember to run the last command (postmap) as otherwise the changes will not be picked up!

Step 5: Edit/Create the /etc/postfix/sasl/passwd file

In this step, we store the SMTP authentication (user ID and password) for Gmail, so that postfix can connect as any other SMTP client to Gmail via an authenticated session.

Note that the file may not exist prior to this step, in which case we will create it.

$ sudo mkdir -p /etc/postfix/sasl    # In case the directory does not exist
$ cd /etc/postfix/sasl
$ sudo vim passwd
$ sudo postmap passwd

Create the following file, replacing <gmailusername> with the ID you use for Gmail (with the “@gmail.com” added at the end), and <gmailpassword> with the password you use to login to Gmail.

[smtp.gmail.com]:587    <gmailusername>:<gmailpassword>

Note that if you use two-factor authenication with Google, then the password to use will be a new application specific password generated via Google’s account settings.

Final Step: Test the settings

We are now good to go. Lets test our settings from the terminal:

$ cd /System/Library/LaunchDaemons
$ sudo launchctl load -w org.postfix.master.plist
$ cd ~                             # Just to be safe, move to your home directory
$ mail <your_id>  # Output of the `whoami' command
# Type in a test email and hit Control-D on a new line
$ mail
# Check whether the email has arrived. Hit 'q' on the '?' prompt to quit

$ mail <your gmail ID>       # Lets now try to send an external mail.
# Type in a test email and hit Control-D on a new line

After the second step above, check your Gmail account for the test mail. If it has arrived, then we have a good configuration.

Summary

Setting up the postfix system on OSX is not particularly hard, but does require some steps. Also, this is just the basic setup to get things up and running. Postfix is an industrial strength mail server has a lot of features (and a corresponding number of configurations). Thankfully, the documentation at http://www.postfix.org/documentation.html is pretty good.

For more details on this specific setup, additional documentation is available at http://www.postfix.org/SOHOREADME.html.

[Updated on 19th Feb 2012]: Corrected a typo.  Thanks to jamrok for pointing it out.


Heretical Confessions of an Emacs Addict – Joy of the Vim Text Editor

Context

As an experiment, I have recently started using Vim as my primary text editor. While I have been an Emacs aficionado for a very, very long time – clocking in at almost 16 years – Vim is something that I have always been curious about, and tinkered with from time to time, while ending up going back to Emacs. This time however, I intend to stick around for a while, and try to get a feel of the Zen of Vim.

The Emacs and Vim Icons

The experience with Vim has been rather pleasant so far. While  configuration is definitely a must (just like Emacs), out of the box experience is not bare bones either. In fact, the default settings are rather good, and one can get a lot of mileage from the vanilla setup.

The USP of Vim is that it is first and foremost a text editor, and does not attempt to be THE kitchen sink. In other words, its core focus is to provide the best and most functions to manipulate text; it is not to provide a universal platform where one of the applications happens to be an editor.

The Editing Model

The very first experience the user gets on launching Vim is that it is a modal editor, with distinct modes for entering text, and editing it.

This is in stark contrast to most other editors, which provide the editing functions via other mechanisms, such as menus or key chords (à la Emacs). While this might seem odd at first glance, it is definitely a key component of Vim’s power, and arguably its success as a text editor.

In addition, the second key distinguishing factor of Vim is that its editing is based on a “noun-adjective-count-movement-verb” model of editing. Nearly every editing command is a sequence of one or more of these primitives, where the intent of the edit operation is usually fully mapped to the encoding offered by various combinations of these primitives.

As an example, to move 5 lines down, and then 3 words forward, to delete the next two words, the following “primitive encoding” can be used:

      5j 3w 2de 

While this is a rather trivial example, it illustrates a key principle of using Vim – the editing model is based on MOVEMENT and MANIPULATION, explicitly and succinctly made via the modal key-chords which apply the same basic foundational model at all scales.

To a large extent, this underlying pair is what the user usually is thinking about, when an editing operation is being thought of. The primitives are just the vocabulary to tell Vim to execute the intent.

The movement itself can be further separated into raw movements (i.e., lines and characters), or be semantic oriented (words, sentences, paragraphs, functions, etc.) This can sometimes be further refined using “adjectives” such as “begin”, “end”, “between”, or “surrounded”. In addition, the count represents multiples of the movement unit being specified, and allows velocity of the movement to be increased significantly.

The verb is of course where the “real action” happens. In a general sense, every verb is really a “change” verb, with specializations of “delete”, “replace”, “convert”, etc.

Conveniences

In essence, the core feature set of Vim (and certainly its spiritual ancestor – Vi) can be adequately expressed by the core model described above.

However, a basic feature does not make a competent editor. This requires other user comforts such as multi-file edits, windowing mechanisms (including split windows), language syntax, external tool integration (spell checks, compilers, shell interaction), extensive customization to fit the user’s need, an usable in-built help system, and many more such features that make regular usage comfortable and transparent. And Vim does have all of these in abundance, with parity with Emacs in most of these areas in terms of feature completeness, and power.

Granted, Vim does not yet have an inbuilt News Reader, or an email client, but the core editing functions are very much there, or easily added via plug-ins.

Conclusions

The feeling I get from using Vim is really about efficiency and focus, while Emacs seems to exude more a sense of power. Both can be frustrating at times, often from over-abundance of features than anything else, and hidden functionality that takes years to internalize before returns are gained.

Vim and Emacs are both excellent editors, and share a lot in common. The complexity, feature-richness, and the high learning curve are the ones that stand out the most. Also, the vibrant community around both editors is a shared characteristic.

Editor wars aside, both are very competent and complex pieces of software, and will serve the user well, provided an investment is made in learning the unique interfaces that both exhibit.

As for me, I do not anticipate leaving the Emacs bandwagon any time soon — especially with significant investments now in the Orgmode work-flow that I have built up over the past few years. However, I do see Vim as another pro-tool in my toolbox, which is definitely going to get a lot more love and use in the days ahead.


The Missing iSync in OS X Lion (and what to do about it)

The short answer to someone looking for a solution: just copy over iSync from your Snow Leopard installation back to the /Applications folder in Lion, and also copy the phone plugin you need back into /Library/PhonePlugins folder. Everything will be as it was. Enjoy.

The longer story requires more explanation …

The Problem

I have been looking forward to upgrading to the latest version of OSX (Lion) ever since the announcements were made in the WWDC earlier this year.

However, in the middle of the whole slew of new functions and features (“250+”), Lion is also leaving behind a few things – notably Rosetta (the PowerPC translation layer) – and the iSync and FrontRow applications.

The demise of PowerPC emulation has been long in coming, and was not really a surprise, given that it has been 6 years since Intel became the official CPU for Apple machines. The main impact really seems to be for Quicken users, which might actually be a blessing in disguise, given how pathetic Quicken really was on the Mac.
isync-2011-07-21-23-25.png
iSync has been removed presumably because it has been one of the “low usage” applications on the platform. iTunes is the center of most of the Mac based synchronization these days, along with MobileMe for the cloud side of things.

However, iSync has been a key feature of the OS for me, as I carry a non-iOS phone (a Nokia E71 actually), and being able to seamlessly synchronize my contacts and calendars with the phone using iSync made it a key part of my workflow (I even have some Automator scripts to make this easier). And iSync has been able to synchronize over Bluetooth, unlike the tethered experience with the iOS devices so far (though OTA sync is on the way with iOS 5 later this fall).

Alas, OS X Lion removes the iSync application from the hard disk once it is installed, and for a time I thought that this was end of the line for the convenience I had with the E71, and would need to go back to the darks days of manual data entry for the addresses (manual entry of calendar entries is too much of a hassle on the phone, and would need to be dropped all together).

The Solution

Before installation of Lion, I had taken a full disk image of my previous Snow Leopard installation using the excellent SuperDuper! Disk cloner.

This was more from a backup and recovery perspective, but allowed me an unexpected solution to the iSync quandary – on a whim, I attached the Snow Leopard disk to the Mac running Lion, and clicked on the iSync application. Voila! iSync works exactly as it should!

In hindsight, this is not really surprising, since iSync really has been a front end for the underlying sync services (which are still around in Lion), and for managing phone and device specific plugins.

All that was needed was to just copy over iSync back into the “/Applications” folder, and also copy the phone plugins (just one in my case) to the “/Library/PhonePlugins” folder.

phoneplugins-2011-07-21-23-25.png

In summary, while Apple in its immense wisdom took out a feature that was useful to some (albeit a minority), getting it to work again was surprisingly easy. Will need to check out the situation with FrontRow next time.

[Update on July 22, 2011]

Looks like a similar recovery process exists for FrontRow as well.  See this article at Macworld.


Follow

Get every new post delivered to your Inbox.

Join 67 other followers