Mavericks and Mail, update

I said in my previous post two weeks ago that I would wait installing Mavericks on my main Macs until an update for the Mail problems arrived. Well, it came out last week (it even installed itself), and seems to have solved the smart mailboxes issues - at least the Unread count was correct on my MacBook Air, where I had Mavericks, and all mails there were visible and accounted for. So, I went ahead yesterday and installed Mavericks on my main office computer (an iMac, large screen, fast Fusion drive, excellent).

Alas, I had not tested it enough on the Air and perhaps not searched enough on the Internet. It was to be expected that Mail spent more time getting up to speed on my main machine, with 200.000 emails archived, it seems to be working still a day later. In at least one set of mailboxes, all mails appear to be empty: itemized and dated, but with no content. However, it is there (visible in TimeMachine), so the program is perhaps still working its way through, if not I can replace them from TimeMachine. Similiarly, the "count badge" for unread mails in a mail box also needs time to update smartly, but in a day or two it will, hopefully. Again, my advice from last time, to recreate the smart box definitions probably holds.

So, patience will prevail. But there are other issues that actually have been widely reported on the web and are clearly bugs and design flaws. There are temporary ways around them, however.

Sending group mail
The main error concerns group mail, that is sending mail to many people at the same time. This is the way it used to be done: In the Contacts (address book) app, you create either a "group" (a fixed group, by dragging cards individually into the group) or a "smart group" (dynamic, defined e.g. as "address cards that contain the word "Class2012""). Either way, you can email collectively to the group by beginning to type the first few letters of the group name in the To: (or, preferably, Blind Copy:) field in Mail. Then all items in Contacts starting with those letters show up in a drop-down menu, you click on the one you want, and it turns into the blue "cartouch" signet of a addressee, with a drop-down menu like "Expand Group" etc. Send, the mail goes to every member of the group.

In Mavericks Mail, you can do this for individuals, but there is also a new method, familiar to iPhone users: To the right of the To: field, there is a blue + icon, which gives access to the Contacts book and its groups. Click on a name, and it is inserted into To:. But that only works for individuals, not groups of any kind.

Now, what happens if you use the old method of typing a few letters? It varies. For fixed groups, it seems to work as it should: You type, select the correct item from the alternatives, and get the regular cartouch, both in the To: and Blind Copy: fields. But smart groups do not work like this. Sometimes you do get the list of choices, but no cartouch appears, nor does it expand into individual members when you ask for this (in Mail's settings). At other times you do not even get the list, and nothing happens when you type out the name in full. Apparently addressing smart groups do not work, which it clearly is supposed to.

But, in fact it appears it does. The feedback you get varies. Sometimes, you do get the list of choices, and when you select it, you will briefly see the name in angle brackets afterwards: "Class2012 <Class2012>" - although the name in brackets disappears once you Ok it. At other times it does not show you anything. But it appears in fact that the smart group function actually does work in both cases: Select the name if given a choice, or type it out if not (making sure you type the name precisely), press Send, and the mail will really go off to all members. At first I thought it only went in the first case (I wrote that ten minutes ago), but when I tried it out also just typing it out, with no indication from Mail it understood it as a group name, I heard back that the mail did go through.
     Why some smart groups give you the minimal feedback, and others not, I do not know. And I have not tested all my smart groups to see if they all actually work - it certainly is an error on Apple's part that they give no feedback, and of course that you have to rely on (precise) typing of a smart group name to select it. I would certainly suggest adding yourself as a recipient to groups you use, to confirm that the messages actually go out to the list
       A workaround can be used to give you peace of mind that it was sent as usual: Go to the menu "Window: Address Panel", find the smart group in question, click inside the list, then select all (Command-A) to select all names, and click on the To: (Copy:, Blind Copy:) buttons to address them individually. Not elegant - not if there are a lot of names - but will work for the moment.

Clearly, this is a bug, but it seems to me also a design error, in that the apparently preferred addressing method, the blue + icon, does not seem to have considered how to include addressing group mails.

There are some other minor bugs: While you can choose Bold or Italics for text you edit, it does not hold: underline works, size change and colour works, but bold and italics are forgotten.

Still hold back?
Now, many people do not use either smart mailboxes or smart groups, so they will not be affected by these problems. But they are not really esoteric, but fairly important and mainstream functions. Every new OS has bugs, that is to be expected, but it is peculiar that Apple released a version of the important Mail program, one that everyone uses, without checking these standard functions better. Personally, I think I will still hold back a few days before I upgrade my last Mac, just in case I find more problems, but these issues can be worked around, as I mentioned, for the moment. On the plus side, as these bugs affect fairly many users and is kind of another strike against Apple, it is to be hoped that they are aware of them and put some pride into quickly solving them. There is indeed reported to be a 10.9.1 on the way, but Apple released the first Mail fix in advance of that (untypically, they normally roll Mail app fixes into general upgrades), so we must hope these other issues will be quickly solved as well.

Mavericks and Mail

New term, new operating system, new problems in the Mac Manor.

Last week, I and I guess most Mac users who had turned on Software update warnings, were invited in large friendly letters to upgrade to the new system, MacOS 10.9 or Mavericks, for free! A novelty, as we remember, it was from System 7.1 in 1992 that Apple started to charge for system upgrades, so after twenty years a more friendly approach is back.

However, before I got round to accept the offer (you must set aside download time and re-organizing time etc. for a couple of hours), I received a warning from TidBits that it might be adviseable to hold off - there were significant problems with in particular how Mail works in Mavericks one should be aware of, before taking the jump.
<http://tidbits.com/article/14219>

It seemed however to affect particular cases, especially those who uses Mail mainly for gmail (Google Mail) and had followed Tidbits' Joe Kissel's earlier suggestions for gmail, which now are broken. I never did that, and gmail is only a secondary address for me, so I thought I should on the one hand be careful, on the other test things out to see how it worked. As it happens, I have a travel Mac, a MacBook Air (on which I write this), with a full Mail setup, but which I do not use daily. I decided therefore to install Mavericks there first, to see what happend.

The install itself is smooth, as you could expect - click, and go away and read a book or magazine for the hour or so the process takes (check in at beginning and end for giving various permission, but in the middle just let it work). I forgot the very useful advice to first download the installer, then quit and copy that installer app on a USB, before installing, that would have saved me the time to download it again for when my other Macs are to be upgraded (the installer deletes itself when Mavericks has been installed). Would have saved me 15 minutes or so, but never mind.

On restart, you have to give various passwords, AppleID and the Mac's admin ID and so on, as usual. I think I made a mistake, because I use different IDs for iCloud and iTunes, and probably gave the wrong one at one point, because I got a blank square covering some window and which I could not click away. But once I saw what was below (System Preferences), I could quit that and reopen, and then (in the menu) of the new iBook app log out of my iCloud account and into the iTunes one, where I have my books. Worked.

Mail issues
However, the main quest was for Mail problems. And yes, there were problems, and yes, they are sufficiently important that I am holding off a bit to see if Apple will fix them with a "10.9.1" bug fix before I install Mavericks on my main machines. If Apple does not, however, it may be possible to work around the issues.

First, a simple one. Each time I openend Mail (which was a lot, in the testing), it kept bugging me for my iCloud password for getting @me.com mail (like everyone, I have such a passive account). It would not remember the password I put into Mail's Accounts pane. The answer was quick, and cited in the TidBits article above: Open System Preferences, the iCloud pane, and then first uncheck the Mail item, and then immediately check it again - that forces Mail to re-configure itself, and you will see that the password item has disappeared from Mail's own iCloud account setting - it now takes it from the general settings, and remembers it.

But then the major one. Joe Kissel's problem was, as mentioned, related to gmail, and had something to do with gmail's special "All Mail" mailbox. I had never messed with that, and could see no problems of the type he mentioned. Probably this, the major point of his article, only or mostly concern those who use Mail mainly for gmail.

However, the second point he briefly mentions, and which has been widely reported on the web, was a major stumbling block for me: Smart mailboxes, which were and still are mainly broken in Mavericks Mail. Smart mailboxes have got several new features, and are possibly built up differently from before, I don't know. However, to explain, this is how I read mail, and why smart boxes are important for my workflow:

I use Mail as a main mail archive, set up with dozens and dozens of mailboxes: for persons, for topics, for mailing lists, for countries or whatever. Anything that I can identity (mainly from regular senders) gets automatically filtered to its mailbox immediately on arrival, whether it is my students, from the department, or newsletters or mailing lists - everything disappears from the In box before I see it, but is of course still listed as "unread" until I read it. The In box only includes mail from irregular correspondents, or those colleagues I want to read first - I always first read through the Inbox, dump spam and answer requests. Then, I have set up a smart mailbox that collects all unread mail that is (a) not in the Inbox (b) not in the high-volume mailing lists, which I read at the end when I have time. But this "Unread" smart mailbox is thus essential, it includes lots of important mail from regular senders.

What happened in Mavericks, is that the Unread list gave a surprisingly high number of items on its badge: about 200, while it should actually be 10-15 since my last mail check. But when I opened the box, it was empty. The menu line said: "0 messages. 196 unread". Now, I could see down in the list of "real" mailboxes that there were a number of unreads down there, but the smart box did not list them. I restarted a few times and tried a few others things, like creating new smart mailboxes afresh, but with basically the same result, all apparently empty. I saw on the net many people who had tried various strange suggestions to no avail, so it was and is a real bug.

Finally, I found a solution that was supposed to work (in the same TidBits article; always check in Tidbits): "Kirk McElhearn found all his smart mailboxes empty, and was able to correct the problem by quitting Mail, deleting the three files beginning with “Envelope Index” in ~/Library/Mail/V2/MailData, reopening Mail, and letting it reindex his messages." I did this,(*) and lo and behold, something happened: Mail put up "Welcome to Mail! For this new version, we need to process your mail to the present version of Mail" [or words to that effect, from my memory]. This is well and good, it is supposed to do this, but it never did when I first started Mail in Mavericks. So here was a possible bug, Mail had skipped its own upgrade process. Anyway, it went to work on my 100.000+ messages, and spent the next fifteen minutes on that. And did it work?

Yes! Or at first appearances anyway. The message count in Unread was down to about 25-30, a bit too many (it included a number of old messages from gmail's Unwanted mailbox), but they were all present and viewable in the mailbox. So, I could just walk through them as ususal and see the badge count drop by one for each. Down to the last. But that one did not go away. I dumped it manually, and it did go away from the mailbox, but the counter remained at 1. The same for the new boxes I had created just before, the all were stuck with one, invisible, unread mail. Now, I found a way to get rid of that minor issue: I went to Edit smart mailbox, took away the rule "Message Is Unread", and then added it again, and then the invisible 1 disappeared. Everything OK. Until the next actually new unread message came in - but not in the Unread box. This time, neither badge nor content shifted, the smart box had simply stopped working. Only the second (old trial) box, where I had not edited the rules, worked (with the invisible 1 in place), all others were broken again. But that one worked - or did it? It included several unread messages that were in the Inbox - which should be excluded. So, it seems to have found all unread messages, i.e. just the first line on the smart box definition, not the ones following.

Last ditch effort: To recreate a smart box afresh, now completely under a properly updated Mail. That kind of works: The invisible 1 was back, but otherwise it seemed to pick up actually unread messages in the mailboxes it should (although some have got strange names, like "Archive" and "Important", apparently from gmail which Mail formerly ignored) and only those. This morning, after one night, the invisible 1 even went away again, but now in the afternoon, it lists 3 after I have read everything. So the badge count is off, apparently haphazardly so, but I can live with that, I guess. I will keep watching for a while, and see what happens.

But the moral appears to be: Make sure Mail actually shows a Welcome to you message and spends time updating your mailboxes (you will notice, a lengthy process), and then re-create any smart mailbox you have had in the old version. For the monent, this seems to work, but there may be other bugs that could raise their heads (but I am very happy that Cmd-Option-T for "Move to [last used Mailbox] again" once again works, saves me a lot of menu trips).

(*) How to open your Library folder? Improvement: It is still hidden by default, but now you can set this to display in an option in Finder's: View: Show View Options menu.