Over the past week or two, I’ve been working on a number of side projects that I’ve started to realize as a new and interesting project in and of themselves.

The first project is that I’ve been working on (re)designing a website with/for my friend Scott, who is a composer/voice actor/podcaster type. While he has a lot of HTML skills he’s never done any of the–even very elementary–web-programging (a la the PHP code/templates that wordpress uses) that one needs to do to have nifty dynamic websites.

So I’ve been working with him to use wordpress to build a basic portfolio site and blog. It’s a pretty straightforward kind of a site, but you have to know wordpress pretty well to know how to design a theme so that all of the right content ends up in the right place. I’ve been using wordpress, more or less since the beginning so it’s the kind of thing that I’m pretty good at.

The second project has been to design and implement a version of the tychoish.com theme for a new wordpress installation that I can deploy for various members of my family who want to use wordpress as a note taking platform. While I use a wiki (with a blogging plugin) for this task personally, the editing interface for wordpress along with its metadata system (tagging and categories) makes for a really ideal notebook-solution. There are other features like comments for annotations and revision tracking for storing document history that have been really helpful, and make this a really good solution.

And of course the fact that I’m building these sites around wordpress makes a couple of things possible. First, it’s open source so we can run these notebooks and portfolio sites on our own (or rented) hardware, and the data is easily exportable into a lot of very useful formats if we ever want to change. Second, the software isn’t very resource intensive to run, particularly for lower volume sites like these, so it can run almost anywhere.

The truth is, that the biggest part of both of these projects was talking to the people who would be using the software to figure out what they already did with their existing websites/notebooks, and then figuring out how to do the same thing with the new solution, and finally also learning enough about what they wanted to do to be able to figure out with they might do that they couldn’t do before.

I guess that makes me something of a consultant.

Weird.

Even more wierd is the feeling that I actually like doing this. And I find it ties together a lot of things that I do. My “day-job,” does similar sorts of things (for different kinds of “clients” with different sorts of problems), and what I’ve done thusfar is very much inline with my musings here about open source and productivity, which is kinda cool.

In a weird way, this is very much inline with my initial forays into the web-world which seemed to center around organizing creative types (in the largest sense) on the internet. This was nearly 10 years ago, but the threads are there. That’s something that I’ve realized as I’ve started new projects and new directions recently: even when I think they’re new and novel, I realize that they resonate with things I was working on and thinking about during high school.

Go figure.

So in the past couple of weeks we’ve seen the proliferation of a couple new “real time services,” for various kinds of data. Enjit brings real time data from friendfeed (which itself aggregates a lot of data pretty close to real time), and then there’s tweet.im which finally brings something approaching real time twitter interaction back to those of us who have been begging for a real time/xmpp twitter interface for a while.

Though to be honest, I think that the lag is a bit more than 30 seconds, but I’m not sure and I’m not going to quibble for now. Actually I’m not convinced that this redeems twitter, given the number of other features that they’ve turned off (can’t delete posts anymore, can’t elect to not receive updates from people you follow, not to mention track) but it’s a start. When they get Oauth and Open Micro Blogging implemented,¹ I won’t worry. But in the mean time, there are people on twitter that I want to be able to talk to, and this is a much appreciated move.

In any case, what this week has taught us is that real time services are here, and that companies and developers are beginning to realize this and provide services based on that. The man said “you don’t need a weatherman to know which way the wind is blowing,” and I don’t think you need an ubergeek to know that realtime is on the way.

Which means, its my turn–as a resident uberworkflow/user interest geek–to parse out what this means. You might think that this means that there are geeks who are wanting as much data as possible as quickly as possible. But I don’t think that’s the case. Really i think it’s about, having as much control over that data as possible.

Ken Sheppardson, one of the folks behind enjit, talked about wanting to have was all about consuming as much data as he could. said ” I only want a notice every hour or so when somebody’s talking about something I care about, but I want it in time to participate.” (Edit Note: I totally flubbed up the reference and introduction to this section and have edited to make me seem like less of a dip. Apologies.)

The secret is that real time means push, and the truth is I think that I read less content and spend less time reading content that comes at me real time, than I do reading the same content that I have to check on in a special client or on a web page. Why?

Because the time/energy spent on checking disappears. So if twitter is coming at me in an IM, I can trust that there’s no reason to visit twitter.com, unless it’s to look at someone new to follow. And it’s easy to tell if I’ve seen something before, and avoid reading the same content that people blast all over the internet again and again. (Ping.fm, how I hate you). And when you get your data real-time, it’s easier to make filtering decisions, which is a good thing.

Converging these data streams in real time/xmpp (ff, twitter, laconica, etc.) means that your data comes to you, not that you get more of it. So from a usage/workflow perspective, I think this is wonderful.

OS X has been a really innovative force in personal computing. It’s highly usable, lots of different kinds of users are able to work with it. It’s compatible with lots of different standards, and it provides a lot of tools to developers that make even the sucky third party software pretty nice. I think if you look at Windows Vista, and the latest versions of KDE and GNOME and some of the other open source user interfaces, it’s pretty easy to see some resonances of OS X.

More importantly, probably, it proved that Unix and Unix-like operating systems were viable and usable for desktop use cases. While we’ve been able to run BSD and Linux on home computers for years, I don’t think we’ve thought of Linux as being something that anyone could run without needing a lot of technical background.

Ubuntu Linux followed this trend, pretty persuasively. Ubuntu makes desktop unix-like experience possible. Which is a really big thing. Chris and I are both using Ubuntu these days for our primary desktop computers, and it’s been really interesting to compare notes. One thing that we keep coming back to, is that despite the fact that the core of the OS is great, the user interface (UI) is tragic. OS X proves that it’s not only theoretically possible to have a nice UI, but it’s possible to do that on a unix-like system.

As an aside, I’d bet good money that Apple has an in house version of Aqua/Cocoa/Carbon/CoreServices (all the UI and application frameworks that make OS X, OS X) running on the Linux Kernel. Betcha.

And by tragic, I don’t mean that GNOME and KDE are unusable, but they’re flawed. GNOME doesn’t use space efficiently, it’s applications are functional but not exceptional (and because of the way the GNOME project is there aren’t many ‘third party’ alternatives), and it feels sort of behind the curve. It works, and it does everything that you might want in a graphical user interface (GUI) but it’s not exceptional.

Thankfully KDE fails for completely different reasons. It’s attractive and usable where GNOME isn’t, and the interface is unique and exceptional where GNOME feels stale and aged. But the applications aren’t nearly as compelling, and it suffers from having an interface/look that’s too flexible, such that it’s pretty easy to get a setup that looks like crap. Not to mention the fact that the kind of rich GUIs that KDE emblimizes don’t mesh particularly well with the mostly hacker audience that Linux (and it) attracts. But that’s a larger critique of the GUI paradigm, which isn’t quite on topic.

So where does this leave us?

I’d say the biggest shortcoming of linux systems is the window management options. I like Awesome and I think there are a bunch of people who might really like it–but it’s not for everyone, and I’m admittedly not up to date with enough of the other options to provide a really clear analysis, but I know that this is the the next big issue for open source operating systems.

I’m not sure that I know enough

When going to graduate school became “someday later” rather than “someday soon,” one of the things I promised myself that I’d do is dedicate more of my time to fiction writing, which I always seemed to cast aside in favor of more seemingly important academic goals. Well not any more. So I have this fiction blog, which I talk about (not nearly?) enough called Critical Futures which is part of my ongoing struggle/effort to write and edit my fiction. And I’m writing another novel-ish project that will probably–but no promises–end up on CF in several months.

When people ask me “what do you do,” I say “I’m a writer,” which is true, not just because I’ve got a novel going on in my head (and who doesn’t?) but because my day-job is also as a writer, so it doesn’t feel so fake, but when I say “a writer,” I almost always think of space ships and distant outposts of human civilization, and not web technology. Not that I dislike the internet or the software that makes it go, but because the space ships are probably the most important/meaningful (to me) things things that I write.

In November, answering “What do you do?” (write.) “What are you writing?” (a novel) seems much less remarkable because of NaNoWriMo.

In the past I’ve been bitter about NaNo, sometimes because I’ve not been in a place where I can get started, or because I feel like it’s teaches the wrong lesson about novel-length writing, or whatever. This year, I’m bitter. To each their own, I say. I’m not participating because I have a project at a crucial stage, I feel like the biggest objective of NaNoWriMo isn’t “get a lot of writing done,” but rather “prove that almost everyone is capable of sitting down and writing a book length thing, if they can get through the crap.” And I know this. I also know that I can write at NaNo-like speeds should I need to.

But.

I am going to, for the month of November, take a brief reprieve from my “don’t write about your writing on your blog” rule, and for the benefit of NaNoWriMo spirit write a little bit about a few writing related topics. I’ll be writing other things as well like linux and open source and knitting, but I think it’ll be cool to reflect a bit on Critical Futures and my current project, and some stuff about my writing process. Because that’s what I have.

How many of you out there are doing NaNo? For the first time? If you write novels in some other way?

I remember once in high school (lets call this 2003/04) I bought an 8.5×11 paper calendar. This was after the age of PDAs (indeed I think I had a PDA at that point), and damned if it wasn’t the best calendaring experience I’d ever had. I kept track of things, I knew what was going on beyond the obvious, it was pretty good.

Other than that, I’ve not really (until now) used a calendar in my day to day work. I have a good memory, and pretty simple commitments, so it’s never been a big deal, and most calendaring tools kind of suck. Actually, really suck.

Now mostly, I could care less, except I have this job now, and the scheduling is really complex. I have meetings with clients, I have intra-office meetings, there are things I have to schedule with other people. It’s intense. So I’ve been trying to figure out what to use, how to connect with other calendars, and generally how to make it all work. I’ll tell you what my uneasy solution has been, but I’d love to hear what you use.

Chandler

The Chandler Project is a really pretty darn interesting open source program. It’s written all in python, it’s designed for collaboration and uses all the open standards. It has a really interesting and useful tagging system. It has a feature-loaded (and not annoying!) web interface that compliments the desktop experience.

And despite this it’s kludgey. It’s not peppy, it uses it’s own paradigm, and like all multi-platform software, it ends up looking just a bit foreign on all OSes (though I’ve not worked very much with it on linux, so to be fair, it might seem less out of place.)

Still, Chandler has become my “home” Calendar Application. There’s something nice about the way that it syncs to Chandler Hub, and all the cool things I mentioned above. And I like it’s dashboard a lot. When I started doing this job, I went from being able to handle a month’s appointments on an index card (or equivalent) to having to having to keep track of tasks and meetings in at least 3 time zones, and Chandler gives me enough flexibility and power to be able to do this without getting in the way.

ICS format

I think the last time I tried to use calendars on the computer, it was before people were seriously using a standard format. So you’d spend a lot of time putting your data into one program, only to have it die a horrible death, or discover something even better and then be stuck in your old program. Not so any more the .ics format is used by (nearly) all programs and by google calender so you can subscribe and sync calendars to your hearts content and it will all work. In all of your programs. Which means, you don’t need to chose the ideal calendar program, as long as it supports .ics files.

iCal

I’ve had a pretty tortured relationship with iCal, Apple’s stock Calendar program. In previous versions of the OS it used to be really crappy. And while it’s much better now, there’s nothing special about it, and it’s just a bit too hard to use it to enter events, and it’s task management is farcical.

Having said that it looks really good. Before you laugh at me and call me an apple fanboy allow me to be clear: data visualization is 60%-75% of a good calendar program (with the rest being performance and data entry/manipulation). The whole point is to be able to see what’s going on throughout your day/week at a glance. Good visuals are key.

The great thing, is that with Chandler Hub / Google Calendars and the .ics format, you can use both programs and still keep all your data together pretty seamlessly

Ideals

Like I mentioned earlier, I’m open to hear what other people use. At least for the moment I’m reasonably happy with what I have. It might be nice to have better command line access to be able to quickly review upcoming appointments and enter new tasks, but that’s mostly a pipe dream. Is there something I should be looking for that I don’t know? Other cool OSS projects that I’ve missed?

Onward and Upward!

I posted a while ago about redoing my assbackards email downloading system. I finally did it. And it works great (and runs a bunch faster). The main sacrifice is that I now have to store two copies of my email directory on my server, but disk space is cheap and git is really effective at saving disk space/transfer speed, so it’s not a really big deal.

It’s also much more straightforward, and I’m going to post the two (2!) simple shell scripts that I’m using along with a couple of recommendations/hints if you want to try this at home. I do recommend that anyone who does this should be familiar with git before attempting this. Sound good?

Great! Here goes.

You need:

• A repository on your server where your email will “land.” This should be a git repository that’s origin is:
• A bare repository that is also on your server. These directories should not be web accessible, and indeed, technically need not be on the same server even, though they need to be always-on servers at static locations, so that you can clone repositories on:
• A repository on your personal computer(s).

Inside of the repository is a great place to keep your scripts (and indeed all settings for mail-related programs.) The two scripts you need are:

syncserver:

#!/usr/bin/env bash

# Get situated 
cd /path/to/mail/

# Pull Changes from the centralized (canonical) repository
/path/togit pull origin >/dev/null 2>&1

# Add in the new mail and push to the central repository
/path/to/git add .
/path/to/git commit -a -q -m \
    "server: mail drop commit, pre-push to central"
/path/to/git push origin >/dev/null 2>&1

syncmail:

#!/usr/bin/env bash

# Do things on the server  synced
ssh USERNAME@DOMAIN.TLD sh /path/to/mail/syncserver \
    >/dev/null 2>&1

# Do things locally to get changes upstreem to central
## Get Situated:
cd /path/to/mail/ 
## Pull new mail down, 
## everyting after the | is to produce a mail notifciation  
##  using growl on OS X

/path/to/git pull origin | \
/usr/local/bin/growlnotify -n mutt -a Mail.app -t \
    "tycho garen (or maybe sam)'s mail status:" \
    >/dev/null 2>&1

## Getting setup and then pushing any changes upstream 
/path/to/git add .
for i in `git ls-files -d`; 
    do /usr/local/git/bin/git rm --quiet $i; 
done 
/path/to/git commit -a -q -m \ 
    "mail changes from NAME_OF_COMPUTER"
/path/to/git push >/dev/null 2>&1

Feel free, of course to change the commit messages and take out the comments. Other thoughts on usage:

1. Have public key sign-ins enabled on any machine that you use regularly. It’s more secure, faster.
2. By all means, run “syncmail” in a launchd daemon/crontab.
3. I’ve used absolute paths, starting with a “/” so that the shell doesn’t have to look for relative paths in order to run, which probably only gives me a marginal speed advantage, but you’ll want to figure out where your executables are and then put the right locations in. It’s a short script.
4. While eventually, you’ll be able to tweak the syncserver file and after two runs, the changes will propagate, you have to put the file there manually (or run the pull yourself.)
5. If you need to download mail from more than one source, you can run fetchmail at the beginning of syncmail

This almost makes sense, finally. The only thing that I’m still trying to figure out is if I were to get, say an iPhone or a BlackBerry into this system. IMAP into my existing gmail account would be ok but not desirable for a number of reasons. I suppose I could forward email from my server/computer to a “mobile” account, but that would be like having an email cadet branch and there wouldn’t be syncing, which seems bad. On the other hand, it means I could pull out only the stuff I really need. It’s a ways off, but it’s pretty complicated.

Onward and Upward!

Aside from doing semi-perverse things with my email retrieval system, one of my most recent technical/digital obsessions has been with a X11-based window manager called awesome. It’s a tiling window manager, and it’s designed to decrease reliance on the mouse for most computer interaction/system navigation purposes.

Unless you’re in the choir, your first question is probably “What’s a tiling window manager?” Basically the idea is that awesome takes your entire screen and divides all of it into windows that are a lot like the windows that OS X, Windows, GNOME, and KDE users are the same. Awesome also has the possibility for what it calls “tags,” but which are akin to virtual desktops (and I think of as slates) which make it possible to have a great number of windows open and accessible which maximizes screen efficiency and multi-tasking while minimizing distractions and squinting.

The second question you might have, given the prevalence of the mouse-pointer paradigm in computing lo these 30 years, why would you want a system that’s not dependent on the mouse? Long time readers of the ‘blag might remember some blogging I did earlier this year about the second coming of the command line interface. The basic idea is that the more you can avoid switching between the mouse and the keyboard, the more efficient you can be. Keystrokes take fractions of seconds, mouse clicks take many seconds, and this adds up. The more complex idea is that text-based environments tend to be more scriptable than GUIs and coded more efficiently with less mess in between you and your data/task. After all, coding visual effects into your text/word processing application is probably a sign that [someone is doing their job horribly wrong][xkcd job horribly wrong].

One of my largest complaints about using GNOME is that it’s terribly inefficient with regards to how it uses screen space. Maybe this is the symptom of using a laptop and not having a lot of space to go around, but most applications don’t need a menu bar at they top of every window, and a status bar at the bottom of every window, and a nice 5 pixel border. I want to use my computer to read and write words, not look at window padding (I suppose I should gripe about GNOME at some point, this is an entry onto itself.) Awesome fixes this problem.

I’m not jumping in to Awesome full time, but I am starting to use it more and learn about it’s subtleties, and hopefully I’ll be able to contribute to the documentation of the project (it needs something at any rate). For a long time I’ve flirted with Linux, but haven’t ever really felt that it offered something that I couldn’t get with OS X, and this changes that pretty significantly.

One of the things that I need to do first is explore Linux equivelents to my remaining OS X-only apps. The most crucial is the news reader, I’m a big fan of NetNewsWire, and I’ve never used a news reader which can top it. As it turns out, between vim and Cream, I’m pretty set in the text editor department (though I need to port over the most important of my scripts and snippets to vim), and although Adium is a port of Pidgin, using Pidgin is painful by comparison, particularly in awesome.

But I have time. I’m doing this becasue it’s interesting interested and weirdly enough, it’s kind of fun.

That’s my story and I’m sticking to it, I’ll be posting more on the subject as I learn more.

[xkcd job horribly wrong]:

Ed: I so I totally meant to post this on Monday instead of this post, but the wires got crossed, and Monday’s post got out early. Sorry. –ty

The one thing that really undermines my argument against preemptive rewriting is the fact that I’m a compulsive outliner, at least for some things.

Since I’m currently in the “planning a new story/project” phase¹ I’ve been thinking about the outlining process a bit.

The difference between blogging and writing is that in a blog post I maybe scribble something on a paper ahead of time, but the rhetorical thread comes out of my fingers and on to your screens pretty much straight through.² Blogging is very much like the research process of fiction writing, in that it’s a process of taking an idea and figuring out what it means, and a lot of the possibilities in whatever thing or situation (blog and fiction respectively). In blogging, you pray someone’s interested or you’ve written something that gets folks irked enough to respond, in fiction you take all this thought work and then write about only the things that really need writing about.

And then I realized the other day, that I write the outlines of my stories–indeed my entire personal wiki notepad–to you, dear blog readers. At some point years ago I got a pretty good idea of who “you” are, and as a result generally my blog posts are all addressed to the same positional reader (even though that reader/group has changed a bit, and my conception of “the reader” has always been pure fantasy on my part.) Even though I’m the only one that reads this document, all the parts are written to the same person. It’s all very strange.

But I digress. Research.

I think the sort of stock advice to the new writer doing research is “don’t get too caught up in planning, writing is about writing, not planning to write,” and while I do think that the reminder to always keep the writing in mind, at the same time, there’s something really important and intense about the planning process that shouldn’t be ignored. It’s a lot of work, there are a lot of words that go into getting a story to the point where you can even think about starting to write in ernest. In the end I think it’s not so much about what you outline or what you read up on, as the act of doing it. The time spent in your head thinking about this world, and this idea.

Speaking of thought work, I have some to do.

Onward and Upward!

In debates between writers who outline voraciously and writers who “just write” and think that outlining kills the energy of a project, I usually come down on the side of the outliners. If I make an outline, even if I don’t hold myself to it closely, I tend to be more focused and run into fewer snags than if I fly by the seat of my pants. Except…

Except that I’m horrible about outlining characters and to a lesser extent settings. I swear if any of my stories were ever performed on stage, it’d be a clear stage except for a background mural vaguely reminiscent of space. And while I tend to have a pretty clear idea of what’s going on in terms of “what happens next” I fly by the seat of my pants every inch of the way in terms of characters.

The good news is that I’m pretty good with character representation and dialogue (that psychology major paying off after all?), so I can fake it well. The bad news is that I don’t often don’t realize that I don’t have a clue what I’m doing until embarrassingly late in the game.

As I was thinking about this last weekend (and forcing myself to write character sketches,) I realized a few things:

1. I suspect that most people who “fly by the seat of their pants” probably do some character development, and that just as I have given myself personal dispensation for not making notes about characters, I think that they probably do the same for making these notes. It’s interesting at any rate.
2. It’s really helpful to make notes about characters to help keep them consistent over the course of a writing project. Duh.
3. My tendency is to make characters too realistic. Characters in stories need to be in great danger, they need to have problems that they can’t think their way out of, and they need to be faced with situations that might radically change the face their world, if not the world itself. The truth is that most of our problems are things we can think our way out of (that would be the downside of the psychology major) and we pass through our lives with only routine levels of danger, and almost all of the situations we experience are alone unlikely to change the world.

Anyway I have some characters to go sketch out…

Though I’ve never asked directly about this, I have sort of hinted around the edges and thought about it a bit, so I figure, it’s worth bringing up. Here’s the question:

You install a new wiki somewhere, and you have a blank “welcome to your wiki page.” How do you go about editing these pages and developing a wiki from there?

The idea behind a wiki is that it provides a quick way to edit and create web pages, without needing to know very much about HTML and associated technologies. Right?

In addition to radically de-centering authorship, wikis also have profound effects on the structure and organization of websites/documents. While much attention has been paid to the former on a social and software level,¹ I think the latter has been mostly ignored.

I think the tendency is to think of the organization of data within a wiki as being largely emergent phenomena. I think this is more true when there are a lot of people starting a wiki and when the size/growth of the wiki is appropriate for the community size.

The truth is that I think this kind of approach ignores this huge data management problem because it’s not an issue for 10% of wikis/documents/collections. I mean, if you throw up a wiki you can’t just let it go the way that Ward Cunningham did for the first wiki (because the people who would contribute and the internet would come to the wiki from a much different place), and sites like wikipedia have a lot of very intentional structure. And while I like the idea of emergent structure, I’m not sure that it actually works in practice.²

And I’m not sure the question ofwhat is in the wiki matters a lot for the organization, because I’m more interested in how much and what kind of structure people create when they’re editing in this kind of document, rather than specifics about naming schemes or the rationale behind a classification system. I guess, if I had to break this question down further I’d be interested in:

• What kind of metadata (categories, tags) do you use?
• How hierarchical is your document. Too much hierarchy and pages get lost at the end of trees, not enough and you have a mess of files and no way to tell them apart.
• Do you use natural phrase WikiWord titles for pages, or do you have a less natural naming scheme?
• How much do you use talk/discussion/commentary pages?
• CamelCase Wiki words or not?
• What kind of search tools do you use? Does this help?

Your thoughts are much appreciated.

Onward and Upward!