tychoish, now without a net

I’d like to talk a bit about what I’ve come up with in terms of data management/research organizational tools. So in my last post on this subject I mentioned that I had hacked together a shell script that did a lot of what I needed. Rather than look for the “one right” OS X tool (which I don’t think really exists at this point,) I’ve worked on collecting extant tools and programs and figuring some way of making something that would really continue to work into the future and that would be more tailored to my particular workflow.

I’m going to outline and present what I’m doing for the public betterment anyone else looking for something like this. Also I hope that people with a similar sort of need/workflow might be able to contribute tools or offer enhancements.

Let me outline the pieces of the puzzle, first:

• OS X/UNIX shell.¹ Having access to all of these great unix tools makes it reasonably easy to write scripts to automate the key parts of this workflow. Also, as most of my system reilies on Apache (which comes installed on OS X) and web servers, having this kind of low level access to the system is great.
• ikiwiki. So this is a great little program that takes a directory of text files, and turns them into a wiki/website via a markdown interpreter. It also connects and automates through subversion, an open source version management tool (it can also use git and others if that’s more your speed.) This part of the project is probably the geekiest and most difficult part to get working, particularly if you don’t have access a good package installer. But it’s possible and totally worth it.
• Several bash/shell scripts that I have written to insert data and clips into the wiki.
  • One script that, with a command in the form of, clip [FileName] [Space delinated tags] will create a page in a research clipping section of the wiki with the contents of the clipboard, and the proper notation for tag organization and the date of collection. The script then opens the file in the text editor. I wish it could capture, more automatically the citation information (author/url) but I think this would require the browser to expose more information than it currently does. But it’s pretty good.
  • One script that create new “tag indexes,” which makes it easier to see all the pages tagged with specific labels. If I were more clever I could probably tie the scripts together so that whenever I added a new tag that didn’t already exist that it would generate the new tag page. Except that this wouldn’t cover all instances of new tag pages, so it’s ok to have separate tag pages. This also helps control “tag sprawl,” and prevent metadata from getting out of control.
• A set very quick functions that let me append text any text file, as well as a quick command to append lines to the end of a general “inbox” or “codex.txt” file for quick thoughts, notes, todos, and tasks. This is outside of the wiki and not a new tool, but it works.
• Fluid - This is a really nifty program that uses OS X tools to build “programs” built around single, site specific websites. Basically this is the ideal bridge between “web apps” and “desktop apps,” particularly once Google Gears begins to work with this kind of app² amazing things are going to happen. Since I’m running the wiki locally, this is moot. There are other non-mac options like Mozilla Prisim, though I don’t have experience with it. This makes the wiki more useable and open I think.

Here’s a file with the relevant scripts and ikiwiki templates that I’m using. It’s rough and kludgy, but if people are interested or willing to contribute (feedback, knowhow, etc.), I’d certainly be willing to work on making this more polished and accessible.

ETA: I just discovered bsag’s textmate plugin, which might be a little more prime-time ready than my script, but I think with ikiwiki’s tagging function and potentially a recourse to spotlight/etc indexing, my solution works better for me. But, bsag is awesome and the text mate bundle is totally way more hard core than my kludge.

permalink • • zero comments
tagged: hypertext • productivity • technology

Every time–it seems–that someone suggests or comments on the future of the digital text (ebooks), someone always says one of the two following things:

1. People don’t like to read words off of screens, the quality/experience is bad and in truth people would prefer to read words off of paper.

2. The codex is–by design and use–an important carrier of information that can’t be replaced by digital technology.

Despite the apparent similarity, I think one of these things (and only one) is true: the second. Lets explore.

For starters, if you really think about it, we all read hundreds of thousands of words off of screens every year (and in some cases, every month or every week). We don’t mind reading off of screens, and particularly with anti-aliasing and flat screens, in many cases it’s easier to read off of screens because readers have more control over the display of text, and yet people don’t really read books off of screens.

The standing argument as to why monographs and novels have never found success, is that there are two many distracting things you can be doing on the computer. Why read a book when you can check your email for the 20th time this hour? This is, I think, why the codex is probably here to stay. And because giving books a physicality is commercially worthwhile, and because we respond to the form.

Having said that, I read a book off of my laptop screen this past weekend, and I have to say that I rather enjoyed the experience. So here are some thoughts on the experience:

The book came to me in PDF format, from a publisher. So the “pages” looked like conventional book pages. There were page numbers and so I was able to locate the text in space on a page, very much like I would a “regular book.” This spatial experience is often forgotten in digital texts, and I think that layer of information helps our minds make sense of longer texts. Also, because I knew the page numbers of the individual pages, and the total number of pages in the document, I was able to calculate my progress, again, helpful in assimilating the data in the text file.

I could read the words comfortably on the screen while viewing the entire page on the screen. Being able to see a snapshot of a page helps me in locating the text in space, which makes reading easier. More importantly, it allowed me to only scroll when I needed to change the page. This is really important to a successful reading experience, in my view.

The final piece of the puzzle is probably approaching an ebook like you would any other text on a computer screen, rather than approaching an ebook like you would a pbook. I read paper books, for the most part, in bed at night. I read things on the computer sitting at my desk. I read the ebook, mostly sitting at my desk. I also tend to read pbooks in longer stretches, reading several thousand words at a time, where as I probably never read more than 1,000 words at a time on a screen without taking a break. You might have a very different method for reading, and that’s fine, but by cycling a novel in with my regular livejournal-blog-email-twitter reading cycle I was able to read a novel in a weekend; whereas it almost always takes me several weeks (if I’m lucky) to finish a novel.

The end result? I’ve started reading another ebook, because it seems worthwhile.

permalink • • zero comments
tagged: hypertext • reading • technology

As I promised earlier, here is the draft of the “terms” that I want to include as part of my pattern for the shawl. It seems reasonable, and it seems to reflect my interests in both promoting freedom and the open exchange of ideas, but more importantly in making it completely clear that I’m not writing this pattern out of philanthropy.

The truth is that by refusing to restrict downloads giving away copies of this pattern means that more and more people will be able to see the pattern and this blog (which is, in the long term, good for me and my pocketbook). The basic idea is that, only a certain percentage of people are going to pay anyway, and with luck by unrestricting the download, more people will see the pattern, and therefore a great number of people will pay. I think/hope the economics work out, but we’ll just have to wait and see.

In any case, below the fold, you’ll see a copy of the draft of the statement of this idea. Please give feedback and commentary either in the comments or via email. I look forward to hearing form you.

Continue reading Pattern Terms

permalink • • coments
tagged: hypertext • interwebs • knitting • media

Ok, here it is: the long awaited first piece from my little fiction project. I’m not sure how well it will go over in this form, but it’s worth a shot. Many of the names and important words will be linked, in the final version, to pages that elaborate and explain characters. Also there are a couple of links that go nowhere in this page, will be linked to other stories and pieces. Including the “letter,” which is mentioned.

This snip, is set in the late 24th century, aboard an explorer/survey ship bound for a nearby star system at very fast (sub-light) speeds.

It’s all below the fold…

Continue reading Breakout: Messages Lost

permalink • • zero comments
tagged: hypertext • science fiction • writing

Some thoughts on the writing project.

So I’ve begun working on the Breakout hypertext, and I think all of my tedious regular expressions work actually made this work out pretty well. I’m really happy with the software, which is something–particularly given the digital nature of this project–that shouldn’t be ignored. Technology and I need to be working with each other not against each other. And I think that’s the case.

What I’m thinking about/working through at this point is a structural concern. My hope is that the sort of “first layer” of the document will be very encyclopedic, in a sort of lighthearted and friendly parody of wikipedia way. That structure, organized mostly around the ships, years/time periods, and characters is all set up and fairly straightforward. (Note, I’m mostly just thinking about the naming schemes at this point.)

Another layer of this project are what I’m thinking of as supporting documents, things written in character’s own voices, or other things that are “in-world” texts. These are all associated with an encyclopedic article, and numbered sequentially, by order of creation. So I have an article for one of the key characters named “frank” and a supporting document named “frank001″¹.

The next layer, that I don’t have figured out is the “fictionalized” story that I’m writing. I want the document to have a lot of little scenes linked into the supporting documents and to the encyclopedic “articles.” I don’t have any notion of how to name these things. Here are my concerns:

• If I associate them with characters, how do I decide that X scene with 2 or 3 characters is associated with a particular character? Particularly, in a way that will make sense to me in 6 weeks and 6 years.
  • If this is the case, how do I distinguish in the namespace between supporting documents and fictionalized narratives?
• How do I give short meaningful names that are distinct and thus identify scenes but that have some sort of systematic scheme. Furthermore how maintain some sort of order without forcing them into a sequential order. Again, using the namespace. So we can’t have scene001, scene002, and so forth.
• While I don’t need the pages to be strictly ordered to reflect the linear story lines, there needs to be some way to organize/grep through the page titles. I’m thinking using the location titles as a way to organize that and then use hidden dotfiles to keep track of the story and the files for my own notes, and then integrate links to the scenes from other pages, using my notes, once I have more meat, as it were, lying around to work with, and there’s a little bit of a Catch-22 here: you need data to play around and get a structure that makes sense, and it’s also hard to write without that structure.

I might have answered my question this is the problem of being an extrovert and a blogger: If you start a question for the crowd, by the end of your post you’ve probably answered it. I would still like input if you have it ;).

permalink • • zero comments
tagged: hypertext • writing

I guess this is in the larger theme of posts about my blogging process, identity, and purpose.

What’s a Wednesday morning before christmas without a little bit of healthy existential angst?

Though the merging of my old TealArt posts into tychoish muddies the water a bit, I think I’m going to pass an important marker in my blogging soon.

Sometime later this week I’m going to pass the 900 post marker. Having 600ish of my TealArt posts helps this, but it’s noteworthy to point out that I’ve posted so much to tychoish in a bit less than 6 months. One of my big struggles with TealArt was finding the time and energy to post. Now I can hardly live without it. That’s the Journaling Instinct, I guess.

I wouldn’t have posted about this except that I’m closing in on another milestone as well, and I figure, what the hell. What’s another odometer effect?¹

Sometime a bit after post 900 (I guess a week or two but that’s just a guess), my weblog writings of the past 7 years will past the 300,000 word mark. Egads. I always was a bit wordy. And the funny thing is that there’s another year or two of data that was lost…

One thing that dave and I have been talking about is how to jump levels in terms of another kind of milestone about readership.

One thing that I’ve been pretty conscious about with tychoish is regular posting. If I want to make a go of this, I figured, posting often is the key to making that work. If there’s never new content, who’s going to come back? It’s not like I don’t have things to say, it’s just a matter of getting it out there.

One thing that I’m seeing is that regular posting will get you to the top of your class, and allow you to make the most of what you already have, but if you want to jump levels you have to do something else. What that something else is, might be another issue.

Probbly the best thing is to be on the cutting edge. If you’re the 1st whatever, it’s easier to make it than if you’re the 20th. But assuming for a moment that we don’t have time machines or ESP…

Part of it is having friends. Getting links from other blogs, even smaller ones drives traffic, and user participation (I guess to backtrack for a moment, I’m measuring blogging success as a function of comments and traffic). Another part is participating in forums and other blog comments which can help a lot. There has to be a strategy out there for choosing the right places to participate and get involved–from a game theory/social dynamics perspective–too big and no one will click on outbound links, to small and no one will see it. For instance, my inbound traffic from ravelry, has gone down slightly has ravelry has grown. I’m still loving raverly, mind you, but I think this is just how the world works.

The other strategy is to work on additional projects. For a while, because there were so few podcasts, doing a regular podcast would drive a lot of attention to your work, but in this vein collaborative blogs, guest blogging, twittering, youtube contributions (and so forth) are all ways that you can sort of draw attention to your blog, but aren’t connected to y our blog, I guess, if that makes sense.

This whole marketing thing is clearly not my thing, but I think I get the concepts on a pretty basic level, so it’s sort of fun to play with the ideas. And of course, I do want tychoish to make it. Even if it doesn’t it’s still a great deal of fun, so it’s not like I’m going to stop… but, it’s worth a shot.

Onward and Upward!

permalink • • zero comments
tagged: Announcements • hypertext • interwebs • journal • TealArt • writing