First Draft Podcast: Liberation Technology
This week: We talk about the accessibility of computational tools, systems, networks, data-driven decision making, neotopology, and pushback from specialists.
Microblog
This week: We talk about the accessibility of computational tools, systems, networks, data-driven decision making, neotopology, and pushback from specialists.
Friend of the show Glen Worthey joins us to talk about DH14, the popularity of digital humanities projects, the humanities savior narrative, and #dhsheep.
I’m stealing an idea from my friend and fellow Husker Brenden Rensink. Brenden just wrote a post summarizing borderlands and transnational history panels—his area of study—at the Western History Association Conference this year. I thought I’d do similar with environmental history.
To reiterate from Brenden’s post, the Western History Association is meeting in Newport Beach, California, between October 15 to October 18 with the theme “The West & the World.” Like Brenden, I’m listing any panel that includes some theme on environmental history even if that only includes a single paper. If I’ve missed a panel or paper, let me know on Twitter.
Conquering the Meat Frontier: North American Ranching in the Nineteenth Century
The Legal and Physical Infrastructure of Southern California Water
The West and the World: Pacific Connections, Pacific Possibilities
Western History in the Imagination and on the Landscape
Teaching “America’s Best Idea”: Lessons from a Century (and More) of National Parks: A Roundtable Discussion with Images and Documents
Borderlandscapes of the Rio Grande
How Labor Shapes the West
Nature, Science, the West, and the World
Rethinking the “Colossus of Roads”: Transportation Infrastructure and the Development of British Columbia
The Mining West and the World
Rethinking Nature with Boom: A Journal of California
Remembering Norris Hundley – Pioneer Scholar, Mentor, Teacher, and Editor of the Pacific Historical Review
And of course, I would be remiss if I didn’t point out digital history sessions. The Technology Committee, of which I am a part, is sponsoring another Six Shooters lightning session this year on Thursday 2:30pm-5pm. There’s rumors of a cupcake reception afterwards. Presenters are limited to six minutes and six slides. We have some great speakers lined up already and we will be announcing them soon.
Finally, I am also chairing the panel “Western Encounters, Alliances, and Experiences: Mormons, Indians, and U.S. Federal Law” on Saturday, 8:30am-10:00am. The full program is here.
If you’re going to be at the Western, get in touch and let’s go grab coffee. Or the beach.
This week: we discuss my experiences working on the Geography of the Post, D3.js, and the challenges of designing, and critically engaging with, interactive scholarly works.
I took a moment yesterday morning to start work on a Ruby script to convert CSV files to JSON. My main motivation for writing the script was to get data into a format that more easily works with D3.js.
D3 has two main ways to load data: JSON or CSV. Although CSV has its benefits, the data is untyped. Because of that, D3 loads all the data as strings rather than differentiating between strings and integers. I could do the conversion on the fly with D3 pretty easily using built-in functions or coerce the data with JavaScript. But sometimes I like prepping data before I begin working with it.
The script builds JSON using the first line of the CSV file to determine the appropriate elements and the corresponding data, as well as checking that integers and strings are converted to their appropriate (typed) formats. And the script seems fairly robust; I noticed hardly a pause when throwing a 14,000-line CSV file at it. There are no built-in specifics to the data mapping; you’ll need to handle that either by modifying the script or handling it in D3.
The script:
#!/usr/bin/env ruby
# Parse CSV files and convert them to JSON.
# Mostly used for preparing data for D3.js. I don't like using untyped
# CSV files, so this script begins to clean things up for me.
# Usage: ./parse.rb /path/to/input.csv /path/to/output.json
# Jason A. Heppler | jason@jasonheppler.org | jasonheppler.org
# MIT License <http://heppler.mit-license.org/>
#
# Created: 2014-07-10
require 'rubygems'
require 'json'
require 'csv'
def is_int(str)
# Check if a string should be an integer
return !!(str =~ /^[-+]?[1-9]([0-9]*)?$/)
end
lines = CSV.open(ARGV[0]).readlines
keys = lines.delete lines.first
File.open(ARGV[1], "w") do |f|
data = lines.map do |values|
is_int(values) ? values.to_i : values.to_s
Hash[keys.zip(values)]
end
f.puts JSON.pretty_generate(data)
endSay I give it an input like this (postwar population of Bay Area cities):
date,San Jose,Palo Alto,Sunnyvale,Mountain View,Santa Clara
1940,68457,16774,4373,3946,6650
1950,95280,25475,9829,6563,11702
1960,204196,52287,52898,30889,58880
1970,445779,55966,95408,51092,87717
1980,629442,55225,106618,58655,87746
1990,782225,55900,117229,67460,93613
2000,894943,58598,131760,70708,102361
2010,945942,64403,140081,72510,112466
And in return I get this:
[
{
"date": "1940",
"San Jose": "68457",
"Palo Alto": "16774",
"Sunnyvale": "4373",
"Mountain View": "3946",
"Santa Clara": "6650"
},
{
"date": "1950",
"San Jose": "95280",
"Palo Alto": "25475",
"Sunnyvale": "9829",
"Mountain View": "6563",
"Santa Clara": "11702"
},
{
"date": "1960",
"San Jose": "204196",
"Palo Alto": "52287",
"Sunnyvale": "52898",
"Mountain View": "30889",
"Santa Clara": "58880"
},
{
"date": "1970",
"San Jose": "445779",
"Palo Alto": "55966",
"Sunnyvale": "95408",
"Mountain View": "51092",
"Santa Clara": "87717"
},
{
"date": "1980",
"San Jose": "629442",
"Palo Alto": "55225",
"Sunnyvale": "106618",
"Mountain View": "58655",
"Santa Clara": "87746"
},
{
"date": "1990",
"San Jose": "782225",
"Palo Alto": "55900",
"Sunnyvale": "117229",
"Mountain View": "67460",
"Santa Clara": "93613"
},
{
"date": "2000",
"San Jose": "894943",
"Palo Alto": "58598",
"Sunnyvale": "131760",
"Mountain View": "70708",
"Santa Clara": "102361"
},
{
"date": "2010",
"San Jose": "945942",
"Palo Alto": "64403",
"Sunnyvale": "140081",
"Mountain View": "72510",
"Santa Clara": "112466"
}
]Note that the data transformation for this particular example probably isn’t the best. A more useful JSON construction would group the data together with child nodes, for example:
[
{
"name": "San Jose",
"values": [
{
"date": 1940,
"population": 945942
},
{
"date": 1950,
"population": 95280
}
]
}
]But if you need a simple CSV to JSON transformation, the script starts getting you there.
Mat Honan in Wired:
The San Francisco Bay Area-based Blue Bottle, along with other formerly regional roasters like Portland’s Stumptown, Chicago’s Intelligentsia, and North Carolina’s Counter Culture helped kick off a brewed coffee movement in the United States. And while all have grown beyond their original city limits, none have really threatened to become the next Starbucks, or even Chipotle. Given its recent investment round, however, along with its purchase of Handsome Roasters in Los Angeles, Blue Bottle is clearly looking to get big. Tonx immediately gives Blue Bottle a much better Web and app capability than it now has.
It’s also a good deal for Tonx, which was attempting to raise more money to purchase its own coffee roaster (it currently has a contract deal where it rents one on the weekends) and open a store front. While neither announced a price, Tonx did abandon a $4 million fundraising round it had been pursuing recently. Presumably, the deal would be on par with that. It’s a big win for the three year-old roaster that’s based in Los Angeles, but lives all over the internet.
Some good news for two of my favorite roasters.
In the early twentieth century, C. W. Post, the cereal tycoon, launched an ad campaign to sell his caffeine-free, cereal-based substitute Postum. The trick of the campaign: convincing consumers that coffee was bad for kids. Jordan Weissmann at The Atlantic tells the story.
I am thrilled to see that the University of Richmond’s Digital Scholarship Lab has launched the Atlas of Historical Geography of the United States. Beautifully designed and wonderful maps both georectified and flat plates. Wonderfully done. What’s not to love?
I am very excited to see the public release of Topotime, a data model, d3.js layout, and Python function for representing complex or fuzzy time. My colleagues Elijah Meeks and Karl Grossner have been working on Topotime for the last few months and I have been eagerly looking forward to its public release.
A nice guide to various coffee brewing methods from Mistobox. Their guide on using the Aeropress is similar to my preferred technique.
Newsletter
Occasional writing on the American West, agricultural history, and political culture.