I đź’™ Spreadsheets, Election Night Precinct Data Edition

I am finally, after sixteen years of post-college experience as a “software engineer” learning to understand and love spreadsheets. So much so that I’ll inefficiently figure out how to do something in a spreadsheet to answer a question as opposed to just writing a script or dropping the data into a proper structured data store with a more programmer focused query language. But spreadsheet formulas, etc. are programming! I am fairly certain that some business spreadsheets I’ve seen are self-aware and planning to throttle us all. Anyway, I can do this. I am a professional.

Tonight I dumped the August 1st primary election night precinct level results CSV file into a Google Spreadsheet and decided to do some programming. You can find the original csv on the elections website – look on the “Download results” tab to see what’s available. The specific file I played with is this one. Note that all the screenshots on this post are using the election night results only. The final precinct results won’t be available until August 16th, sob, though at that point I can just replace one tab in my spreadsheet and voilĂ ! It will all update.

Continue reading “I đź’™ Spreadsheets, Election Night Precinct Data Edition”

Migrated the site … again.

Having my website be static using Jekyll as generator was nice. Except that I couldn’t really edit and publish posts on mobile. I’ve been working a lot, and doing lots of politics things, and have a toddler, and have miscellaneous other things going on so I honestly spend 99% of my non-work computer time on a mobile phone. The majority of posts in the last year or more have been composed in whole or part on my phone and published on Medium, then gradually moved over when I had time to spend.

So, on the advice of my friend Cate who, full disclosure, works at Automattic as their 📱👑 (emoji for head of mobile development), I moved my blog to hosted WordPress. Even using my own domains (I also have an owl-themed sub-site), it’s only a bit more expensive and I get a quite functional mobile app. Plus I can change the colors when I get bored without having to go mess with CSS. I’m really not all that good at the theming and CSS and stuff.

Unfortunately while there are numerous guides and posts out there on how to migrate your blog from WordPress to Jekyll, there are none that I could easily find for the other way round. Fortunately, I found someone who migrated from Jekyll to Medium. That process involved using Jekyll to generate a WordPress (WP) export.xml file. Unsurprisingly, you can use a WP export file to import into WP too. So I followed the author’s tool and it went pretty smoothly, dumping my entire blog (essentially) into one big file, formatting and all. It did not import comments but that’s okay. They were a headache to moderate because I don’t believe in having comments unless you moderate and I’ve had comments turned off on all newer posts (and disabled on older ones). If people are dying to get to their comments, they are in Disqus … somewhere.

The only annoyances were that self-hosted media didn’t import right, all my footnote anchors broke and none of my tags carried over. The media presumably broke because all my posts had image source values like /images/something.jpg and the import process couldn’t infer where they came from. Most media I have in blog posts is actually on Flickr though so it was only a few posts. The tags were pretty quick to fix as I just don’t have that many and I didn’t even have to refer to the original site very often to remember which ones. The footnotes on the other hand was kind of obnoxious to fix and I had to go through each post and edit the HTML directly. I was going to go through all of them and at least glance to make sure they looked right, but this made it a bit more time consuming. Footnotes: never again1.

But done! Getting my DNS (how your computer figures out how to talk to web sites) squared away was probably the most stressful part because despite doing computers for a living, me and DNS aren’t good friends. Being able to post on mobile will be đź’Ż and maybe instead of twitter threads or long facebook posts, I’ll just write a darn blog post instead!


  1. As if I’d give up footnotes.

Tech Execs Bow Down

So this happened:

deray_tweet_20161214
‘Deray tweet: Their faces say it all.’

 

shaft_tweet_20161214
‘Shaft tweet: #trump does not meet the ethical bar for employment and would be fired for his actions, words and deeds from all of the firms represented’

 

My frustrated mid-day comment after foolishly opening twitter to find out about this tech company leader meeting with the President-elect was to ask:

r343l_tweet_20161214

‘All of those retweets. I realize we often have to work with people we don’t like or who did wrong things. But is there a line not to cross?’

A friend commented that this is happening and folks need to work with him and some work from the inside and so on. And yes, of course that’s correct.
But I didn’t hear most of these people loudly criticizing anything he said during his campaign (or not very often). Or loudly criticizing what he said he wants to do. Or demanding Republican leaders show some standards.

This is a president that is promising to deport millions of people, to profile and monitor people for their religious faith, who thinks all black people live in violent ghettos, that Americans who weren’t born here (and let’s be honest aren’t white) can’t be trusted, that women are just pieces of ass. He has an absurd number of financial conflicts of interest. He literally asked a foreign state to hack his political opponents and we’ve discovered that a foreign state probably did hack his political opponents. He called for his political opponent to be locked up … for alleged crimes that people he is considering for his administration actually committed. He lies incessantly. He has assaulted multiple women. He threatens the media for reporting on him negatively and wants to change libel laws to make it easier to silence people. He has white supremacists (yes, Nazis) who advise him. He can’t be bothered to learn about what doing the work of a president entails and he’s including his kids in political meetings.

Okay I’ll stop.

Why exactly should tech leaders go meet with him without a hint of criticism? Are they already so afraid of what he will do if they demand that he first show some sign of contrition or apology or even the slightest hint that he’ll change his behavior, words and policies? Did any of them negotiate concessions before lending their credibility and influence to him?

This is how fascism works. All these captains of industry who won’t use their very real power to demand better of our leaders are instead making nice.

We look at history books and talk about how different companies in fascist Germany supported and helped the Nazis and why that mattered. Do you think those leaders were all committed believers in racial purity from the start? Or did many think they could play nice, keep their heads down, try to work behind the scenes and eventually things would work out — but really their silence allowed the entire system to become controlled by fascists?

Now I’m not a big, successful tech company executive so I don’t know what it takes to excel as one. But then, none of these tech company executives has had to exercise their skills in this environment. This isn’t just a change of political winds. This is someone who threatens the order of society. He, painfully, has the near full support of the party in the majority and seems to respect none of the norms of American government or politics.

I think fairly positively of many of the men (mostly men) who met with the president-elect. I honestly don’t believe most of them are fascists or bigots or personally support any of the next president’s policies.

But how can we expect the “little guy”— all of us just plain Americans with very little power or money comparatively— to stand up to fascism and bigotry if the most powerful in our society are signaling that they won’t?

We got here because some political leaders have been saying the same kinds of things Trump says for a long time. They weren’t challenged, especially in the last eight years as opposition to every act of the first black president became a rallying point. Now they will run the country, led by the man who popularized the racist conspiracy that President Obama wasn’t born here.

We’re not guaranteed a happy ending. The system works because people make it work and try to make us better. That striving for a more perfect union of our founding mythology that we so desperately want to be true. I believe in the best vision of America and the idea that people can collectively improve. But some days it is really hard.

Raising the Bar on Leaving

My shields went down the morning of October 19th, 2015. I went online and found a piece by Jay Carney. I’d been at Amazon more than five years. It was possible I was going to work there for much longer. I had my problems with the company. There were things I would change. I’d found places where I could thrive and work past the problems. But this piece destroyed what was left of my shields. That was the day that made it much more certain that I was leaving Amazon someday. I just needed a few more pushes and the right opportunity. One came quickly. Opportunities in tech are not hard to find.

Carney’s Medium post came out of nowhere. The controversy about the “New York Times article” had mostly died down. I was still asked about it sometimes and I gave my stock answer: I can believe that everyone in the article had those experiences but I’d largely had good experiences. So I didn’t expect to see my employer publicly attacking people for what I saw as valid criticism in an article published months earlier. I didn’t expect a senior leader to publish such an uncharitable, aggressive, privacy-violating response at all.

sculpture_apollo
‘Public art outside Amazon building in Seattle’

I knew people who had bad managers. Some I knew had been “managed out”. In one particular case, someone I didn’t even know came to me with a story of how their manager was disengaged, didn’t give good direction, blamed them for bad outcomes (that they had actually pointed out the risks of beforehand) and let the more senior engineers do whatever they wanted. Ordinarily, I might assume that there was probably a bit more to the story than that, but I’d tangled with their manager’s manager before. I thought he was an asshole and was willing to tear people apart in meetings, hide his own group’s failings and shift blame (and had behaved in a pretty obviously sexist way in front of me). It was no surprise he’d support (or ignore) one of his managers do that to the most junior member of a team. Many of the stories in the Times article were like that: it hadn’t happened to me, but I knew it could happen somewhere in the company. It’s a big place.

So I was especially upset that Carney chose to “clarify” that someone who said they’d gotten very bad feedback in the “Anytime Feedback” system had only received positive feedback via that system. The NYT reported that her manager had made that claim. Employees don’t have access to this feedback and a manager can easily mislead about the source of feedback. Most managers include quotes from Anytime Feedback in their yearly performance review documents. Good ones only include quotes that they think are representative of the employee’s actual performance and that are constructive. So Carney writing that she hadn’t actually been given negative feedback was petty. She could only know what her manager told her and it’s an obvious sign of a bad manager to falsely blame negative feedback on anonymous reviewers. Wouldn’t it be better to talk about what Amazon is doing to train those managers to do better and hold them accountable? If they were so rare, it would be easy!

I also didn’t expect to see so many of my coworkers publicly and internally enthusiastic about this new offensive form of public relations. Thankfully many saw it for what it was: an aggressive use of asymmetric power to bully people into being quiet about problems. I usually didn’t talk about Amazon online at all for fear my honest viewpoint would be taken as trashing my employer. Carney made me believe that even if I left the company I still had that to fear. Several people the Times got on record had both positive and negative things to say about Amazon. Their positive opinions of Amazon didn’t save them.

At Amazon I’d experienced or heard about all the standard sexist bullshit. Scantily clad images of women in presentations. Senior leaders with questionable posters on their office walls and harmful words in their meetings. Software with jokes mocking women embedded in their very names. Men (mostly men) who talked over or ignored me. Men who hit on me (or other women). Men who harassed their coworkers. Men who assumed I wasn’t technical even in meetings where everyone invited was required to be technical. People who said “diversity” hiring was lowering the bar. Women promoted more slowly than men. Women doing the tedious work to maintain a service while men went off and built new, usually unnecessary, things that got them promoted. Women doing extra work to make the place better. Men who said they wanted to help improve the company culture but never showed up. Women whose proposals were ignored. Groups of a hundred or more with no woman above entry level in any role. Women who left. So many women who left. But I was still there.

rainy_seattle
‘Rainy Seattle skyline’

A day or so after the original NYT article, one employee had published a public response about how the Times didn’t reflect his experience. In it he casually dismissed the idea that women were treated differently. Jeff Bezos then endorsed it, citing the post in his message to the entire company. I let it go at the time. I’d seen and heard a lot. I have a thick skin and in general Amazon’s culture is one that I mostly can get along with. Carney’s aggressive, smearing post made it all make sense. It explained a lot. An inclusive work culture is not a priority for senior leadership because they don’t really believe the culture has problems. Some of Jeff’s leadership team (the all-male senior vice presidents) probably believe there are significant problems. I stopped believing Jeff thought it was important.

Maybe Carney’s post didn’t make it inevitable I would leave. I liked my team a lot. I was learning a lot. I didn’t quite understand what I was thinking and feeling. Problems that I thought I’d made peace with started to frustrate me again. I was too often angry about things that used to be mere annoyances I could laugh about. I was hyper-critical of company statements public and private. I was even more impatient than I had been about the glacial pace of company cultural change. At the next company all-hands meeting, this aggressive public relations stance was endorsed in front of everyone, souring my joy that the women in engineering group was finally being promoted company-wide at the meeting. It just justified decisions I’d at that point already made.

Jay Carney, how many people have left or are going to leave because of that one post? How many more will stay silent, slowly letting it eat at them? How many are more cynical and less committed because they know senior leadership is willing to publish your performance review to prove you wrong? How many would never risk saying anything like this but wish they could?


I wrote most of this months ago. I still think Amazon can be a good place for someone to work (see my last post). You can learn a lot. There are many interesting and smart people there. I don’t think it’s Evil with a Capital E. No company is. Companies are made of people trying to achieve many different goals. I don’t think it’s a good place for me to work right now. I don’t want to work somewhere senior leadership doesn’t seem to realize how damaging that post was. I raised this concern with HR before I left. It didn’t sound like leadership thought this response was a problem. Maybe in the time I’ve been gone they’ve changed their minds and realized Carney’s post was a mistake. Maybe I’m not risking anything by posting this. Maybe.

Dreaming the Same Dream

No job is perfect. That seems like an obvious thing to write. But if you read tech job postings, you’d believe that every job is one where you build a product that is dramatically changing the world, the work environment is better than all others and your coworkers are all smarter than everyone else (echoes of Lake Wobegon where all the children are above average). Oh, also they pay you better than everyone else and the stock might make you rich!

The reality is a lot of tech work is incremental change. You’ll be doing many small and sometimes boring things. Many products and business plans are awful. And no one talks honestly about culture or the people who support bad ones. But many tech jobs are good enough depending on who you are, when you’re looking, what you need and what you can tolerate. But if you can find one where you believe in the product it’s pretty great.

I was at Amazon for just over six years on three different teams. It was good for me, professionally and personally. I learned about scale, orchestration, software maintenance, working with people, coaching people, controlling my own behavior, being more deliberate and more. The last team I was on was especially great. My manager was awesome (and is – if you’re looking for a developer role and want to work at a big company, she’d be a great choice). She gave me space to learn and grow, both technically and professionally. The one downside of the role is that I was not incredibly enthusiastic about the team’s product. I loved its technical goals and even liked the product goals. But fundamentally “retail” has never been something that is that important to me.

That’s funny because the entire time I worked at Amazon, I worked in Retail, either services used to make the website run or actual frontend features as with my last team. But jobs compensate you with more than some product you’ll walk up hill barefoot in the snow to help build.

I realize to some this may sound completely uncontroversial and melodramatic. But to read discussions in tech circles we expect people to be completely devoted to a company’s product. In reality there are many compensations for a job and I’m not even talking about money yet! Companies with huge legacy, both in terms of software and products, are not always going to find people exactly excited about the full and complete “mission” of every team. For one, it’s hard for a candidate coming in to even understand what the job might be about. For another, it’s okay to not love every last thing about the job or the team’s goals or the org’s goals.

Let’s repeat that: it’s okay to not be 100% super excited, committed and believe in your job or the company’s product.

But I do recommend not hating the product and having other reasons that the job is good enough. It helps for getting up in the morning. You might even make a list of what’s important to you and arbitrarily rate what a job brings in those areas. A big company can compensate for a product you don’t love by having:

  • Money, benefits, duh.
  • Working hours or oncall responsibilities that fit your life right now.
  • A team you work with well.
  • A manager that supports your career goals.
  • The opportunity to learn how to solve problems at scale.
  • A larger internal technical community. You’re less “on your own”.
  • The ability to change teams when yours isn’t working out.

Obviously many of those a small company can offer. A startup might additionally offer a product that you are excited about such that you’ll compromise on some of those. When I started thinking about changing jobs and specifically Glowforge, I got excited about an actual product more than I’d ever been. I’ve been working professionally for over 15 years and this is the first place I’ve been really excited about the product. Not just technically interesting or a thing that I think is worth existing but actually something I want to see in the world. It’s a version 0.1 of the matter compiler from The Diamond Age or replicators from Star Trek. I get up in the morning many days and remind myself I’m literally helping to create science fiction.

When you join a company or any group endeavor, you’re fundamentally working together to build something bigger. Entrepreneurs (and artists and writers and so on) all start something new because they have a dream of something they want to exist in the world. When you then ask someone else to join your company, and especially when you’re selling a vision of changing the world, you’re asking them to dream your dream for a while. Let’s be honest about what that dream is and that maybe the parts that get you up in the morning aren’t always what gets everyone else up. So despite my raving about how awesome my job is now, I wish we were more honest that a lot of work, even in “tech”, is pretty routine. That’s okay. The “boring” stuff keeps society working and is worth doing too.