Recently in opinion Category

Security profiling: TSA

| No Comments

Being of a gender-nonconforming nature has revealed certain TSA truths to me.

Yes, they do profile.

It's a white-list, unlike the police profiling that gets people into trouble. There is a 'generic safe-traveler' that they compare everyone to. If you conform, you get the minimum screening everyone gets. If you don't conform, you get some extra attention. Some ways to earn extra attention:

  • Don't look like your government ID.
  • Wear your hair up, or in braids (they've seen those kung-fu movies too)
    • Yes, they put their gloved hands in your hair and feel round. Anyone with dreads knows this all too damn well.
  • Fly with a name other than the one on your government issued ID.
  • Have body-parts replaced with things, such as a prosthetic leg, or knee (if going through metal detectors).
  • Have junk when there shouldn't be junk (or so they think).
  • Have breasts when there shouldn't be breasts (or so they think).
  • Have breast prosthesis instead of actual breasts (mastectomy patients love this).
  • And many more.

Here is an exercize you can try the next time you fly in the US. When you get to the other side of the scanner (this only works for the porno-scanners, not the metal-detectors), while you are waiting for your stuff to come out of the X-ray machine, look at the back of the scanner. Watch the procedure. Maybe put your shoes on slow to catch it all. You'll notice something I've noticed:

There are always two officers back there, a man and a woman. When someone steps in to get scanned, they have to either hit a button to indicate the gender of the person being scanned, or are presented with a side-by-side with both genders and the officer has to chose which to look at. They have a second, maybe two, to figure out which body baseline to apply to you, and those of us who are genderqueer confuse things. I fail the too-much-junk test all the time and get an enhanced patdown in my inner-thighs.

Yes, but with PreCheck you can skip that.

This actually proves my point. By voluntarily submitting to enhanced screening, I can bypass the flight-day screen annoyances. It's admitting that I no longer fit the profile of 'generic safe traveler' and need to achieve 'specific safe traveler' status. That, or I can have my bits rearranged and conform that way. Whichever.

The H-1B and I-140 labor-market

| No Comments

A former employer of mine was a big user of H-1B and I-140 visa-workers. They were big enough we had international development centers, and often those employees came to the US HQ. As an Ops-type, this was pretty awesome since we had follow-the-sun support for our stuff. It also meant holidays in other countries impacted our operations in ways that I'd never experienced before. It was a very diverse workplace, which I enjoyed immensely.

However, the big flaw revealed itself when said company had a really bad quarter and decided to reorganize.

What 'reorganization' involved:

  • A near complete musical-chairs round in the Executive VP, and VP offices.
  • A restructuring of the company into completely new divisions.
  • Completely closing one of our foreign development offices.
  • A serious downsizing of the US workforce.


For those of us who are full citizens, we have a completely fluid job-market. This company was in one of the major tech-hubs (the DC metro area, a.k.a. NOVA, a.k.a. 'Ashburn VA', a.k.a. "us-east-1") where cloud-talent was experiencing a shortage and salaries were rising quickly. Unsurprisingly, a large percentage of our senior cloud-trained talent left the company rather than deal with working for a convulsing organization. Me and most other senior systems-engineer types had cold-calls from both Google and Amazon. Some of us took them up on that offer (not me).

It was during this time that I learned about how different the labor-market was for visa-workers.

Both the H-1B and I-140 visa require employer sponsorship. If you lose your employer, you need to get another job (transfer the visa sponsorship) within a specific time or you have to leave the US. Per my fellow coworkers, that time is four weeks. And that's four weeks to first-day-on-the-job, not four weeks to acceptance-letter-is-signed. Four weeks in which to do a complete job hunt with a company big enough to be able to deal with visas and the Department of Immigration. That is nearly impossible.

As a result, all of my coworkers here on visa were job-hunting just in case they got a layoff. All of them. Half of them had married (always other visa-workers) and bought houses here. Due to the DC metro area being one of the most expensive housing markets in the US, you need two full-time incomes in order to afford anything of any size; so having one of the earners leave the country was a recipe for financial disaster.

My department was one making buckets of money for the company, so we didn't take a layoff at all. Even so, we lost about a quarter of our people due to better offers coming in on that just-in-case job-hunt. The other quarter left due to not wanting to put up with the reorganizations.

Some visa-workers in other departments got layoffs. The rumor mill said they were offered a paid ticket back to their home country in addition to whatever other severance benefits they were giving out. No help with the expense of an international move.


After the layoffs were done and the reorganizations had completed to the point that the org-chart wasn't being updated on a weekly basis, upper management got down to the serious business of trying to keep the people they had. Part of this effort was to rebase salaries to market averages. There were some big movements.

One coworker had been out of the job market for about seven years to raise her son. When she got back into it, she was hired on as an 'associate systems engineer' at a salary of around $70K/year. Fast-forward five years, and she was still at that job-title, but doing a senior systems-engineer's work and getting paid $73K. They reallocated her up to a 'lead systems engineer' title and a salary of $100K.

Another coworker was here on visa and hired as a 'systems engineer'. He was being paid $85K. The full citizen in the chair next to him, also a 'systems engineer', was being paid $102K. After discussing salaries one memorable afternoon (this is legal, don't let anyone tell you otherwise), we learned that all of our visa-people were underpaid versus the citizens. When the reallocation came around, most of this disparity went away.

Why did it get that bad?

Capitalism, of course. The job-market for visa-workers is limited to those companies with skilled enough HR departments to deal with immigration paperwork. This greatly reduces the number of companies they can work for, which in turn reduces upward pressure on salaries. It also means that those companies who have figured out the paperwork problem have access to a skilled job market with less salary pressure than the greater one, so they tend to go deep into it.

The other factor at play here is the internal raise process. As has been mentioned elsewhere, you get your best raises when you change companies. While salaries in the overall job market had been raising, internal ones were not raising as much. People who don't job-hunt until provoked were falling behind their peers and not noticing because talking salary is taboo.

How can we prevent this disparity?

By talking salaries more often. The visa job-market is fundamentally different than the one for full citizens, but our costs-of-living are still the same. By talking about salaries and visa-status the labor market's supply, us workers, can learn which companies are known to exploit visa workers and which are more likely to treat them the same as the citizens. It means being pushy during negotiations, but that's how you stop this kind of exploitation.

Paying for the web

| No Comments

An uncomfortable realization

| No Comments

I have a thing against Apple.

They're not the first tech company I have felt that way towards, but it is a definite thing now.

I've been snarking for a while now that...

But it's more complex than that. I've made quite a lot of money off of Microsoft; the first 13 years of my resume read like a Microsofty (the Novell stuff is ignored these days). Exchange. Web-serving. Clustering. Scaled up file-serving. Yep, made a lot of money with it.

And yet, when the most recent iPhone announcement hit my inbox(*) my reaction was definitely not the one I had about, say, Windows XP back in the day. Apple is not actually a Microsoft, and there are many people out there who will gladly fight me for saying it might be. But in terms of market-share for desktops... 90%? Those are numbers Microsoft was rocking for a long, long time. Admittedly, this is at a startup, not at a 1000+ employee company where manageability trumps individual employee preferences. But small companies become big companies...

My reaction to that email was a vengeful unsubscribe. Apparently I didn't want to be seen to be one of them.

I have Apple in the home. There is an iMac. We have Wifi from Apple because the iMac wouldn't stay attached to anything else. But it isn't anything I use on a daily basis(*).

At work I'm one of four non-Apple users. Three of us are Linux-desktop people, and we have a lone Windows user (who is a pretty chill dude). At least one of the other Linux users seems to feel the same way I do about Apple. So we're out there.

I used to be pretty cool with people using whatever, and believed I could use anything if given time. Heck, I was dual-stack Microsoft/Novell at a time when you were either one or the other. And I was dual-stack Linux/Microsoft at my last startup, which is a kind of unholy union people don't admit to but are actually kind of in demand.

But apparently I draw the line at OSX and iOS. When did that happen?

(*): I have an iPhone. It was a gift from a previous employer. Current Employer doesn't pay for telecoms, so I'm using it as my workphone. Thus, I get Apple marketing mail. And use it daily, because it was a free phone. When Apple decides this model is gauche and it becomes unusable, I'll replace it with an Android and feel relieved.

Or, a post I'd never thought I'd make seeing as I'm a sysadmin.

But it seems I'm the senior git expert in my team, so I'm making it. So odd.

There are a series of questions you should ask among your team before moving a repo over to git. Git is a hell of a toolbox, and like all toolboxes there are nearly infinite ways of using it. There is no one true way, only ways that are better for you than others. These are a series of questions to help you figure out how you want to use it, so you can be happier down the road.

Q: How do you use the commit-log?

History is awesome. Looking back five years in the code repository to figure out WTF a past developer was thinking about writing that bit of spaghetti code is quite useful if that commit includes something like, "found weird-ass edge case in glib, this is the workaround until they get a fix." That's actionable. Maybe it's even tied to a bug number in the bug tracking system, or a support ticket.

Do you ever look through the history? What are you looking for? Knowing this allows you to learn what you want out of your source-control.

Q: What is the worth of a commit?

A commit in Git is not the same thing as in SVN, Fog, or ClearCase. In some, a commit, or checkin, is a pretty big thing. It takes reviews, or approvals before it can be made.

This question is there to get you thinking about what a commit is. Commits in git are cheap, that changes things. Knowing that you will be facing more of then than you had in the past will help guide you in the later questions.

Q: Is every commit sacred, or you do you value larger, well documented commits more?

Practically everyone I know has made a commit with the message of 'asdf'. If you're grinding on a stupid thing, it may take you 19 commits to come up with the two lines of code that actually work. In five years, when you come back to look at that line of code, the final commit-message on those lines might be '

a1bd0809 maybe this will work

Not exactly informative.

bdc8671a Reformat method calls to handle new version of nokogiri

That is informative.

Most projects value more informative commits over lots of little, iterative ones. But your team may be different. And may change its mind after experience has been had.

Q: Should new features be all in one commit, or in a few modular commits?

Some features are quite large. So large, that rebasing them into a single commit leads to a diff of hundreds of lines. Such a large feature means that the history on those files will be slathered with the same initial-feature-commit with no context for why it is that way.

Is that good enough? Mabe it is, maybe you're more interested in the hotfix commits that are fixing bugs and explain non-intuitive behavior and workaround. Maybe it isn't, and you need each sub-feature in its own. Or maybe you want every non-fixup commit.

This is where your approach to the history really informs your decision. If you know how you deal with the past, you will be better able to put process in place to be happier with your past self.

Once you've thought about these questions and your answers to them, you'll be better able to consider the deeper problem of branching strategy. Git is notoriously lacking in undo features, at least in shared repos, so getting this out of the way early is good.

Public Cloud (AWS, Azure, etc) is a very different thing than on-prem infrastructures. The low orbit view of the sector is that this is entirely intentional: create a new way of doing things to enable businesses to focus on what they're good at. A lot of high executives get that message and embrace it... until it comes time to integrate this new way with the way things have always been done. Then we get some problems.

The view from 1000ft is much different than the one from 250 miles up.

From my point of view, there are two big ways that integrating public cloud will cause culture problems.

  • Black-box infrastructure.
  • Completely different cost-model.

I've already spoken on the second point so I won't spend much time on it here. In brief: AWS costing makes you pay for what you use every month with no way to defer it for a quarter or two, which is completely not the on-prem cost model.

Black-box infrastructure

You don't know how it works.

You don't know for sure that it's being run by competent professionals who have good working habits.

You don't know for sure if they have sufficient controls in place to keep your data absolutely out of the hands of anyone but you or nosy employees. SOC reports help, but still.

You may not get console access to your instances.

You're not big enough to warrant the white glove treatment of a service contract that addresses your specific needs. Or will accept any kind of penalties for non-delivery of service.

They'll turn your account off if you defer payment for a couple of months.

The SLA they offer on the service is all you're going to get. If you need more than that... well, you'll have to figure out how to re-engineer your own software to deal with that kind of failure.

Your monitoring system doesn't know how to handle the public cloud monitoring end-points.

These are all business items that you've taken for granted in running your own datacenter, or contracting for datacenter services with another company. Service levels aren't really negotiable, this throws some enterprises. You can't simply mandate higher redundancies in certain must-always-be-up single-system services, you have to re-engineer them to be multi-system or live with the risk. As any cloud integrator will tell you if asked, public cloud requires some changes to how you think about infrastructure and that includes how you ensure it behaves the way you need it to.

Having worked for a managed services provider and a SaaS site, I've heard of the ways companies try to lever contracts as well as lazy payment of bills. If you're big enough (AWS) you can afford to lose customers by being strict about on-time payment for services. Companies that habitually defer payment on bills for a month or two in order to game quarterly results will describe such services as, 'unfriendly to my business'. Companies that expect to get into protracted SLA negotiations will find not nearly enough wiggle room, and the lack of penalties for SLA failures to be contrary to internal best practices. These are abuses that can be levered at startup and mid-size businesses, quite effectively, but not so much at the big public cloud providers.

It really does require a new way of thinking about infrastructure, at all levels. From finance, to SLAs, to application engineering, and to staffing. That's a big hill to climb.

The No Asshole Rule is a very well known criteria for building a healthy workplace. If you've worked in a newer company (say, incorporated within the last 8 years) you probably saw this or something much like it in the company guidelines or orientation. What you may not have seen were the principles.

Someone is an asshole if:

    1. After encountering the person, do people feel oppressed, humiliated or otherwise worse about themselves?
    2. Does the person target people who are less powerful than them?

If so, they are an asshole and should be gotten rid of.

And if we all did this, the tech industry would be a much happier place. It's a nice principle.

The problem is that the definition of 'asshole' is an entirely local one, determined by the office culture. This has several failure-modes.

Anyone who doesn't agree with the team-leader/boss is an asshole.

Quite a perversion of the intent of the rule, as this definitely harms those less powerful than the oppressor, but this indeed happens. Dealing with it requires a higher order of power to come down on the person. Which doesn't happen if the person is the owner/CEO. Employees take this into their own hands by working somewhere else for someone who probably isn't an asshole.

All dissent to power must be politely phrased and meekly accepting of rejection.

In my professional opinion, sir, I believe the cost-model presented here is overly optimistic in several ways.

I have every confidence in it.

Yes sir.

The thinking here is that reasoned people talk to each other in reasoned ways, and raised voices or interruptions are a key sign of an unreasonable person. Your opinion was tried, and adjudged lacking; lose gracefully.

In my professional opinion, sir, I believe the cost-model presented here is overly optimistic in several ways.

I have every confidence in it.

The premise you've built this on doesn't take into account the ongoing costs for N. Which throws off the whole model.

Mr. Anderson, I don't appreciate your tone.

It's a form of tone policing.

All dissent between team-mates must be polite, reasoned, and accountable to everyone's feelings.

Sounds great. Until you get a tone cop in the mix who uses the asshole-rule as a club to oppress anyone who disagrees with them.

I'm pretty sure this new routing method introduces at least 10% more latency to that API path. If not more.

I worked on that for a week. I'm feeling very intimidated right now.

😝. Sorry, I'm just saying that there are some edge cases we need to explore before we merge it.


Okay, let's merge it into Integration and run the performance suite on it.

This is a classic bit of office-politics judo. Because no one wants to be seen as an asshole, if you can make other people feel like one they're more likely to cave on contentious topics.

Which sucks big-time for people who actually have problems with something. Are they a political creature, or are they owning their pain?

The No Asshole Rule is a great principle in the abstract, but it took the original author 224 pages to communicate the whole thing in the way he intended it. That's 223.5 pages longer than most people have the attention-span for, especially in workplace orientations, and are therefore not going to be clued into the nuances. It is inevitable that a 'no asshole rule' enshrined in a corporate code-of-conduct is going to be defined organically through the culture of the workplace and not by any statement of principles.

That statement of principles may exist, but the operational definition will be defined by the daily actions of everyone in that workplace. It is going to take people at the top using the disciplinary hammer to course-correct people into following the listed principles, or it isn't going to work. And that hammer itself may include any and all of the biases I lined out above.

Like any code of conduct, the or else needs to be defined, and follow-through demonstrated, in order for people to give it appropriate attention. Vague statements of principles like no assholes allowed or be excellent to each other, are not enforceable codes as there is no way to objectively define them without acres of legalese.

Paternity leave and on-call

| No Comments

It all started with this tweet.

Which you need to read ( Some pull-quotes of interest:

My manager probably didn't realize that "How was your vacation" was the worst thing to ask me after I came back from paternity leave.

Patriarchy would have us believe that parenting is primarily the concern of the mother. Therefore paternity leave is a few extra days off for dad to chillax with his family and help mom out.

Beyond a recovery time from pregnancy, much of parental leave is learning to be a parent and adjusting to your new family and bonding with the baby. I can and did bond with the baby, but not as much as my female coworkers bonded with their babies.

I should also state, that I don't just want equality, I want a long time to bond with my child. Three months or more sounds nice. Not only can I learn to soothe him when he's upset, put him to sleep without worrying about being paged, but I can be around when he does the amazing things babies do in their first year: learning to sit, crawl, eat, stand and even walk.

At my current employer, I was shocked to learn that new dads get two weeks off.


At my previous startup, paternal leave was under the jurisdiction of the 'unlimited vacation' policy. Well...

Vacations are important. My friends would joke that the one way to actually be able to take vacations was to keep having children. Here the conflation was in jest, and also a caricature of the reality of vacations at startups.

We had a bit of a baby-boom while I was there. Dads were glared at if they showed up less than two weeks in and told to go home. After that, most of them worked part-time for a few weeks and slowly worked up to full time.

This article caused me to tweet...

The idea here is that IT managers who work for a company like mine with a really small amount of parental leave do have a bit of power to give Dad more time with the new kid: take them off of the call rota for a while. A better corporate policy is ideal, but it's a kind of local fix that just might help. Dad doesn't have to live to the pager and new-kid.

Interesting idea, but not a great one.

Which is a critique of the disaster-resilience of 3-person teams. I was on one, and we had to coordinate Summer Vacation Season to ensure we had two-person coverage for most of it, and if 1-person was unavoidable, keep it to a couple days at best. None of us had kids while I was there (the other two had teenagers, and I wasn't about to start), so we didn't get to live through a paternity-leave sized hole in coverage.

Which is the kind of team I'm on right now, and why I thought of the idea. We have enough people that a person sized hole, even a Sr. Engineer sized hole, can be filled for several to many weeks in the rotation.

That's the ideal route though, and touches on a very human point: if you're in a company where you always check mail or can expect pages off-hours, it doesn't matter if you're not in the official call-rotation. That's a company culture problem independent of the on-call rotation.

My idea can work, but it takes the right culture to pull off. Extended leave would be much better, and is the kind of thing we should be advocating for.

You should still read the article.

This showed up today.

I get that. The little white lie that it's all right, I wasn't offended. The lying silence where the, "check that bullshit," should have been. The desire to belong to the in-group (or an in-group, even if it's an in-group of one) is probably baked into our genetics. Those that arbitrate membership in the in-group set the standards by which membership is granted. So long as there is power there, the little internal betrayals needed to achieve membership, or if that isn't possible, satellite membership, can be justified.

For a while. Until the price starts getting too high.

If the in-group is in all of the positions of both power and employee redress? That's a lot of incentive to shut the fuck up and laugh like you mean it.

And if you keep poking at it, because shuting the fuck up and laughing already is becoming very hard, you lose in-group status.

This is a very human progression, we've been doing it since pre-history. The modern workplace is supposed to be set up to deal with toxic managers and hostile work environments, but cronyism is incredibly corrosive. It takes active push-back to fend off, and of the corruption is deep enough that just costs you your job.

Most corporate severance agreements include something called a non-disparagement clause, which means, in effect:

The severed employee agrees to not say bad things about the Company, or cause material harm to the Company's business through their actions.

And accusing a manager of being a harassing asshole is the kind of thing that could trigger that clause. By telling the world about her experience with this manager, naming names, and calling out the toxic culture of that particular work-unit, she can be considered to be causing 'material harm' and could face serious legal consequences. If Google wants to be assholes about it, of course. But the language is there in the agreement specifically to scare ex-employees out of doing things like this.

The internal system was stacked against her, and the court of public opinion was also stacked against her by the very company that had the bad culture.

I'm guilty of making the same kind of calculations. I didn't seek in-group status as firmly as Kelly did, and it got me fired in the end. It turned out well for me, but was pretty traumatic at the time.

While I was there I did consciously choose to not call out jokes, behavior, or other things that offended me, specifically because I needed to stay on good terms with the in-group. I never got to crying, but the little niggling things did add up. It meant I didn't stay long at company events, didn't follow on after-work outings to bars, and generally stayed quiet a lot of the time. It was noticed.

Smartphone ecosystems have definitely reached the level of complexity where we have to worry about hostile apps. And they're following the pattern shown by the Internet over the years in that there are classes of hostile actions:

  • Known/Allowed, also known as ad/revenue streams. App owners have to pay the bills somehow, and purchase fees only go so far.
  • Known/Disallowed, also known as malware following known exploits. For this we have scanners.
  • Unknown, apps doing things they shouldn't, by ways that aren't in the scanners yet. Evil, evil little beasties.

If there is one lesson about information security that has been true since the beginning, is that it's the victim's fault for getting owned. Really, look at the press following hacks: hacks are entirely the fault of the defending entity for not being good enough. If you just followed accepted security standards, this would never happen. Never mind that transitive trust models in very complex IT infrastructures are nearly impossible to fully secure, especially ones that involve humans, it's still the victim's fault.

Those 'accepted security standards' are somehow lacking in the app-stores, especially Android. It's like the app-owners don't really want you to secure yourself.

What would be very nice in these phone OS security system would be selectable permission filters. Don't want to allow bluetooth-access to any applications except those you whitelist? Don't want to share your contacts with an app that seemingly has no need for it? A limited version of this is in iOS, but as I'll get to in a moment it only goes so far.

There are two methods of denying access to capabilities, and we already have a good example of this two-tier model in the firewall world:

  • Notifies connections of no-connection.
  • Pretends there is nothing there.

The first method is nice for applications since they learn quickly to stop trying. The second is nice for defenders because it means potential attackers have to wait for timeouts before marking a IP:Port tuple as up/down. When it comes to phones, there are two ways to deal with selectable permissions:

  • Notify the app that they don't have rights to that thing. Apps know they're being banned.
  • Lie to the app and provide a stub service that returns nothing or a simple carrier-signal. Apps will have to do tests to see if they're banned.

IOS uses the first model. If you've ever seen a, "turn on bluetooth for an enhanced user experience," modal, that's what happened. I believe that Apple standards say that applications have to honor those settings in that they still run and don't quit in a huff over not getting your identity goodies. You may not be able to do much, but they'll still run.

Android currently doesn't have selectable permissions (out of the box; there are some apps that try to provide it), you decide whether or not an app can be allowed to do it's full list at the time you install it. This can be problematic, especially if circumstances require that you install certain apps, but you want to disable certain capabilities. Such as having only one phone with both work and email on it, and you'd rather they didn't wipe it when they fire you.

That's where things like XPrivacy can come in handy. This only runs on a rooted device, but it provides the stub-services needed to prevent apps from quitting in a huff over not getting the ability to remove accounts on the device, lie about Bluetooth/NFC/Wifi access and state, or falsify 'network' location data. Things like XPrivacy allow us to provide those very 'accepted security standards' that reduce victim-blaming after incidents. It would be awesome if this came stock, but we can't have everything.

Other Blogs

My Other Stuff

Monthly Archives