Kellie Ottoboni - Data Scientist

Demanding Leisure in the Time of COVID

2021-04-05T00:00:00-07:00

November 2020 was the lowest time for me during the COVID-19 pandemic. My boyfriend left to spend a month with his faimly across the country, while I stayed home in our apartment. Aside from a few days visiting my sister, I spent the month without human contact.

I played Youtube, Spotify, and Netflix constantly to drown out the silence. Sure, I still spent 12 hours every week in virtual video meetings for work, but staring at someone’s eyes on a screen hardly substitutes for in-person connection. The winter days were short, and indoor activities outside of my home were off limits at the time. Social media swallowed up my downtime, and I’d go to bed thinking “today was another day of nothing, like all the rest”.

The moments when I actually DID things were when I felt my best. I’m not talking about exercise; I view that as a minimum baseline of taking care of myself. I mean the times when I set out on a mission to create something and poured my full attention into this one thing. Following a watercolor tutorial on Pinterest. Trudging through the Berkeley Hills to photograph the small bits of beauty I’ve come to love. The moments spent creating, not consuming.

Finding purpose through leisure

In Digital Minimalism, Cal Newport suggests filling time with “demanding leisure” instead of “passive consumption”. It’s a no brainer, but in our modern world, most people tend to default to passive consumption if they’re not being mindful about it.

The loneliest moments I’ve felt during the pandemic have been underscored by a lack of purpose. Even when my boyfriend is home, there are days when I shut my work laptop at 5 o’clock and ask myself, “now what?” My work is cognitively demanding and I usually feel the urge to stop thinking entirely. The easiest way to do so? Popping open my personal laptop and mindlessly reading or watching content online.

Passive consumption could not be easier. Social media companies literally optimize their apps to maximize the time users spend scrolling. Instagram tries to show you the images and captions it thinks you’ll engage with the most, Facebook throws the most clickbaity links in your feed, and Twitter invented the infinite scroll. Whether it’s images, essays, doom-filled news, 140-character quips, or stupid videos, you can pick your poison – there’s an app for everything.

Once in a while you might learn something useful that sticks with you, or you might see something that sparks inspiration to go do something in the real world, but usually you don’t retain what you consume in short form content. After a scrolling session, you walk away having achieved nothing. It’s as though the time spent online disappeared into a black hole, never to be remembered or revisited. It’s an activity that adds zero value to life.

Demanding leisure is the opposite of vegging out. It means doing activities that require skill, planning, or social connection. It might not be the first thing you think of as leisure, since in our modern culture we tend to associate leisure with relaxing. Think back to just about any book you’ve read that was written before 1950 – how did the characters spend their leisure time? They were probably reading literature, playing instruments, learning new languages, doing needlework, or playing outside. None of these things is “relaxing” per se, as they require focus and energy to partake.

I think of consumption vs leisure similar to the distinction between type 1 and type 2 fun. Type 1 fun is happy and fun while you’re doing it, like going out for tacos or playing a game. Type 2 fun is challenging while you’re doing it, but satisfying at the end. Things like hiking mountains and writing this blog post are type 2 fun. Demanding leisure is like type 2 fun. It leaves you with a lasting feeling of purpose and satisfaction, as opposed to the fleeting interest of passive consumption. It adds value to life and causes you to grow.

Cultivating demanding leisure

Just like restrictive dieting often leads to rebounds and overeating, trying to cut out social media cold turkey has not been a successful strategy for me to cultivate a more purposeful life outside of work. Instead, I’m trying to intentionally fill my evenings and weekends with creative pursuits and working towards goals.

This is so, so hard to do during this pandemic. Many businesses aren’t operating in their usual capacity, events that we’d normally participate in are cancelled, and it’s hard to do anything social. I think everyone deserves a little grace for not “making the most” out of this unusual time.

I’ve been consciously spending more “demanding leisure” time on a few things, like learning about local plants and fungi, training to run longer distances, and wildlife photography.

I encourage you to try this: pick one night this week to skip your usual digital wind-down rituals and instead, spend that time on demanding leisure of your choice. How does that feel?

Practicing Essentialism in the Tech Industry

2021-02-14T00:00:00-08:00

Hello readers! It’s been a long time since I posted on this blog. Years, actually. During that time, I finished my PhD at UC Berkeley and started working at Pinterest. It was an adjustment, to say the least. I traded my 10 minute walk to campus for an hourlong bus ride across the Bay Bridge; freedom to work from anywhere with Wifi for butt-in-chair from nine to five; the pursuit of knowledge for the pursuit of measurable business impact.

At the beginning, it still felt like academic work in some ways. I did data analyses on my own, munging tables in Jupyter notebooks for hours. That feeling dissipated as I gained more responsibilities. Today, I am the tech lead of a group with 2 other data scientists. My time is split 50/50 between my own work and “tech lead” work, overseeing our group’s progress towards goals and unblocking the others so they can do their best work. With more responsibility comes more meetings, more interruptions, and more work infringing on the 50% of time I’m supposed to spend on my own projects.

A few months into tech leading, my mental health was in the trash. I was perpetually stressed and felt like no matter how much I worked, it was never enough. It bled into my personal life: I wanted to “turn off” and just veg out every night instead of doing other things that normally spark joy. I had a difficult time being present and listening attentively to things not work-related. For the first time in my life, I experienced the Sunday Scaries, which sometimes started as early as Saturday night. I knew something had to change.

I started working with a coach to manage my anxiety and she hit me with a difficult truth: it isn’t the work that’s making you stressed, it’s how you’re thinking about it. I had let my inner perfectionist off her leash and she was dragging me through the mud, leading with the belief that I had to do it all, perfectly, or else people would think I’m incompetent. Anyone would feel stressed out about work if that were their mental model of the world!

Around the same time, I read Essentialism_ by Greg McKeown. The premise is that in order to accomplish the most essential tasks in our lives, we have to say no to everything else. By saying no to all the extra things, we actually give a resounding yes to the most meaningful ones: spending quality time with loved ones, devoting time to our hobbies, and delivering the most impactful projects at work.

Do less, but better. I was sold on this approach to reign in my inner perfectionist, tamp down my stress, and ultimately do better at my job.

Here are 5 ways that I’ve started practicing Essentialism at my job in 2021.

Stop being connected all the time, pt 1: minimize Slack and email distractions

Tech companies have built their products to give us intermittent rewards, in the form of notifications. These intermittent rewards keep us coming back for more, making us hooked on our devices. I hate to admit I’m guilty of this: I feel excited when I pull up Gmail and find a new email.

Without even realizing it, I find myself checking every platform where I might get a notification, on repeat throughout the day. It’s distracting, and 95% of the time, no message I get requires immediate action. If there’s a small question from someone, I feel like I should respond right away “while I remember.” It detracts from what is essential.

Here are a few steps I’ve taken to minimize distracting notifications and my proclivity to check them:

I deleted Slack from my phone. It was convenient when we worked in the office, but now I find that I have no need for it anywhere besides on my work computer.
I have a 10 minute limit on email on my phone, using Screen Time for iPhone. I usually spend most of those 10 minutes reading newsletters in the morning, and after that I have to manually override a “Time’s Up” blocker in order to check my email. I still check sometimes, but the blocker forces me to pause and be deliberate.
I turned off as many Slack nudges as possible. Here’s a good guide on ways you can reduce notifications from Slack. When I’m focused on something, I try to remember to pause alerts for DMs.
I simply close the Gmail tab in my browser. If I see there’s a little (1) in the tab name, I feel compelled to check it immediately. But when I don’t have the tab open, I forget about email altogether for hours. Out of sight, out of mind.

Stop being connected all the time, pt 2: quit checking your email at night

I almost never intend to work after hours. When I do, this doesn’t apply. Otherwise, there’s literally no point in checking emails at night except to ruminate about what you need to do in the morning. Just don’t bother.

Optimize your calendar for focus blocks.

I struggle to start on something if I know I only have a short amount of time to work on it, which presents a problem on days when I have meetings interspersed with 30 minute breaks. I get nothing done besides little tasks, and I don’t touch the essential ones. By deliberately optimizing my calendar for blocks of 2+ uninterrupted work hours, I’ve reclaimed focus time for essential projects.

I signed up for Clockwise, a Chrome extension that optimizes your calendar for focus blocks. You can turn certain meetings on “autopilot” and it will optimize the time of that meeting for all attendees. It’s helpful for 30-minute one-on-one meetings, which can usually be moved to a better time slot.

To protect my time even further, I have blocked off Wednesdays on my calendar and request that people check with me before scheduling Wednesday meetings. There are also org-wide focus blocks during the week on our shared calendar. If someone schedules a meeting during these blocks, I suggest we move it.

Use the Pomodoro method and practice meticulous attention

As I was thinking about ways to reduce my mental clutter, I found this Zen Habits blog post where the author talks about the practice of meticulous attention, an application of mindfulness to work: notice each thing you’re doing as a distinct task, then commit to giving that task your full attention.

To implement a meticulous attention practice for myself, I’ve been using a Pomodoro timer. It divides time into 25-minute focus blocks, with 5 minute breaks in between them. The idea is stupidly simple, but effective: knowing there’s a designated break coming up soon when I can indulge in every distraction that I want motivates me to stay on track during those 25 minutes. I downloaded the Marinara Chrome extension, which pops up a tab when your focus time or break is up.

Set more small goals and checkpoints towards bigger goals

During 2020, my team ran a monthly planning meeting to assign everyone projects for the month ahead. It was difficult to estimate how long tasks would take and to assign an appropriate amount of work for one month. Instead, we often ended up assigning far too much work, with no midpoint milestones and little guidance on how to prioritize.

In 2021, I’ve moved my team to a bi-weekly planning cycle. So far, it seems to be helping us focus on fewer tasks at a given time, while giving us the agility to evaluate progress and strategy more often. Having a smaller number of milestones at any given time is good motivation for me and helps cut down on the perfectionist voice in my head that wants me to do more, now.

How I Traveled the World in Grad School

2019-03-16T00:00:00-07:00

If you follow me on social media, you might’ve seen that I’ve been traveling a ton this past year, and most of it has been related to my grad school work. In my five years as a PhD student, I’ve visited five states and five countries for conferences and other events. As someone who didn’t travel much as a kid, I’ve been loving these opportunities!

However, travel is terribly expensive and I make pennies as a PhD student. And what’s more, my advisor doesn’t have any grants that he can use to pay for my travel expenses. Finding funding has KEY to making this all possible. I wanted to write down some tips because people ask me all the time, how do you travel so much? It’s pretty simple: seek out opportunities and ask for money.

Tip #1: Speak at small conferences in cool places

Here’s the first place where people go wrong. It’s tempting to try and get your papers into the big, prestigious conferences where all the big name professors attend every year. That’s a mistake: your chances of getting your work accepted there as a grad student are pretty low unless your faculty collaborators are well-connected. We can pretend that research quality is important, but the reality is that academia is all about connections.

Instead, look for smaller conferences. Seriously, whatever niche you’re in, you can find a conference for it. Chances are that your advisor is connected to people in that community if they’ve been researching this subject for a while, so there’s your in. Plus, you have a better chance of making meaningful, lasting connections at small conferences.

You’ll need to have some work to present at this conference. Don’t be discouraged from submitting your work to a conference if it’s not in tip top shape. Sometimes it needs to be a complete, polished research paper, but often it doesn’t. Lots of conferences have special options for grad students too, like short talks and posters. As long as you’re presenting something at this conference, you’re set.

I've been spoiled rotten during grad school, getting to travel to conferences in the pretties places. #EVoteID2018 pic.twitter.com/HwPmb9yXGr
— Kellie Ottoboni (@kellieotto) October 6, 2018

Tip #2: Apply for scholarships, fellowships, and grants

Now, how are you going to get to this conference? Expenses add up: flights, a place to stay, conference registration, and food. For conferences in the U.S., plan to spend around $1000. If you’re going abroad, it’ll be more like $2000.

There are tons of different ways to cobble together money:

Scholarships are usually one-off payments, unrelated to travel.
Fellowships tend to be more long-term awards, where you’re part of a cohort of fellows. They usually come with a stipend and sometimes an allocation for conference travel.
Grants are one-off payments for a specific purpose. The Graduate Division at UC Berkeley has conference travel grants for grad students. Professional societies sometimes offer travel grants as well.
Sometimes, conferences will offer student travel grants that you can apply to.

I’ve used all of these sources of funding, sometimes in tandem for more expensive international trips.

You have to apply for all of these, so it’s good to always have a written statement of purpose and a faculty reference handy.

You have to google around for lists of scholarships and whatnot. Some universities compile lists. But always be on the lookout for them!

Tip #3: Partner with multi-site organizations

Outside of traditional academic departments, lots of campuses house organizations that are funded privately or through campus. As part of the Berkeley Institute for Data Science, I’ve gotten to travel to different states for our annual MSDSE meeting with UW and NYU. The institute covered this travel for me.

Is there a group on campus that you can join that aligns with your research? This is also an awesome way to expand your professional network outside of your department.

Good morning Mohonk! #2016DSSummit pic.twitter.com/o7u0Hx0WRQ
— Kellie Ottoboni (@kellieotto) October 24, 2016

Tip #4: Just ask for money

I ran out of grants to apply to a few months ago after my (very expensive) trip to E-VOTE-ID in Austria. But travel opportunities kept popping up! I had to step outside my comfort zone and just ask organizations if they’d be willing to fund my trip.

For someone to give you money, you must provide them value. How does your attendance at whatever event impact the organization?

For instance:

I got a stipend to spend a week in Padova in exchange for teaching a short Python course.
I got asked to speak at a conference, and told them that I’d love to if they could book my flight and hotel for me.
I asked a nonprofit to send me to an international conference to present work directly related to their mission.

Asking for money is uncomfortable, but the worst they can say is no.

Explaining pseudo-random number generators and sampling ballots for Lansing. 130 absentee, 130 Election Day ballots. #MIRLA pic.twitter.com/0Nt0rKkXrn
— Kellie Ottoboni (@kellieotto) December 4, 2018

Tip #5: Tack some additional days onto your trip

So you’ve gotten your travel all set up, great! Now that you have the means to get to your destination, if you have some extra money, tack on a few additional days to your trip. The flight is the most expensive part, usually. If you can afford a few more nights in a hotel or Airbnb, why not?

To get to E-VOTE-ID in Austria, I had to fly through Zürich, so I added a few days in northeastern Switzerland to my itinerary before and after the conference. Tiny Airbnbs in Europre tend to be pretty cheap; a hostel would have been even cheaper.

You might have to explain your extra time if you have to fill out reimbursement forms. Don’t expect that your funding source will pay for extra days on your trip.

Random Problems with R

2019-01-22T00:00:00-08:00

I gave a lightning talk at the SF R Ladies meet-up about a problem with R’s sampling algorithm. Check out my slides here!

As part of my dissertation, I dug into the pseudo-random number generators and sampling algorithms used by common statistical packages. Along the way, I found an issue with the way R generates pseudo-random integers using the sample() function. I’ll give an example where we’d like to generate integers uniformly on an interval, but sample produces 2x as many odd numbers as even ones. Our short paper on the problem is on arXiv.

Internet voting, security, and transparency at E-VOTE-ID

2018-10-08T00:00:00-07:00

Last week, I attended my first voting conference: E-VOTE-ID. I’ve presented at statistics conferences before but never an interdisciplinary one like E-VOTE-ID. It brought together people working on electronic voting issues from a whole range of disciplines: legal studies, sociology, cryptography and security, voting systems developers, former election officials, and one statistician. This guy!

I learned an incredible amount in 4 days, none of it having to do with my research directly. It was awesome! There were no parallel sessions, so we had everyone at the conference in one room for all the talks. The most interesting part, for me, was hearing how every country has its own challenges and addresses them so differently. Here’s a recap of some of the most memorable things I heard.

E-VOTE-ID Day 0: the PhD Colloquium

I was one of six grad students who gave talks on e-voting research. I heard talks on how to verify security properties of a new e-voting protocol with some crazy cryptographic properties, a history of security problems with paper ballots in the United States, inefficiencies in the way Estonia integrated internet voting into their electoral process, and a discussion of how Germany painted themselves into a corner legally.

The German issue is a funny example of how non-technical people try to do the right thing but fail when they don’t consult technical people. Germany had some issue in one of its elections and so passed a law in 2009 with the goal of making elections more transparent. Instead, they’ve essentially made it impossible to use any sort of electronic equipment. They require that “the average voter” can understand anything that leads from ballots being cast to a final tally. Problem is, even most educated people can’t understand security and cryptography under the hood, so they can’t use any of those techniques to make things private. But they are required to implement some security by law – ballots have to be completely private and cannot provide voters a receipt of how they voted. They call this the transparency-anonimity dilemma.

I won the best PhD workshop paper award! Check out my slides from the talk here.

E-VOTE-ID Day 1

I had NO IDEA about how everything in Estonia is so technologically advanced. Frankly, I had to check on the map to be sure where this tiny country is. But in the past 20 years, Estonia has introduced a smart ID card for every citizen that plugs them into an entire internet-based system called the X-Road. Banking, healthcare, and voting are all done on the X-Road. They’ve had internet voting since 2003 and participation rates are above 30% now. They studied the costs associated with internet voting, early voting, and election day voting, and found that e-voting is 20x cheaper than all other options combined. Crazy.

In Nigeria, the norm is for the losing party to bring a law suit. They don’t accept that they’ve lost. It’s interesting because a delegation of elections officials from Nigeria visited Berkeley last year and spoke to my advisor and me about ways they could improve their elections. I think it’s all about making the process transparent while people are voting and auditing things afterwards. The person delivering the talk on Nigerian elections proposed having external (foreign?) impartial people monitoring the election as it happens.

Jason Thompson gave a lively talk about internet voting for Americans. Even as an American, I didn’t know this was a possibility! It’s meant for American citizens living abroad, of which there are about 9 million including military. Each state can choose how to do its remote voting. Shockingly, 28 states ask voters to return ballots by email and they make waive your right to a secret ballot. Emails aren’t end-to-end encrypted, PDFs can be changed easily, and voting this way isn’t necessarily private. To demonstrate, Jason sent in his ballot in front of everyone at the conference!

I found this article independently the same day: Pennsylvania and other states are locking foreign voters out of their elections websites. Probably some IT specialist doesn’t understand firewalls and Americans abroad don’t necessarily know how to VPN into a U.S. network.

E-VOTE-ID Day 2

We heard a great talk about lessons learned about internet voting in Norway. The talk was given by a cryptographer who acknowledged the fact that e-voting is way more than just the cryptographic protocols! No matter how well the security works on paper, there are challenges putting it into practice. Norway has done remote voting experments in uncontrolled environments, including local elections in 2011 and parliamentary elections in 2013. They’ve stopped using it because the government is too worried about coercion happening when people vote from unregulated environments.

They had a cool natural experiment: the incompetent election workers somehow printed the wrong return codes on ballot receipts. Voters should have been able to use their code to verify that their vote was cast correctly – but they got someone else’s code. Based on the number of complains they received, they could estimate that 85% of voters tried to verify their vote. This is huge and unexpected!

Another lesson learned: openness helps with mistakes and confusion. If you explain everything in great technical detail, then journalists aren’t interested. People are looking for a good story and if you don’t give fodder for negative press, they don’t care.

In many talks from many countries, researchers gave evidence that goes against our stereotypes. They found that internet voting is not more attractive to young people. Early adopters of internet voting in Estonia were older. Surveys of voters in Norway, the Netherlands, and Canada showed the same thing. At first, this was surprising, but the more I think about it, it’s not: perhaps young people are more aware of the security issues involved with electronic devices, while older people are more willing to trust them?

We heard a lot about blockchain voting. I’ll just leave this here.

Blockchain voting (in many cases) is like fridges with TVs: solving problems that don’t exist. #EVoteID2018 pic.twitter.com/9xeLbzYR5c
— Kellie Ottoboni (@kellieotto) October 4, 2018

E-VOTE-ID Day 3

This morning, I gave my presentation on SUITE, a new method for risk-limiting post-election audits of stratified samples. Check out my slides here!

We heard not one, but two interesting talks from Vanessa Teague on measuring the evidence that outcomes are correct in elections that use instant runoff voting. The way that votes are distributed to candidates in these kinds of races (mostly done in Australia, but sneaking into some American elections) is incredibly complicated. What can look like a close margin to the naked eye can actually be a close race, based on early rounds of eliminated candidates. She showed that in such cases, it’s essentially impossible to do a risk-limiting audit, but when margins aren’t too close then it’s statistically quite simple.

The conference ended with an awesome lunch and perfect early fall weather in Bregenz. I haven’t been to that many conferences, but this was among my favorites.

I've been spoiled rotten during grad school, getting to travel to conferences in the pretties places. #EVoteID2018 pic.twitter.com/HwPmb9yXGr
— Kellie Ottoboni (@kellieotto) October 6, 2018

From Paper to Program: Challenges of Implementing Permutation Tests

2018-06-12T00:00:00-07:00

I gave a talk about a short book I’m writing at the 4th Conference of the International Society of Nonparametric Statistics. Please check out my slides!

As the field of data science grows and computational resources abound, it is increasingly common for people to analyze their data using permutation tests. Introductory statistics textbooks often teach parametric statistics exclusively; when they do teach permutation tests, they are often portrayed as being model-free. In fact, permutation tests do require assumptions, and this approach to teaching omits crucial considerations. Without appropriate pedagogical material about issues of implementation, people are more likely to misuse permutation tests. To fill this gap, we are developing a short open access textbook. Topics from the book include: how controlling for covariates can improve statistical power, illustrated using clinical trial data with multiple time points and study sites; how to design a permutation test for a complex experiment, illustrated with a study to assess gender bias in teaching evaluations; and computational limitations of sampling algorithms and pseudorandom number generators for big data. These examples are written in accessible language, shown using real-world data, and accompanied by R and Python code. Our materials should supplement, not replace, instruction of permutation test theory, and will enable intermediate statistics students, data scientists, and domain researchers to design the best nonparametric tests for their observational or experimental studies.

My grad school “crisis of faith” and how I got through it

2018-04-16T00:00:00-07:00

A few years ago, I was pretty unhappy in my PhD program.

I started off wanting to be a high-profile academic researcher. I was enticed by the glory of being a big name scientist and making earth shattering contributions. But once I started the PhD, that vision started to overtake my other interests. Successes in my classes and research became the main source of my self esteem.

That’s how so many of us get to graduate school in the first place: we put so much effort into schoolwork that it becomes a big part of our sense of self. But when we start to struggle instead of succeed, our self esteem suffers. To make matters worse, I was having a hard time in my personal life and instead of dealing with those things head on, I compartmentalized them and poured myself into the PhD work instead.

I lost interest in my hobbies and spent all my time working. Before starting the PhD, I would go to the farmer’s market every Saturday and cook myself elaborate meals just for fun. Eventually, I stopped cooking for fun and ate the same meals every day to save myself time. My nights and weekends were spent reading papers. I didn’t realize it at the time, but I was putting all of my eggs in the academic basket.

Around the middle of my second year, I started to lose my footing. Grad school is a struggle for everyone, no doubt, but it’s even harder when you are questioning whether you belong. This was the first time I really felt that I was a woman in a male-dominated field.

I felt inadequate because my research interests weren’t as “theoretical” as most of my classmates. At least anecdotally, men tend to enjoy more theoretical work while women tend to prefer applied research. This was certainly my experience. It wouldn’t be a problem on its own but it became increasingly clear that projects I found interesting were not ones that would be publishable in the top journals or would be considered when applying for faculty positions. I really enjoyed things like writing software packages, but you don’t get much academic recognition for doing it.

I felt out of place because I didn’t carry myself with the same confidence as my (male) colleagues. There’s a big confidence gap between men and women: many men won’t hesitate to say something or volunteer, even if they might be wrong or partially qualified. Many women do the opposite. I never spoke up in discussions for fear of being shot down and talked over by arrogant or unforgiving colleagues. And often, I simply wasn’t interested in the minutia of the mathematical theory anyway. My imposter syndrome was at an all-time high.

I presented my early stage research in one of these seminars and got totally ripped to pieces. I didn’t know the whole body of literature on the topic and instead of helping me out, a few of the poeple there berated me with questions I couldn’t answer on the fly. It felt like the opposite of the collaborative environment that drew me to Berkeley and really exposed the “old boys club” mentality that I was immersed in.

I thought I wanted to go on to be a professor — essentially doing more of the same, as my career — until it wore me down too hard.

Are your values in alignment with your work?

One week later, I was at a beer garden in Portland. I was having a drink at 2 in the afternoon with my best friend in the PhD program on our spring break trip. I chose not to think about my research project for a few days while I regrouped. With the taste of IPA on my lips, I realized “I don’t even want to do this.”

Talking it through with her, I realized that it was not my incompetence holding me back but my motivation. This lifestyle was not in alignment with my values, what I think is important, or what makes me happy. I don’t feel driven to compete to be the smartest, or to do math for math’s sake. To feel fulfilled, I want my work to directly impact the lives of others, even if it’s in small ways. Playing the academic game is not for me.

I was in the middle of my PhD when I realized this. I could have quit and done something else, but I didn’t throw in the towel. Instead, I adapted.

I’m fortunate that I was in a position where I could mold my lifestyle and my work to better fit my values. And I’m so glad I did. My physical and mental health are way better than they have ever been, and my research is taking off despite putting a little less energy into it!

A few months ago, I organized a lunch for the women students, faculty, and staff in the department. The discussion was centered around honoring your values and defining what success means to you. Woo woo, yes, but the discussion was uplifting.

I read this book about the history of women in science posited why women in STEM might compromise their values more than men.

In this we see two sides of the image-making coin: [the media] deliberately had shaped Curie into a maternal martyr who had used science for womanly ends; but this portrait also evoked an image of a superwoman, too smart, too dedicated, too focused, and too talented to be emulated by ordinary women…Institutional sexism remained unchallenged, as women told themselves simply to work harder or publish more.”

I asked the participants to pick 2-4 values that resonate with them and to reflect on whether they uphold those values in their work life. If the way that “success” is defined at work doesn’t match up with success as determined by your values, then maybe something is off.

What did I do?

Well before I thought about my values explicitly, I started changing the way I approched grad school to better align with them.

1) I started giving myself nights and weekends off. No sense in working 24/7 on something for which you aren’t getting paid much and don’t really enjoy. Besides, taking breaks makes you more focused and creative when you are working!

2) I worked with my advisor to find research projects that I really wanted to work on. I had to find my voice to say no to things I didn’t want to do and find the courage to dive into new problems.

3) I sought out opportunities outside the PhD to do things that matter to me: teaching workshops, writing blogs and science articles, and consulting with people doing food research on campus. Having side projects has been a lot of fun, expanded my skill set, and created opportunities for me in the future.

4) I’ve been proactive about seeking research funding, so I can really focus my energy on the projects I want, and seeking travel opportunities related to my work (hello, Italy 2018!!)

5) I stopped caring so much. After realizing I didn’t want to pursue a career in academia, I stopped worrying about how my work appeared. No more pressure to publish in top journals, and with that there was a lot less pressure to do the kinds of work I don’t enjoy. Without my perfectionist tendency in the way, now I actually do better work because I’m not afraid to make mistakes along the way.

Working towards a better grad school experience

Over the last ten years, 95% of male students have completed the PhD in my department while only 75% of female students have. There might be a small numbers game at play (in that there are never very many women in my program), but I’d posit that something else is going on.

I think that many students go through a crisis of faith at some point in the PhD. But perhaps the female students are having a harder time getting through this difficult period.

Unfortunately, people don’t tend to talk about these feelings openly while they’re going through it. It’s scary to be vulnerable and share the feeling that maybe you don’t belong. It’s no good if we all suffer side by side, but silently, when we could lean on each other for support! In sharing my story, I hope others will reflect on what really makes them happy and have the courage to speak up and demand it.

Open Source Licenses Explained

2018-03-26T00:00:00-07:00

I ALWAYS forget to put a license on my work until someone reminds me. I’ve learned over and over that it’s important, but I think the reason why it hasn’t stuck is that I was never taught why it’s important.

So that’s the goal of this super quick blog post: to explain what a license is and why it matters that your open source software or other materials has one.

What is a license?

A license is a legal document that specifies the conditions under which someone may use a software or written materials. This includes rules for downloading, modifying, or redistributing the materials. All products should have some sort of license, but it’s especially important for people in the open source community to be aware of licensing issues. Most projects don’t have lawyers on hand to handle copyright issues and the whole idea behind open source is to making things open for people to use and contribute to!

Why do you need a license?

Copyright and intellectual property laws apply to all works of software and written materials. Without a license, a work is under exclusive copyright: people who are not the copyright holders cannot use, modify, or share the work. A license can relax these restrictions.

As the author, you want to put a license so:

people can legally use your work! Imagine academics using your software in their papers; that wouldn’t be allowed without a sensible license.
some licenses protect you from liability if your software or materials cause problems for its users.
people can contribute to your work in such a way that preserves your copyright permissions

As a user:

you want to be sure you’re protected against copyright infringement. The license lies out instructions so you don’t use software improperly, share materials without permission, or fail to properly cite the authors.
you can always technically fork a GitHub repository and download the code. But without a license, exclusive copyright supercedes: you do not have permission to use, modify, or share the materials.

How do you choose a license?

There’s a website for that!

But this site only lists a few of the possible licenses you could use. There are tons of premade licenses for different types of works (e.g. software, scientific papers). You don’t have to write your own - just search for a prewritten one that fits your criteria.

This website breaks down possible considerations into three categories:

Permissions - what are users allowed to do with the materials?
Conditions - what do you require a user to do in exchange?
Limitations - what are users not allowed to do?

Certain licenses are more or less stringent in each of these categories. For my software projects, I tend to use the BSD-2 or MIT license. They’re both permissive licenses that limit the author’s liability. For papers, I use a Creative Commons license, specifically CC-BY-4.0. Creative Commons allows people to share and adapt the work freely, with proper attribution to the authors.

Why There Isn’t More Evidence That Pesticides Disrupt Your Hormones

2017-12-08T00:00:00-08:00

I’ve really been gotten on the crunchy bandwagon this year – buying high quality grassfed meats, organic produce, paraben-free beauty products, and swapping out plastic food storage containers for glass ones. Up until recently, I was skeptical about the evidence that these choices really make a difference for your health.

The main reason I’ve been so skeptical about these kinds of products is that the scientific evidence that the amount of chemicals present in our day-to-day products directly causes worse health outcomes tends to come from poorly done statistical analyses. Frankly, I know too much about how these things can go wrong to believe the numbers they report. They often rely heavily on parametric modeling assumptions that are rarely satisfied.

I looked at two studies of pesticides and fertility recently. My instinct was to rip them apart. And yet, I couldn’t help believing their qualitative results. I want to dig into the reasons for this and to provide some thoughts on what kinds of studies and reporting would be done in an ideal world.

The studies: pesticides and infertility

The first study I read was about semen quality. 155 men at a fertility clinic were given dietary surveys and asked to give semen samples. They analyzed semen quality in a number of ways and correlated that with the amount of pesticides consumed. The authors found a nonlinear dose-response effect: more typically-pesticide-heavy foods in diet was associated with lower sperm counts and other infertility outcomes. That sounds like a plausible finding to me, however…

Noisy data

They used a single 24-hour dietary recall to determine what fruits and vegetables each man ate in a day. They assumed this was representative of their past and future diet. The authors took several USDA pesticide measures for various fruits and vegetables and aggregated them into a single score for each food item. Then they dichotomized the produce into “high pesticide” and “low-to-moderate pesticide” foods. This is pretty crude. They checked robustness by moving the threshold be called “high pesticide” up and down, but this still assumes that the underlying pesticide consumption score is valid.

Fruits and vegetables with a total score greater than or equal to 4 (i.e. at least one of three measurements was in the third tertile) were considered to be high pesticide residue foods while fruits and vegetables with a total score <4 were considered low-to-moderate pesticide residue foods. Based on these criteria 14 fruits and vegetables were categorized as high pesticide residue produce, and 21 as low-to-moderate pesticide produce.

Their Supplementary Table 1 lists the produce categories and their scores. The highest scoring are celery, strawberries, bell peppers, and spinach while the lowest include peas, grapefruit, dried prunes, and onions. There’s one strange thing in the list that sticks out to me: cooked spinach is a different entry from raw spinach. That’s because they mapped “cooked spinach” in peoples’ dietary surveys to “frozen spinach” in the pesticide data list. But using frozen spinach isn’t the only way to cook it! I imagine there’s a lot of noise in the way they map food frequency questionaires to the pesticide data.

Not externally valid

The second thing that makes me uncomfortable is that they sampled men from a fertility clinic and not from the general population. The results of this paper cannot be generalized to the public because the study population was at a fertility clinic where, presumably, the men giving samples are already having trouble with their fertility. The authors don’t mention this fact in the abstract.

I’m not even going to comment on the statistical models. I guess they’re less offensive because the authors aren’t claiming causality, but I still worry whether the estimates mean anything. Linear models tend to be “asymptotically consistent,” meaning they estimate the right thing if you have infinite amounts of data, but with a sample size of 155 men I’m not so sure we’re in the asymptotic regime.

Another example of these issues

I’m even less inclined to believe the numbers from this second paper on pesticide intake and pregnancy outcomes. This study has all the same problems as the other: they use the same kind of methods to classify fruits and vegetables as high- or low-pesticide foods, they sample women from fertility clinics, and they use generalized estimating equations to estimate the probability of live births and of clinical pregnancy for all these women undergoing assisted reproductive technologies. These models assume that the dose-response is linear, and I’m wary of any parametric model that estimates probabilities.

But I believe it anyway.

I think the trends are compelling. While the dietary survey measure is noisy, I’ll concede that it probably does give some insight into how many pesticides and endocrine disruptors people are eating. The study populations are people who are undergoing infertility treatments, who presumably have compromised endocrine systems. This suggests that imbalanced hormones are susceptible to becoming even more imbalanced due to endocrine disruptors in food.

The way the results are reported, precise numbers about how humans are affected, is a big part of why these studies are offputting to me. My advisor calls this “quantifauxcation”: situations in which a number is, in effect, made up, and then is given credence merely because it is quantitative.

I find the results meaningful in a qualitative way, but not in the way that the popular media reports them. Take it at face value: people with fertility troubles seem to be negatively affected by pesticides. The findings say nothing about how an otherwise fecund person would respond. But it suggests a mechanism by which these pesticides are harmful!

Why are these studies so hard?

There are a myriad of things that make it challenging to pin down the effect of pesticides. Some of them are due to the nature of the problem and some are just fundamental problems with science in our society.

There are ethical and logistical challenges to studying health outcomes.

This is a problem for all kinds of health outcomes and diseases. In a perfect world, you’d have a random sample of the population, you’d randomly assign them a certain diet with measured pesticide levels for a fixed amount of time, and you’d follow them and measure their fertility until they died. Unfortunately, studies like this are limited to convenience samples of people, who eat whatever diet they want, and you only get to measure them a few times. Since diet isn’t randomly assigned, researchers try to control for confounding factors by using modelling approaches. Modelling is the best they can do, but there are no guarantees that the results are right.

Incentives to research endocrine disruptors are low.

Who wants to fund research that says that the status quo is hurting us? There are powerful industries and lobbies with lots of money that want to keep pesticides in use. There’s no powerful organic food lobby that can compete with “Big Food”. Funding may be limited for this reason. And when research on the harmful effects of pesticides does get funded, it’s never enough money to carry out the kind of study one would like to do (a randomized control trial over a long time horizon). Limited resources force people to use fewer subjects and lower quality measures of exposure, resulting in lower-quality research.

The burden of proof is on consumers, not producers.

Here’s the fundamental problem: it’s up to consumers to prove that these chemicals are harmful. It seems relatively easy for companies to introduce new pesticides. They ned to show that in lab settings, the chemicals pass certain toxicity tests. However, these tests only tell us about short-term effects in the lab. They don’t say anything about potential long-term effects of eating low doses of pesticides over many years. These things enter the market, and it becomes the consumers’ job to show that they are harmful. Once products are in use it is difficult to get rid of them.

Without changing how pesticides are studied, farmers will continue to spray their crops with endocrine disrupting chemicals.

I’d like to see some changes in the way people think about studying the harm of chemical exposures.

Simply put, do better studies. I realize this kind of effort is difficult and takes a tremendous amount of funding and effort. But the evidence from a small observational study like the ones here is not convincing. I’d like to see more patients, more external validity, and better measures of pesticide exposure. If researchers from different schools could form a consortium and pool their resources, I bet that more compelling findings would come out.
Take into account prior knowlege when reporting statistical significance. David Colquhoun makes this suggestion in his comment on “Five ways to fix statistics”. If there has been lots of prior work showing that a chemical is harmful, then even if your study does not reach statistical significance at some pre-specified level, you may want to give more credence to a marginally significant result.
My suggestion for observational data would be to focus on the qualitative results rather than the precise numeric estimates when reporting these studies. I can believe that a positive association is real, but I lose confidence when I hear probabilities being thrown around.

And in a perfect world, regulatory agencies would respond to this kind of evidence and pull products immediately to be examined more rigorously. It would not be such a battle to prove that these chemicals are dangerous. Until then, we have to be vigilant and make smart choices for ourselves.

Questions to Consider about Your Product from a VC Partner

2017-11-27T00:00:00-08:00

I participated in my first hackathon two weekends ago. I use code to do data analysis most of the time, not write apps or websites. For me, it was more of a fun learning experience and I got to see what kinds of work are expected and rewarded.

One of the judges for the hackathon was a principal at a venture capital fund. The questions he asked the hackathon teams during judging were clearly the type of questions one would ask a founder about their startup. It was super useful to hear those questions and to think about how I can use them to sell my research and future project pitches.

Who else is doing it?

You need to show that you’ve done your research about the market. Your product/research isn’t exciting if someone else is already doing it. You need to put your own spin on it and offer something unique. If you can’t say that your idea is the “Uber for dogs” or something, then you haven’t made a great case.

How many people will use this?

VCs want to invest in products that are highly impactful. That means tailoring your product to a specific group of people who are most likely to use and benefit from it. Very often, we get excited about what we’re working on and think that everyone else will be excited about it too. But that’s not very effective: in trying to please everyone, you end up pleasing no one. Funders want to see that you know your target audience and that you’re tailoring your product to them appropriately.

How much impact do you expect?

Again, VCs want to see that your product will be highly impactful. They also want to have metrics by which they can gauge impact. You don’t want to say something generic like “my apps helps people do this one particular thing”. They want to hear “x million people have used my app in y cities around the world”. They want to hear how many dollars you’ll earn, or how many tons of CO2 emissions you’ll save, or how many hours your users will save. The more specific, the better. It shows that you have performance goals and you’re actively working towards them, not just plodding along.

How will you know if it’s working?

This is related to metrics for impact. This is why you need to lay out your expected impact clearly with measurable outcomes. If you’re not hitting those outcomes, then you can figure out what’s going wrong and make a plan to do better. Without this kind of feedback, you’re putting your product out there and hoping that your target audience is adopting it. The funders are less likely to have faith in your product if you don’t have a plan to detect when it’s not doing well.

Applying the ideas to academic work

Thinking about these questions highlights one of my frustrations with academia: people work on obscure research that has little impact outside of their field. I’m just not sold on why it’s important. Most research papers, I’d guess, are read by tens or maybe hundreds of people who work on similar, related questions. Even when I do read a paper that I think is intellectually interesting, I often don’t see how it’ll be put into practice: lots of statistical methods sit in academic papers and never get turned into code that’s widely distributed or used. I try to avoid that kind of work and instead make sure that I do research where the potential impact is clear and quantifiable, publish things online where people can find them, give good talks and write clearly so my work is accessible, and write open source code that people can download and use. In the future, I’d like to set better metrics and outcome goals for my work.