Home Archive About
Home Archive About


2016-10-05 - Knowledge base completion

This blog post is mirrored on on eClub’s blog.

My name is Michal and I came to eClub Prague to work on an awesome master’s thesis. I am interested in AI and ML applications and I sought the mentorship of Petr Baudiš aka Pasky. The project we settled on for me is researching knowledge base completion.

You may already know something about knowledge bases. Knowledge bases, also known as knowledge graphs, are basically knowledge represented as graphs: vertices are entities (e.g., Patricia Churchland, neurophilosophy, University of Oxford) and edges are relations between entities (e.g., Patricia Churchland studied at University of Oxford).

A small neighbourhood of Patricia Churchland in a hypothetical knowledge base

Knowledge graphs are useful[citation needed]. The most famous knowledge graph is Google’s eponymous Knowledge Graph. It’s used whenever you ask Google a question like “Which school did Patricia Churchland go to?”, or maybe “patricia churchland alma mater” – the question is parsed into a query on the graph: “Find a node named Patricia Churchland, find all edges going from that node labeled studied at, and print the labels of nodes they point at.” Look:

How Google uses knowledge bases

Google’s Knowledge Graph is based on Freebase (wiki), which is now frozen, but its data is still publicly available. Other knowledge bases include DBpedia, which is created by automatically parsing Wikipedia articles, and Wikidata, which is maintained by an army of volunteers with too much free time on their hands.

Because knowledge graphs are useful, we would like them to contain all true facts about the world, but the world is big[citation needed]. And since we want to represent everything in knowledge graphs, we’d need a really big number of editors. Real-life knowledge graphs miss a lot of facts. Persons might not have their nationalities assigned, cities might be missing their population numbers, and so on.

Someone had the bright idea that maybe we could replace some of the editing work by AI, and indeed, so we can! When we try to add missing true facts into a partially populated knowledge base, we are doing knowledge base completion.

Researchers have thrown many wildly different ideas at the problem, and some of them stuck. For example:

  • Extracting relations from unstructured text. This means I take a knowledge base, some piece of text (I’m using Wikipedia articles), and I try to fill the gaps using the text.
    The canonical approach is to train a classifier for each relation type, for example the relation actor played in movie. The classifier takes a sentence as its input, and outputs a number between 0 and 1, which represents the classifier’s estimate on the likelihood this sentence represents the relation. So, on sentences like University of Oxford is in Oxford, UK. or Marie Curie died of radium poisoning., we’d like to see low scores. On the other hand, sentences like Arnold Schwarzenegger played in the movie Terminator. should score close to 1.
    A problem with this approach is that we’d need a really large training set to train our classifier well, and who has the time to label 10k+ sentences those days? Fortunately, we can use a neat trick called distant supervision. Interested? Read about it in this paper.
  • Mining for graph patterns. In real life, we know that if Peter is the father of John and Kate is John’s mother, it’s pretty likely that Peter and Kate might be married. So, if our knowledge base contains the facts Peter is the father of John and Kate is the mother of John, then if the knowledge base doesn’t say that Peter is married to Kate, we could expect that to be an error of omission. On the other hand, if we add the fact that Peter is married to Marie, that counts as evidence against Peter being married to Kate.
    There are algorithms that look for such patterns (and more complex ones) in the incomplete knowledge graph and then use these patterns on the same graph to assign likelihoods to missing relations. One is called PRA (Path Ranking Algorithm), another one SFE (Subgraph Feature Extraction). Matt Gardner has an implementation of both.
  • Embeddings. This means that we invent a space with, say, 50 dimensions, and somehow represent entities and relations within that space. We choose this representation so that the embedding then informs us about which relations might be true, but missing in the knowledge graph.
    For example, say we represent people as points in a 2D plane, and we represent the is mother of relation as a step to the right and up, and the is father of relation as a step to the right and down. Of course, you can’t really represent all the complexity of family relationships this way, but if you tried to get as close as you could, you’d end up with a figure where two people who are siblings would probably end up close together. Behold: embedding according to is mother of and is father of just told us something about who is sibling of who!

In my thesis, I plan to replicate the architecture of Google’s Knowledge Vault (paper, wiki).

Google created Knowledge Vault, because they wanted to build a knowledge graph even bigger than The Knowledge Graph.

The construction of Knowledge Vault takes Knowledge Graph as its input, and uses three different algorithms to infer probabilities for new relations. One of these algorithms extracts new relations from webpages. The second one uses PRA to predict new edges from graph patterns. The third one learns an embedding of Knowledge Graph and predicts new relations from this embedding.

Each of these different approaches yields its own probability estimates for new facts. The final step is training a new classifier that takes these estimates and merges them into one unified probability estimate.

Finally, you take all the predicted relations and their probability estimates, you store them, and you have your own Knowledge Vault. Unlike the input knowledge graph, this output is probabilistic: for each Subject, Relation, Object triple, we also store the estimated probability of that triple being true. The output is much larger than the input graph, because it needs to store many edges that weren’t in the original knowledge graph.

Why is this useful? Because the indiviual algorithms (extraction from text, graph pattern mining and embeddings) have complementary strengths and weaknesses, so combining them gets you a system that can predict more facts.

Simplified Knowledge Vault architecture

My system is open-source and extends Wikidata. The repository is at https://github.com/MichalPokorny/master.

So far, I have been setting up my infrastructure. A week back, I finally got the first version of my pipeline, with the stupidest algorithm and the smallest data I could use, running and predicting end-to-end!

The system generated 116 predictions with an estimated probability higher than 0.5. Samples include:

Subject Relation Object Probability Is this fact true?
Northumberland occupation Film director 0.5913 False
mathematician occupation Film director 0.6201 False
Jacob Zuma member of political party Zulu people 0.5159 False
Mehmed VI country of citizenship Ottoman Empire 0.5479 True
Brian Baker country of citizenship Australia 0.5523 False
swimming occupation Film director 0.6229 False
West Virginia member of political party Tennessee 0.5107 False
Tamim bin Hamad Al Thani country of citizenship Princely Family of Liechtenstein 0.5289 False
Sheldon Whitehouse member of political party Washington, D.C. 0.5086 False
Liberation Tigers of Tamil Eelam country of citizenship United States 0.5349 False
Italian Communist Party country of citizenship United States 0.5545 False
Henri Matisse country of citizenship French 0.5004 False
Lawrence Ferlinghetti country of citizenship United States 0.5471 True
Michael Andersson occupation Film director 0.6283 False
Brian Baker country of citizenship Canada 0.5187 False
John E. Sweeney member of political party Republican Party 0.5036 True

Okay sooo… not super impressive, but pretty good for a first shot. At least it does a bit better than rolling a bunch of dice :)

It’s basically a logistic regression over a bag of words. The dataset are 10 000 Wikipedia articles about persons. My task now is to use smarter algorithms to get better results, adding other algorithms (graph patterns and embeddings), running over a larger dataset and fleshing out the architecture.

I’d enjoy talking at length about the various design choices and cool tools I’m using, but I was told 1 A4 page would be quite enough, so I’ll cut my proselytization short just about now.

Let’s get back to work. Have a fine day, and may your values be optimally satisfied!

2016-09-22 - Tracking thesis progress

I’m currently writing my master’s thesis. I’d like to have some idea of how to answer questions like: How well am I doing this week? Have I been stalled for some time? How fast can I write when I’m in the zone? How long until I’m finished?

To help me answer those, I’m gathering data about it using a cron script that logs the number of pages of my the PDF of my thesis and the number of lines of code I’ve written every hour.

This is a technique I copied from a video from the excellent screencast series Destroy All Software. The author uses it in several videos. One variant of this data collection idea he uses is going over the Git history of a project and collecting data ex post by checking out every historic revision and running a data collection script over it.

My use of this technique lets me produce a pretty graph that looks like this:

Other really nice uses of this technique include asking:

  • How many tests were failing in each revision?
  • Which authors commited how many lines of code, over time?
  • How has the speed of a POST request to /clients/create evolved over time? Which changes affected it positively and negatively?
  • How has the speed of my website’s rendering evolved over time?

My crontab line looks like this:

0 * * * * log-pagecount

The log-pagecount script in question is dead simple:

set -e

cd /home/prvak/master/text
make >/dev/null
COUNT=`pdfinfo /home/prvak/master/text/thesis.pdf | grep Pages: | cut -d\  -f11`
DATE=`date +%Y%m%d%H%M`
echo $DATE $COUNT >> /home/prvak/misc/master-pagecount-log.log

echo "---- CUT ----" >> $SLOC_LOGFILE
cloc /home/prvak/master/code >> $SLOC_LOGFILE

The lines-of-code counting is done by the cloc tool (AlDanial/cloc on GitHub). In Arch Linux, you can install it by running pacman -S cloc.

Then, in ~/misc/sloc-count-to-log.py, I have a simple script that changes the format of the lines-of-count logfile:

with open('master-sloc.tsv', 'w') as f:
    for line in open('master-sloc-log.log'):
        line = line.strip()
        if line.startswith('SUM'):
            f.write("%s %s\n" % (date, line.split(' ')[-1]))
        if line.startswith('2016'):
            date = line

Then, I have this Gnuplot script in ~/misc/plot-thesis-stats.gnuplot:

set datafile separator ' '
set xdata time
set y2tics
set timefmt "%Y%m%d%H%M"
set key right bottom
plot \
	"master-pagecount-log.log" u 1:2 w lines axes x1y1 title "Pages", \
	"master-sloc.tsv" u 1:2 w lines axes x1y2 title "SLOC"

To see the pretty graph, I do this:

~ $ cd ~/misc
~/misc $ gnuplot
gnuplot> load 'plot-thesis-stats.gnuplot'

I have enjoyed procrastinating by writing this post more than looking at my script munching away training samples for my classifiers.


2016-09-19 - Trip to Chernobyl

I went to visit Ukraine and the Chernobyl exclusion zone in particular with some old and some new friends. It was awesome and I can wholeheartedly recommend the folks from Expedition Club, who kindly arranged the trip for us. I’ve wanted to do this way back since I first played S.T.A.L.K.E.R., probably about 9 or so years ago.

I have a bunch of photos on my Facebook.

Aside from Chernobyl, we also spent 2 days in Kiev. I didn’t enjoy the atmosphere and architecture too much - it’s full of socialist high-rises. Not saying Prague isn’t, but Kiev looks like 90% of it was built in the 80’s. The Statue of Mother Ukraine, which is slightly larger than the Statue of Liberty, has a pretty good museum of WWII inside. They also show off army vehicles (mostly tanks, and at least one helicopter) nearby. The statue and its surroundings are a pinnacle of socialist realism.

Then, we spent 2 days in the exclusion zone, and after that, we went to Neptun, which is a boat-turned-fishing base/hostel/restaurant in the delta of the Dnepr. The way there was really adventurous: let me tell you, you don’t know bad roads until you see the villages of Ukraine. When we finally arrived to the riverbank, a speedboat from Neptun took us and our things for the night and we sped towards Neptun. The ride there around all the tiny islands of the Dnepr delta under the moonlight was amazing. The next day, we said goodbye to Neptun and all its fuzzy kitties, doggies and puppies and headed back to Kiev.

On the way back to Kiev, our gearbox decided to kick the bucket - shifting gears was an ordeal for Ondra. Accidentally, we made a wrong turn at one point and headed toward a military facility and got stopped at a checkpoint. All right, let’s turn around and head back. Only the gearbox decided to not let us shift into reverse. So, we stood in front of the gate for like 20 minutes, Ondra frantically trying to shift to reverse and the guard getting more and more amused. Slowly, more and more soldiers from the checkpoint looked at the spectacle, until one hero went and helped us shift the gears, letting us leave a full squad of ~15 laughing servicemen. Fun times!

On our second day in Kiev (after Chernobyl and Neptun), we ziplined from one end of Dnepr to the other one, which was fun, if somewhat shorter than expected. We also saw some old churches, the Majdan and so forth.

More experiences: Ukraine is crazy cheap, which I’m saying having spent most of my life in Czech Republic, already a pretty cheap country. Once you figure out the alphabet, the language is very easy to read. Speaking in Czecho-Slovako-Polish is also a good supplement for English.

Radiation exposure

We couldn’t agree about how large the dose you get in Chernobyl actually is, so I decided to do a little bit of research. The second day of our visit in the Zone, I wrote down the value of the dosimeter roughly every 10 minutes plus every now and then.

Time μSv/hour Notes
08:47 0.15
09:05 0.21
09:26 0.21
09:47 0.48
09:57 0.30
10:08 0.18
10:29 0.32
10:38 0.18
10:48 0.30
10:53 1.00 Entering campground
10:57 1.24
11:14 1.53 Couldn’t find Petra, who had the dozimeter :)
11:18 0.95
11:22 0.95
11:28 0.27
11:35 0.44
11:39 0.32
11:42 21.7 Measured in hot spot around school by the village Kopachi.
11:43 2.65
11:49 0.91
11:51 6.79 Driving through Red Forest
11:52 12.6
12:00 0.30 Hospital
12:04 118 Measurement of highly radioactive glove in hospital (77 μSv/h a bit further by) - some idiots went to the basement where they kept and buried the first-response firemen and used the glove to hold one boot of the firemen. And then left the glove upstairs. And the glove is itself dangerously radioactive. Didn’t go nearby.
12:10 0.20
12:20 0.20
12:25 1.10
12:26 3.06 Small hot pocket by Pripyat river shipyard
12:26 3.06 Small hot pocket by Pripyat river shipyard
12:28 0.96
12:28 ~15 Random radioactive piece of wood our guide pointed out. (“Don’t step on this.”) Didn’t go nearby.
12:39 1.40
12:50 0.14
12:59 0.70
13:09 0.31
13:16 0.87
13:17 1.06
13:20 0.84
13:25 0.74
13:30 0.22
13:42 0.35
13:45 0.47 ~2 minute drive through Red Forest; next few entries show range of fluctuations.
13:46 2.68
13:47 1.58
13:49 0.27 Out of Red Forest
14:00 0.15 Lunch
14:24 0.27
14:33 0.35
14:35 58.8 Random piece of graphite from the cooling rods, laying in grass by unfinished cooling tower.
14:41 2.5 Radiation in unfinished cooling tower. It wasn’t yet sanitated, so the background radiation level is much higher than elsewhere.
14:43 3.40
14:54 0.39 Out of cooling tower
15:04 0.95
15:15 0.31
15:27 0.21 Coming back to Chernobyl town
15:40 0.15 Chernobyl town
15:51 0.15

The background radiation level in normal cities is (ballparking it) about 0.12 μSv/h. My idea was I would just look at the dosimeter every 10 minutes and then approximate the dose we received by representing each 10 minute interval by the value I measured by its end.

After seeing what the dozimeter did during the short ride through the Red Forest, I think that might be the wrong approach to go about it – the dose you receive in Chernobyl might not be dominated by the elevated level of background radiation (which is only 2×-4× higher than in uncontaminated areas, from my data) - the more important factor might be very short intervals spent in radiation hot pockets, which would be hard to pick up by 10-minute measurements. Look at the Red Forest ride: during it, the radiation levels were 20-30× the normal levels.

Next time, I’ll just bring my own dosimeter. With logging, and blackjack and hookers.

2016-07-14 - Summer

So, what have I been doing since last time?

Effective altruism in Czech Republic

I decided I want more effective altruism in general, and also in Czech Republic in particular. Me, Jirka Nádvorník, Radim Lacina and Dan Hnyk started working on organizing a conference on effective altruism in Prague, probably some time in fall. The work is fulfilling and interesting: I’ve never done anything similar and I think I’ll learn a lot from this. Our current idea is a weekend with 1 day of events aimed at educating the broader public about EA and related crucial issues, then 1 day of an “unconference” mostly aimed at community building.

Unfortunately, progress has been pretty slow so far.

Our main priority is securing a venue, then fixing a date. We quickly agreed on a sweet venue in Prague we’d really like to have (Paralelní polis), but unfortunately communication with them has been sluggish. I think we didn’t dedicate enough attention to this - we should have contacted potential venues sooner and more broadly. We’ll fix that.

We’d like to talk with the organizers of EA Global to see if we could make our conference an official EAGx event, but communication has been slow. We’re (so far unsuccessfully) trying to reach anyone from LEAN (Local Effective Altruism Network), which is the org behind Effective Altruism Hub. It seems they tried to start a small Czech online EA community, but it didn’t take off and it seems inactive now, so we’d like to see what could we do about that.

We’d like to use the conference to bootstrap the Czech EA community: have EAs know each other and talk to each other, regular meetups, materials in Czech, talking to Czech influencers and media. We are working on establishing a small non-profit. We currently just have a bare-bones site at efektivni-altruismus.cz.


I talked with my psychologist a few days after publishing Recent icky personal stuff. The session was basically just my huge mind-dump, maybe just a bit more verbose than that public post. There was relatively little back-and-forth - I mostly gave a long, prepared spiel and got just a few questions. I was a bit surprised by what in particular interested my psychologist: my ideas about polyamory and my experiences with recreational drugs. Discussing both made me a bit anxious. It’s probably nothing to analyze, just something to expect when dumping personal stuff on strangers.

The next step is a personality test, which I’m taking a few days from now. It should take about 4 hours. I was wondering what’s taking so long to test, and there’s an interesting reason the test is so long: it’s supposed to be tedious, because when you’re tired, your answers are less biased (e.g., you put up less effort to look like a nice person).

I was mostly well the last month. I felt sad a few days and cried a few times, because of my own anxieties, because I worry about everything, because sometimes I feel I have no idea what I’m doing. Mostly around personal relationships. Working on it.

Actual work

I apologized to my thesis advisor for not doing any work yet, explained I have been depressed, and promised to send him a draft text for my thesis assignment the next day. I still didn’t do that, and it was some weeks ago. That was all I did about finishing my degree. I sometimes feel a bit bad about not making any progress. Last year, I had hoped I would be finished by now.

I didn’t get the H-1B visa, so I’m not moving to Mountain View this year. I’ll probably have the option of working in some office in Europe. I’m currently optimistic about the prospect. If I want to finish my master’s, employment would probably have to wait until February 2017.

Many questions I need to answer, many things I want to do, many tradeoffs I’d rather not have to decide.


I was in Rome for a week. I’m not usually one to highly value museums and historical buildings, but I really enjoyed Rome’s antiquities. People in Rome have been building the city since ~900 BC to the modern day. Everywhere you dig, you’ll eventually find something Etruscan or Roman or at least medieval. After you make it through all the Renaissance built on top of that. The remains are aesthetically nice, but what I enjoyed more was a feeling of frisson I got from the inscriptions.

Take a look at this one. (Quoted from the Colloseum website.)

Decius Marius Venantius Basilius v(ir) c(larissimus) et inl(ustris) praef(ectus) urb(i) patricius consul ordinarius arenam et podium quae abominandi terrae motus ruina prostravit sum(p)tu proprio restituit.
Decius Marius Venantius Basilius, very famous and well known prefect of the city, patrician, regular consul, repaired the arena and the podium which a dreaded movement of the earth had cast into ruin, at his own expense.

What gives me frisson is this thought:

Once upon a time, there lived a man called Decius Marius Venantius Basilius. He was an actual human being, like you and me. There were probably people he hated, and people he loved. He had friends, he had hobbies, sometimes he felt good and sometimes he felt bad. And, 1500 years later, we know he was there. There was a horrible earthquake. And Decius Marius Venantius Basilius went and had it repaired at his own expense.

History is not finished. We’re writing it every day with our actions.


I’m slacking off a bit with running and jiu-jitsu. Win: I’ll be running my first 10k in September. Fail: I mostly stopped going to jiu-jitsu the last 2-or-so weeks. I’ll try to pick up the slack.

I’m visiting Vienna next weekend to see their local rationality meetups, talk to a few people who might be interested in our EA conference, and to generally hang out.

In September, I’m going with a group of friends (mostly from high school) for a trip around Ukraine, mostly to see the Chernobyl exclusion zone. They’re putting a new sarcophagus on the NPP this year, so this is my last opportunity to see it the way I remember it from STALKER :)

2016-06-18 - Recent icky personal stuff

So far, 2016 has been a pretty interesting year of me. I discovered a lot about myself and I had both the happiest and saddest time in years. I hope you like icky personal stuff, because this post will be unusually full of it.

The first time in my life, I had a crush on someone and I told them about it. My previous method of dealing with love was mostly hoping that maybe senpai will notice me. That, of course, didn’t ever work. I don’t claim rationality.

I was lucky and a relationship followed for a few months. I felt very much in love. But there were problems that weren’t properly dealt with and I didn’t pay nearly enough attention to. And so, suddenly, poof, no relationship no more.

I had depression for the following ~4 weeks. (I concluded this was actual depression from my results on the Beck Depression Inventory.) At first, I mourned losing someone I loved. After I first successfully applied CBT on myself (with the help of talking it over with others), I wrote Updating. You can read that with X set to something like He loves me.

I noticed that the bad thoughts I have been thinking were familiar - I have thought some variations of them many times in the past. A hard one to verbalize feels like a searing mental pain from being unloved, or from being rejected. Some were variations on Nobody likes me and I have no real friends. These bring back painful memories of the times people have been mean to me, mostly in my childhood.

Then, I looked very hard for answers, probably thinking maybe there was some way I could fix whatever broke the relationship, or at least have it not happen again.

I realized that this was probably a textbook dependent relationship on my part. I sincerely felt that this totally best person was the center of my world. I derived my feeling of satisfaction with my life from the relationship.

I came to the conclusion that a spiral of my dysfunctions played a large role in the breakdown of the relationship. It starts with one tiny problem that makes me feel a little bit insecure, because it looks like a threat to a relationship that’s the center of my world. One way I deal with insecurities is emotionally detaching, which creates communication problems. Communication problems lead to more problems, which make me feel more insecure. And more detached. And when I’m detached, I can completely miss emotional clues. Or the fact that things are not going well and that the current trajectory leads downwards. Which might be why the breakup surprised me so.

I turned my attention inward.

Two young fish are swimming in a fish tank. An older fish comes by and asks: “Morning, boys, how’s the water?” The two young fish swim on for a bit, and then one looks at the other and goes, “What the hell is water?”

On proper reflection, I have a lot of issues. When your issues have been with you long enough, you don’t see them floating in the corner of your vision. The way I saw them was looking backward. Once I knew what to look for in my personal history, I saw them at work undoing my happiness all over the place. Noticing the way you think is harder than it sounds. I’m working on spotting my issues at work more readily so I can course-correct faster.

Why am I writing this? The true reason is just because I want to, it feels good to talk about myself. I also have a few rationalizations.

A: I did well for myself in school and in work and some people look up to me, admiring my abilities in CS and math. And yet, I, the guy who can program his way out of anywhere and who never got a C on an exam at uni, have problems. This sends a message: nobody’s perfect, everyone has problems and it’s OK to talk about them.

B: Maybe you’ll recognize someone you know or maybe even yourself. Maybe me talking about things I’m working on will help you see opportunities for your own personal development.

Yay, issues!

Social anxiety

One symptom I call (for now) social anxiety. It doesn’t usually manifest as actually feeling anxious when dealing with strangers. I just avoid doing anything that means dealing with strangers. Say, even something as stupid as buying new pants. I never actually think OK, I need to avoid buying new pants because I don’t want to deal with people. I just never go buy new pants. Ask me why? I’ll probably whip up a rationalization.


Another one is the detachment I mentioned earlier. I think I developed that in middle school. I had a crush on a girl and I couldn’t for the love of god bring myself to do anything about it. Then, by a coincidence of my stupidity and my classmates’ nosiness, my entire class found out I liked boy parts. I haven’t been very well-educated about sexuality by then, so my thought process was mostly OK, now everyone thinks/knows I’m gay. That’s a bad thing. People think badly about me because of that. Nobody can find out, especially not my parents. If people found out, I could never be happy again. Also, now since everyone thinks I’m gay, any chance I ever had of being with the girl I love is ruined beyond repair. (Of course, probably not a single one of these statements was actually true.) So, I felt I couldn’t be helped by anyone, so I didn’t talk to anyone. And, maybe to prevent my parents from finding out how sad and vulnerable I felt (because that would only harm me further), I started acting like a tough, emotionless, hostile robot. If you see me spitting venom with an empty expression on my face, it probably means I feel threatened on some level. (It just so happens Spitting Venom by Modest Mouse is one of my favorite songs. Synchronicity.)

Fig. 1a: Me, uncertain about the future and detached:

Fig. 1b: Compare to happy me a few weeks later:

I think the most common reason I go detached is that I feel I’m being left out or ignored for some reason.

Aside: In the end, I think the aforementioned thought process is what eventually led me to believe I was gay, which I still believed in 2015. But, on proper reflection, while I seem to prefer man parts, the fact of the matter is that there are men I like and there are women I like. Would I say I’m bisexual? Maybe it’s the right word to describe my experience. But I no longer think a label on my sexuality is an important part of my identity. (Yet I still exclaim that in a public post. I like showing off. Shoot me.)


I think I placed too much value on my work even before high school. Even on my first small side-gigs I did for random people, I put in very long hours for peanuts and I tried to make everything perfect. I also always liked learning scientific stuff. In high school, I took a lot of electives and worked more side jobs.

By the end of high school, I think I got the idea of becoming a Successful Software Engineer, which I worked towards up until now.

In my university, you basically get a free reign over your schedule. I got to work and optimized my first-year and second-year schedules. To the extent that my schedule looked like a huge rectangle all over every day of the entire workweek. (I even wrote a schedule optimizer to optimize my schedule for me. Hello, Twilight Sparkle.)

A 3-year bachelor’s costs you 180 credits. After 2 years, I had 211.

On top of that, I kept on taking side jobs, and in my second year, I started my internship at Prizeo. My schedule slipped and I ended up going to work as one of the first people and leaving very late. I worked on useful features, yes. But I actually spent huge chunks of my time doing useless busywork I created for myself in retrospect. Like fixing bugs nobody cared about, refactoring the packing optimizer the 8th time, changing spaces for tabs. Embarassingly enough, I also looked a lot at GitHub’s contribution charts. I was almost always first in number of commits and pull requests. I know, right? If I remember correctly, I probably had a vague expectation of recognition. Did I realize there was anything pathological about what I was doing? Nope.

I also started tracking my net worth in a graph, which I looked at several times per day. It’s nice to see your score go up, isn’t it?

At Google in Paris, it got even more out of control. I almost never got around to doing anything I liked outside of work, so I spent my time at work morning to night. I got a slight twinge of satisfaction when I got a peer bonus for working so well.

In general, I think most of the work I did I didn’t do for money to do things with. That was just a side-effect. It was probably just so I could have less free time alone to fill with feelings of sadness and loneliness.


That might have been my first unhealthy (unhealthiest, too) coping mechanism. I’ve had problems with overeating since the first years of primary school. Food, especially sugar, is satisfying and numbing, and it feels better to eat than to think about how everyone hates you for being fat (and a smart-ass).

And so on

Very low impulsivity: I often am just about to do something or say something, but stop myself before I do. Probably something about fear of criticism or rejection. I also think I often act not the way I want to or the way I am, but the way I think other people expect me to act. There’s an interesting concept called true self and false self on Wiki.

Scarcely having fun on purpose: Before 2016, I almost never did anything fun myself because I wanted to. Lunch in a nice restaurant? A walk in a park on a sunny day? This easy fun thing I had on my bucket list for ages? Nope, let’s just browse Reddit all day. I seem to act as if I were incapable of having any fun on my own, so I end up having no fun on my own, which undermines my self-esteem.

“I am unacceptable”: I act as if I can’t open myself up to other people, because then they would hate me or reject me. Before I came out as gay, I thought something like “if people knew I’m gay, they’d hate me, so I must be careful so that nobody can tell”. After that, there’s always a ton of irrelevant bullshit my brain can substitute. “Hey, remember stupid embarassing thing #71? I bet everyone would shun you if they knew.” The consequence is problems opening up to other people, which sucks. I even do that with close friends, family, and romantic interests or partners.

“Other people don’t like me”: thinking that is an easy way to get people not to like you. Sample intermediate steps: “Huh, this random sentence she just said could be possibly interpreted as criticism/teasing/… . She doesn’t like me, so it indeed is, and that’s a purposeful attack on me.” A: “Let’s fight back.”, B: “Let’s be passive-agressive to her now.”, … “I have no real friends” is another thought that can ruin my enjoyment of any activity, especially one I’m enjoying with my friends.

Bad self-image: I think my idea of my looks and fitness developed in high school at the height of obesity, and I didn’t yet properly update after fixing that.

Abuse of recreational drugs: I have used marihuana, and I think at least some of it was for the numbing effect.

“I need a relationship to be happy”: This one undermines happiness and self-worth when you think it while single.

I read that “I have wasted my life” is also frequent and destructive. The Near Future is practically dripping with it.

I like helping people. In high school, I got a ton of satisfaction when I wrote exercises on complex numbers for Khan Academy, which helped a lot of people learn, and I dreamt of working there one day. I might have a tendency to attach myself to people I can help. I have strong feelings about effective altruism. “Helper’s syndrome” might be part of my mix.

Finally, I easily get jealous when I perceive someone as being more successful than me, when it comes to relationships, friendships and doing fun things.

What now

Save a day or two here and there, I have been mostly OK for the last few weeks. I’m seeing a psychologist in a few days to get some professional advice and possibly treatment. (Since I have been reading Feeling Good just before I became full-blown depressed and reading more psych stuff afterwards, there’s some possibility I’m just imagining most of it and being a hypochondriac with confirmation bias. We shall see.) There’s some serious self-improvement I’m working on.

I’m purposefully organizing my free time to maximize fun. I’m using a Pleasure-predicting Sheet, which is a technique from Feeling Good by David Burns. (It’s intended to build confidence that you don’t need other people to have fun, but I also use it to force myself to have fun on purpose.) I push against my social anxiety by doing more people things and seeing that people bite way less than my stupid lizard brain thinks. I took on running and jiu-jitsu - sports are supposed to help with depression. My self-esteem is particularly low when it comes to my looks and my fitness and running a 5k in 26:30 helps with that. I use Zombies, Run. Talking also helps.

My original plan for this year was finishing a thesis before October, grabbing my master’s, and then working for Google as a software engineer (I got an offer).

Stupidly (and workaholically), I spent the winter semester working on Factorify instead of on my thesis. In the summer semester, I had my failed enmeshed romance, and then my depression. Now, I mostly don’t work on it. I usually procrastinate or start getting bad feelings after an hour or two. I make a very small step every week or two. If I pushed myself, I might be able to finish it this year, but it’s not very likely. I’m also no longer so sure I want the degree enough.

I’m ~99% certain I didn’t get the H-1b visa for Google Mountain View, so it’ll probably come down to deciding whether would I want to work for them in Europe, and if so, where and when. When I’m in a good mood, I think Zurich or London would be nice. But I worry that might be just a cached dream of the career-obsessed past-me and that moving would just put me under more stress than I can safely handle. On the other hand, making software used to make me happy before. Would I be happier on a year-long vacation without any programming? One question I’ll have to reconsider as well is: Do I actually want to work in software?

I think “Yes”. I have at least one other idea. Do I want to work in AI safety? The question of the importance of AGI has been lingering in my mind for months. It’s confusing to my intuition, which is, of course, not a LessWrong-rational argument. And I don’t (yet?) defer to LessWrong-rationality when confused. I applied to MIRI’s 2016 Summer Fellowship, mostly hoping it would convince me either way, but I didn’t make the cut. (My impression is that I didn’t have enough research and math credentials, which means there’s probably someone who has more of those than me working on AI safety now, which is good news.) Before my depression, I was slowly reading up on MIRI material and I was hoping of getting more information and meeting AGI researchers once I got to Bay Area. Alas, no Bay Area for me, and since my depressed brain is vulnerable to the issue, I’m putting it on hold.

For now, I’m leaving the question of my future open until it comes knocking. I don’t like to think about it too hard. Wherever I go (or not go) next, I want to feel comfortable in my skin.

Thank you for reading. I thrive on your attention.

2016-05-05 - JVM Tool Interface

At school, I gave a short, roughly 15-minute talk on JVM Tool Interface. It’s a part of the JVM that people use to build debuggers, profilers, and similar tools that affect or observe the running JVM. I focused mostly on the interesting technical parts: the architecture, how do they make the JVM run fast while being instrumented, how they deal with native code, and such.

2016-05-01 - Updating

When I first read Yudkowsky’s other magnum opus Rationality - AI to Zombies, I didn’t really understand why was he so obsessed with looking for the truth, attacking false beliefs with huge flaming swords and why he made such a huge deal out of letting go.

I think I understand now.

What is it like to have a precious false belief?

There are no huge neon signs saying “YOU’RE WRONG AND YOU SHOULD STOP BEING WRONG”.

When nothing you see goes against your precious belief, you may take pleasure in thinking about your cherished belief about and how beautiful the world, as seen by you, is.

Holding a belief does not feel like thinking “I believe X”. Holding a belief feels like X is actually the case and acting upon that assumption.

When you see things that go against your cherished belief X, you don’t go “I’m confused. Let’s do some rationality. If X, I want to believe X. If not X, I want to believe not X. Let’s collect evidence and figure out what’s going on here.”

When you see evidence against your cherished belief X, it hurts to look at. You try not to confront it for what it is. You close your eyes and think it will go away. You think you can’t even stand to think that maybe not X. You cry and scream and beg for help, because it feels like X is what the whole world depends on. You grasp for straws, look into every corner, because you want X so much, because you hope you will find evidence for X if you want it enough.

“Hell is not other people. Hell is yourself.” – Ludwig Wittgenstein

Indeed. What hurts is not the way the world outside your skull is. What hurts is your thoughts that X just must be the case, that X is so good, I want X so much, why can’t I have it, everything depends on it, please let me have X! States of the world are not sad. Emotion comes from your cognitions. When you think thoughts that hurt, you can torture yourself very efficiently. Quite possibly, most of the pain in my life actually came from within.

It’s quite likely the world will refuse to bend itself to your will, even if you think X is the most important thing in your world.

I think there are, broadly, two ways to approach this.

One way is to just refuse to think not X and to keep torturing yourself, because every time you think about anything that reminds you in any way of your cherished belief X, you will only be reminded about all the clues that not X, and through you can try all you want to cover up all the huge billboards saying “NOT X” because they’re too painful to look at, they will stay there.

The first approach will only bring you more pain.

The second approach is to look the truth in its eyes. Think “not X”. Not “but I want X so so much, it’s the most important thing in the world, please, let X be the case, let me wake up and see that I imagined all the evidence, it’s so painful, please help me”.

Don’t flinch or look away.

Not X.

And not X is not sad.

Sadness is an emotion. Not X is an objective proposition about the world.

Now, why would you feel sad about not X? Maybe you want X. But is feeling sad helping you bring X about?

Is refusing to accept not X helping anything you care about in any way? Or is it only causing you more pain than the fact that not-X itself? Do you want to be miserable, or do you want to be serene and happy? If X were not the case, would you want to be miserable, or would you want to be serene and happy, so maybe you could do something about not X?

Is your belief I want X so much and I can’t be happy unless X so important that if not X were the case, you would rather keep believing you can’t be happy unless X and be miserable every time you thought about it?

So I updated. It felt like the hardest thing in the world, but it was well worth the effort. I now accept that not X. Not X. Not X is not the end of the world. Being sad about not X would not help anything I care about, so I’m not sad. I think this has something to do with the rationalist meme of letting go.

I’m not certain that my way of dealing with evidence that not X fully generalizes to any problems you might face. I just know I really really thought, I believed I could not accept that not X and that not X would hurt. But when I stopped fighting the truth, it ended up being not as painful to accept as I thought it would be. Coming to terms with it was certainly much better than the alternative for me.

And the next time I think I can’t bear to look at Y because Y would be so painful and because Y is more important to me than anything, I will remember the last time I thought I depended on something. I will remember how absolutely central to everything X seemed, and how fighting against not X made me cry for hours. I will remember how empowering it was to stop fighting, and how stupid it was in retrospect to try to hide the truth from myself. I will remember that I thought I could never be happy about anything in the world again if not X, and that the actual enemy of my happiness was not not X, but my sadness about not X.

Not X and it’s OK. The world is still a pretty neat place.

Interesting. When I decided to start writing every now and then last January, I thought I’d mostly have things to say about technology.

2016-04-07 - EA pledge for 2017

The universe does not love us.

If it did, it probably would not have us suffering and dying all the time.

Okay. Do we have a problem with this?

I guess we don’t. It’s just a fact of life that you will be hurt. And of course, you have to die. Your family and friends and everyone you care about? Yes, they have to be in pain. If you’re lucky, it will be just every now and then. Do they also have to die? Oh, yes, they do. Possibly, painfully so. And let me tell you about, say, neurodegenerative diseases. When your best friend gets one, she will spend several last years of her life with her personality and memories wasting away, and frightened to death. And at the end, she will, of course, die. And when she does, you will never see her again.

But that’s OK. It’s a fact of life, so it’s OK. It’s not like you can really do anything about it, so it would be useless to try. Hey, look at this new shiny widget! Isn’t it really shiny? Come on, you already feel like everything is OK, right?


I will no longer allow this.

I can’t protect everyone. I am just a human and I don’t have the ability. But I promise that if I could protect everyone, I would not allow a single person to die or suffer. I want everyone to live as happily as possible and as long as they want to, and I want nothing to prevent people from enjoying life.

I can’t protect everyone, but I can try my best.

I promise to give at least 10% of my pre-tax income between October 1, 2016 and October 1, 2017 to fighting the Arch-Enemy.

I am sorry I am not yet strong enough to commit more. I hope to become stronger one day.

We will fight. If you’d like to join us, we would be honored to fight by your side. We are called effective altruists. Some of us fight diseases that kill in the developing world. Some of us fight to save animals. Some us fight to end death itself. All of us fight to remove suffering from the world.

The philosopher Peter Singer did an excellent TED talk on the effective altruism and you should watch it:

Ben Kuhn has a good list of effective altruism reading material. I also personally recommend Nick Bostrom’s Fable of the Dragon-Tyrant (Czech translation).

If you think there’s any chance it might a good idea to try to do good, please don’t dismiss it easily. If you’d like to learn more or if you’d like to talk with me about this, I would be happy to talk.

2016-02-02 - HackCambridge 2016

Last weekend, I went to HackCambridge with Petr to hack on things. Some things could be organized a bit better (like the WiFi at the main venue being hilariously broken), but we ended up having a lot of fun. Luckily, a bunch of my homies from Dropbox also happen to study in Cambridge and it was great to catch up with them.

We built a cool game. It’s called The Deadlock Empire and it lets you slay dragons and learn concurrency - what could me more awesome?!?!

Just look at this presentation!

It’s at https://deadlockempire.github.io. You can laugh at our poor coding practices on GitHub (deadlockempire/deadlockempire.github.io).

The Deadlock Empire has a sequence of challenges for practicing concurrent thinking. Each challenge gives you several threads, each running their own source code in C#, and your objective is to play the mischievous Scheduler: take seemingly-innocent parallel code and run it so that it breaks.

You can step each thread line-by-line (and undo your steps), and some non-atomic instructions can be expanded into atomic parts (i.e., i = i + 1 translates to a separate load and store). Sometimes, you have to do this to properly crack the challenges, like in Tutorial 2: Non-Atomic Instructions.

If you feel bored by how simple the first challenges are, scroll down a bit. The Deadlock Empire quest at the bottom should keep you busy for a while :)

I think it took us around 18 hours of actual work to write The Deadlock Empire: both of us had a bit of shut-eye early morning and we didn’t have enough time by the end to start working on the really ambitious features, so the last few hours were just a commit here and there, fixing up a font or an obscure bug.

There aren’t many shiny technologies: it’s plain HTML5 and JavaScript with jQuery, Bootstrap, and a few other things. The game has a simple “internal virtual machine” that executes simple instructions, like “increment this global variable”, “lock this lock and block this thread”, and so on. We used pseudo-C# as the challenge language mostly because of our familiarity with its ideas about concurrency, but it would be straightforward to write in pseudo-Java instead.

In the upcoming days, we’ll add more synchronization primitives and polish the experience. We want this game to be more fun, educational and useful.

I now have a favorite bookstore. Next door from their regular store, Cambridge University Press sells their damaged books. (Even through the only damage I found on them was the red ‘DAMAGED’ stamp at the front page.) Awesome books on advanced mathematics and CS, philosophy, sociology, biology. For 3 pounds a paperback or 7 pounds a hardcover. People grab them by the stack. Note to self: bring big suitcase next time.



We also won the Best Hack in Gaming/VR award (judged by Improbable) and Bloomberg’s Favorite Hack.

2016-01-10 - Stock correlation calculator

I have some stocks and I try to keep my portfolio reasonably balanced. One thing you may want to know about is how much do your assets correlate. The correlation of stocks A and B is just the correlation of the daily returns of A with the daily returns of B, or, in other words, how much do the two stocks tend to move in the same daily direction.

Usually, high correlation means that the two stocks are affected by the same type of news. For example, you could reasonably expect AAPL (Apple) and GOOG (Google) to be highly correlated, because both are technology mamooths.

One reason why you might want to keep the correlation of the stocks in your portfolio as low as possible is safety against bad news. If China bans computers, everything related to computers goes out of the window, but e.g. manufacturers of chemicals might remain more or less unaffected.

There are some online tools that can calculate the correlation of stocks, but I didn’t like any of these enough. Some of them only let you compare two stocks at a time, but I actually want to see the whole correlation matrix of everything I have. Some only let you make N queries and then they tell you to go give them your money. So, I wrote my own. It’s at https://stockton-prvak.rhcloud.com.

It prints correlation matrices. I also played around with visualizing the stocks as “planets” that are attracted or repulsed based on their correlation. The visualization is still pretty buggy, but fun to look at. If you’d like to see how I did this, have a look at https://github.com/MichalPokorny/stockton.

You can find more in the archive.