Introduction
The other day I attended the closing ceremony for the FY2023 round of IPA’s Mitou program (the Exploratory IT Human Resources Project).
They gave me a very grand plaque
The details of what I did in Mitou are in the project overview (in Japanese) and my final presentation (in Japanese), but in short, I was developing Erg, a programming language that transpiles to Python.
I had already been working on it before I was selected for Mitou, and during the Mitou period I added features, fixed bugs and built development tools (a package manager, a registry and so on) to make it more practical. Put that way, it might sound like most of the work was done before Mitou, but this business of making it practical turned out to be pretty tough. Partly as a proof of concept, I wrote the package manager in Erg itself, and bugs in the compiler came pouring out one after another; to fix them at the root, I even ended up overhauling the compiler’s architecture.
What follows is less a technical article than a rambling piece, mostly about what I felt over the course of Mitou, plus some… advice? for people thinking of applying.
About Me
I’m a grad student who became a first-year master’s student in April 2024. My major is physics, and in my lab I work on quantum field theory and thereabouts. I’m also interested in quantum information and quantum computing.
You might wonder why someone like that ended up in Mitou developing a programming language, but programming has been my hobby since high school. I’m one of those people who played around with Python and Nim, wrote Nim packages and put them up on GitHub, and came away thinking that programming languages and OSS are really fun. I was fairly confident I could program well enough on my own, so I chose the physics department at university because I wanted knowledge that I didn’t think I could pick up by teaching myself. Looking back, that was a pretty arrogant way of thinking.
The Road to Applying
It all goes back to April 2020. I’m part of the generation that took the full brunt of COVID: right after I started university (the Japanese academic year begins in April), there were no classes at all in April and May. What I started during that time, wanting to try my hand at something, was developing a programming language. For the first year I just plugged away at writing a parser. I had an urge to write everything from scratch myself without relying on libraries, so the parser is a handwritten recursive descent parser. Using a parser generator would have cut that year of development considerably, but it gave me time to think the syntax and such through carefully, so I don’t think it was wasted.
At first I wasn’t building it around any particular concept like the current one, but when it came time to decide whether the implementation would be a compiler or an interpreter, I hit on the idea of piggybacking on an existing language, namely Python. Nim is a language that looks a lot like Python, but it isn’t compatible with it, and it’s statically typed. Nim was really where I first learned about type inference, and it made me think: statically typed languages being a pain to write (I had C and the like in mind) isn’t an inherent problem, is it? With the right design, maybe you could make a statically typed language that’s easy to write and compatible with Python? Wouldn’t everyone want a language like that? That’s the concept that carries through to Erg today. The “Python has these problems, so…” spiel is something I trotted out a lot during Mitou, but actually I wasn’t thinking along those lines at all back then.
Once the parser was done I just wanted to get something running, so I skipped the checks for the time being and built the backend (the code generator). Also, at the time I was scheming to speed up Python itself with static typing and had half-built an interpreter, but that’s a non-starter unless things are statically typed, so my plan was to finish it as a compiler first. Of course, I later realized that the compiler alone was more than I could handle.
By August 2022 the type checker, ownership checker and side-effect checker had more or less taken shape, so I made the repository public. I remember being surprised when it collected nearly 2k stars in the first week.
https://ossinsight.io/analyze/erg-lang/erg#overview
After that the growth in stars settled down, but I was convinced this was going somewhere and that my ideas had been right all along, so I kept developing day and night. It looks like in August and September I was putting in up to about 12 hours a day. Of course, when you go at it like that, the people around you eventually notice something’s off. However many stars Erg collected, they weren’t going to turn into money, and my family scolded me for wearing myself down, body and mind, over a mere hobby. That put an end to the insane development pace, but my reaction was: well, if that’s what you think, I’ll just have to make it more than a mere hobby. So I decided to apply to some competition or other.
Applying to Mitou was my first choice pretty much from the start. Programming languages are actually a traditional theme in Mitou that goes all the way back to the very first round. At the time, programming language projects had been selected three years running, and I figured I should ride that wave. I also looked into Inno-vation (異能vation), but Mitou is better known, and it’s more flexible about how you can use the money you get. There’s also Cybozu Labs Youth, where, like Mitou, you get to develop whatever you like and receive money you can use freely, but Mitou wins on both name recognition and the amount of money.
I really do think Mitou’s terms are exceptionally good compared to similar programs. The amount you receive has been going up year after year, which I think is probably thanks to all the things Mitou alumni have gone on to achieve. The great thing about Mitou is that you’re allowed to build low-level stuff1, or to put it bluntly, stuff that doesn’t exactly smell like money. That’s because Mitou is also about developing talent.
So Mitou it was. That was around November 2022, and I think I submitted the application around the beginning of March the following year.
Selection
As I mentioned in the previous section, Erg already had a clear-cut track record of 2k stars, so honestly I was confident I’d be selected.
The screening was held over Zoom. It follows the usual pattern of giving a presentation for a set amount of time in front of the PMs (project managers, who select and mentor the projects) followed by Q&A, and my backup slides came in handy during the Q&A. Anticipating the likely questions and preparing backup slides in advance is a presentation technique I use a lot, and it would go on to help me many times over the course of Mitou.
Events After Selection
Boost Meeting
About a week into the Mitou period, the first meeting (the Boost Meeting) is held. This is where you first learn about the other selected projects.
In the evening there’s a sort of social gathering with the selected creators, alumni, PMs and others, and everyone chats away happily. Apparently alumni who worked in the same area the previous year tend to be invited, and this time someone who had done a programming language project the year before came along, so we had a fun chat about programming languages.
After the Boost Meeting there are no big meetings for a while, but PM meetings (something like regular progress reports) are held roughly once a month. Alumni sometimes get invited to these too, and they’re sometimes held jointly with projects under another PM.
8th Station Meeting
This is held around late November. The name (after the stations on the way up a mountain such as Mt. Fuji) makes it sound like a meeting where you show something that’s 80% complete, but according to the PMs, “this is where the real work begins,” and in practice the vibe is that any percentage is fine as long as you push hard based on the reactions you get at the 8th station. Application projects sometimes pivot, after all. For low-level projects the sheer amount of work tends to be what counts, so there was a sense that I’d better have more results to show than an application project would. Fortunately, thanks to the head start I had from before Mitou, my presentation went off without a hitch, and from there it was a matter of how far I could raise the quality.
Final Presentations
The final presentations are usually held in mid-February. In effect, this is your “delivery date.”
From the end of the year through mid-January I focused on writing my undergraduate thesis, and once it was done I went straight back to development without a second glance. I’d heard alumni stories along the lines of “it worked properly for the first time on the morning of the final presentation,” which terrified me, so I set my real deadline a week earlier, and looking back, I think that was a good call. The reason is that you also have to think about how to present your work. Going in cold is far too risky, so most projects first show their presentation to their PM and get feedback. In my case that feedback was really valuable, but if you’re the type who keeps polishing right up until the last minute, you can’t really do this.
As for the presentation itself, well, I think it went pretty well, as far as I’m concerned.
Reflections
After the final presentations, I talked with others from my cohort about how it had all gone by in a flash. Objectively, that’s only natural: we were asked to build something no one had ever done before (zenjin mitou, “untrodden by anyone,” the same 未踏 as in the program’s name) in just eight months. And with several peers each building something like that, plus people and organizations backing them… it was an extraordinary situation, unlike anything I’d experienced in my life.
We were made to write a self-assessment of our growth in the final report, and I think what grew was my ability to triage. With only about eight months, I think it’s rare for a project to achieve everything it originally envisioned. If anything, a project whose whole course you can foresee at the proposal stage risks being judged as not “unexplored” enough. In my case, development of the native code backend ended up only halfway done. The native code backend was a backend I tried to implement so that Erg could emit binaries in addition to Python, since that should make things faster. I named the project “Development of a statically typed programming language that can be transpiled to Python” with this native code backend in mind, but I ran out of time. I got it to the point where it could compile Fibonacci code and the like, but compiling the package manager, which was the goal, stayed out of reach. I decided that improving the quality of the package manager and the other development tools was more important, and stopped working on it. On the flip side, this shift in direction led me to build things I hadn’t originally planned, so it wasn’t all bad. Knowing when to make calls like this is what matters, I think. It’s probably the same in real-world jobs. Then again, Mitou is a job too…
I’d like to keep chipping away at the native code backend after Mitou and release it.
Some Slightly Negative Stuff
It wouldn’t feel very real if all I did was rave about how great Mitou is, so let me talk about some slightly negative things too. None of this is likely to be very useful to you, so feel free to skip this section.
First, getting paid to develop Erg was wonderful, but at the same time it also became a shackle. Specifically, I stopped touching any project other than Erg, because both in terms of results and in terms of money, that time should go to Erg. I’m sure some programmers develop OSS in their spare time, but in Mitou you’re fully remote with flexible hours, so the line between work and time off tends to disappear. Every time you boot up your computer, the choice is always staring you in the face: work on Erg, or something else? If you pick Erg, it’s the right choice, but a bit of frustration builds up. Well, the answer is to just put up with it until Mitou is over, and now that Mitou is over, I’ve gradually started working on other projects again.
Also, 1,440 hours (the program’s target number of working hours) is simply a lot. The Mitou period is 264 days, so even if you worked every single day without a break, that’s about five and a half hours a day. Obviously that’s impossible, so you end up doing about 10 hours a day for days on end during long vacations. I suspect 1,440 hours assumes a project with multiple people. The money is paid out based on the hours you actually work, so if you don’t put them in, you simply don’t get paid for them. I stubbornly saw it through, but if you’re applying with a solo project, do your best without overdoing it. For the record, no matter how busy I am, my rule is that I never pull all-nighters to get through tasks and always get at least six hours of sleep, so I didn’t have any physical or mental health problems during Mitou. You can get through Mitou even with rules like that.
What’s Next
I’ll keep developing Erg going forward, though probably not at full throttle. Language features that are still unimplemented include
- A multi-stage computation system (≈ type-checked macros)
- User-defined compile-time functions
- User-defined polymorphic types
The first one in particular is under active development right now, and so far I’ve implemented if, for and import macros. As for why I want to add a multi-stage computation system, as I also wrote in this post,
Ergを啓蒙していくにあたってやはりPythonと文法がかけ離れていることが障壁になりうると考えていて、しかしこれ以上文法を増やしたくもないので、Scalaのようなマクロを追加することを考えています
— Shiba (@s_sbym) February 19, 2024
Translation: In spreading the word about Erg, I think the fact that its syntax is so far from Python’s could be a barrier. But I don’t want to add any more syntax, so I’m thinking about adding Scala-like macros.
it’s all about usability. It’s something I wasn’t really conscious of before Mitou: a sense of the product’s market. It’s not that I’m trying to make a business out of Erg, but thinking about the fact that other people might actually use this language, and how to improve it so that they will, is one of the ideas that deepened for me during Mitou.
Also, I haven’t really done much in the way of outreach so far, so I’d like to start. I’m thinking of attending a Rust or Python conference, but if you’re involved in running some other conference and would be willing to let me promote Erg, I’d be happy to hear from you. Eventually I’d like to be able to hold workshops and such. For that I’ll need to gather some users first, though.
Also, there’s something called Mitou Advanced that I could apply to as well, but there the vibe is that it has to be a business or something already in practical use, so once again, I need more users…
This drifts away from Mitou a bit, but I’d also like to talk about my main line of work. I had written code for numerical calculations in my research before, but since doing Mitou I’ve come to feel more strongly that I’d like to bring programming, or rather computer science, into my research as well. Specifically, I’ve become more interested in quantum computing. I recently read a paper suggesting that lattice gauge theory calculations, whose computational cost grows exponentially on today’s computers, might be handled nicely on a quantum computer, and it got me interested. I haven’t taken lattice gauge theory yet, but I’m studying it alongside quantum computing.
There also seems to be a research area called quantum programming languages, which, well, I’m extremely interested in, but it does seem to be CS territory. I’m studying that too, but I doubt it’ll get me a degree…
It seems I’m more interested in what we could do once quantum computers are “complete” than in how to build them, or what can be computed with what exists today. This may be getting way ahead of things, but if you look back at history, theories like the lambda calculus were developed for classical computers before the actual machines existed. So I think there’s value in doing basic research ahead of the times. And beyond that, I simply think it’d be better for the field to have a vision along the lines of “here’s the world that awaits us once quantum computers are complete,” rather than research being driven by a motivation like “let’s develop quantum computers because computation will probably get faster.”
Anyway, back on topic.
Miscellaneous
Here are a few odds and ends for those who are thinking of applying to Mitou.
Paperwork
Here’s the paperwork you’ll need to deal with.
- Reviewing and signing the contract documents (at contract time)
- Daily work reports (weekly)
- Progress reports (monthly)
- Confirming requests for advance payments (irregularly)
- Travel and accommodation expense claims (irregularly)
- Final report (February to March)
I’m not sure how much detail I’m allowed to go into, so I’ll keep it vague, but the daily work reports aren’t as tedious as they sound, since the Mitou administrative office staff help you out. I was keeping a private work diary of my own separately from the reports, so I’d put them together on the weekend based on that. There’s a rule that you can’t keep working on the same task for three or more days, which was quite a painful constraint when I was fixing a really tough bug, but I got by by tweaking the wording a little to reflect my progress or switching to another task for a while. For progress reports, too, each PM usually holds a monthly progress meeting, and submitting your slides from that meeting counts as your progress report. Confirming advance payment requests is basically just a matter of sending a “confirmed” email. As for travel and accommodation expenses, I never ended up claiming any. Conferences and meetings are mostly held in Tokyo, so if you live in Tokyo your fares fall below the minimum amount you can claim. Only the final report is fairly long and a bit of a slog. I’ve put all my documents together in a GitHub repository (in Japanese), and you’ll be made to write about that much (mine is on the longer side, though). I’d been made to write my undergraduate thesis a few months earlier, so compared to that it was nothing, but if you haven’t been through that kind of baptism of fire yet, it might catch you off guard.
Income
The money you get from Mitou is officially called 付託金 (futakukin, roughly “entrusted funds”). If you need a budget to carry out your project, it comes out of the futakukin, and money spent this way can be claimed as expenses when you file your tax return. Be sure to keep receipts and other proof. If you’re a good negotiator, you can also use various connections after being selected to borrow the equipment you need. One person in my cohort got a company on board and had them provide a high-performance computer rental and so on. That’s not something you can count on, though. The name “futakukin” makes it sound like you can’t use it freely, but you can use it completely freely. For software development projects the costs are zero, so it’s effectively a salary. However, as I’ll explain later, it’s not actually employment income, so your income tax deductions are smaller.
IPA pays the futakukin in several installments a year according to your work record (advance payments). People often say part-time job wages are tax-free up to 1.03 million yen, but that comes from the 650,000-yen deduction for employment income plus the 380,000-yen basic deduction. Mitou’s futakukin doesn’t count as employment income, so your deduction is just 380,000 yen. In other words, anyone receiving more than 380,000 yen from IPA (which I think is pretty much everyone) has to pay tax. I’ll get into this in the next section on tax returns too, but since the tax rate changes depending on your income, you can also ask to defer receiving the advance payments. In my case, as a solo project I’d receive at least 1.3 million yen a year however it was split, but it would never exceed 1.95 million yen, so I didn’t request a deferral.
Taxes
It was my first time filing a tax return (確定申告, kakutei shinkoku, Japan’s annual self-filed return), but it went more smoothly than I expected. That’s probably partly because all I did was build software.
Just for reference, here’s a rundown of how I filed my return. The process may change in the future, so treat this as a rough guide only. Having a My Number Card (Japan’s national ID card) makes it much easier, so I recommend getting one. If your smartphone can’t read the My Number Card, I think you’ll also need a card reader.
Open the National Tax Agency’s tax return preparation site (in Japanese) and press “Start creating” (作成開始).
- Use a smartphone
- Create the return and other documents for Reiwa year X (the Japanese era year)
- Create the “financial statements / income and expense statement” (「決算書・収支内訳書」)
- I initially picked “Income tax” (所得税), the leftmost option, and was told at the very end that I needed to attach an income and expense statement, so I went back, chose this one instead, generated the .xtx data and attached it
- Link with Myna Portal (the government’s online services portal)
- Scan the QR code and read your My Number Card
- Enter your revenue (the futakukin)
- The income category is business income (miscellaneous income seems to be fine too? If you’ve filed a notice of business commencement you can apparently get special deductions and so on by filing a “blue return,” but I didn’t file one, since it probably doesn’t make much difference)
- In my case revenue = income, but in general income = revenue − necessary expenses, etc.
- Enter your deductions
- I didn’t have any. Withholding tax was 0 too (IPA doesn’t withhold tax; the exact details are in the contract documents)
- You’ll be sent two pages of the return plus the income and expense statement, so check them
- Pay
- I chose payment by direct debit
If you choose direct debit, the tax is automatically withdrawn from your registered bank account around late April. By the way, as for the tax rate, as stated here (in Japanese), mine was 5%. I’d often heard office workers say things like “my salary is 300,000 yen but my take-home pay is only 200-something thousand,” so I assumed they’d take around 10%, but it wasn’t that bad. After that, the bills for resident tax and the like arrived, and they took quite a bit.
Advice
If you’re a university student right now, I think it’s best to do it sometime in your first to third year. For some reason Mitou ‘23 had quite a few people who, like me, were fourth-year undergrads at the time, but between grad school entrance exams or job hunting and your thesis, you’ll have a lot on your plate. There were also quite a few grad students, so if you’re going on to grad school, doing it after you get in might be fine too. If you’re a high school student, I’d say weigh your application against university entrance exams and such. There was someone selected at high-school age, but I believe they weren’t attending a regular high school. Apparently if you get results in Mitou Junior (the youth version of the program), some universities will give you a recommendation for admission, so it might be a good idea to do that and then apply to the main Mitou program afterwards.
Apparently Mitou projects can be broadly divided into the problem-solving type and the life-work type.
The problem-solving type is a project that tackles a problem that’s hard to solve with existing technology alone, and on the surface, Erg falls on this side. The life-work type is a project packed with the proposer’s obsessions and personal philosophy. The term “life work” is one that PM Yoichi Ochiai used, and apparently the projects he takes on are of this type.
I said “on the surface” because I simply thought the former framing would be easier to explain; in reality, I think the Erg project has a lot of life-work aspects too. Implementing languages is likely to remain a lifelong hobby of mine.
In that sense this dichotomy isn’t always appropriate, but when applying to Mitou, I think it’s good to be at least vaguely aware of which angle you’re going to push.
For the problem-solving type, you’ll probably need to dig deep into how serious the problem is, the prior art, the shortcomings of the prior art, and the advantages and novelty of your proposed project.
For the life-work type, I think it’s good to explain how interesting the project is, how passionate you are about it, and how much you’ve already done.
Or, though I haven’t seen many people do this, maybe you could push both aspects. Well, it’s Mitou, so doing things your own way is best.
Related Links (in Japanese)
- https://matsuuratomoya.com/blog/2020-03-08/mitou2019-2/
- https://skishida.github.io/mitou2019/
- https://arailly.hatenablog.com/entry/2020/03/18/190502
- https://xar.sh/post/1466554357/
- https://maruuusa83.hateblo.jp/entry/2018/05/10/145925
Footnotes
-
Programming language implementations are generally treated as low-level software, but I don’t really think of myself as a low-level person. I’m interested in static type systems and language design, so if anything I see myself as a theory-layer(?) person. Not that I have anywhere near enough expertise, though. ↩
COMMENTS