Quite a while ago now, Lars spoke about writing the scaffolding of your projects as a first step. We've covered all sorts of things like how to lay out your codebase and recently about motivation for projects in the first place.
And all this is good. You've decided you want to write something awesome, you have a design which you think will lead to an amazing program which will wow the world. But it's going to take you about six months of engineering before you can get to the result you've designed for. That's an awfully long time before you know if you're doing the right thing.
Lars spoke of TDD (test driven development) in his scaffolding article and that's an excellent way of ensuring that everything you write is validated. Sadly it does nothing to guarantee that everything you write is useful or desirable in implementing your final solution. There are a number of ways of bridging this gap and today I'm going to talk about one of them.
The "Minimal Viable Product" is a process whereby you develop the least you possibly can to give yourself an edge or unique selling point; and the same thing can be applied to software engineering. You can shorten the feedback loop, reduce the chance of developing something undesirable, and get to a program which is of use to you as soon as possible; if you focus your early development effort on a minimal viable project.
You should take time, after you've done your high level architecture and design discussions to gather by a whiteboard and come up with the absolute bare minimum of your feature set which you could implement and produce something which can actually be used for a real-world use-case.
Then, after you've implemented your MVP you've likely had to amend your design documentation a good amount (possibly even fundamentally). Fortunately for you, you come out of the process with something which does something useful (and thus can be effectively evaluated) and every change you make to your design as a result has a grounding in the reality of implementation.
Your MVP could be thought of as a prototype. Indeed some developers prefer, at this point, to throw away the MVP and reimplement it again with the insights gained during the implementaiton. What is important about an MVP though, is that once you've been around the prototype loop a few times, you keep the MVP code and evolve your final project out of that codebase. This critical difference between an MVP and a pure prototype allows you to bring online continuous testing and continuous integration and deployment if appropriate, the moment you have your MVP ready (or even before). And remember, if it's not tested, it doesn't work.
Your homework this week is to look at some of the designs or READMEs you've written of late, and think about what an MVP might be which could lead you to greater insights into the problem, or even toward your final solution.
Daniel recently discussed the Minimum Viable Product development approach.
As discussed in his article it has the disadvantage that it results in a lot of code that will be thrown away after being deemed to be insufficient.
An alternative approach is to do some analysis up-front and determine the full set of functionality and split the work into sub-projects, so that each component is a project of value by itself.
For example, you may be writing a daemon in python which needs to be able to safely create and read PID files.
You would then need to investigate which projects exist for doing so, evaluate whether they are sufficient, and if not either develop a replacement or contribute to an existing project.
For an ambitious project this can provide more good in the short term, since while it will take a while before the project's main goal is achieved, some of the component parts are already of value.
This can be better for motivation than the Minimum Viable Product approach if the developers involved tend towards obsessing over minutiae and having the "correct" solution for every little part of the project.
However it is particularly vulnerable to losing motivation from introspection of the current state of the project compared to its goal, since it takes longer to get to something demonstrable if feedback on the individual components is not readily available.
This may be viewed as an agile vs. waterfall development methodology, since it requires a large amount of analysis up-front to determine which components are required and which should be split out into separate projects, but those individual components may be reusable.
Similarly, if it turns out that the goal of the project was bad, then at least the component parts of the project are salvageable.
If the project does eventually fail, then at least lessons will have been learned for future attempts.
A lot of people keep a diary. This is a time-tested way of ensuring future historians have juicy rumours to read and write books about. A journal is like a diary, but it's meant to be more serious and less saucy.
I've been keeping a journal for some years now. It's been good in several ways:
It's a private place where I can vent entirely without any inhibition. As long as I don't leak the content, I can write anything there, things I couldn't share even with my closest friends and loved ones. In my journal I don't need to be fair or balanced or diplomatic; if calling my boss a poophead helps me, I can do that in my journal. More importantly, apart from name-calling, I can be open in my journal about my hopes and dreams, and speculate as freely as I need to about all sorts of crazy ideas. If I want to fantasize about writing my own Debian installer, my journal is where I'll do it. In my journal I don't have to worry about people misunderstanding me, or attacking any vague, half-developed crazy ideas, and ridicule me about them for the next several years.
My journal is a safe place. (This is one of the reasons why my backups are encrypted.)
It's a place to keep an external memory. One of the the things I put in my journal is a stream of consciousness while developing. This allows me to answer questions of the form "what the bleeding heck was I thinking then designing this software" with quotes from my younger self. Perhaps more usefully, this can be applied to debugging as well: tricky problems often involve a lot of data to be kept to fully understand what's going on, and a written journal is a better place for that than the brain.
A bug tracker is usually not a good place for this, or not the full stream of consciousness. Most of that stream is necessary for the process, but other people shouldn't be flooded with all of it, only the actually relevant parts.
I also put all sorts of other bits of information into my journal. In fact, over the years it has developed into a personal knowledge base, where I can find a bunch of things that are relevant to me, but not necessarily easy to find online. As an example, what's the command to do TOTP on Linux that works with Amazon? (
oathtool --base32 --totp, not too difficult to find but easier in my journal.)Some information is private in nature, such as who were the people I had that interesting lunch with at that conference that one time.
There are a myriad of tools for keeping a journal, or personal knowledge base. There's wikis of various flavors, plain text files in git, online document services, etc etc. You can keep it on paper as well, though that makes grepping harder. As I prefer to not pour my heart and soul into a service run by other people, my journal is an ikiwiki instance that I run on my laptop, which renders a static HTML that is served via Apache on my laptop (and only to my laptop). This satisfies my needs for ease of use and privacy.
Using a wiki engine for this is nice, because linking adds a lot of
power and can make finding relevant information faster. Using the
ikiwiki inline directive, which produces pages by collecting other
pages based on a pattern, I get pages for particular people (link to
the person, the person's page includes the page linking to them),
topics (e.g., projects), tags, and more. Quite nifty, and I'm afraid I
can't show you.
Keeping a journal takes a bit of effort, of course. It also takes time for a journal to become useful: having diary entries for a week probably doesn't help much. Having them from a decade changes this in a qualitative, not just a quantitative way. Do you remember what you got your loved as a present seven years ago? I don't even remember what I got as a present last year.
Give it a try. It doesn't have to be perfect, but you need to keep doing it. Report back in a comment below in four years from now and tell use if it was helpful to you.
Time is a big enough subject that I'd never finish writing about it.
Fortunately this is a Linux technical blog, so I only have to talk about it from that perspective.
In a departure from our usual article series format, we are going to start with an introductory article and follow up starting from basics, introducing new concepts in order of complexity.
What is time?
Time is seconds since the unix epoch.
The Unix epoch is midnight, Thursday, 1st January 1970 in the UTC time zone.
If you know the unix time and how many seconds in each day since then you can use the unix timestamp to work out what time it is now in UTC time.
If you want to know the time in another timezone the naïve thing is to apply a timezone offset based on how far or behind the zone is compared to UTC.
This is non-trivial since time is a political issue, requiring global coordination with hundreds of countries coming in and out of [daylight saving time][] and various other administrative changes, time often changes.
Some times don't exist in some timezones. Samoa skipped a day and changed time zone.
A side-effect of your system clock being in Unix time is that to know local time you need to know where you are.
The talk linked to in https://fosdem.org/2017/schedule/event/python_datetime/ is primarily about time in python, but covers the relationship between time stamps, clock time and time zones.
http://www.creativedeletion.com/2015/01/28/falsehoods-programmers-date-time-zones.html is a good summary of some of the common misconceptions of time zone handling.
Now that we've covered some of the concepts we can discuss in future articles how this is implemented in Linux.