Previously we discussed control structures in our scripting languages and we have touched on chunking up your code into procedures/functions/subroutines. Today we're going to take that a little further and discuss the ways in which our chosen languages group up those pieces of code into what is commonly referred to as a 'module'.

Conveniently, Python, Lua, and Perl all refer to collections of reusable code as modules so we don't need to learn a whole bunch of different words for the same concept today. However each language has different syntax (shocking, I know) for both: the creation of, and use of, modules.

Perl modules

Perl uses the Paamayim Nekudotayim to separate the scope of modules, so the Perl module which provides access to a useful piece of code designed to dump out data structures is called Data::Dumper. To use that in your own Perl code, you might do:

use Data::Dumper;

Perl is, as you might expect if you've gotten this far in our discussion of scripting languages, a little odd when it comes to building your own modules. Fortunately the Perl community is pretty awesome and they have written a very good simple module tutorial.

Python modules

Python uses the . to separate the scope of modules, so the Python module which provides you access to the system's path-related functions is called sys.path. To use that in your own Python code, you might do:

import sys.path

Python's module syntax is slightly less complex than Perl's: namely you just have a .py file and it's implicitly usable as a module straight off. Python extends this slightly by allowing for a special file called __init__.py which acts as the content of a module whose name is actually a directory. This allows you to group modules into what Python refers to as a package. You can learn more about this here.

Lua modules

Lua uses . as Python does. However Lua doesn't come with many modules by default, but if you happen to have it installed, then the Lua module containing code representing a multimap in the Penlight library of code is called pl.multimap and you would use it as follows:

local mm = require "pl.multimap"

Lua's module syntax is mid-way between Python's and Perl's. Unlike Python, you must explicitly return your module's contents, however unlike Perl you do not have to include a lot of other stuff in order to make it work. There's a reasonable Lua module tutorial available to help you with this.

Challenge

Your homework for this time, is to take some time to investigate what modules are available on your computer for the various languages you are playing along with, and see what they have to offer you. If you're feeling 'advanced' then you should proceed to write your own code module, perhaps containing some of the software you've written thanks to your experimentation in the previous installments of this series.

IP is the Internet Protocol, which defines how to talk to computers on the Internet.

The two ways you can send data over it are the TCP protocol, and the UDP protocol.

TCP is for reliable, ordered and stream-based transport, UDP is for sending packets of data over IP without these guarantees.

To create a UDP socket, you use the socket(2) system call, with the AF_INET and SOCK_DGRAM options.

int udp_socket = socket(AF_INET, SOCK_DGRAM, 0);

To recieve packets you need to bind(2) the socket, as you would a TCP socket.

int ret = bind(udp_socket,
               (struct sockaddr*)(&(struct sockaddr_in){
                   .sin_family=AF_INET,
                   .sin_port=htons(1234),
                   .sin_addr=htonl(INADDR_ANY),
               }),
               sizeof(struct sockaddr_in));
if (ret != 0) return 1;

You don't need to call connect(2) to send messages over UDP, you then just call sendto(2).

ssize_t ret = sendto(udp_socket,
                     argv[0], strlen(argv[0]), 0,
                     (struct sockaddr*)(&(struct sockaddr_in){
                         .sin_family=AF_INET,
                         .sin_port=htons(1234),
                         .sin_addr=htonl(inet_addr("127.0.0.1")),
                     }),
                     sizeof(struct sockaddr_in));

Now on the recieving end, you can use recvfrom(2) to read the packet.

You need to specify a buffer to write the message to, and how large that buffer is. If your protocol doesn't define ahead of time how large that buffer is, you can query this with MSG_PEEK|MSG_TRUNC:

ssize_t msgsize = recvfrom(udp_socket, NULL, 0, MSG_PEEK|MSG_TRUNC,
                           NULL, NULL);
char *msgbuf = malloc(msgsize);

You can also get the address of the sender of the packet by providing src_addr and addrlen to the recvfrom(2) call.

struct sockaddr_in recvaddr;
socklen_t recvaddrlen = sizeof(recvaddr);
(void) recvfrom(udp_socket, msgbuf, msgsize, 0,
                (struct sockaddr*)&recvaddr, &recvaddrlen);

You now have all the data of the message, which may have been variable length, and know where it came from.

You could use this to send a reply to the sender, or just log it as follows:

printf("%zd bytes containing %s from %s:%d\n",
       msgsize, msgbuf,
       inet_ntoa(recvaddr.sin_addr),
       recvaddr.sin_port);

Program listings for the client and server can be read at udp-server.c and udp-client.c.

When learning to program, or when learning a new programming language, framework, or other tool, it is good write some practice programs to start with. These can be any programs one is interested in, and do not need to be meant to be used for real. Sometimes one can't think of anything. Here's a list of ideas.

• The very basics:
  • hello: a program that just shows the message "hello, world" to the user. This is useful, because it means you can write the source code file, compile (if it's that kind of language), and run the program, and see the output.
  • Simple Unix-style commands: echo, cat, sort, sed. These can be made as hard as one wants, but the simplest versions are often quite simple.
• System software:
  • find: Look for files based on names or metadata.
• Data structures and algorithms:
  • Word frequencies: Read input, count the number of times each word in the input occurs, and write the 20 most common words.
  • Data compression: in its simplest form, this is run-length encoding, but can be as sophisticated as one wishes. Data compression algorithms can be quite tricky.
  • Lisp: implement a version of the Lisp programing language. Lisp has a very simple syntax, but a powerful sematics, making it particularly interesting to implement.
• Miscellaneous:
  • An interactive text editor.
• Web apps:
  • Show time until/since a point in time.
  • A clone of Doodle, for having a group to choose a time or place or other such thing.
  • A clone of Twitter.

What are your favourite practice programs, either from the list above or otherwise?

Last time, we discussed the grouping of code together into modules and packages and at the end I encouraged you to investigate your computer for software modules and also to have a go at putting your own modules together.

Today we're going to look into the ways in which our chosen languages offer other people's code modules for your delight and enjoyment. This is typically referred to as a language's package database and it is one of the many ways of getting hold of modules written by others without use of your operating system's package manager.

Perl

Perl is a very well established langauge and community. For a long time, the go-to place for Perl modules has been CPAN (The Comprehensive Perl Archive Network). Perl comes with a special module called CPAN which can be used to access the archive of modules without needing any other tooling. You can interact with it by running:

$ perl -MCPAN -eshell

Follow through the configuration, let it have a jolly good fiddle and then it'll let you search for and install perl modules on your computer.

Python

Python's community went for a commandline tool rather than a REPL, and then being a super-special community they schismed a bit and so there's both pip and easy_install to choose from. You can install Python modules as simply as:

$ pip install <packagename>

Lua

Lua, having a much smaller and to some extent younger community around it has only managed to grow a single package manager. It is called luarocks and is closer in behaviour to pip than to CPAN.

$ luarocks install <packagename>

is all you need to get something installed for use with Lua.

Challenge

Today's task is to become comfortable using the package managers of the languages of your choice. Learn how to install and remove packages, and how to search and retrieve information about different packages in the respective databases. Then think of something fun to play with, install those packages, and have a jolly good time coding with the libraries you didn't have to write yourselves.

How open-source can go wrong

There are a number of ways that Open-source projects can go wrong, where by "going wrong" I mean that it's not receiving bug reports and fixes, it's not seeing patch contributions, and it's not seeing improvement.

If it's a trivial piece of software, or is only intended for a very small niche (e.g. "me and my gaming group"), then that's fine, of course.

Here are some things that have gone wrong in projects I've worked with, which might draw a few parallels with the Truisms.

Nobody uses it

I've seen projects with nobody using them for a number of reasons:

• It's hard to integrate it into a running system and use it. Maybe it needs some arcane systemd setup which isn't part of the default installation. Maybe it takes a lot of effort to get it to build and install properly.
• It depends on another piece of software, which isn't finished yet.

There's no plan for improvements

Maybe the maintainer has minimal interest in this project, or is too busy with newer projects. If there's no community to pick up the slack, the project will flounder.

The maintainer isn't interested in contributions

If the maintainer is the only one making changes to the project, then the only fixes and improvements are what the maintainer is interested in. This can be a problem because development will occur more slowly, but is also a problem because the problems the maintainer sees are different from what inexperienced users see.

And finally

That's all the ways open-source can go wrong that I've personally experienced, doubtless there are many, many ways for open-source to go wrong, just like there are many ways for software to go wrong.

What ways have you seen open-source projects gone wrong?