Indentation

So, while digging around in some external library code I stumbled across this method:

def format(self, value):
    if isinstance(value, set):
        value = list(value)

    return [
        self.container.output(idx,
            val if (isinstance(val, dict)
                    or (self.container.attribute
                        and hasattr(val, self.container.attribute)))
                    and not isinstance(self.container, Nested)
                    and not type(self.container) is Raw
                else value)
        for idx, val in enumerate(value)
    ]

You can see it is very difficult to understand what is going on there.. Please do not try to understand what the method is doing or where it comes from. That’s not the point I am trying to make here - just look inside self.container.output.

The method self.container.output gets the parameter idx and either val or value based on some conditions. These conditions are the cause why it is hard to understand. It is val if three conditions are met, but the first one contains further nested conditions where one of them is again nested… Complicated.

So I took the liberty to reformat the code without changing the logic of it (just whitespace changes). It turned out to this:

def format(self, value):
    if isinstance(value, set):
        value = list(value)

    return [
        self.container.output(
            idx,
            val
            if (
                isinstance(val, dict)
                or (
                    self.container.attribute
                    and hasattr(val, self.container.attribute)
                )
            )
            and not isinstance(self.container, Nested)
            and not type(self.container) is Raw
            else value
        )
        for idx, val in enumerate(value)
    ]

Well, yes, it became much longer, but on the pro side my PEP8 linter in the editor won’t complain about bad-continuation anymore. (It still complains about the Redefining built-in ‘format’, but anyway…)

Basically I removed all the hanging indents - now it is much easier to understand how these conditions interact. That single or moved one indentation step further to the right, so one can clearly see, it has nothing to do with the ands on the outside.

Then I remembered some talk I watched years ago - If your code is easy to grasp visually, the chance someone else understands it is way higher.

Ok, let’s give it a try - how about some dirty script?

#!/usr/bin/env python3

from sys import stdin, stdout
from string import whitespace

if __name__ == '__main__':
    for line in stdin:
        stdout.write(''.join(
            char if char in whitespace else '#' for char in line
        ))

    stdout.flush()

It replaces everything but whitespace with an # character.

The first example turns out like this:

### ############ #######
    ## ################# #####
        ##### # ###########

    ###### #
        ##########################
            ### ## ################ #####
                    ## #########################
                        ### ############ ###########################
                    ### ### ########################## #######
                    ### ### #################### ## ###
                #### ######
        ### #### ### ## ################
    #

And the second one like this:

### ############ #######
    ## ################# #####
        ##### # ###########

    ###### #
        ######################
            ####
            ###
            ## #
                ############### #####
                ## #
                    ########################
                    ### ############ #########################
                #
            #
            ### ### ########################## #######
            ### ### #################### ## ###
            #### #####
        #
        ### #### ### ## ################
    #

Just by looking at these outputs without comparing it to the original code (don’t scroll up) it is suddenly possible to see the different nesting levels, and how they act together.

In my opinion, the second one shows it more clearly - although it takes up much more horizontal space. But I think this is worth it - scrolling through text on the screen takes much less effort than reading the same code over and over again…

Think of it next time you are trying to write very dense code!

So long!

Updates

So, here are some updates. I am still alive, and I have not forgotten about that website.

Originally I had the plan to write something new each month. It worked well in the beginning, but then stuff happened.

Primarily: I have a new job!

It’s a very good one, the company is pretty good, and I pretty much enjoy it there. Part of the company culture is the willingness to learn something new, so there are many places where I can bring in my knowledge. On the other hand my head is exploding, because there is so much new stuff for me to learn. I have the feeling I am improving my skills every week.

Very good.

Along with this new job came the relocation to Leipzig – finally some decent city, it’s not that big, but big enough. There are a lot of very interesting people and every day I am somewhere else to discover new stuff to do. It is a lot of fun!

That is the reason I rarely find time to take care of this website. I am very sorry. But slowly my life sorts itself out, so there will be more time in the future for personal stuff like this. Maybe I can come back to former glory and write something new on a monthly basis…

In other news: The development of the Slick Hugo theme continues. Not that fast as it used to be, primarily because I don’t have so much time to do so, but also because it slowly gets to the point where it is finished.

Worth mentioning is maybe the switch to TypeScript for all the client side logic, introduction of Hugo Pipes for CSS and JavaScript delivery for fingerprinting and of course some adjustments to keep up with recent Hugo versions.

Also – Today I killed the 447 days of uptime of the server this page is running on. I think it is a quite impressive number and also a little bit scary..

It is pretty good that some system can run stable for such a long time (have I mentioned that I love FreeBSD?!?) but very scary that all the time no kernel patches were applied…

Well, now it’s over, both userland and kernel are now fully updated and patched and everything seems to run fine again.

I discovered some issues while rebooting - It is somehow not possible to send DNS requests from within newly started jails, because my firewall is not aware of the network interface of the jail. So the webserver inside the jail cannot apply OCSP stapling and waits endlessly… Sad.

This is something I have to dig into in near future, so rebooting (and thus updating the kernel) won’t be so much pain anymore.

Maybe this is stuff for the next entry here, we will see..

So long!

Weird JavaScript

This is something I stumbled across a while ago. Someone on the internet said he was asked this question during an job interview:

Can you think of a situation where this is true:

a == 1 && a == 2 && a == 3

Well, yes.

You have to ask the question, what a is.

Is it a Number? Obviously not.

The solution is quite simple if you know how JavaScript works.

You can define a to be an Object and overwrite .valueOf:

const a = {
    i: 1,
    valueOf: function() {
        return this.i++;
    }
};

if (a == 1 && a == 2 && a == 3) {
    console.log('yeah');
}

I think this is very awkward!

Or like someone else on the internet said:

Sometimes when I’m writing Javascript I want to throw up my hands and say “this is bullshit!” but I can never remember what “this” refers to.

So long!

Generate htpasswd entries with bcrypt

This week I was digging around in the configuration of my webserver.

For some internal pages I am using Basic Auth. Looking at the password hashes I noticed they start with $apr1$.

What is that? The Apache documentation about password formats tells us that this is some kind of MD5 hash.

Thanks, but no thanks (It’s a mystery to me why I decided to use this..).

So let’s strengthen security by using blowfish!

It is possible to create bcrypt hashes using the htpasswd command. This is some utility that ships alongside Apache. I am no Apache user, and I am not willing to install it just for one small task. So what do we do?

There are some alternative implementations of just htpasswd (htpasswd.py in python and Apache::Htpasswd in perl) but they don’t support blowfish as far I can see. Sad. Alternatively, there are a lot of online generators out there. But someone must be totally crazy to use them…

Python to the rescue! If you have the bcrypt package installed, you can use this script to generate the hash:

#!/usr/bin/env python3

import bcrypt

print(
    bcrypt.hashpw(
        input().encode(),
        bcrypt.gensalt(rounds=10)
    ).decode()
)

I am using input() here, so the password won’t show up in plaintext inside the ~/.python_history file when using the interactive prompt.

If you save the script inside some file, you may use it like this to append a new entry:

echo "user:$(./htspass.py)" >> .htpasswd

So long.

Fnord Day

Happy fnord day!

23.5

Live CSS editing on page

So, I just stumbled across something really cool!

It is possible to edit CSS live on page!

If you have some <style/> tag, you can add some style attribute to itself. First use display: block, so it shows up on page.

And then the second trick: Add the contenteditable attribute, to make it changeable.

<style style="display: block;" contenteditable>
    /* ... */
</style>

That’s it - new styles get applied immediately.

Demo

So, go on and change some of the CSS below:

I think this is awesome! Please don’t ask me, where it could be useful – still, awesome!

You could also do that for <script/> tags, but to trigger the Javascript again you’ll have to reload the page and then all the changes are gone.

So long!

Calculate IPv6 link-local from MAC Address

While preparing for my final exams I stumbled across some question:

Please calculate the IPv6 link-local Address from the following MAC Address…

Good question.

Well, it’s pretty easy if you know how to do it.

Let’s say you have the following (hypothetical) MAC Address:

5f:4e:3d:2c:1b:0a

First thing, you have to add some ff:fe in the middle:

5f:4e:3d:ff:fe:2c:1b:0a

Now for the tricky part - the calculation. Look at the first part, in this case 5f. Convert it from HEX to BIN (see table below if you need help):

      5        f    (HEX)

0 1 0 1  1 1 1 1    (BIN)

The second to last bit needs to be flipped. Or more technically speaking the result needs to be XORed with 0 0 0 0 0 0 1 0.

0 1 0 1  1 1 1 1

0 0 0 0  0 0 1 0    (XOR)

0 1 0 1  1 1 0 1

Lastly, convert it back to HEX:

0 1 0 1  1 1 0 1    (BIN)

      5        d    (HEX)

Now we get the following:

5d:4e:3d:ff:fe:2c:1b:0a

As a final step add the prefix fe80, fill it with zeros and format it like a proper IPv6 Address:

fe80::5d4e:3dff:fe2c:1b0a

As I told you - pretty easy.

BIN - HEX table

XOR table

The other way around is similar – chomp both fe80:: and ff:fe, then use the same XOR operation as above.

If you know all this above, you can easily spot generated link-local addresses, when they contain some ff:fe.

There are some operation systems around that use randomly generated ones - so don’t be confused if your machine does not generate one like above. I don’t know if this is compliant with the spec…

So long!

Update:

Just fixed some mistake – the final address was incorrect, I forgot to add leading zeros:

Changed: fe80: → fe80::

Further – If you want to know more, take a look at those RFCs:

• 4291 IP Version 6 Addressing Architecture – Section 2.5.6 Link-Local IPv6 Unicast Addresses

• 4862 IPv6 Stateless Address Autoconfiguration – Section 5.3 Creation of Link-Local Addresses

• 7721 Security and Privacy Considerations for IPv6 Address Generation Mechanisms

• 4941 Privacy Extensions for Stateless Address Autoconfiguration in IPv6

So, yes, using randomly generated link-local addresses is compliant with the spec…

Catchall Mail

As a proud owner of a domain coupled with a mail server I have the freedom to use any mail address below that domain. So a long time ago I decided to use a catchall address. This is great, I can use it to track who is leaking my mail address (either by accident or on purpose - does not matter).

So when I register somewhere using shabby-webshop@example.org and, after a certain period of time, spam arrives on that address I know what to do: Cancel my account there and reject all further mails to that address. Maybe also poke around how this happened - but that’s optional…

But catchall addresses have some huge drawback:

After some time you completely lose track which addresses are in use, how often mails arrive there, and so on. All I have is my list of blacklisted addresses - I don’t know anything of the active ones.

This has to stop - so today I started to fiddle around in my mail setup to track active addresses.

I am using procmail to sort the mails directly on the server anyway - so why not start here? The first rule inside the definitions looks now like this:

:0 c
| /some/path/to/monitor.py

It passes a copy of the mail (note the c flag) to the monitor.py script and continues normally afterwards.

Inside the monitor.py script I am using something similar to this:

#!/usr/bin/env python3

from email import message_from_file
from email.utils import getaddresses
from json import dump, load
from os import path
from sys import stdin

FILENAME = 'monitor.json'

# read the mail from stdin
mail = message_from_file(stdin.read())

# parse the recipients
recipients = []
for field in ('To', 'Cc', 'Bcc'):
    recipients.extend(mail.get_all(field, []))

# load existing content from json file
result = {}
if path.exists(FILENAME):
    with open(FILENAME, 'r') as jfile:
        result = load(jfile)

# count the addresses
for _, addr in getaddresses(recipients):
    result[addr] = 1 + result.get(addr, 0)

# store result in json file
with open(FILENAME, 'w') as jfile:
    dump(result, jfile, indent=4)

Please note this is just some example code - I’ve added error handling and much more. Please do not just copy & paste that if you intend to use it!

The result is a json file with that content:

{
    "info@example.org": 23,
    "mail@example.org": 42,
    "something@example.org": 5
}

I am totally looking forward to look into the json file after some time. Hopefully this will help me to find some addresses I am using and already have forgotten about. And much more important: Hopefully this script won’t eat up all my mail.

If I won’t reply for some time, you know what happened… 🙈🙉🙊

So long.

Compound Words

In programming you always have the problem to join words together keeping the syntax intact and to keep at least some amount of readability.

Those compound words got some names, so everyone knows what you are talking about.

You might be familiar with:

• CamelCase - Maybe the most famous one, most people might know it from Wikipedia

• snake_case - Very popular in Python, but don’t fall for that. The name of the language refers to Monty Python, and not the snake.

• kebab-case - This is my favorite name - Not really common because it is not easy to distinguish it from a minus sign. So you’ll find it mostly in markup languages (Pretty common for HTML class names and/or CSS)

But there is one which was new for me:

• beercase - I guess it is inspired by two glasses clinking together. 🍻

You can grow old, but you never stop learning.

Cheers