Adding multiple content filters to an email server with postfix and dovecot Pigeonhole/Sieve

A year ago, my friend Laura was wishing that email providers could do some tone filtering and reject messages that are too mean. Since I run my own email server, I thought it might be simple to set something like that up easily. Turns out, it’s not that hard, but it wasn’t exactly trivial to figure out.

My setup

I run have postfix running to receive messages from the internet. It passes them through SpamAssassin, which inspects the messages and adds a few headers that indicate whether or not it’s spam. Then it passes them on to dovecot , which stores the messages in mailboxes and then tells with my email client, Thunderbird, that I’ve got mail. I like this setup because I feel like I have a bit more control over my data. Besides, it’s fun!

The plan

The original request is here shown below. I figure, if I could just have the message go through a second filter after it goes through spamassassin, I could make it a custom script that counts swear words.

 

text post showing the idea
The Original Idea

 

How to do multiple content filters on postfix

Learning materials include:

A lot of answers were along the lines of “Just get amavisd to do the other filter.” Maybe that’s the right answer but I’m not running amavisd and didn’t feel like setting it up just for this.

Then I stared at the postfix config for a while, in particular at this line:

That tells postfix to pipe everything off to spamc for content filtering. Then, once the filtering is done, the -e sendmail thing (according to the man pages for spamc)  sends the filtered mail to sendmail, which delivers it to dovecot for filing. Aha! I bet you can just send the processed mail to an intermediate script that does its thing and then sends it on to sendmail! Turns out this works perfectly. Here’s my script (WARNING: it has swear words in it!)

As you can see, this just expects an email to come in on stdin, checks it out, and sends it to the stdin of sendmail.

This is not particularly efficient, because it will take some time for python to process some huge attachment looking for swearwords in binary. I wouldn’t put this on a large-scale system. But the pathway works so adjust to suit your needs!

Your postfix config line changes to:

Processing in Dovecot

Now that we have a header that indicates swearwords (‘X-Too-Many-Swearwords’), we can take actions with it. The Dovecot PigeonHole extension interprets the Sieve language which is used to do stuff based on email content. I use it to filter emails server-side into folders and stuff, and to reject emails flagged as spam into my spam folder. You can actually adjust the filters directly from Thunderbird with an extension, so that’s cool (uses ManageSieve Dovecot extension too).

Here’s the user-level Pigeonhole/Sieve filter that deals with swearwords and does some other stuff. The spam filtering is in the global one, which I configured to run before this.

That last clause simply rejects the email so I never even see it and tells the author to send a new email. Nice! Totally works. Gmail marks the response as spam though… typical!

Anyway, so that was fun, if not totally useless. I had enough trouble configuring that I hope this info helps someone do something actually useful! Good luck.

The Infopanel: a simple MQTT-connected display system for weather, traffic, pictures, animations…anything!

I got one of those RGB LED matrix things for my birthday and wasn’t sure what to do with it. Then I found this awesome library which has Python bindings and can control it nicely even from a Raspberry Pi. Conveniently I had a spare Raspberry Pi 1 B+  sitting around so I hooked it up. After playing around for a while, I got the demos working.

UPDATE: Full documentation of infopanel is now available.

But I needed to connect it to home-assistant to really make it valuable to myself. So I wrote a little program called Infopanel (available for free on github) that lets you do some things:

  • Get data directly from a MQTT broker for getting live data (e.g. travel times in traffic, weather conditions) and for command and control. This allows me to connect the screen to my home-assistant home automation system.
  • Assemble various built-in elements like giraffes, animated text, rainbow text, pictures, animated gifs into various scenes that rotate through on the screen to display the information in various fun and/or useful ways.
  • There are Temperature and Duration sprites that you can define high and low values of so they’re red when they’re bad and green when they’re good, and anywhere in between.

You can set the scenes to be just random or you can control them through MQTT.

It’s intended to be very configurable but since it’s brand new some extra development is needed to make everything perfect. Send in your ideas and requests and code changes!

A relatively complete example configuration file is in the repo. That demonstrates using MQTT, connecting MQTT topics to various sprites, building your own frames of animation by hand, and adding in gifs and images from file paths. Note that you have to set an environmental variable or two to get the fonts right and whatnot.

Jeep Cherokee Laredo 1991

Note: This is a thowback post, revitalized from the old partofthething.com where it was posted by me in like, 2004. I took out most last names.

My car:

My car is the ultimate car ever. It’s the greatest thing on Earth. No, really. It is. Sure, sure it’s old but that’s not what matters anymore. It’s the past that counts…and the stereo system. I grew up in this car. Yes sir I did. I always sat in the back right seat driving all over the place since I was seven years old. Usually Tom Petty was playing. you see, it used to be my mom’s car. When she got a new one, it became my sister’s car. And then, it became my car.

Continue reading Jeep Cherokee Laredo 1991

Getting live alerts when your website is visited with Apache, MQTT, and home-assistant

I have a website or two and sometime wish I could get notifications whenever someone visited them, just for fun. Well I did it, and now I can get beeps in my home whenever anyone visits. It’s kind of cool to hear it go off, though normally it will be annoying, so we need a switch for it.

Continue reading Getting live alerts when your website is visited with Apache, MQTT, and home-assistant

Adding rigor to information analysis in the era of Fake News and Alternative Facts

Collecting information about our surroundings, analyzing it to make conclusions, and then performing actions based on those conclusions is the definition of life itself. Organisms generally care most about their food source, dangerous strangers nearby, environmental changes, and family planning. For example, the rabbit pays attention to nearby gardens, the closest rival warren, any nearby foxes, human constructions, and potential lover rabbits*. The human was concerned with roughly the same things for years immeasurable, but since the Bronze Age, as we spread out globally and invented neat technologies, things became a bit more complex.

Continue reading Adding rigor to information analysis in the era of Fake News and Alternative Facts

Running fortune in a Python virtualenv

I learned on reddit the other day about the sudo “insults” capability where it throws shade at you when you mistype the password. I configured it everywhere I could, but I wanted more, so I came across cowsay and fortune.

$ fortune | cowsay
/ SHIFT TO THE LEFT! SHIFT TO THE RIGHT! \
\ POP UP, PUSH DOWN, BYTE, BYTE, BYTE!   /
 ----------------------------------------
        \   ^__^
         \  (oo)\_______
            (__)\       )\/\
                ||----w |
                ||     ||

Needless to say… wow.

Continue reading Running fortune in a Python virtualenv

Enlighten your old furnace with a Raspberry Pi, Home Assistant, an ESP8266, and some relays.

My mom has one of those on/off furnaces (EDIT: actually it’s a boiler) that heats up water and circulates it through pipes around the house that have little radiator fins. She wants it to turn on before she wakes up so it’s not so cold in the morning. In this post, I explain how to turn a normal furnace into a smart furnace for only a few bucks.

Furnace controls in Home Assistant

Continue reading Enlighten your old furnace with a Raspberry Pi, Home Assistant, an ESP8266, and some relays.

Getting IP camera motion events into Home Assistant to trigger things

Oh this is exciting! I’ve been trying to figure out how to get motion events from my IP camera into my home-assistant instance running on my Raspberry Pi, and I just did a successful test! It works! Hooray. Briefly, I set up an email server on the Pi, have the camera email the Pi, have the email server trigger a script which parses the email for key words and sends MQTT signals as appropriate, at which point the home-assistant MQTT client sees them and triggers automations (like blinking a light to scare people off). Here’s how I did it.

Camera-based motion sensors in home-assistant

Continue reading Getting IP camera motion events into Home Assistant to trigger things

Cheap vacuum pump for home physics experiments

As documented earlier, I made one of those CRT-in-a-winebottle things. I used a cheap 2-stage mechanical vacuum pump for it. Well, it was so good that my boss wanted to use it as a science demo at some dinners he gives, so it’s been permanently transferred to work. So I need a new pump, and a new chamber!

Continue reading Cheap vacuum pump for home physics experiments

Fixing my motorcycle’s charging system

I got my first motorcycle,  a 1997 Suzuki GS500, in 2007. It was great for a long time. I cut my motorcycle maintenance teeth on it, starting with gas line issues, shim adjustments, and an oil leak fix. Things escalated to taking the pistons out and head off to get the top-end rebuilt. Issues kept popping up more frequently than I liked so I sold it in 2012 and got a brand new Suzuki DL650A V-Strom ABS (aka “Glee”), feeling like I had had enough maintenance for a while. I only felt a little shame when I took it in to the shop for some routine maintenance

Well…I guess I’m back in the routine. After I totaled it but kept it, I got warmed up doing basic maintenance stuff on it. So when an issue came up recently, I decided to fix it myself.

UPDATE Jan 2017: Whaddaya know? The stator was just recalled due to a tendancy for this to happen.

Trouble a-brewing

Continue reading Fixing my motorcycle’s charging system

Raspberry Pi + Home Assistant DIY Burglar Alarm

I just configured a pretty slick burglar alarm with the open-source Home-Assistant platform running on my Raspberry Pi. It can be armed to trigger when a door is opened and/or when a motion sensor goes off.

Arm the burglar alarm

Then, a sequence of events happens:

  • An IR LED turns on my stereo
  • A sound file I cobbled together starts playing. It starts with a computerized voice saying that your entry been detected and recorded, then it beeps for a while (giving you time to disarm), and then it goes into a blaring siren noise. Quite the escalation!
  • A light blinks on and off a few times
  • A camera takes a series of snapshots of the area
  • An email is sent to my phone with the snapshots, showing me what’s going on

Continue reading Raspberry Pi + Home Assistant DIY Burglar Alarm

Adding IR send and receive capabilities to my home-assistant based smart home

Or, how to control your A/C from your phone

You may have seen my earlier post about my new home automation setup. Well I just upgraded it! I just finished testing out a new infrared remote control send and receive capability and it is pure awesome. Here’s what I can do now from within home-assistant:

  • Turn on and off my new DeLonghi portable air conditioner from Costco, and set the temperature to whatever
  • Turn on and off the TV and press any buttons from its remote control
  • Turn on and off the stereo and virtually press any buttons from its remote control (which I never even had!)
  • Use an old random blueray remote control to set scenes, turn lights on and off, etc. (basically do anything under home-assistant’s control)

This post is about how I did it on my Raspberry Pi.

Update: A video!

Annotated setup with IR sensors

Continue reading Adding IR send and receive capabilities to my home-assistant based smart home

My experience totaling my motorcycle after slipping on streetcar rails

I went out for a quick motorcycle ride one night just as it started to rain. I turned South from Pine onto Broadway towards the gas station and the next thing I knew I was sliding along the ground in the middle of the road. I wasn’t being careful enough and had slipped on the new streetcar rails.

A guy helped me lift the bike and get it out of traffic. I was dazed but the only damage I could see was a broken side-stand. The bike started fine and I rode it a few blocks home, defeated. I leaned it up against the wall and checked for more damage. There was a little scuff on the engine and the side case was scratched. And there was a small ding in the front wheel from hitting the center divider. Not bad. Looks like the crashbars did their job!

Continue reading My experience totaling my motorcycle after slipping on streetcar rails

Using an Aeotech recessed door sensor in home-assistant

I went over to my friend’s house last night to help get home-assistant on a Raspberry Pi working for his z-wave door sensors and siren. The Ecotech sensor showed up fine but the Aeotech recessed one (Recessed Door Sensor Gen5 , ZW089-A) did not show up by default. After poking around in the logs for a while, we realized that it was sending BASIC_SET command classes instead of BinarySensor commands. Apparently (I thought I had fixed this already), home-assistant doesn’t recognize the BASIC_SET commands just yet. But looking at the config file for this device, we discovered that config item 121 changes which kind of report this sensor sends:
Continue reading Using an Aeotech recessed door sensor in home-assistant

Building a low-pass filter to remove alternator whine in my car

Ever since I installed a mobile ham radio in my Subaru, there has been a pretty noticeable whine that revs up and down with the engine. I got a few complaints that the whine was on my transmitted signal. I can also hear the whine through my subwoofer, though it is quieter. I needed a low-pass filter to let the DC from my battery make it through to the radio while blocking any alternator-induced AC coming along for the ride. I was going to just buy one but then I searched the web a little and found that it was fun and easy to build my own!

Continue reading Building a low-pass filter to remove alternator whine in my car

Determine your location from relative FM radio signal strengths

In this post, we’ll take a brief measurement of regular old FM radio stations and try to determine where we are. It’s like a GPS but with local FM transmitters instead of satellites. I did this just for fun. I wonder if it could be used for indoor location and stuff? It is nowhere near as accurate as GPS. But whatever.

calibrated_result1Reading FM radio signal power

The first step is to get a reading of all the nearby radio stations. I used gnuradio and a HackRF One software defined radio. A simple flow-chart that takes the FFT and dumps it to a file is all I needed. I had to throttle the I/O or else my computer would freeze. I used 16M sample rate to have as wide a bandwidth as possible.

The gnuradio flow-chart used to gather data
The gnuradio flow-chart used to gather data

The file sink saves the FFT results in 4-byte integers and just has one 1024-length vector after the other. In a few seconds, I had a 50 MB of data. I did all the post-processing in an external Python script.

Continue reading Determine your location from relative FM radio signal strengths

Home automation with Z-Wave, Home-Assistant, Aeon Multisensor, HUE lights, and a Raspberry Pi 2

I’ve always dreamed of having a “smart-home.” There’s just something cool about being able to flip switches and read sensors and have a program turn a light on when you open the door, but only if it’s dark. This post is about home automation.

UPDATE: I made a video demoing everything:

Continue reading Home automation with Z-Wave, Home-Assistant, Aeon Multisensor, HUE lights, and a Raspberry Pi 2

Why your 32nd birthday is a momentous ordeal

Turning 32 may not seem like a very exciting birthday, but it is! In fact, it is the most momentous birthday of all, because it’s the first time in your life that you can’t show your age on a single hand.

Counting in binary.

First, a refresher. Binary, (popularly known as the number system of computers which think in terms of ons and offs, as represented by ones and zeros) is base-2, whereas good old decimal is base-10.

This means that in the decimal system, we have:

One 0001 100
Ten 0010 101
One hundred 0100 102
One thousand 1000 103

Whereas in the binary system, we have:

One 0001 20
Two 0010 21
Four 0100 22
Eight 1000 23

Continue reading Why your 32nd birthday is a momentous ordeal

My new color-changing LED lights

My wife got me an early birthday present: Philips HUE Color-Changing LED lights! They’re incredible. Look:

Right now I’m controlling them with my tablet, which is fine for changing colors and setting light alarms. I can even control them from afar thanks to my VPN. I will soon hook them up to my nascent Raspberry Pi powered home automation system (using the open-source Home Assistant program) that will allow me to trigger them based on various events like motion, doors opening, weather changing, or whatever. The future is here!

Each bulb uses 9 Watts but makes the same amount of light as a 60W conventional bulb, so they’re very eco-friendly. Furthermore, since they’re LEDs, they should last for 15 years of normal use! They communicate with a little hub that uses the ZigBee protocol (like Bluetooth but lower power and lower speed, good for home automation stuff).

Downside: They’re pretty pricey. I expect their coolness to drive demand enough to bring supply up. Expect these bulbs for $10 in the next few years.