Given those headers, I can get lists those spam scores with a simple chained command in the Junk folder and then also in normal inbox folders<\/p>\n
grep X-Spam-Status: * | awk '{print $3}' | awk -F'=' '{print $2}' > \/home\/nick\/trash_ratings.txt<\/pre>\nThis gives files that have Spam and Ham ratings. Then I just plotted them with matplotlib:<\/p>\n
\"\"\"Plot some info about email server SpamAssasin scores.\"\"\"\r\n\r\nimport matplotlib.pyplot as plt\r\nimport numpy as np\r\n\r\nFILES = ['spam_ratings.txt', 'trash_ratings.txt']\r\n\r\n\r\ndef read(files):\r\n data = []\r\n for fname in files:\r\n print('Reading ' + str(fname) +'.')\r\n with open(fname) as f:\r\n vals = []\r\n for line in f.readlines():\r\n try:\r\n vals.append(float(line))\r\n except ValueError: # occasional bad values show up\r\n pass\r\n data.append(np.array(vals))\r\n return data\r\n\r\n\r\ndef plot(data):\r\n spam, ham = arrays\r\n n, bins, patches = plt.hist(spam, 50, normed=1, \r\n facecolor='red', alpha=0.75, label='Spam')\r\n n, bins, patches = plt.hist(ham, 50, normed=1, \r\n facecolor='green', alpha=0.75, label='Ham')\r\n plt.grid(color='0.90')\r\n plt.xlabel('SpamAssassin Score')\r\n plt.ylabel('Frequency')\r\n plt.title('Spam Scores of my E-mail')\r\n plt.legend() \r\n ax = plt.gca() \r\n ax.axvline(x=5) \r\n ax.annotate('Spam Threshold = 5', xy=(5, 0.2), xytext=(0.6,0.5),\r\n textcoords='axes fraction',\r\n arrowprops=dict(facecolor='black', shrink=0.05,width=2))\r\n plt.show()\r\n\r\n\r\nif __name__ == '__main__':\r\n arrays = read(FILES)\r\n plot(arrays)\r\n\r\n \r\n \r\n \r\n<\/pre>\nResults:<\/p>\n
![\"\"](\"https:\/\/partofthething.com\/thoughts\/wp-content\/uploads\/plot.png\")
<\/a>All the red below the threshold gets into my inbox and I have to file it manually, so that’s no good. I can retrain SpamAssassin to do better with its Bayes filters and that improves things for a while, but for now it’s clear that I probably shouldn’t reduce the threshold much because I do have ham coming in all the way up to 5.0 (most of which are encrypted Facebook notifications).<\/p>\nI had a few outliers in the green at first with very high spam scores that I clearly had just forgotten to put in the Junk folder. I found them with a series of commands:<\/p>\n
First, print out the exact spam score of the violators:<\/p>\n
grep X-Spam-Status: * | awk '{print $3}' | awk -F'=' '{print $2}'\u00a0 | awk '$1 > 5 {print ;}'\r\n<\/pre>\nThat gave a 28.3. Then I can just do a simple grep to find the exact message and refile it correctly:<\/p>\n
grep \"score=28.3\"\u00a0 *<\/pre>\nAfter retraining, I’ll see if I can get ham with lower scores.<\/p>\n
Ah yes, running your own email server is fun!<\/p>\n","protected":false},"excerpt":{"rendered":"
I use SpamAssassin on my e-mail server to flag spam messages that come to my addresses. It uses a series of checks on each message and determines a Spam Score. If the Score is above a user-defined threshold, it adds a header that says that it is spam. Then dovecot files it away into a … Continue reading Spam statistics from my email server<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"activitypub_content_warning":"","activitypub_content_visibility":"","activitypub_max_image_attachments":4,"activitypub_interaction_policy_quote":"anyone","activitypub_status":"","footnotes":""},"categories":[3],"tags":[],"class_list":["post-1323","post","type-post","status-publish","format-standard","hentry","category-computers"],"_links":{"self":[{"href":"https:\/\/partofthething.com\/thoughts\/wp-json\/wp\/v2\/posts\/1323","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/partofthething.com\/thoughts\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/partofthething.com\/thoughts\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/partofthething.com\/thoughts\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/partofthething.com\/thoughts\/wp-json\/wp\/v2\/comments?post=1323"}],"version-history":[{"count":3,"href":"https:\/\/partofthething.com\/thoughts\/wp-json\/wp\/v2\/posts\/1323\/revisions"}],"predecessor-version":[{"id":1388,"href":"https:\/\/partofthething.com\/thoughts\/wp-json\/wp\/v2\/posts\/1323\/revisions\/1388"}],"wp:attachment":[{"href":"https:\/\/partofthething.com\/thoughts\/wp-json\/wp\/v2\/media?parent=1323"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/partofthething.com\/thoughts\/wp-json\/wp\/v2\/categories?post=1323"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/partofthething.com\/thoughts\/wp-json\/wp\/v2\/tags?post=1323"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}