BOPM/FAQ
From Blitzed
[edit] General Information
This page is intended to answer the most frequent questions we get about BOPM. If you can't see the answer to your question here, please investigate the other support options
[edit] What is BOPM?
BOPM (Blitzed Open Proxy Monitor) is an open proxy monitoring bot initially designed for Bahamut and Hybrid based IRCDs. The bot is designed to monitor an individual server (all servers on the network have to run their own bot) with a local o: line and monitor connections. When a client connects to the server, BOPM will scan the connection for insecure proxies. Insecure proxies are determined by attempting to connect the proxy back to another host (usually the IRC server in question).
BOPM is written ground-up in C language, concept derived from wgmon. It improves on wgmon with HTTP support, faster scanning (it can scan clients simultaneously), better layout (scalability), and dnsbl support.
[edit] Will BOPM work for me?
BOPM has a few requirements:
- An IRCD which presents connection notices in a format which BOPM can be made to recognize. As long as your IRCD has connection notices which include the IP address of the user then this will be possible via the connregex config directive.
- A host with full connectivity for all the ports you wish to scan, i.e. is not transparently proxied -- many domestic internet connections have port 80 transparently proxied and this produces completely unpredictable results, sometimes as severe as 100% of clients being K:lined!
- A unix OS with GNU Make, GNU autotools, an ANSI C compiler etc..
- Permission from your users to portscan them for open proxies.
To install BOPM will require you to have a basic understanding of how to build software with GNU autotools. BOPM has been developed under Linux and FreeBSD but should work on any reasonably modern UNIX OS. No Windows ports are planned; please do not ask. If BOPM does not work on your UNIX OS, please contact us and we will try to help you. What is the latest version of BOPM?
The current version recommended for production use is BOPM 3.1.2. There is currently no development version except for what can be got out of CVS.
[edit] Where can I download it?
Download instructions can be found on the main BOPM page.
[edit] Blitzed is small but my network is large, how do I know BOPM will work?
It is true that Blitzed is rather small, and we did at first only write BOPM for ourselves. However, since January 2002 many other networks have taken an interest in BOPM and we have been receptive to their needs.
BOPM now counts some of the largest IRC servers in the world amongst its users. Admins on EFnet, IRCNet, Freenode and others use BOPM. Some of these servers deal with 400 client connections per minute under normal load. By way of comparison, at the time of writing the average Blitzed server has 75 users on it and it handles ~0.8 connections per minute; the entire network has ~1000 users on it. Similar sized networks will probably have a similarly tiny connection rate.
We suspect that some of the other larger networks use our software without telling us. Some IRCD developers are also starting to take an interest in linking their software with libopm.
If you are from a major IRC network, we would be very interested in talking with you about how we can help you cut down the amount of insecure proxy abuse on your network. In return you can help us grow our blacklist to benefit ourselves, other IRC networks and the internet community.
[edit] Can I get access to the insecure proxy blacklist that BOPM uses?
Possibly. If you can demonstrate a legitimate need and we feel we can trust you with the information. Please apply by email to opm-xfr@blitzed.org. In the email you send you will need to explain why simply querying each IP address you see against our DNS-based blacklist is not sufficient for you. Note that several large IRC servers on EFnet and IRCnet work precisely this way without any operational problems, so if you need access to our list for IRC purposes then please explain why this method is still unsuitable for you.
If you are reading this because you asked us on IRC, please don't continue to argue with us there. The only way to get access to this list is to send mail as described in the previous paragraph.
People like this need not apply:
*** CGI446 (cgiirc@pcp01269185pcs.midltn01.ct.comcast.net) has joined #blitzed
<CGI446> Hello
<IZS> hi
<CGI446> a place I can download a list of the blacklisted proxies?
<grifferz> for what purpose?
<CGI446> is there*
<CGI446> Is there gritterz?
<grifferz> for what purpose?
<CGI446> My bopm on my network seems to be not functioning properly.
<CGI446> And I want to test it on the blacklisted Proxies.
<grifferz> I'll give you the address of 1 proxy
<grifferz> for testing purposes
*** Teh1337 (rep@pcp01269185pcs.midltn01.ct.comcast.net) has joined #Blitzed
*** CGI446 (cgiirc@pcp01269185pcs.midltn01.ct.comcast.net) Quit (Quit: CGI:IRC 0.5.1 (EOF))
<grifferz> 80.235.36.208
<grifferz> is a socks 4/5 on port 1080
<grifferz> and a port 80 http post
<Teh1337> where do you get that info?
<Teh1337> What list does BOPM check?
<grifferz> from our blacklist
<grifferz> opm.blitzed.org
<Teh1337> I tried to open that website
<grifferz> what website?
<IZS> its not a website
<Teh1337> opm.blitzed.org and nothing happens
<Teh1337> it doesn't give a list
<Teh1337> Then how do you view the list?
<IZS> you dont
<Teh1337> Why?
<grifferz> why would you need to?
<Teh1337> My proxy site is losing hits and I need to update it with up to date proxies.
<grifferz> oh dear, that wasn't the original reason you gave me
<grifferz> so which is the lie
<grifferz> the first thing you said, or the new thing you're saying?
<Teh1337> I just need a list. where can I get it?
<IZS> <@IZS> you dont
<grifferz> Teh1337: you are lower than a snake's dick and should be taken outside and
shot by every respectable internet user.
<grifferz> you will get no assistance here
<Teh1337> If you made hundreds of dollars a month with it
<Teh1337> you would too
<Teh1337> I hope this network survives
<Teh1337> Does Bopm check 1000 users proxies at once?
<Teh1337> when they all connect instantenous?
<Teh1337> on a botnet?
<grifferz> yes
<Teh1337> 1000+
<grifferz> yup
<Teh1337> I hope so.
<grifferz> is that a threat?
<Teh1337> Seeing as though it didn't survie the other network
<Teh1337> that ran bopm
<Teh1337> Flood?
<Teh1337> I dn't flood
<Teh1337> I'll ping the fuck out of the network
<grifferz> oh dear
<Teh1337> Why the hell would I flood?
<Teh1337> thats pointles
<Teh1337> accomplishes nothing
*** Teh1337 (rep@pcp01269185pcs.midltn01.ct.comcast.net) Quit (Quit: )
[edit] How do I get BOPM out of your CVS?
First of all, do you really need to do this? We can't support your use of unreleased CVS versions of BOPM; they are for developers and those who know how to use CVS. For that reason, this FAQ will not go into detail about the use of CVS itself.
All of our releases come set up ready to do anonymous CVS. If you have already downloaded a version of BOPM, you can change into its base directory and type:
$ cvs up -dP
Unfortantely the CVS Root has recently changed, so this may not work on released versions.
If you want to get the latest CVS from scratch, try the following:
$ cvs -d :pserver:anon@cvs.blitzed.org:/ login Password: (just press return at password prompt) $ cvs -d :pserver:anon@cvs.blitzed.org:/ co bopm $ cd src $ cvs co libopm $ cd ..
After this, the build instructions are the same as if you had downloaded a release.
[edit] Compilation
If you have a problem in configuring or compiling BOPM, hopefully you will find its answer here.
[edit] I type make and it says something about a missing separator..?
BOPM requires GNU Make. This error is commonly seen on BSD systems whose make utility is not usually GNU Make. GNU Make is often installed as gmake on non-Linux systems, so any time you would type make, type gmake instead.
If that does not work, try to find out if your make really is GNU Make or not. Your sysadmin should be able to help.
[edit] make install went fine but I have no idea where it has put anything!
As of BOPM 3.0, we now use GNU autotools to configure, make and install. What this means is that BOPM installs to a prefix. The prefix is kind of the base directory under which all the other BOPM files go, and it defaults to $HOME/bopm/ i.e. a directory called bopm inside your home directory.
If you don't want BOPM to go there, you can change it by using:
$ ./configure --prefix=/some/other/directory
Once you have done the make install (provided there were no errors) you will have several subdirectories inside PREFIX. You will find the BOPM binary as PREFIX/bin/bopm, and the sample config will have been installed as PREFIX/etc/bopm.conf. You don't need to move these files; BOPM 3.x does not run all out of one directory like BOPM 2.x did.
[edit] During ./configure it said something about set owner/group Operation not permitted..?
The short answer is: this is just a warning, so just ignore it.
The longer answer is: this happens most often on BSD machines which have the /tmp filesystem mounted with some particular mount options that cause all files put there to have their owner/group changed. GNU Autoconf notices this and when it tries to move some files out of /tmp back to where they should be, it tries to reset the ownership. Unless you are root this will fail, hence the warning. It is harmless, however.
[edit] I reran ./configure with different options but they do not seem to have been picked up.
Make sure to do a make distclean if you ever need to change ./configure options, otherwise you will experience strange things such as all the files being installed in a new PREFIX, but the config file still being read from the old PREFIX.
[edit] Configuration
This section hopes to answer various questions regarding BOPM's configuration.
[edit] How do I make BOPM recognize my IRCD's connection notices?
The connregex config directive tells BOPM how to recognize a connection notice. There are plenty of examples for major IRCDs in the same config file, and more will be added here as they are contributed between releases.
Please note that the connregex is a fairly complicated POSIX regular expression. The consequences of having an incorrect expression range from BOPM not doing anything to BOPM actually crashing. As a result, only experienced BOPM users should try to edit or create their own connregex -- we will not support you if you try to use anything but one of the suggested settings of connregex!
[edit] How do I make my scans come from a particular IP address on my server?
You need to use the vhost directive inside each scanner {} block. Note that this was broken in the BOPM 3.0 release. It has been fixed in version 3.0.1.
[edit] How do I make BOPM work with this DNSBL?
We can't really provide documentation for every DNSBL there is, mainly because the policies and procedures of those DNSBLs may change from time to time. So, this section will give links to the relevant documentation for some DNSBLs.
[edit] Tor DNSBL
Tor is an effort by the Electronic Frontier Foundation to provide completely anonymised Internet use. As a result the possibility for abuse of poorly-designed protocols like IRC is immense. There exists a Tor DNSBL with instructions for configuring in BOPM.
[edit] IRCD Compatibility Notes
Some IRCDs need some additional or specific settings to work with BOPM.
[edit] Bahamut
You may want to give BOPM operflag F (available since bahamut 1.4.34). This allows it to send IRC commands (like KLINEs and PRIVMSGs) unthrottled, resulting in much better response under heavy attacks.
O:hostmask:password:nickname:OF:class
Of course you need to make sure that BOPM uses it by setting itself umode +F through the BOPM mode configuration option.
The following connregex is known to work for bahamut 1.4.35 and prior:
connregex = "\\*\\*\\* Notice -- Client connecting: ([^ ]+) \\(([^@]+)@([^\\)]+)\\)";
The following connregex is known to work with both 1.4.36+ and 1.8.x:
connregex = "\\*\\*\\* Client -- Client connecting: ([^ ]+) \\(([^@]+)@([^\\)]+)\\)";
[edit] UltimateIRCd
We have successfully made BOPM work on UltimateIRCd (tested with 2.8.8 and 3.0.0.b3) using the following connregex:
connregex = "\\*\\*\\* ^BConnect/Exit^B -- from [^:]+: Client connecting on port [0-9]+: ([^ ]+) \\(([^@]+)@([^\\)]+)\\) \\[([0-9\\.]+)\\].*";
Note those ^Bs are actually control-B characters as used on IRC to make text bold. To insert them using vi type control-V followed by control-B. This procedure will be familiar to you if you have ever tried to put bolds into an IRCD MOTD or similar.
[edit] StarIRCD/SorIRCD
connregex = "\\*\\*\\* Notice -- Client connecting on port [0-9]+: ([^ ]+) \\(([^@]+)@([^\\)]+)\\) \\[([0-9\\.]+)\\].*";
[edit] Conference Room
We were under the impression that Conference Room does not have connect notices that contain the IP, but instead contain the host (which is sometimes the IP if it didn't resolve properly, but not always). This meant that CR would not work with BOPM. However, Roby (idolatry2000[at]ticino[dot]com) tells us that this now does work. The connregex he uses is:
connregex = "\\*\\*\\* Notice -- Client connecting on port [0-9]+: ([^ ]+) \\(([^@]+)@(([^\\)]+))\\).*";
We can't see how this would work, since it specifies a connection notice that always shows the host as an IP, but Roby says it does. Please let us know how you get on so we can clarify this FAQ, but if it doesn't work for you there's nothing we can do, you'll need to ask the CR people.
[edit] UnrealIRCD
UnrealIRCD works with BOPM since at least v3.1.2. All versions of Unreal need the following to be uncommented in bopm.conf:
perform = "PROTOCTL HCN";
in the IRC {} section. Once this is done, a connregex as for Bahamut/Hybrid will work for Unreal too, for scanning users connecting to the local server.
prince[at]avalon[dot]zirc[dot]org also tells us the following:
- Unreal 3.1.3 seems to require opers to be global if they are to see connection notices.
- Later versions of Unreal require different user modes to be set on BOPM for it to see connection notices. You will need to use
mode = "+sc";
in your bopm.conf.
We would appreciate clarification of these points, and if they are still valid with current versions of Unreal.
[edit] Using Unreal for proxy scanning a whole network
Since UnrealIRCD 3.2.1, an option has been added to see "far connect notices" in hybrid-compatible notice format (HCN). This allows an oper on one server to see connects to all servers, and consequently will allow a BOPM to protect the whole network.
This does require a modified connregex:
connregex = "\\*\\*\\* Notice -- Client connecting.*: ([^ ]+) \\(([^@]+)@([^\\)]+)\\) \\[([0-9\\.]+)\\].*";
Thanks to webmaster[at]universe-x[dot]de for this info.
[edit] Operational Notes
This section aims to answer questions you may have about the operation of BOPM.
[edit] Is there a way to see what is going on in more detail? A debug mode?
Yes!
Run it like: bopm -d. Add more -d's to get different levels of debug:
-d
- General debug over actions BOPM is performing.
-dd
- All IRC traffic. Use this mode to see if BOPM is able to correctly oper itself or if it is seeing connect notices, for example.
-ddd
- Scan content logging. This will also print every line of text that comes from proxy scans. This is not likely to be useful to anyone except developers.
Remember: fatal errors should always be logged in bopm.log!
[edit] My BOPM keeps giving errors saying "Lookup error" every time someone connects
The most common cause for this is that your DNS setup is wrong.
Please note that opm.blitzed.org is now deliberately returning Server failure (SERVFAIL), see OPM status. If your problem is not with opm.blitzed.org, read on.
Your system resolver library and the common DNS diagnostic software like dig or host look in /etc/resolv.conf to find a list of nameservers and ask them one at a time the question you gave it. As long as one of them responds, it works (albeit sometimes slowly if you have listed a number of broken nameservers).
BOPM, however, being a somewhat more intensive DNS application, does not use any system libraries for its DNS needs. It uses a slightly modified version of the FireDNS asynchronous DNS library. FireDNS also reads /etc/resolv.conf, but instead of using each nameserver in turn, it sends the query to all nameservers simultaneously. Whichever one replies first is the answer that is used, and any errors with any of them will be reported.
What this means is that it is quite common for a broken nameserver to end up in your /etc/resolv.conf, you don't notice because you don't happen to see anything running slightly slower (even though it probably is), but then you run BOPM and get a bunch of errors.
If you are seeing something like this then you need to look in your /etc/resolv.conf and test each nameserver individually, if only to rule out this situation which is very common amongst BOPM users.
For example, if your /etc/resolv.conf looks like this:
domain example.com search example.com nameserver 192.168.0.53 nameserver 192.168.1.53
Then you should test a typical BOPM query against both 192.168.0.53 and 192.168.1.53:
$ host 2.0.0.127.opm.blitzed.org 192.168.0.53 Using domain server: Name: 192.168.0.53 Address: 192.168.0.53#53 Aliases: 2.0.0.127.opm.blitzed.org has address 127.1.0.31 $ host 2.0.0.127.opm.blitzed.org 192.168.1.53 ;; connection timed out; no servers could be reached
So there is your problem: 192.168.1.53 is not reachable. There could be other errors you will see, but whatever error you see, you should take that nameserver out of /etc/resolv.conf until it has been fixed.
/etc/resolv.conf can normally only be edited by root, so if you don't want to wait you can instead make a file called firedns.conf in the same place as your bopm.conf. You would just put IP addresses in this file (no need for the "nameserver" part), and then BOPM will only use those IPs.
If, however, trying each of your nameservers did not produce any error, then this is not the problem.
[edit] BOPM doesn't detect any proxies / doesn't work!
Before you can ask anyone to help you with a problem such as this, you need to give us more details.
BOPM needs a couple of things to be able to scan users who connect to your server. Firstly, it needs to be opered and have the correct umode(s) for seeing connection notices. Secondly it has to be able to translate a connection notice into nick, hostname and IP address.
So, firstly check that your BOPM really is opered. /whois will probably tell you. If it isn't, running it in debug should help.
Next, be sure that it is seeing and parsing connection notices properly. You'll need to use debug mode to discover this. Check that you can see the connection notice, and that BOPM says that it has caught it. BOPM logs
IRC REGEX -> Regular expression caught connection notice. Parsing.
when it sees a line that it recognises as a connect notice.
If you get this far and you've verified that both of the above are happening, now it is time to ask for more help. To avoid being harshly redirected back here, please mention that you have read the FAQ and verified that your BOPM is both opered and seeing connect notices.
Note that BOPM 3.x does not add K:lines (or whatever) when it finds proxies in a manual scan (bopmnick check 1.2.3.4). This is a change from BOPM 2.x.
[edit] Is there a crontab script to restart BOPM?
An example script which will require only slight editing is included in BOPM's release files as contrib/crontab/bopmchk
[edit] Feature Requests
Some common feature requests we get, and what we intend to do about them.
[edit] Can BOPM scan users connecting to any server on my network?
The answer right now is "maybe". There is also some doubt over whether you would actually want to.
Remember that you can run multiple BOPM on one machine. Each BOPM can connect to a different IRC server to protect that server's users. This can be helpful in situations where you have a Windows IRCD (because there is no Windows version of BOPM), where shell process limits prevent the running of BOPM, or where port scanning from a particular host is not desirable.
We do not however regard this to be an ideal solution. We would prefer to write a version of BOPM that connects as a server and parses NICK commands in order to scan all users. Such a thing would be quite easy to write using libopm and the shell of BOPM, but we don't really see a huge demand just yet.
As for whether or not you would even want to do this...
Advantages
- Central maintenance - only one piece of software to install, configure, maintain, upgrade.
- Less abuse problems - only one IP address to publicise as a scanner, only one parent ISP to explain your need for proxy scanning to.
Disadvantages
- Easier to attack - only one scanner machine to attack, and then the whole network is unprotected. In the scenario where the scanner is on the IRCD machine, attacking this machine also denies access to IRC.
- Single point of failure - should the net-wide scanner die for any reason, the whole network is vulnerable.
Note that some Unreal IRCD users suggest using the +F mode (far connect notices) to try to make one BOPM scan the whole network. The authors of Unreal IRCD tell us that this will not work since remote servers send the hostname, not the IP, and so this would require modification to Unreal or BOPM or both.
We've now heard of several networks using the BOPM Helper Module for Unreal IRCd. This module shuld be loaded on all servers, and will then allow a BOPM on one of the servers to proxy scan users from them all. We don't personally use this software but we know a few people who do and it works for them, so try it out if you want to have just one BOPM.
We are now also told that since v3.2.1, UnrealIRCD can present far connect notices in hybrid-compatible notice format. This should allow one BOPM to monitor the entire network, see Using Unreal for proxy scanning a whole network above.
[edit] Can BOPM stop trojans and other abusive bots?
Not unless they are coming at you from open proxies, no.
Please please please don't ask us to add checks for anything that you regard as "bad" without first considering exactly how the "bad thing" is detected. BOPM does not work on the principle of "this looks bad, let's ban it". BOPM works on the principle of "I know how to use a bad thing, so I will try to use you as if you are one. If you allow me to use you, then I know you are bad and I will ban you."
Many people at this point say things like, "yes, but there are ports you can scan...". They are wrong. In order to fit in with the BOPM philosophy, BOPM would have to know what port the trojan was on, how to access the program on that port, and how to use it to do something. Most trojans are passworded, most don't even listen on any ports but instead take their commands via normal messages in IRC channels. It is simply not realistic.
If you know of a piece of software that is reasonably common and has a known protocol for making a TCP connection out, then we can probably add it to BOPM. The purpose of BOPM is to defend IRC servers against mass attacks using well known insecure proxy servers such as those that are found on many web pages and traded between those who would abuse them. We will not bloat it by adding checks for certain patterns of nick/username/realname or the like. Such checks need to be flexible, and are better done in other software written for that purpose. We are only interested in independently verifiable insecure proxy software.
[edit] Can you make a DNSBL that contains drones, trojans and other hostile bots?
We could, but we will not, for reasons detailed in the previous question.
DNS-based blacklists are very dangerous things. The organization running the list is giving its guarantee that all of the hosts it contains are "bad" in whatever way. It is of utmost importance that the criteria for being listed are well defined. It is preferable if the list is maintained in an automatic fashion to avoid any accusations of people being added or removed for personal reasons.
All of that is quite easy with something as clear-cut as insecure proxies. Either a host has an insecure proxy on it, or it does not. If it does then logs of the proxy being (ab)used can be kept to justify the listing. If the host can be shown to no longer have an insecure proxy, the host can be removed.
No such procedures can be followed when it comes to trojans, drones, "bad people" and so on. Listings would often be completely arbitrary, and down to the personal judgement of someone. Once something is listed there would be no way to decide if it is "fixed" or not. We believe it would soon become a complete and utter shambles and we want no part of it.
If you do however wish to make a blacklist of whatever sort of hosts you like, do feel free, but don't ask us how to do it.
[edit] I like using DNSBLs but I want local control over what is banned, will you implement this?
We'd really rather that you picked sensible DNSBLs to use -- many DNSBL aimed at email such as the CBL (and therefore Spamhaus XBL) and NJABL are not recommended for use on IRC because they list a large amount of dynamic IP space with long expiry periods (if at all).
However, we recognise that it may sometimes be necessary to allow users online even when they are blacklisted by a reputable DNSBL. For this we suggest using Mark Bergsma's whitelists patch.
[edit] Contributed Patches
This section lists the patches that have been contributed to the BOPM project. Please note that we cannot support anyone who uses any of these patches, they are in no way official and come with no form of guarantee.
- Improvements to logfile handling, and channel rejoin feature
- Modifications for ircnet use
- DNS whitelists support
- HTTP GET proxy detection
[edit] Other Support
This section contains details about other ways of getting help with BOPM. Please make sure you have thoroughly read all documentation that comes with BOPM (e.g. the README file) -- and this FAQ -- before looking elsewhere.
[edit] Mailing Lists
There are several mailing lists related to the BOPM project.
Support queries and feature requests should be directed to the BOPM mailing list. CVS commit emails are also sent here, and this is the place for patches to be contributed (unless they are massive, in which case please just post a link to your patch).
Closely associated with BOPM is the OPM DNS-based blacklist (DNSBL) that we maintain. All users of BOPM who report their proxies to OPM should subscribe to opm-announce, a read-only list for important announcements regarding the OPM service.
The opm-talk list is for discussion and feature requests regarding the OPM service.
Note: There is NO IRC support for BOPM!