Bootland Server Serously Ill

I know that user problems with this boot-land.net/winbuilder.net multi-domain server have already been mentioned here many times in the past, but, at the risk of beating a dead horse, I would respectfully suggest that some kind of help is badly needed. In fact, the situation appears to be getting worse rather than better and I'm certain that I'm not the only one who would gladly provide whatever assistance might be acceptable to the owners/administrators.

Even if the "Internal Server Errors" are left aside and without mentioning publicly any vulnerability aspects, it would seem apparent that loading factors alone are a major contributor to the ongoing issues. In that context alone, there's plenty of disc space and bandwidth available for the asking and I, for one, would happily contribute some. I feel quite sure that others would do the same if that would help to spread the load a bit more efficiently. It seems a shame that such a useful site is being so badly crippled by needless obstacles to its availablility.

Hi Arvy.

This matter was also mentioned on the header of our weekly newsletter: http://boot-land.net...Newsletter.html

---------

You're right about needing help.

Our troubles are related to the resources allocated by MySQL processing that is being drained by the increase of visitors over the last months, so what we need is simple.

A dedicated server with at least 1Gb of RAM and a Pentium class IV processor.
---------

We have excellent bandwidth conditions on our server that are currently supporting around 4Tb for month without any worries but mySQL would be better placed at an external location as it is shared with other accounts on this hosting provider and I suspect that they limit the available DB resources.

Renting a dedicated server is too costly and would rely on donations or google ads which is not an acceptable policy to keep our site clean and independent.

To avoid costly rentals I volunteer myself to buy a dedicated server with good conditions exclusively for this task but I do live on a small island with no available quality data centers and the costs of locating a server on my country or the rest of Europe are also a bit too prohibitive for an healthy long term maintenance.

So what we really need is someone somewhere capable of "allowing" our server to be placed on their farm and possibly provide some basic support occasionally (check hardware conditions, change any broken component, etc).

This is an heavy responsibility and therefore would prefer to it myself once I get back to mainland over the next year where I should have good conditions to find a reliable data center near to where I live so that everything can be periodically monitored. I also keep a few childhood friends/professors that work in my local hometown university and they might be available to help completely for free since our work and knowledge on this site is shared to the public domain.

----

As solution, maybe someone else knows closely enough some IT staff at an university that can help with the necessary internet conditions to provide good network quality or perhaps some company that would be interested in providing support in return of a logo image placed on the "Recommended" box.

Recently I got in contact with R1Soft and another software company that keep a tradition of helping sites with hosting and such but our requirements are very specific and so we're a bit difficult case to help as some would say.

In either case, I'm quite sure that things will get better sooner or later.

Let's continue the talk about possible solutions and see what can be done.

I've also disabled most of the addons on the board such as mkportal, this increased the overall performance but now it just feels a bit empty without the left column.

Any solutions for this matter?


We might end up not needing any external server with some luck.

A dedicated server with at least 1Gb of RAM and a Pentium class IV processor.

Sorry Nuno, but that seems wrong.
I think some software solution should be good enough to get our troubles handled. Throwing more hardware at a problem is usually the fastest solution, but seldom really necessary.
How about two databases for instance? One holding only the data of the last week or so and a big one which holds the rest.
The smaller one would be way more used and have at the same time a better performance.

Sorry I can't help with a server farm for you, Nuno. I once talked Bell Canada into co-locating a server for me at their network centre, but the vice-president who let me do that got "transferred" to another job shortly afterward. Anyhow, I'm a long way away from your physical location and I can certainly understand that you'd prefer to manage your own hardware.

It sounds like you've done some analysis of the current loading issues and have a plan for dealing with it when the opportunity is right. I guess, in the circumstances, there's not really much an "outsider" like me can do to help unless there's some non-critical stuff you'd like to offload on a temporary basis. I have unlimited MySQL databases available if that would be of any use to you. Have you optimized you own dB tables recently?

As for the "empty" forum pages, I personally don't mind the missing sidebar at all. And it does seem to help with the performance issues. But others may have a different opinion.

mkportal...
but now it just feels a bit empty without the left column.

Yes it does feel a bit Empty. I do notice a faster reponse time.

Hi folks ,

I'm sure whether the problem is at my end but I am having a great deal of trouble accessing Boot-Lands web-site (near impossible practically locked up), even with this original layout yet I am able to access other sites quite easily! Also the Web pages take a very long time to load & at the end often crash, also I'm being re-directed to the chat forum all the time!

Can I make a very selfish suggestion, if these problems are due to the high frequency of hits on the site can you not make it that only members are allowed access to the site, they can still be anon. I notice it on other forums you are not allowed to view the forums unless you have entered a pass word & user name. Put up a popup that informs the casual visitor as to the reason why this has had to be introduced. It seems unfair that the registered users have to suffer at the expense of visitors.

Or split the server into two areas one for registered users and one for visitors & cap the amount of visitors that are allowed to visit at one time. At least the site would remain functional, as it is all are suffering. I know this suggestion seems harsh but it would benefit the registered users even it was adopted for a short period of time until a longer term solution can be found. Not sure if these suggestions are practical but just ideas anyway? 1 further thing I prefer this colour scheme its easier on the eyes things stand out more

Regards & Respect,

ispy

Or split the server into two areas one for registered users and one for visitors & cap the amount of visitors that are allowed to visit at one time. At least the site would remain functional, as it is all are suffering. I know this suggestion seems harsh but it would benefit the registered users even it was adopted for a short period of time until a longer term solution can be found. Not sure if these sugg 1 further thing I prefer this colour scheme its easier on the eyes things stand out more

As known, I'm also a 'black and white painter' And I think that ispy's suggestion is very worth to be considered carefully.

Peter

Generally speaking, trying to second guess any site administator's analysis of server problems is a fool's errand for anyone else without admin privileges of his own. In any case, I don't doubt Nuno's conclusions about the resources that are currently available for MySQL processing being a bottleneck. It may not be the only one, but, if Nuno thinks it's the primary factor, I certainly couldn't argue with his being in the best position to know.

Even so, I can't help wondering if there might not be at least some measures that could be taken to alleviate that problem whilst awaiting the opportunity to implement a hardware solution. I know that my own SQL databases can get very messy very quickly, even although I use multiple smaller dBs rather than a single large one for my site. That's why I asked the question about recent optimization. Perhaps I or someone else could assist in that area if trusted with access via phpMyAdmin or some similar management tool.

Hi Arvy ,

Good Point! Point taken! Appologies extended! Offence removed!

R&R,

ispy

The forum database is centric, there is no change of splitting the load across different external mySQL servers, nor I have available separate mySQL servers with capacity for this task and I also lack the time or conditions to experiment anything on the live board since it is not a good time to perform extensive changes on the functioning.

I've disabled mkportal to improve performance and it only seems to have made things worse, so regardless of my actions, the more resources allocated, the more spam bots are sent to try get their way inside the forum.


It would be normal to require more resources as more visitors come to boot land but it isn't normal to see this sort of massive visitors all coming down at the same time from no apparent referral site.

We've already had a few episodes like this in the past. They were related to prolonged spam farm attacks and I'm now convinced that our current situation falls under this category based on the given symptoms.

This is just a deduction, even worse if this is an intentional DDoS attack with the intention to bring down the server.

-------------

If we had a server with 8Gb of RAM then it would likely suffer the same issues that we face at this moment.

There are some "smart" techniques to fool spam bots but most of them are related to sites that can easily have their page contents cached. On our there are thousands of pages that would need to be cached which would use even more resources than the ones currently available.

MySQL is our weak point, last time something like this happened I relocated the blog onto blogspot and disabled the boot land wiki that were both based on mySQL.

-------------

Making a distinction between registered members and guests is also not a solution. It would mean a lot of work only to conclude that the same problems still exist, meaning that spam bots don't would still hog the server resources looking for possible weak points in the forum security.

One solution would be taking down the site for a couple of days. This would likely calm them down but I'm guessing that it wouldn't leave the regular boot land members anywhere happy about this.

-------------

So, at the moment all we can do is wait for things to calm and get better.

For those who remember a similar occasion last year, these spam operations may last up to 15 days..

So, at the moment all we can do is wait for things to calm and get better.

For those who remember a similar occasion last year, these spam operations may last up to 15 days..

On several sites there is a randomly generated string shown as picture, therfore unreadable for any crawler.
The user must type the string in order to get access..
Is a way like this possible?
What's about 'serious' search engines in this case?

Peter

We would be seriously hurt on page rank classification and all our pages wouldn't be indexed by google anymore.

All our talks wouldn't show up on the search engines, meaning that it would be almost as if they never took place.

-----

A password would only hurt human people, bots would still hurdle a huge amount of resources just by reading index.php by the thousands (do you remember those days in the summer of 2007?)

We are better off waiting for them to get bored and go pick on some other site..

Hi Nuno ,

Don't know if this is associated but there is a lot of hits at the same time in the online visitors, same article & 39 pages

Also would something like this help:
http://sourceforge.n...ojects/sandtrap
This CGI program can trap malicious robots that spider your website. The program works by blocking access from the bot's ip address. It can also provide an unlimited number of false e-mail addresses to muck up databases of email harvesting bots.

An internet security expert recently has stated that there is a "Flash" CAPTCHA in development that generates a shockwave flash image with letters that spin and revolve and tumble at various rates and in different directions. This would make it impossible in current technology for a spambot to use OCR because the character is constantly changing shape.
CAPTCHA's are being defeated!
I'm Clutching at straws really!

R&R,

ispy

I've also disabled most of the addons on the board such as mkportal, this increased the overall performance but now it just feels a bit empty without the left column.

Any solutions for this matter?


We might end up not needing any external server with some luck.

Try cleaning a few things to make the database smaller.
First thing you could delete are logs in the database:

Email Logs,
Email Error Logs,
Bot Logs,
Task Manager logs,
and Warn Logs.


Logs can chew up a considerable amount of space in a database.

Persistent Db Connection

"persistent" => "1"

Persistent Db Connection

This may make your forum faster by keeping the database connection active so it
can skip reconnecting everytime.

One interesting thing I have seen that can shrink a database is "IPB PM Flush Tool"

"This script attempts to connect to your IPB Database using the information in conf_global.php
and "Flushes" or deletes all Private Messages stored for all members. Very useful if you need
to shrink your MySQL database a bit. It is strongly reccommended that you allow your members
to back up their saved PM's before running this tool."

http://tools.invisio...ndex.php/f/4223

But it is for Invision Power Board v2.0

"This script attempts to connect to your IPB Database using the information in conf_global.php
and "Flushes" or deletes all Private Messages stored for all members. Very useful if you need
to shrink your MySQL database a bit.

Does that mean:

(I still have all of my PMs with their attachements etc.)
If I (and all other members too) purge my PMs, that can help a lot?

Peter

... so regardless of my actions, the more resources allocated, the more spam bots are sent to try get their way inside the forum.

That is unquestionably a major part of the problem along with some quite serious server vulnerabilities that I mentioned to you in our earlier exchanges via private email. I think we should probably avoid any public discussion in that area, but, as I said at the time, the world is unfortunately full of people who consider any "loophole" something to be exploited for purely malicious purposes.

There are some "smart" techniques to fool spam bots but most of them are related to sites that can easily have their page contents cached. On our there are thousands of pages that would need to be cached which would use even more resources than the ones currently available.

Have you considered Michael Hampton's "Bad Behavior" spam blocker? I've had considerable success in customising it for use with several content managment systems. If you think it might help, I could probably build you a version to work with your Invision board.

MySQL is our weak point, last time something like this happened I relocated the blog onto blogspot and disabled the boot land wiki that were both based on mySQL.

MySQL shouldn't create more vunerabilities than any other type of interactive process, provided that all of the inputs and queries are properly sanitized. That's not really a MySQL issue. It depends on each individual application and some authors of PHP and other scripts, whether using MySQL or not, are definitely more conscientious about doing so than others. Generally speaking, sanitizing inputs can be accomodated quite simply with appropriate preg_replace functions, but I can't claim to be sufficently familiar with Invision's code to comment on it in particular.

Anyhow, regardless of what approach you choose, I wish you and all of your users (except the dishonest ones) the very best of good luck.

Does that mean:

If I (and all other members too) purge my PMs, that can help a lot?

Peter

thats a lot better idea

thats a lot better idea

I just freed more than 1100 PMs (hopefully with attachements)

Peter

I just freed more than 1100 PMs (hopefully with attachements)

Peter

Site working great now, thanks Peter!

Seriously, the site is working great all morning (Mon, local time) after many weeks of trouble. I don't know if it's going to last or what has helped but normally at this time the site would be unusable for me. Regarding the mkportal removal; I, for one, don't miss the clutter

Thank you Nuno for your continued attempts to better the situation

The only medicine, i know against spam bots are timeouts.
One starts with a reasonable timeout, between two consecutive requests from the same ip.
And then eighter increases it bit by bit with each new attemp that falls into the timeout, basicly building up to a complete block.
Or one jumps directly to a large timeout of a couple of minutes, while sending a html page which shows a countdown. (Only to the request which caused the big timout, further requests are simply ignored.)

A protection like this usually does not affect human visitors at all.

Hi folk ,

Some incidental info grabbed off the net:

What I've found to be the most effective, is to make forum #1 a category instead of a normal forum, then place my publically accessible forum inside that category, that way they register, but never get to post anything, because all the spam bots are trying to post into forum #1.

Spammers can hire humans that will register manually at 1000's of forums for them and spam the forums for a small fee. I've seen the sites. Given the increased obscurity of registration processes, it's a growing market - having humans do the spamming for you, which will always defeat the captcha and other registration requirements.

Take a look at this:
http://www.botmaster.net/more1/

Regards,

ispy

That's a good idea Peter, I've also wiped out my personal message box, less 1247 pm's hogging the DB.

Also, the max PM per member has been reduced to 100.

I'm looking for more optimizations but it seems that the bots got tired for today, should we expect another visit tomorrow?

-----

ispy, it's really easy to create your own bot and no need to avoid captcha's, just make your script download index.php inside a loop to drain resources with multiple connections. I'll read the links provided and see what can be used for our own good.

I'll also remove mkportal from the main page. Will code our own solution without resort to their CMS software.

The left column will be added back but we'll be using an internal block method that doesn't hog the server.

If it's war they want, it's what they'll get..

EVERY protection can be circumvented, if one is willing to through enough resources at it.
Since spammers are in the business to make money, they can't afford to throw unreasonable amouts of resources at a protection or they go out of business.
Make the price high enough for a purchase and the consumers will stay away.

Today, I clicked some forum links but I got an annoying chat page.