Smalltalk › Instantiations

How to find out the actual memory usage of an Image at runtime?

_‹ Previous Topic Next Topic _›

Classic

List

Threaded

3 messages Options

jtuchel

How to find out the actual memory usage of an Image at runtime?

We are seeing a spooky bot hammering onto our Seaside App:

kontolino.de:443 136.243.16.7 - - [25/Jan/2015:07:34:31 +0100] "GET /Buchhaltung?_s=GAzGw-mQ5X65ufSV&_k=bcOwu4ti2QL4EEs0&3 HTTP/1.1" 302 229 "-" "Mozilla/5.0 (compatible; MJ12bot/v1.4.5; http://www.majestic12.co.uk/bot.php?+)"

This is spooky for several reasons:

1) This bot sends really a lot of requests over a few minutes
2) It sends _s and _k parameters in its requests, so it either is good in imitating things or somebody knows about Seaside
3) When this wave of requests comes in, our server runs out of Memory and Linux starts killing processes, starting with Apache
4) When this happens, it doesn't answer ssh or http
5) our hoster's console tells me the server is up and running and fine
6) The only thing that helps is restarting the server

The good news:

1) Nobody ever connected onto our machines from strange IP addresses
2) Seaside never starts a DB connection, because there is no logged on user, so the _s and _k parameters are either outdated or random (Our application logs every DB connect and much more for auditing reasons - I am glad it does!!!)
3) Our DB Backups are tested and reliable, so we don't loose any data
4) We have a very good alarm system that tells us the server is not responding, long before we get calls from customers

So it seems the bot is just trying to find weak spots and hasn't found one. For now, we closed the machine for MJ12 bots completely. We don't care if there is a real search engine that is being imitated by bad guys. We just don't want that shit on our machine.

So now of rmy VAST related question...

Here's our current theory on what exactöy is going on
1) Server gets a lot of requests from the bot
2) Seaside creates a lot of 302 Redirects and WASession instances
3) Image gets VERY big very fast
4) DB2 is first to find out it's running out of Memory
5) Linux starts killing processes

We could now do several things:

1) log each WASession creation in an application log
2) write the actual memory size of the Image every 30 seconds or minute or so (we currently write a heartbeat file every minute, to see whether the server is still running. It writes a heartbeat during the attacks asif all was good)
3) Hope that locking out this one bot is the solution to all our problems and will one day bring global peace

There are several drawbacks to these:

1) Would only prove the theory that lots of Sessions get created, but not that they eat a lot of memory
2) We have no idea how to determine how much Memory the Image actually uses
3) is very unlikely to stand for long.

I'd like to work on 2)

So.... How do I measure the amount of MBytes my Image actually uses at a given moment in time?
Is there a (performant) way to tell which classes (resp. instances thereof) fill most of that memory?

This would be the only way to prove that it's our Seaside App that starts the downward spiral before the server goes out of business.

Every hint is welcome

Joachim

--
You received this message because you are subscribed to the Google Groups "VA Smalltalk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at http://groups.google.com/group/va-smalltalk.
For more options, visit https://groups.google.com/d/optout.

Louis LaBrunda

Re: How to find out the actual memory usage of an Image at runtime?

Hi Joachim,

I use the following code (well something close to it) on windows to get the memory being used by a program. I realize you are using Linux but maybe you can find something similar.

Lou

	"Get the Memory Usage from the OS."
	| memoryUsage processMemoryCounters size rc pmc |

		size := 40.
	processMemoryCounters := ByteArray new: size.
	processMemoryCounters uint32At: 0 put: size.

	rc := OSHandle getCurrentProcess getProcessMemoryInfo: processMemoryCounters cb: size.
	"WorkingSetSize is the fourth entry."
	rc ifTrue: [memoryUsage := processMemoryCounters uint32At: 12].

On Monday, January 26, 2015 at 10:54:04 AM UTC-5, Joachim Tuchel wrote:

We are seeing a spooky bot hammering onto our Seaside App:

<a href="http://kontolino.de:443" target="_blank" rel="nofollow" onmousedown="this.href='http://www.google.com/url?q\75http%3A%2F%2Fkontolino.de%3A443\46sa\75D\46sntz\0751\46usg\75AFQjCNG6DyQPGJnjFLSuV_OQQkOUbwPzxA';return true;" onclick="this.href='http://www.google.com/url?q\75http%3A%2F%2Fkontolino.de%3A443\46sa\75D\46sntz\0751\46usg\75AFQjCNG6DyQPGJnjFLSuV_OQQkOUbwPzxA';return true;">kontolino.de:443 136.243.16.7 - - [25/Jan/2015:07:34:31 +0100] "GET /Buchhaltung?_s=GAzGw-mQ5X65ufSV&_k=bcOwu4ti2QL4EEs0&3 HTTP/1.1" 302 229 "-" "Mozilla/5.0 (compatible; MJ12bot/v1.4.5; <a href="http://www.majestic12.co.uk/bot.php?+" target="_blank" rel="nofollow" onmousedown="this.href='http://www.google.com/url?q\75http%3A%2F%2Fwww.majestic12.co.uk%2Fbot.php%3F%2B\46sa\75D\46sntz\0751\46usg\75AFQjCNFdVquAvzICRbRM5rDcBBfoH8mGSA';return true;" onclick="this.href='http://www.google.com/url?q\75http%3A%2F%2Fwww.majestic12.co.uk%2Fbot.php%3F%2B\46sa\75D\46sntz\0751\46usg\75AFQjCNFdVquAvzICRbRM5rDcBBfoH8mGSA';return true;">http://www.majestic12.co.uk/bot.php?+)"

This is spooky for several reasons:

1) This bot sends really a lot of requests over a few minutes
2) It sends _s and _k parameters in its requests, so it either is good in imitating things or somebody knows about Seaside
3) When this wave of requests comes in, our server runs out of Memory and Linux starts killing processes, starting with Apache
4) When this happens, it doesn't answer ssh or http
5) our hoster's console tells me the server is up and running and fine
6) The only thing that helps is restarting the server

The good news:

1) Nobody ever connected onto our machines from strange IP addresses
2) Seaside never starts a DB connection, because there is no logged on user, so the _s and _k parameters are either outdated or random (Our application logs every DB connect and much more for auditing reasons - I am glad it does!!!)
3) Our DB Backups are tested and reliable, so we don't loose any data
4) We have a very good alarm system that tells us the server is not responding, long before we get calls from customers

So it seems the bot is just trying to find weak spots and hasn't found one. For now, we closed the machine for MJ12 bots completely. We don't care if there is a real search engine that is being imitated by bad guys. We just don't want that shit on our machine.

So now of rmy VAST related question...

Here's our current theory on what exactöy is going on
1) Server gets a lot of requests from the bot
2) Seaside creates a lot of 302 Redirects and WASession instances
3) Image gets VERY big very fast
4) DB2 is first to find out it's running out of Memory
5) Linux starts killing processes

We could now do several things:

1) log each WASession creation in an application log
2) write the actual memory size of the Image every 30 seconds or minute or so (we currently write a heartbeat file every minute, to see whether the server is still running. It writes a heartbeat during the attacks asif all was good)
3) Hope that locking out this one bot is the solution to all our problems and will one day bring global peace

There are several drawbacks to these:

1) Would only prove the theory that lots of Sessions get created, but not that they eat a lot of memory
2) We have no idea how to determine how much Memory the Image actually uses
3) is very unlikely to stand for long.

I'd like to work on 2)

So.... How do I measure the amount of MBytes my Image actually uses at a given moment in time?
Is there a (performant) way to tell which classes (resp. instances thereof) fill most of that memory?

This would be the only way to prove that it's our Seaside App that starts the downward spiral before the server goes out of business.

Every hint is welcome

Joachim

Seth Berman

Re: How to find out the actual memory usage of an Image at runtime?

In reply to this post by jtuchel

Hi Joachim,

So you are interested in an analysis of Smalltalk heap memory?

Have you looked at EsMemoryModel? Specifically, the EsMemorySegment class whose instances describe all the segments in the heap. From here you can use the API on the instance side to get the information you need. For example, you might want to start with "EsMemorySegment allSegments" to get the view of the heap. The longer you hold on to these, the more chance they will be outdated because the GC could run with each additional "step" you talke in smalltalk which may change the layout. But it should be good enough for what you are trying to do.

Performant walk of the segments? If you mean something better than linear..then no...it will be a linear walk of the segment. However, this should be fast enough. See the EsMemorySegment>>do: API for an object walk.

There are some tools that can use this framework like EsMemoryMonitor. But if you are after something specific and need to evaluate a "picture" at a specific point...I would use this lower level framework.

Hope this helps.

-- Seth

On Monday, January 26, 2015 at 10:54:04 AM UTC-5, Joachim Tuchel wrote:

We are seeing a spooky bot hammering onto our Seaside App:

<a href="http://kontolino.de:443" target="_blank" rel="nofollow" onmousedown="this.href='http://www.google.com/url?q\75http%3A%2F%2Fkontolino.de%3A443\46sa\75D\46sntz\0751\46usg\75AFQjCNG6DyQPGJnjFLSuV_OQQkOUbwPzxA';return true;" onclick="this.href='http://www.google.com/url?q\75http%3A%2F%2Fkontolino.de%3A443\46sa\75D\46sntz\0751\46usg\75AFQjCNG6DyQPGJnjFLSuV_OQQkOUbwPzxA';return true;">kontolino.de:443 136.243.16.7 - - [25/Jan/2015:07:34:31 +0100] "GET /Buchhaltung?_s=GAzGw-mQ5X65ufSV&_k=bcOwu4ti2QL4EEs0&3 HTTP/1.1" 302 229 "-" "Mozilla/5.0 (compatible; MJ12bot/v1.4.5; <a href="http://www.majestic12.co.uk/bot.php?+" target="_blank" rel="nofollow" onmousedown="this.href='http://www.google.com/url?q\75http%3A%2F%2Fwww.majestic12.co.uk%2Fbot.php%3F%2B\46sa\75D\46sntz\0751\46usg\75AFQjCNFdVquAvzICRbRM5rDcBBfoH8mGSA';return true;" onclick="this.href='http://www.google.com/url?q\75http%3A%2F%2Fwww.majestic12.co.uk%2Fbot.php%3F%2B\46sa\75D\46sntz\0751\46usg\75AFQjCNFdVquAvzICRbRM5rDcBBfoH8mGSA';return true;">http://www.majestic12.co.uk/bot.php?+)"

This is spooky for several reasons:

1) This bot sends really a lot of requests over a few minutes
2) It sends _s and _k parameters in its requests, so it either is good in imitating things or somebody knows about Seaside
3) When this wave of requests comes in, our server runs out of Memory and Linux starts killing processes, starting with Apache
4) When this happens, it doesn't answer ssh or http
5) our hoster's console tells me the server is up and running and fine
6) The only thing that helps is restarting the server

The good news:

1) Nobody ever connected onto our machines from strange IP addresses
2) Seaside never starts a DB connection, because there is no logged on user, so the _s and _k parameters are either outdated or random (Our application logs every DB connect and much more for auditing reasons - I am glad it does!!!)
3) Our DB Backups are tested and reliable, so we don't loose any data
4) We have a very good alarm system that tells us the server is not responding, long before we get calls from customers

So it seems the bot is just trying to find weak spots and hasn't found one. For now, we closed the machine for MJ12 bots completely. We don't care if there is a real search engine that is being imitated by bad guys. We just don't want that shit on our machine.

So now of rmy VAST related question...

Here's our current theory on what exactöy is going on
1) Server gets a lot of requests from the bot
2) Seaside creates a lot of 302 Redirects and WASession instances
3) Image gets VERY big very fast
4) DB2 is first to find out it's running out of Memory
5) Linux starts killing processes

We could now do several things:

1) log each WASession creation in an application log
2) write the actual memory size of the Image every 30 seconds or minute or so (we currently write a heartbeat file every minute, to see whether the server is still running. It writes a heartbeat during the attacks asif all was good)
3) Hope that locking out this one bot is the solution to all our problems and will one day bring global peace

There are several drawbacks to these:

1) Would only prove the theory that lots of Sessions get created, but not that they eat a lot of memory
2) We have no idea how to determine how much Memory the Image actually uses
3) is very unlikely to stand for long.

I'd like to work on 2)

So.... How do I measure the amount of MBytes my Image actually uses at a given moment in time?
Is there a (performant) way to tell which classes (resp. instances thereof) fill most of that memory?

This would be the only way to prove that it's our Seaside App that starts the downward spiral before the server goes out of business.

Every hint is welcome

Joachim