Memory usage viewer

[laurens]

My suggestion is to make a memory usage viewer in cpanel or just a way to make it possible to view it,

the reason for it is

because people could get problems because they get over that amount that is allowed so admin's could check there memory usage to upgrade before getting suspended,

[GatorEricS]

That's a pretty good suggestion, but I don't know that it will be all that useful. I have never seen a user cause issues from using too much memory. CPU, yes, memory, no.
(Of course, now that I say that, someone will inadvertently do it)

[laurens]

Not sure, but i thought the hosting could get suspended for taking to much memory for 20-30 seconds,

They could place a sort kind of processor usage meter there to,

Because Sometimes people have good running site with high traffic then they get problems because it is suspended if they can see it comming they can buy a dedicated server before it get suspended

Because dedicated takes atleast 24 houres,
and that is a big downtime if you got a good site

[skeetr]

Originally Posted by laurens

Not sure, but i thought the hosting could get suspended for taking to much memory for 20-30 seconds,

They could place a sort kind of processor usage meter there to,

Because Sometimes people have good running site with high traffic then they get problems because it is suspended if they can see it comming they can buy a dedicated server before it get suspended

Because dedicated takes atleast 24 houres,
and that is a big downtime if you got a good site

What you are missing from Erics response is that people dont usually have problems by using too much memory. Most people that run into problems end up using too much CPU long before using too much memory. Using too much memory can happen, but for the most part, using too much CPU seems more likely to happen.

So maybe we could request a CPU usage viewer AND a Memory usage viewer.

Out of curiosity ERIC....how difficult would this be to make happen?

[whatrevolution]

Not difficult. You're only needing to safely display data you can get from the shell. Same thing we've been talking about for nearly two months.

[laurens]

linux support that kind of stuff so wont be to hard though,

[GatorEricS]

Well, the issue would be calculating cpu usage across a specific time span while using only the process data from a specific user. It's easy to display the load, or the cpu usage across the whole system. For each user, we would have to:
1. get a process listing
2. parse out the processes for just that user
3. grab the cpu usage for each process into an array
4. calculate the average usage across the array
5. do it again, several times, across a specific time span
6. report the data.

For a single user, it's not that big of a deal. We could easily write an internal tool to do this on a variable basis (i.e. $ cpusage uname) (which gives me an idea). However, if we did this for every user, through cPanel, we would run the risk of everyone hitting it at once, which - needless to say - could cause substantial load on the machine.

It's a good idea, it's just not feasible on this scale.

The other thing that a lot of people fail to realize is that a graph, or a system like this isn't all that helpful to the end user. 90% of the time, your site (anyone's site) will have negligible load and cpu usage. Another 9% of the time is caused by a bad query, a hanging script, or a ddos. About 1% of the time, it's due to an abusive domain.

This is why we don't auto-suspend for people exceeding cpu usage, we try to take it on a case-by-case basis.

[skeetr]

Originally Posted by GatorEricS

Well, the issue would be calculating cpu usage across a specific time span while using only the process data from a specific user. It's easy to display the load, or the cpu usage across the whole system. For each user, we would have to:
1. get a process listing
2. parse out the processes for just that user
3. grab the cpu usage for each process into an array
4. calculate the average usage across the array
5. do it again, several times, across a specific time span
6. report the data.

For a single user, it's not that big of a deal. We could easily write an internal tool to do this on a variable basis (i.e. $ cpusage uname) (which gives me an idea). However, if we did this for every user, through cPanel, we would run the risk of everyone hitting it at once, which - needless to say - could cause substantial load on the machine.

It's a good idea, it's just not feasible on this scale.

The other thing that a lot of people fail to realize is that a graph, or a system like this isn't all that helpful to the end user. 90% of the time, your site (anyone's site) will have negligible load and cpu usage. Another 9% of the time is caused by a bad query, a hanging script, or a ddos. About 1% of the time, it's due to an abusive domain.

This is why we don't auto-suspend for people exceeding cpu usage, we try to take it on a case-by-case basis.

Thank you for taking the time to enlighten all of us Eric. It is much appreciated by me and hopefully many others.

Since this topic has been discussed by quite a few people for about a couple months, hopefully this will shed some light on it and people can start understanding a little bit more as to why this isnt already in place.

Thanks again.

[skeetr]

Originally Posted by whatrevolution

Not difficult. You're only needing to safely display data you can get from the shell. Same thing we've been talking about for nearly two months.

Exactly why I have asked.

No where has anyone from HG actually gone into detail as to what it would take to implement this on the servers, or even why they havent implemented it already, until now.

[whatrevolution]

Originally Posted by skeetr

No where has anyone from HG actually gone into detail as to what it would take to implement this on the servers, or even why they havent implemented it already, until now.

Agreed; that's why the thread wouldn't fall off.

I appreciate everything that all HG employees have done to carry themselves this far. I've never questioned that, either... I'm just a blunt person.

Originally Posted by GatorEricS

Well, the issue would be calculating cpu usage across a specific time span while using only the process data from a specific user.

Originally Posted by GatorEricS

It's a good idea, it's just not feasible on this scale.

That is why I began my thread on this subject by showing two different ways this is done. Your way is included, which is stringing together the available collection of command line tools. You are right, that is not feasible on your scale. That is something we can do individually, though; which I've seen mentioned recently by other peers and gators.

RRDtool was an example more appropriate to your scale.

Given time and effort, this can be done.

Originally Posted by GatorEricS

The other thing that a lot of people fail to realize is that a graph, or a system like this isn't all that helpful to the end user. 90% of the time, your site (anyone's site) will have negligible load and cpu usage. Another 9% of the time is caused by a bad query, a hanging script, or a ddos. About 1% of the time, it's due to an abusive domain.

This is why we don't auto-suspend for people exceeding cpu usage, we try to take it on a case-by-case basis.

The usefulness of any given feature of your service is directly proportional to each customer's knowledge and training. Smart, even nerdy features, will attract their like-minded users if the feature is marketed correctly. Then you'll be gathering customers by the usefulness of the graphs; rather than expecting everyone to find use in everything your system can do... which has not ever happened.

It makes great sense for the bulk of your company to remain focused on serving and building for the popular majority of your customers. I assume you're also large enough now to hire a person or three to begin growing a successful research and development office.

In any case, we're really faced with wether or not a customer can be fairly expected to comply with this system resource usage term of service. The judgement is made by you, on your machine, which is proper. Yet, we will do well, and build better systems, if your way of enforcing such a term of service is to involve us in that technical process ... not just the administrative blame game that happens during suspension after things break.

[riddler]

also PHP supports that and there are free scripts out there to tell you that exactly. You dont need it to be embedded in cPanel...

[GatorEricS]

Although there are tools out there that can do this, they are all "compile-time executable", in other words, they will execute and display information taken when they are loaded. I use a small php script on my private server, and on my HostGator hosting, that displays load avgs, memory usage, etc... The problem is that - while it works wonderfully for me, or even a handful of users - if every user on the machine were to be hitting the script at the same time, it could literally bring the machine down.

The security administrator (GatorJamyn) and I were talking about this issue, the other day, actually. We both agree it would be a great idea, but until we can find a way to parse the data, graph it, etc... without bogging down the machines, it's just not feasible in our environment. (I am keeping my eyes open for a solution, though, as well as asking some contacts I have at cPanel if they can spend some time tossing about ideas. Perhaps we'll see something like this in a future version of cPanel.)

[whatrevolution]

That is why we take the samples periodically, adding that as processing overhead which is paid for by the community of users renting the machine. If that overhead forces a reduction in capacity, then a machine running that software simply costs more and is part of some separately "packaged" hosting service.

The script that each user gets data from should not be lively running the process which polls the hardware for current load data. The data is collected separately into a round robin database, then the users could only query that database simultaneously; which is not going to hurt your servers. So, the designer and admin of such a system will decide how frequently the software can poll the hardware for this data, thus how timely and accurate the available information in the database is. The frontend of that is not resource expensive, at all.

So, your code polls for then stores the data to some other database machine. The frontend simply queries that other machine, which carries that load easily. Obviously, any specialized processing such as graph generation should also not be done on the machine for which the graph is being generated. Sure, you do that in realtime on a desktop machine with gkrellm, so perhaps that is where such an assumption would come from; but it's not so in serving. Most of the code for this system should not be running on the machine being audited... in fact, just think about that... what if the IRS let us audit ourselves?

Potentially hazardous resource usage would only occur if you tried to bundle too many tests together in a single cronjob. That would be bad, and isn't even in the scope of this; you'd do that yourself, in your own code, during unit testing and such.