From: Zotter, E. David <david@...>
Subject: RE: [NADUG] Re: how many engines are you running on your typical online/web cluster for mrinterview?
To: "NADUG@yahoogroups.com" <NADUG@yahoogroups.com>
Date: Wednesday, September 10, 2008, 5:05 PM

To clarify:

 

Cluster 1 – version 4.5:

One of our clusters has 9 engines per app server for a total of 36.

 

This setup is sitting on 4 physically very large servers with a ton of RAM (not including DB Cluster).  Each physical server is hosting two virtual servers (1 web and 1 app) via VMWARE.

 

We actually limited each engine to 50 interviews for a total of 1800, in an effort to better distribute the load across all available hardware.

The cluster doesn’t seem to bring engines online when they should, so we often see the first app server struggling the most unnecessarily….when the engines towards the end are hardly used at all.

 

This cluster has really performed extremely well.  The only time there are issues is with an occasional project with a 13-20 meg MDD which seems to throw the cluster into a tail spin every once in a while.

 

 

Cluster 2 – verison 5.5:

The other has 4 engines per app server for a total of 8.

This one has 2 physical web and 2 physical app (not including DB Cluster).

 

We’ve limited each engine is to 200 per engine for a total of 1600

 

 

Both clusters are using Enterprise versions of OS and SQL.

 

We’re somewhat trying to understand the best mix.  The second cluster is to understand how this performs without the VMWARE layer.

 

 

 

What I’d really like to see is a cloud solution, where I can deploy and suspend web or app servers on the fly..attach them to the cluster when needed, etc..  Much like what http://www.gogrid. com/ does…  

The thing that stinks now is that I have to plan for the peak loads, which is extremely difficult to do with mrinterview (as it entirely depends on the mix of projects, the size of the MDD files, and so on).  It would be great if I could pay for capacity on demand like a utility.   I bet we’re only a few years away from this.

 

 

 

-David

 

 

 

 

 

 

From: NADUG@yahoogroups. com [mailto:NADUG@ yahoogroups. com] On Behalf Of ko_mike.gates
Sent: Wednesday, September 10, 2008 12:34 PM
To: NADUG@yahoogroups. com
Subject: [NADUG] Re: how many engines are you running on your typical online/web cluster for mrinterview?

 

--- In NADUG@yahoogroups. com, "Zotter, E. David" <david@...> wrote:
>
> How many engines are you running on your typical online/web
cluster for mrinterview?
>
> One of our clusters has 9 engines per app server for a total of 36
and the other has 4 engines per app server for a total of 8.
>
> We're trying to figure out if it is better to have more engines
doing less work each or less engines doing more work each.
>
>
> I'm curious to hear what others are doing.
>
> Thanks,
>
> -David
>

Good question David. We've deployed a couple different
configurations at this point, although if I'm honest, I can't say we
have a lot of solid reasoning behind the choices we've made.

We currently haven't extended beyond six engines per app server yet
(total number of engines per cluster varies typically from 18 to
24). One factor that has acted as an artificial threshold of sorts
is the need to use an Enterprise-level OS for servers with more than
4GB. But if I thought I could deploy 16 engines on a single server
with no downside, the extra costs for the Enterprise OS would be a
no-brainer to me.

We are following the general guideline communicated to us long ago
by SPSS to allocate 1GB of physical RAM per engine. With the use of
virtualization, we've been able to take advantage of that more
readily, but still face the OS memory limitation if we deployed with
the Standard OS.

The key driver for adding engines at this point seems to be memory,
memory, and memory. Some engines will struggle with a relatively
low number of live interview sessions because of a huge per-session
memory footprint attributed to a project or two. This potential for
variance at the project-level is very frustrating when trying to
project the capacity of a given cluster.

Do you segment out your app servers from the other tiers? How many
app servers are you using in your cluster configurations?