Sign in with
Sign up | Sign in
Your question

Infrastructure for an educational institution

Tags:
  • Infrastructure
  • Business Computing
Last response: in Business Computing
Share
May 1, 2014 6:31:44 PM

Hello!

I'm the IT responsible of a Brazilian college (so excuse my bad english =P) and in need of building a "datacenter" for my Campus.

We have about 1200 students, 60 professors and an administrative staff of ~35 persons. We should be growing on the next few years, and the expectative is a total of 1800 students, about 75 professors and ~45 on the administrative staff;

Well, in the beginning of the year we had some serious problems that almost caused data loss. So I asked my director some money to build a decent infraestructure, since it's ever been a problem.

He allowed me to do so. I'm thinking of something like:

2 servers - each one with two Xeon V2630 V2, or stronger, 64GB of RAM and 2 300GB SAS disks. I am think about the IBM x3650M4 or DELL Poweredge R620;

1 storage - dell powervault DAS or ibm storwize v3700, filled with 12 SAS disks of 1TB each, 12 NL-SAS disks of 3TB each - everything in raid 60;

1 backup unit - HP MSL 2024 or dell powervault;

1 server to work as a backup controller - that would be a small one - I'm thinking about the DELL R210II;

2 switches - with GB support (would two 10gb switches be worth it?)

And that's the basic.

If there's enough money, I'll buy two R210II to work as DNS servers (ns1 and ns2) and one R520 (filled with enough ethernet adapters to provide a good throughput of at least 2 or 3GB) to work as a firewall (I'll use pfsense on it).

Everything should have at least 5 years guarantee.

I'll also buy one Windows 2012 Standard edition license, but we will work mostly with linux.

Now my questions are:

1) Is it worth to buy the vmware licenses? Since it's really expensive, I am looking at XEN. Is XEN a reliable alternative?

2) Is it worth buying the subscription of Oracle Linux? We've been working with CentOS and it's been doing the job well, but I'm still thinking of buying something with support. Is it worth it's price?

I Have, let's say, about "115k ~ 120k dollars" to spend in this project (but think about brazilian prices, almost twice the american ones). Would you change something in this project? Any advices?

Thanks in advance! =)

More about : infrastructure educational institution

May 2, 2014 10:18:25 AM

I think you need to add what areas are you trying to cover with this server infrastructure upgrade.

Are you trying to provide only new file storage & backup? Are you trying to upgrade mail servers? Are you trying to incorporate enterprise wireless to give students Internet access?

What are people expecting you to accomplish with your upgrade(s)?

m
0
l
May 2, 2014 12:09:52 PM

As 2Be has said above we need a better understanding of what this infrastructure is supposed to address. Hopefully this isn't supposed to be the network, servers, and storage for the entire campus services. I know the small community college here that we do some work with has nearly ten times the number of storage and server resources available for running the campus, and our total number of students isn't quite what you are discussing here.

So what infrastructure are you wanting to address with this upgrade and what is the expected end result or solution?
m
0
l
May 2, 2014 1:06:25 PM

Hi guys, thanks for the replies.

Well, yeah, this is supposed to be the entire infra for the Campus. Actually our resources, now, are extremely poor - having only four proliant ml350g servers with 32GB of Ram each and nothing more!

This infrastructure is supposed to adress our domain, host our website and educational services (such as moodle) and host some administrative tools (one VPN, GLPi, Proxy servers, Reverse proxy servers, Samba 4, multiple mysql databases, logs, etc). The city hall (huge city: 1.3 million inhabitants) has also approved a law for a municipal week of science and technology and we will need to host the website (in which people of the city - and also outside it - can register to watch the courses and speeches professors will do. After it, those people will also take a certify of participation in the event using this same website) and I'm expecting about 5 to 10 thousand participants (their own estimatives based on similar events of the past years).

I'll need to run about 30 to 40 virtual machines on it.

All of the students machines (about 400 computers) need to autenticate in the domain, but this was becoming and extremely slow process and the students were losing class time in and order of 10 to 15 minutes just to log in the computer, so I just told my co-workers to remove all the students machines from the domain and they are now locally loging in it.

The students do not have institutional e-mails yet, since there's no sufficient infraestructure for it; with this upgrade, I plan to do implement a basic mail for them in the next year.

Concerning wifi, they already have access, but I'm also upgrading it, since it's a poor one. I'll use ruckus equipaments, but this is a far easier issue for me - in this case, the servers will only authenticate their users and be used as proxy and logger.

The professors and the administrative staff also use the domain so they can save their files and share it among their respective co-workers. We did network folders for each one and one network folder for each sector (where everyone that's located in the sector have access). Permissions of using printers, scanners and stuff are also all controlled by Samba.

This being said, I will not have enough money to build the whole infraestructure this year, so that's why I'm trying to do it the most scalable possible.

I'll use VCloud or OpenStack to build a kind of cloud with High Availability. I'll start with two servers (If I go for OpenStack, I'll probably have money to buy one more server this year... hard choice), one highly expandable storage (Both Storwize and Powervault have capabilities of going to almost 1PB - that's more than I'll need for the next few years) and a backup infra. After building this basis setup this year, I'll add servers to it (as processing nodes) and buy more capable storages in the years to come.

I already have the nobreaks and energetic resources needed.

So, what would you guys tell me to do? Any advices? Seems quite hard to do something really good with the financial resources I have; but that's what I have and I need to do my best with it.
m
0
l
May 2, 2014 2:02:24 PM

I get the impression that you are running a Linux domain along with other Linux services. It also seems like you need more help on the PDC since you had previous unsatisfactory performance on logins. So I would recommend beefing up the servers which will service the user logins.

The next step is to figure out your priorities. To me, #1 would be anything that would affect your funding for the years to come. That means your public-facing event will need to work as flawlessly as possible. I would investigate possibly a new router, load balancer(s), and a new web farm. Since the event is only 1 week, that equipment can also server as your everyday working equipment as well. In fact, the router will likely give you VPN access as well.

"I already have the nobreaks and energetic resources needed." -- I think this means you have the power backup (UPS) and enough power to keep everything running. So then look at upgrading your backbone switches. You may not need 10Gb, but you can get modular switches that you can upgrade to 10Gb in the future. Also check to see if the cabling in your offices/sectors need upgrading. Perhaps you will need to install some fiber links to keep everything going fast.

Dell's PowerVaults are good for external storage. As you mentioned, most are expandable as well. I like Dell PowerEdge servers; I would recommend either PowerEdge R520 or R720, with Xeon 2643v2 or 2667v2. They are 2U, 2P servers. You might even be able to get by with local storage before you buy a bigger SAN. SANs have their own needs, so if money is getting smaller, then you might be able to save here before paying $10,000-30,000 for a larger SAN. The PowerEdge R720xd can even support up to 24 2.5" drives, so that might work as your primary file storage.

Those PowerEdge servers would have plenty of power for virtualizing a lot. I've haven't worked with Xen, but I have worked with VMware and Hyper-V. One thing I like about the Datacenter Edition of Windows Server is that you can host as many Hyper-V guests as your server's hardware can handle and not have to pay any extra licensing costs. So even though you have a lot of Linux, that might be something to help out in overall costs.

So many areas for you to go with - you should call Dell/HP and make their reps work for you by creating several proposals to address your needs.
m
0
l
May 2, 2014 3:34:02 PM

All good suggestions here from 2Be. His last line is especially what I would recommend if you haven't already done so. Contact the experts at HP/Dell, and explain what you are wanting to do. They have staff who plan and implement this sort of thing day in and day out, so they can give you a very good idea of what sort of hardware and things to look at.

This sort of project really is above my head. From the ground up a datacenter to support this load is greater than what I have more than just looked at or tinkered around with, so I honestly may not be a whole lot of help. But I do know that you want to start with a solid foundation for your storage and virtualization because it can be a HUGE investment not only money but in time to try and migrate from it if you need in the future. There may be educational pricing advantages for you going through the big name virtualization companies like Microsoft for Hyper-V or VMWare for ESXi. Still, yes, it is going to cost in licensing. It's hard to justify that licensing cost when you are needing to utilize every cent possible to purchase your necessary equipment. I think you are making a good choice, though, trying to utilize open source, linux based, or other free alternatives where possible to save money for the first step. For instance, while a pfSense or Untangle box may not be the most robust thing out there for a campus, it may be much cheaper to implement than a massive Sophos firewall router unit or something else which can greatly cut into your overall budget.

It comes down to finding a balance that is cost effective and efficient right now, but can also be upgraded as you go to give you more room to grow. Find the key areas that need to be addressed first to put your money. From the sounds of it right now that is mainly storage and servers. It's not the best solution, but you can build a SAN using iSCSI from a good quality NAS turning it into your SAN for your server infrastructure. Set up additional SANs for storage of general data and files. Set up a network that supports gigabit throughout, but keep in mind as you grow that you will need to possibly upgrade your backbone to 10 GbE in the future. Still, if you can get away without it right now (and perhaps stick to a few link aggregation gigabit pipes if needed) then that can save you money on switching infrastructure to begin with.

And lastly on the servers, while it might not be the most recommended solution, you can consider implementing whitebox server solutions such as Supermicro systems which can help bring the cost of your server system down. In the past comparing doing a custom Supermicro system to a pre-built HP or Dell server didn't save very much money, but if you are looking at multiple servers then the savings may end up being worth it. This of course just means some more work and responsibility for you though to ensure the compatibility, maintenance, and warranty of the servers.
m
0
l
May 2, 2014 3:39:33 PM

I don't recommend IBM products they are way over priced and quality is not consistent, IBM is a consulting / solution company, not a system builder. They have lost so many of their customers recent years, they ended up selling their server department to Lenovo,

Including Google, Amazon, Facebook, NASA, most laboratories and data centers use Supermicro server products nowaday, their products are very high density & way more cost-effective, they have 6U 112nodes blade server and 4U 72 3.5" Bay dual CPU file server ect.

Just buy a supermicro SC837-E26-RJBOD1 3U 48-Bay DAS system with Dual Expander ($1,890) And add your own harddrives:

Seagate SAS 1TB $135 ea
Seagate SAS 4TB $335 ea
HGST Ultrastar SAS 6TB Helium Drives $689 ea ( yup, filled with helium and makes your voice sound funny )

See, you can build the SAN yourself for just $ 6,500-$8,500, instead of paying $12,000-$20,000 for a prebuilt one.

m
0
l
May 2, 2014 5:21:35 PM

2Be_or_Not2Be said:
I get the impression that you are running a Linux domain along with other Linux services. It also seems like you need more help on the PDC since you had previous unsatisfactory performance on logins. So I would recommend beefing up the servers which will service the user logins.

The next step is to figure out your priorities. To me, #1 would be anything that would affect your funding for the years to come. That means your public-facing event will need to work as flawlessly as possible. I would investigate possibly a new router, load balancer(s), and a new web farm. Since the event is only 1 week, that equipment can also server as your everyday working equipment as well. In fact, the router will likely give you VPN access as well.

"I already have the nobreaks and energetic resources needed." -- I think this means you have the power backup (UPS) and enough power to keep everything running. So then look at upgrading your backbone switches. You may not need 10Gb, but you can get modular switches that you can upgrade to 10Gb in the future. Also check to see if the cabling in your offices/sectors need upgrading. Perhaps you will need to install some fiber links to keep everything going fast.

Dell's PowerVaults are good for external storage. As you mentioned, most are expandable as well. I like Dell PowerEdge servers; I would recommend either PowerEdge R520 or R720, with Xeon 2643v2 or 2667v2. They are 2U, 2P servers. You might even be able to get by with local storage before you buy a bigger SAN. SANs have their own needs, so if money is getting smaller, then you might be able to save here before paying $10,000-30,000 for a larger SAN. The PowerEdge R720xd can even support up to 24 2.5" drives, so that might work as your primary file storage.

Those PowerEdge servers would have plenty of power for virtualizing a lot. I've haven't worked with Xen, but I have worked with VMware and Hyper-V. One thing I like about the Datacenter Edition of Windows Server is that you can host as many Hyper-V guests as your server's hardware can handle and not have to pay any extra licensing costs. So even though you have a lot of Linux, that might be something to help out in overall costs.

So many areas for you to go with - you should call Dell/HP and make their reps work for you by creating several proposals to address your needs.


Hi!

Yeah, I already called Dell and IBM. I have had a visit of a salesperson and a technician from Dell, but they recommended me a VRTX solution, but it wouldn't be as scalable as I would need - anyway, it's still a good hardware. I told them what I would like to build and they are about to send me the costs to see if it fits my budget. Also, the XEON 2667v2 seems to have a great power, but at a price tag of $2,668.00 it may be kind too expensive for me (don't even know how much it costs here) - anyway, that's where I'm willing to spend money, and if it fits the budget I'll surely buy four of these.

About the my backbone switches, I'll surely upgrade them. But, as I said, I won't have money to everything in one shot so this will be done in the "part 2" of this project. The router, anyway, Is an extremely good one.
I'm also thinking about the load balancer, but I don't really have idea of what to buy, or if I could do it in the firewall using pfsense or something like that. Still have to do more research. What would you recommend me to do the load balance?

"SANs have their own needs, so if money is getting smaller, then you might be able to save here before paying $10,000-30,000 for a larger SAN.". Well, I really thought about it, but I'm definitely going to build a SAN - I think it's the time for us to do it here. I'm aware it may have more costs, but as I'm going to build a H.A. cloud infraestructure, I'll do the "baseline" and after increase it.

Thanks for the advices!


choucove said:
All good suggestions here from 2Be. His last line is especially what I would recommend if you haven't already done so. Contact the experts at HP/Dell, and explain what you are wanting to do. They have staff who plan and implement this sort of thing day in and day out, so they can give you a very good idea of what sort of hardware and things to look at.

This sort of project really is above my head. From the ground up a datacenter to support this load is greater than what I have more than just looked at or tinkered around with, so I honestly may not be a whole lot of help. But I do know that you want to start with a solid foundation for your storage and virtualization because it can be a HUGE investment not only money but in time to try and migrate from it if you need in the future. There may be educational pricing advantages for you going through the big name virtualization companies like Microsoft for Hyper-V or VMWare for ESXi. Still, yes, it is going to cost in licensing. It's hard to justify that licensing cost when you are needing to utilize every cent possible to purchase your necessary equipment. I think you are making a good choice, though, trying to utilize open source, linux based, or other free alternatives where possible to save money for the first step. For instance, while a pfSense or Untangle box may not be the most robust thing out there for a campus, it may be much cheaper to implement than a massive Sophos firewall router unit or something else which can greatly cut into your overall budget.

It comes down to finding a balance that is cost effective and efficient right now, but can also be upgraded as you go to give you more room to grow. Find the key areas that need to be addressed first to put your money. From the sounds of it right now that is mainly storage and servers. It's not the best solution, but you can build a SAN using iSCSI from a good quality NAS turning it into your SAN for your server infrastructure. Set up additional SANs for storage of general data and files. Set up a network that supports gigabit throughout, but keep in mind as you grow that you will need to possibly upgrade your backbone to 10 GbE in the future. Still, if you can get away without it right now (and perhaps stick to a few link aggregation gigabit pipes if needed) then that can save you money on switching infrastructure to begin with.

And lastly on the servers, while it might not be the most recommended solution, you can consider implementing whitebox server solutions such as Supermicro systems which can help bring the cost of your server system down. In the past comparing doing a custom Supermicro system to a pre-built HP or Dell server didn't save very much money, but if you are looking at multiple servers then the savings may end up being worth it. This of course just means some more work and responsibility for you though to ensure the compatibility, maintenance, and warranty of the servers.


Hi!

Yes! As I said to 2be, I've already contacted experts at Dell and IBM - now I'm just waiting them to send me the costs proposal of what I asked them.

We already use free alternatives here. As we just have one processor per server, we are using the free version of ESXi. Anyway, as I'm buying servers with two processors, I would need a paid license. That's why I'm looking at XEN. As you mentioned, pfsense is my first choice for a budget firewall!

For the cloud solution, i'm looking at OpenStack or Solaris OpenStack... but I still have to do a lot more research on that. If you have any advices corcerning this topic, it would be of great help! =)

The iSCI solution that you suggested, that's exactly what I'm thinking right now. I thought of a fibre channel solution, but it's really expensive, so I'll be upgrading it in the next few years - for now, this is enough. Some link aggregation should do the work if needed.

Supermicro is really cool. Their prices are insane, but I'm kind of "scared" about it, since it's market share and support is completely incipient here in Brazil. I found just three distributors of their products here, so I'm stiking with Dell, IBM/Lenovo or HP.

Thank you for the help and advices!

RackMountProcom said:
I don't recommend IBM products they are way over priced and quality is not consistent, IBM is a consulting / solution company, not a system builder. They have lost so many of their customers recent years, they ended up selling their server department to Lenovo,

Including Google, Amazon, Facebook, NASA, most laboratories and data centers use Supermicro server products nowaday, their products are very high density & way more cost-effective, they have 6U 112nodes blade server and 4U 72 3.5" Bay dual CPU file server ect.

Just buy a supermicro SC837-E26-RJBOD1 3U 48-Bay DAS system with Dual Expander ($1,890) And add your own harddrives:

Seagate SAS 1TB $135 ea
Seagate SAS 4TB $335 ea
HGST Ultrastar SAS 6TB Helium Drives $689 ea ( yup, filled with helium and makes your voice sound funny )

See, you can build the SAN yourself for just $ 6,500-$8,500, instead of paying $12,000-$20,000 for a prebuilt one.



Hi!

Yes, supermicro has crazy low prices. $8500 for that kind of technology is really cheap compared to other products. Anyway, didn't found much of it here. =\

Thanks for the help!
m
0
l
!