Aqua Star Incorporated is a privately owned seafood company headquartered in Seattle, Washington. They are one of the leading suppliers of frozen seafood in North America, and one of the largest retail frozen seafood brands.

Website: http://www.aquastar.com
Address: 2025 First Avenue, Suite 200, Seattle, WA 98121
Phone: 1-800-232-6280

I was recruited by Aqua Star in November of 2006 to serve as their sole Systems Engineer, as such I handled all Systems & Network engineering, implementation, and administration activities.

I was challenged with rectifying numerous technology issues that plagued the company, and establishing a robust technology infrastructure that would scale with the company's growth expectations.

Aqua Star was in serious need of some systems engineering expertise when I joined the company. It was apparent that heir prior engineers/administrators never had a plan. Every addition, change, or upgrade was just handled as a one-off project with no thought given to the future needs or requirements. I actually heard that the prior engineer would just buy whatever hardware was on sale that week.

While there was some thought given to network security when switches and network gear was deployed, everything beyond that was just the default config. No segregation or optimization of network traffic. The poorly designed, deployed network was the root cause of many issues Spiration was experiencing. The most noticeable was the jitter experienced when using the VOIP phone system and the slow Internet connectivity.

Server hardware was inconsistent, and mostly configured incorrectly (e.g. Exchange server with one large RAID 5 array for OS, Data, EDB, Etc., and a fax server with critical data running on a RAID 0 array (and not being backed up))..

Top fifteen issues present when I joined Aqua Star:

• Wired network was just a flat /24 network using daisy-chained default-config switches.
• Wired network had only 1Gbps for backbone connectivity (network traffic bottleneck).
• VOIP phone system performance issues (related to the improper network design, deployment).
• VOIP phone system hardware was obsolete, and approaching EOL.
• Wireless network failed almost daily, used consumer-grade hardware, and was not secure.
• Power conditioning, backup, and distribution was almost non-existent.
• Server/Telco room did not have adequate cooling, seismic bracing, or cable management.
• Absolutely zero client and server security patching.
• Client and server provisioning performed via obsolete tools and external hard drives.
• Inadequate Internet connectivity for company size/needs.
• Legacy Exchange 2003 server that failed on a weekly basis.
• Antivirus/Antimalware was not centrally managed/monitored, or even maintained.
• No hardware or software standards for clients, servers, or network devices.
• Servers, clients, and network gear were not adequate, and were not configured correctly.
• No server virtualization solution in use, server resources not utilized efficiently.

During my four and a quarter year tenure at Aqua Star I systematically rebuilt or replaced all existing infrastructure and introduced new technologies and methodologies as well. The end result was a completely re-designed, properly-engineered, robust, reliable, and scalable information technology infrastructure solution. Additionally, I deployed a standardized, scalable remote office solution that integrated the remote offices with the corporate office.

Security, stability, and reliability of the network and systems was the first priority. Over the course of the first six months I addressed and rectified the majority of the core issues. I migrated the legacy Exchange 2003 environment to Exchange 2007 and new, properly-engineered hardware. I implemented centralized anti-virus (SEP), security patch (WSUS) management for all client and server devices, and brought everything current (most devices had not been patched since they were deployed).

Next was the server room expansion, which also included the resolution of the inadequate cooling solution, the implementation of a proper power infrastructure, seismic bracing, cable management, and much more. This expansion was key in building out a enterprise-class scalable infrastructure that would serve as the foundation for future changes, upgrades, and additions as the company grew.

To use the limited resources as efficiently as possible, I engineered and implemented a server virtualization solution. Initially this was Microsoft Virtual Server, and later Hyper-v 1.0, then 2.0. As the virtualization environment grew I implemented Microsoft Virtual Machine Manager (SCVMM) to effectively manage the solution.

The 100% manual method of tracking assets in use when I joined Aqua Star was inefficient, and prone to human error. To rectify this I created a custom asset tracking solution that utilized a web frontend, SQL database backend, and numerous scripts to collect data and populate the database. This improved accuracy greatly, and allowed for custom reports and metrics to be generated.

The following are just some of my accomplishments at Aqua Star:

• Engineered, deployed an enterprise-class, scalable, datacenter and remote office infrastructure solution.
• Engineered, implemented a completely new power solution for the information technology infrastructure (Circuits/UPS/PDU).
• Corrected wired and wireless network issues.
• Established hardware and software standards for client, server, and network devices.
• Migrated their legacy Exchange 2003 environment to Exchange 2007.
• Implemented LCS 2005, later upgraded to OCS 2007, and integrated it with Exchange and the phone system (RCC).
• Deployed server virtualization infrastructure for efficient utilization of limited resources.
• Deployed iSCSI Storage Area Network (SAN) for centralized, scalable storage (Dell/EqualLogic).
• Implemented disk-to-disk-to-tape backup solution (Symantec Backup Exec/Promise iSCSI NAS/HP LTO fiber tape library).
• Developed an Information Technology asset tracking solution (custom coded IIS-based front-end, SQL database backend).
• Deployed Microsoft Windows Server Update Services (WSUS) for server and client OS updates and security patching.
• Deployed a network based imaging solution (WDS) to increase efficiency and reduce hardware provisioning timeframes.
• Migrated from legacy T1 ISP connectivity to fiber circuits and MPLS solution (Masergy).

Server Room Re-design/Expansion:

• Redesigned server room was roughly twice the length and two feet wider than original layout.
• Cooling was accomplished via two 3 TON A/C units, standard Hot/Cold aisle configuration was implemented.
• A total of six dedicated 208V 50AMP circuits were installed to power the infrastructure.
• Each server cabinet had a dedicated APC SmartUPS 8000VA UPS for power conditioning and backup.
• Each server cabinet featured an HP Modular PDU solution for power distribution within the cabinet.
• Power umbilicals from two of the cabinet UPS devices were utilized to power equipment in the Telco racks.
   Note: This provided redundant power for the telco equipment (in case of single UPS/circuit failure).
• Two Gigbit switches in each cabinet provided redundant server network connectivity, and a 4Gbps backbone.
• A redundant layer-3 switch stack was utilized for core network connectivity and routing.
   Note: Cisco 3750X with StackWise. 4Gbps Etherchannels for server network uplinks, 2Gbps Etherchannels for client access layer switches.
• VLANs were implemented to segregate network traffic and allow for prioritization.
• Legacy T1 ISP connectivity replaced with fiber circuit and MPLS service through Masergy, single T1 retained for failover.
• Server virtualization implemented to effectively utilize resources.

Corporate Office Infrastructure Re-design/Upgrade:

• Replaced legacy client access switches with POE capable units (allowed for simplified cabling at user workstations).
• VLAN implementation allowed a user's workstation and phone to share one network port (reduced switch count, and simplified cabling).
• Centrally managed wireless solution rectified connectivity and speed issues, security was addressed via certificates and RADIUS.
   Note: Cisco 4400 series WLAN Controller and Cisco Light-weight Access Points (remote offices managed through 4400 as well).
• Client wireless configuration, certificate distribution handled via GPO to provide a stress-free, seamless experience for end users.

Remote Office Infrastructure:

• Designed, implemented a standardized remote office infrastructure solution (none existed prior to this).
• Single server solution that provided core domain services, file & print services, and simplified remote administration.
   Note: Core domain services configured to failover to corporate office, file services used DFS and was primarily administrative.
• Phone system managed from central office, with local failover during any WAN outage event.
• MPLS WAN solution with end-to-end network traffic prioritization allowed for jitter-free inter-office voice/video calls.
• Wireless LAN monitored/managed from central office.
• Dial-in device with serial emulation for troubleshooting during outages.

Like all companies, Aqua Star went through many changes during the four plus years I was there. Some of these changes were related to the 2008 financial crisis, others were just standard evolutionary changes that every business goes through as they grow. After some management shake-ups in 2010, the environment was no longer the same, and it became just another job rather than the company I previously enjoyed working for.

I resigned from Aqua Star in early 2011 to pursue other opportunities. It was a hard choice, but it was the right one.

Server/Telecommunications Room, Front - Before: (This was actually after a bit of cleanup (new server cabinet and UPS/PDU))


Server/Telecommunications Room, Front - After: (Server room cleanup/expansion complete, new infrastructure design in place)

Patch Cable Management - Before: (This was my predecessor's idea of cable management)


Patch Cable Management - After: (After I cleaned things up a bit)