NEWS

Paul Henry is truly a Security Hero, he is most well known for his expertise and leadership in digital forensics, but he is actually well grounded. We are thankful he is willing to invest the time to participate in the Security Hero project

Paul, tell us a bit about your early formative years.

Growing up my father worked on large industrial and power plant boilers in NY and NJ. As a teenager, I would often go with him to work on weekends and during breaks from school. I had my initial exposure to process control and developed a good understanding of troubleshooting the underlying logic of burner management systems at a very early age.

Burner management sounds valuable, heaven knows they apply the heat to us in the security field; what can you tell us about high school?

In high school I seemed to master auto shop and often arranged side jobs working on my other teachers' cars in the school shop. I found myself quickly getting bored with my other subjects and spent every available minute in auto shop.

I built my first custom Harley at 17 and have built a total of 4 custom Harley Davidson motorcycles; one placed in the bike show at Sturgis in 1989, and another one won the radical custom classification at the Key West bike show in 1999. However, while I sold my Harleys after a bad accident in 2007, I've maintained my love of fast cars: my most recent project was a 2002 Z06 Corvette with over 600 RWHP that turned a quarter mile time of 9.4 seconds. Two custom vehicles I built were featured in Low Rider Magazine.

Wow, I have not stuck with it, but when I was in the Navy, I loved the automobile hobby shop; I had a 65 Mustang convertible with a 289 bored out to 302 that I, somehow, managed to never get a ticket with. But, we have to get jobs in the real world to pay for our toys, how did you start work?

My first real job was working for a company in the Carolinas, Boiler Equipment Company, doing start ups on new industrial and power plant boiler installations and troubleshooting existing boiler plants. The systems I worked on for Boiler Equipment Company were early technology, primarily Bailey and Westinghouse Mercury Ring Balance and Pneumatic process control systems, and they used relay logic for burner management and support systems. I went through the evolution from the traditional Mercury Ring Balance and Pneumatic control to analog electronic systems and was excited how much more could be done with the move to analog controls. While at Boiler Equipment Company, I was sent to my first electronic process control vendor training at Cleveland Controls up in Ohio. I primarily commissioned Westinghouse Hagen, Bailey and Foxboro systems at Boiler Equipment Company.

The work for Boiler Equipment Company was very regional and there were only so many large industrial and power facilities in the area. I quickly discovered that I would not have the ability to expand my professional growth only working locally, so I left Boiler Equipment Company and took a position with Southern Technologies (STI) based in Altamonte Springs Florida that worked nationally. I continued to live in the Carolinas and traveled extensively. I specialized in commissioning analog boiler control systems and worked large projects for STI, such as commissioning new controls and burner management systems at Tinker Air Force Base in Oklahoma City OK and Ft Benning in Columbus GA. Work slowed at STI and I quickly took a position at a local company near their HQ in Florida called Mid Florida Boiler.

Working at Mid Florida Boiler gave me my first opportunity to design my own process control and burner management systems. I based my systems on what I had learned about analog process control and had taken my first step in computer based control by replacing old relay logic based burner management systems with PLC based systems. The work for Mid Florida Boiler brought me back to Ft Benning in the installation of additional control system enhancements to the systems I had originally commissioned for STI some years back. I quickly found my self in the "regional rut," limited to working a day's drive from central Florida, and I was quickly getting bored with the limited work I was exposed to.

OK, so you made the leap from analog control to programmable logic controllers, what is the next step in your life?

An opening came up with a company based out of Chicago called Indeck Power Equipment Company to commission steam plants both nationally and internationally. While the position was a step backward in responsibility, it offered more long-term opportunity and I jumped at the chance to expand my horizons. The systems being used by Indeck at that time were primarily older pneumatic process control systems along with relay logic based burner management. One of the first large projects I started up was a Power system at the Koppers Company in Toledo OH that utilized Coke Oven Gas, a by-product of creating coke for steel production. The project utilized a manufacturer supplied control system from the control system vendor Foxboro and had previously been commissioned by Indeck and the vendor representative, but it was having several performance and reliability issues and was, basically, a "problem" project. After getting an understanding of the issues they were having with the system, I literally re-designed it in the field making several modifications to the underlying control system architecture based on what had worked for me in the past, and successfully commissioned the modified system to the great satisfaction of the client.

Awesome, design and troubleshooting skills, how did you jump onto the digital train?

My success at Koppers allowed me to develop a good personal relationship with Gerald Forsythe, CEO at Indeck. We had several conversations about our use of manufactured pneumatic and relay based systems while the rest of the industry had evolved to using digital process control along with digital burner management systems. I had taken a number of programming courses at a local community college and had developed a working knowledge of programming in both machine language and C. I had been spending, literally, every available moment of my personal time learning everything I could about these new "digital systems," and I was able to describe their capabilities and benefits in plain English / layman's terms in several conversations with Mr. Forsythe.

I had previously put together the control capabilities we had been providing in our pneumatic control systems expressed as a mathematical model in anticipation of eventually evolving to digital control. The system controlled everything from the gas compressors and the boilers to the steam turbine and the generator, and offered two levels of redundancy in both the process control systems and the burner management systems. I used ISA standards for the process control system and NFPA standards for the burner management system, and combined them with my personal experience in commissioning previous systems to provide a state of the art solution. This became our standard and was used for complete control of 10 MW Cogeneration facilities.

Jeepers, Paul, it sounds like you were on your way to being the number one guy in digital controls used in power generation, but we know your story has a turn in it somewhere to get you to where you are today. What was your next step towards IT and IT Security?

After our success on the 10 MW cogeneration facilities, I was promoted to Manager of the Control and Instrument Group at Indeck and we standardized on providing state of the art digital control solutions for our fossil power plant solutions. When Indeck began moving away from their own traditional brick and mortar business systems, it was simply natural that I also played a role in their adoption of computer business systems, and I used my previous experience to assist with the systems administration of their business network.

OK, so you are starting to really dig into the business of business as well as programming, process control and experience briefing technical issues to business executives; it sounds like you are putting "the package" together.

After a total of nearly 12 years at Indeck Power and being involved in projects in the USA, Europe, Asia and Africa, I felt had gone as far as I could go technically within the industry and decided that I needed to expand my horizons. An opportunity with a digital control system vendor called RTP Corp in Florida that specialized in the real-time control of nuclear power systems became available, and I applied for the position. One of the more intriguing parts of the job offer was the "stock options" (RTP was a part of a publicly traded company.) I was hired immediately, and they relocated me to their HQ in Ft Lauderdale Florida.

RTP Corp was a relatively small company at the time, and I reported to the CEO Sal Provanzano. While there, I wore a number of different hats - handling everything from assisting with supporting their enterprise network to designing NRC 1E qualified control systems, writing thought leadership articles, handling speaking engagements and handling pre sales meetings with resellers and clients. One of my favorite accomplishments at RTP was the design of voting scheme based Rod Drop safety shut down system. Simply put, the Rod Drop control system was responsible for safely shutting a reactor down even after a complete failure of not just one, but two of the four separate computers used in the system. It was a relatively short career at RTP Corp after 18 years at Indeck, but it was certainly intense: within two years of joining RTP Corp the company was taken private, so I cashed out my options and started looking for my next adventure. It is important to note that by this time, I had begun to see the value of third party certification and had gone ahead and earned my MCP+I and MCSE Microsoft certifications on my personal time at RTP Corp.

My CEO, Sal Provanzano, felt I should capitalize on my recognition as an expert in process control and challenged me to begin speaking at public conferences' and to begin writing articles. I was an invited speaker at my first public event in 1987 at the Pittsburgh ISA conference and spoke on "Common Mode Noise and Common Mode Rejection in Process Control". I enjoyed it and made public speaking a large part of my work from then on. I also published, on average, one article per month in Power industry publication including ISA magazine as well as Power Magazine.

By this time, I had spent a majority of my time in process control, but had learned a great deal about TCP/IP as it became the backbone for my control system designs; I also had a considerable amount of administrative and audit time under my belt. Further I had gained a great deal of experience in properly securing my digital control systems, utilizing both OS and application hardening as well as firewalling.

I think your CEO was wise; you are a powerful public speaker today and that comes from ability, of course, but also experience. So, you have a strong knowledge of TCP and your Microsoft credentials, is it time to leave the process control world, now that you are the big fish?

One of the firewall vendors I had a good experience with, CyberGuard, was located just a short distance from RTP Corp; I had heard that the company was having some management issues and had just been delisted from NASDAQ. I knew the product itself was solid and felt it presented a potential opportunity, so I contacted CyberGuard and expressed my interest to come on board. I met with Robert Perks and then with CEO Robert Carberry, and they explained that I would have to take a significant pay cut, but they would provide a significant number of performance based stock options if I would come on board. I accepted the position and left RTP Corp on good terms, moving over to CyberGuard as a regional manager to handle the SE Region.

In my exit interview with Sal Provanzano, I learned something about myself from a manager's perspective that still sticks with me today. He told me that it took him a while, but he had finally figured out what made me tick: I was only happy doing things that other people could not do. I committed 150% to everything I did at RTP, but he always noticed that I seemed to be attracted to those tasks that others could not do. If the task was one that could simply be handled by any other employee, it never got my full attention.

I hate to give up a SANS business secret, but I am often coaching folks like Eric Cole to focus on doing the things that only Eric can do. That seems to be a hallmark of the successful people in our industry. OK, so now you are at Cyberguard, took a paycut, but have options if things go on the upside. What's next?

Timing is everything and I was about to learn how quickly things could change at a public company. Within weeks of leaving a job that could have afforded a stable future, I found myself at a company, for less then half my previous pay but a handful of stock options, where the CEO I had just interviewed with two weeks earlier was terminated by the board of directors, and was not yet replaced, and learned that other changes were certain to come. A new CEO, David Proctor, formerly with IBM, was brought on board. I worked in various roles while CyberGuard assessed their best "go forward" strategy and put together their new management team.

It was during this time I had met Chuck Phillips at CyberGuard. Management at CyberGuard was thoroughly impressed with Chuck and his network security knowledge as well as his many certifications, including both Microsoft and his CISSP certification. I sat through a number of his customer training classes and learned more with every course I attended. Chuck suggested that I had the necessary experience to meet the requirements for the CISSP because of my work at Indeck and RTP Corp, and that I should go for the certification myself to complement my Microsoft certifications. Shortly thereafter, I flew myself to New Orleans to take the CISSP exam at a CA World conference. The CISSP was a tough exam and nearly everyone that took the exam left after the 6-hour exam wondering if they passed - it was brutal. It was 6 weeks after I took the exam before I learned I had passed it and was granted my CISSP certification.

Back in the day, the CISSP by exam, as opposed to being grandfathered was a big deal, so you have your networking expertise, your OS skills, and your CISSP credential, you are definitely ready to play in the early infosec days. I bet the next steps are exciting, please continue!

The upper level management team at CyberGuard by this time was in place and numerous people were terminated as the new CEO Dave Proctor built his management team. The position that I was originally hired for had been given to an employee with more seniority them myself. I quickly found myself in a new role handling presales support globally. Simply put, my job was to explain the Application Proxy and Orange Book B level OS based technology offered by CyberGuard and how if afforded a higher level of security then the popular tasteful packet filter in use.

I took advantage of a policy at CyberGuard offering to pay for one industry certification annually for employees and I took my first SANS course. It was the SANS Firewall course and GCFW certification and I was blown away by the amount of knowledge that was dispensed in the weeklong course. The only way to describe it was like drinking from a fire hose, and I was really shocked to see not one other person involved in sales in attendance. It was all hands-on security geeks in attendance and I felt I had really stumbled onto something here. Clearly, the most successful route to success in sales was solving a customer's problem and SANS training provided an invaluable amount of industry intelligence that I could put to use on a daily basis in my role at CyberGuard. One issue I quickly ran into, however, was a CyberGuard policy that CyberGuard retained the exclusive copyright to every thing that any employee wrote. Hence, I was not permitted to submit a written practical to achieve my official certification at SANS. Back in those days, to receive a certification, you had to write a paper called the practical. All I could do was accept a certificate that acknowledged I had attended the training. It was a major setback professionally for me, but I felt the value of the knowledge was worth the effort, even without an official SANS certification.

A little more than a year into my new career at CyberGuard, Marty Ryan was made the VP of sales; in a meeting with Marty, he candidly told me that, because of my lack of seniority, he had considered letting me go but he felt that I had a high level of technical knowledge and that my certifications would give me the necessary credibility to take over the Asian region and be successful at CyberGuard. I jumped at the opportunity, and within a week of being given the position I found myself in Singapore putting together a team.

Well now, there is a unique management technique: I am considering letting you go, but instead will promote you to be lead of Asian operations. Can't wait to hear what happens next! (And, I think Marty Ryan is now VP Sales and Marketing at eDMZ, right?)

Correct, he is. Well, I decided to use the very same solution sales methodology that I found successful at CyberGuard and RTP over the years. Don't hard sell the products; simply show the client how the underlying technology solved the client's issues. I also recognized that, while every one of my direct reports had been in the industry for years, if you were going to be successful in this space, you had to have third party certification. I mandated that every one of my direct reports, as well as the principals at any partner who would sell our products, had to achieve their CISSP certification within 12 months, or be terminated. Guess what, every single one of them achieved their certification.

During my first year of handling CyberGuard in Asia, while offering to assist clients in solving their network security problems, I often found myself being asked to assist clients with everything from network security audits to incident response for those who that had suffered network intrusions. I quickly checked with SANS to see what courses were available that would allow me to better assist my clients and learned about the SANS Advanced Intrusion Analysis course and the GIAC GCIA certification. I arranged to take the course the first available chance I could and, again, was overwhelmed with the abundant knowledge dispensed by SANS that I could immediately put to use. We finished out the year in Asia by more than doubling sales for the region - clearly, the solution sales approach using credible third party associates was the key to our success.

During that first year, one partner really stood out: Quantiq International, based in Singapore, run by a woman with incredible drive, ambition and a quest for excellence in her solution offerings named Kwek Hong Sin. Quantiq very quickly was appointed as my exclusive distributor for Singapore, Thailand and Malaysia. Hong Sin took the same approach that I did in business - you create opportunity by solving problems and every problem afforded an opportunity. A good example of this was early on in her sales efforts; Hong Sin was challenged by a prospect that they would recommend a competitor's product over her CyberGuard offering because, in Singapore, the prospect could find numerous people that had training and certification in the competitive product, but relatively few that had any familiarity with CyberGuard. Hong Sin immediately initiated an effort to "train the trainer" with CyberGuard training her personnel and then she proceeded to offer free training through her new trainers to anyone working for government or enterprise clients on the administration of CyberGuard firewalls. Within a year, Hong Sin completely removed the objection and had trained hundreds of Singapore, Thailand and Malaysian network administrators on the CyberGuard firewall products. Because of Hong Sin's determination, CyberGuard Firewalls became the de facto standard within Singapore Government high security applications.

Wow, I had never realized CyberGuard had that level of penetration in Asia. But, security is changing rapidly and CyberGuard and, in fact, firewalls themselves, are starting to reach limits, what happens next in your career?

Increasing awareness was found to be the most useful tool in building our customer base in Asia. From Japan, south through Singapore and over to Australia and New Zealand, we alerted users to the increasing inherent risks of the Internet and the failure of traditional technologies to meet the threat by showing real hands-on examples, not simply using Fear Uncertainty and Doubt (FUD). Our reputation for solving customer's problems, not just hyping products, drove our success. My direct reports and I made a point of speaking at every network security industry event within the region, typically once per month, and we spoke at local events for my channel partners on almost a weekly basis. The network security events in Singapore had grown to the point that, since they were so well attended, they were being hosted at National University of Singapore (NUS), and the event at NUS quickly grew to be one of the better-attended network security educational events in the region. My last speaking engagement at NUS had over 1000 in attendance.

During my second year of running CyberGuard in Asia, I found myself being asked to assist law enforcement regularly with investigations of network breaches and I again called on SANS to help me round out my knowledge, so I was up to the task. I scheduled to attend the SANS Forensic course with the GCFA certification. This was easily the most impressive training I had received from SANS to date. While most of the forensic training from vendors concentrated on how to use a specific product in an investigation, SANS training provided the foundation of knowledge necessary to understand what happened behind the mouse click with a commercial forensic product. I returned to Asia armed with my SANS training and used the ability to assist with the investigation of network breaches to assist with the deployment of CyberGuard solutions that afforded the ability to have prevented the intrusion in the first place. CyberGuard solutions quickly became well known within government and law enforcement circles across Asia. It was during this period that CyberGuard had another change at the helm with the CEO David Proctor being replaced by CEO Scott Hammock and VP of Sales Pat Clawson. CyberGuard was now moving into an acquisition mode, acquiring technologies that complemented their product offerings.

By the end of the third year of managing Asia at CyberGuard, the team I had put together had taken annual revenue in sales from well under a million to over 8 million. I had a team of direct reports with respected third party certification, and my reseller partners had also taken the opportunity to increase their credibility with third party industry certifications that came with the unmatched depth of knowledge available only from SANS. I personally had added the CISA certification to my list of credentials. CyberGuard had acquired Snap Gear and WebWasher and was integrating the products into their offerings.

By this time in addition to your CISSP, Microsoft certs, SANS training and now your CISA, you have a lot of experience speaking to user groups and are becoming one of the "names" in industry. Very exciting times - can't wait to hear the next chapter in your journey.

I had now worked with Pat Clawson VP Sales for about a year or so and he had now been appointed CEO at CyberGuard, replacing Scott Hammock. Pat Clawson recognized that what we had accomplished in Asia needed to be replicated globally. I was promoted to Corporate Vice President and was tasked with raising awareness of the CyberGuard differentiators globally. My responsibilities included writing thought leadership articles and speaking at industry and regional events, as well as direct involvement with strategic clients. Within two years of my promotion, I added the CISM and the CISSP-ISSAP certifications to my list of credentials. Reporting directly to CEO Pat Clawson allowed me to respond to any area of the globe that Pat felt needed assistance and have a direct impact on revenue. I had collected numerous performance-based stock options at strike prices ranging from $.50 to $8 per share over my many years at CyberGuard and had begun selling them off at an average stock price of $16 share, banking nearly $1 million toward my retirement.

Thanks for being so explicit about this Paul. To any younger or even, not so young folks reading this: it is easy to see retirement as something in the distant future, but time flies by so much faster than you can ever believe. It is imperative that you do something about your retirement, starting this year. As social security collapses, and it will, you are going to be reading horror stories in the paper about older people in poverty. Put something aside this year for your future.

As our revenue continued to grow, CyberGuard attracted the attention of our primary competitor, Secure Computing, and was quickly acquired. This represented a huge accomplishment for all involved as we had taken a company that had been delisted from Nasdaq and literally rebuilt it, to the point where it was purchased for just under $300 million. The merger was initially described as a merger of equals.

I was the only senior executive at CyberGuard that came directly over to Secure Computing after the merger. My new role reported to the VP of sales as VP of Technology Evangelism. I handled speaking engagements at industry trade shows, wrote thought leadership articles, handled media interviews and was directly involved with strategic accounts. Sales continued to grow, however, not at the rate desired by Secure Computing.

Right! This is where Paul Henry comes into his own, as a brand, and you start becoming a big cheese in forensics as well, yes?

Shortly after the acquisition of CyberGuard in 2005, Secure Computing also acquired CipherTrust in 2006 and several CipherTrust personal were placed in senior positions at Secure Computing as part of the acquisition agreement. I felt the acquisition diluted my opportunity for advancement and I began planning for my personal future. I had decided to increase my knowledge in forensics by repeating the SANS forensics training a second time. Following two rather rapid acquisitions, together with a declining economy and rumors that Secure Computing was positioning itself for sale, I left Secure Computing in May of 2008. Important to note that in 2008, I was also voted in as Vice President to the board of directors of the Florida Association of Computer Crime Investigators (FACCI) and also achieved my CCE forensics certification.

OK, so at this point, Paul Henry is making opportunities for Paul Henry, you are pretty much out on your own, right?

After leaving Secure Computing I spoke with a few vendors who had products that I felt I could have significant impact with, but I have not yet found one with a culture that I felt would facilitate the levels of growth and, quite simply, the level of fun I had experienced back at CyberGuard. In order to keep my skills sharp, in June of 2008 I formed Forensics & Recovery LLC; I continued my network security work and began speaking and writing for companies under contract with retainer agreements, i.e., under a retainer agreement, I am the Security and Forensic Analyst at Lumension and handle blogging, speaking engagements, interviews and writing through leadership articles, as well as video and audio podcasts. I have also spoken at network security conferences on behalf of multiple vendors including McAfee, Fortinet and Aladdin. I am also consulting as the Senior Forensic Analyst For Precision Discovery on Worldwide Engagements and have provided eDiscovery support on multiple cases for Document Technologies Inc HQ in Atlanta.

But you are staying well rounded yes? Not just fully going down the digital forensics path?

I continue to keep a finger on the pulse of network security and provide penetration tests regionally for local clients, including a recently completed penetration test at the University of Tampa and CIP Audit for Seminole Power, a regional utility; and as I am finding VoIP to be a major part of my network security work today, I again looked to SANS to provide me with the hands-on experience and knowledge to master VoIP. I attended the SANS VoIP training in March 2009 and have expanded my technical offerings to now include penetration testing of VoIP networks.

Speaking engagements since leaving Secure Computing have averaged 2 or more per month included the keynote at HackerFest, tracks at the MECTF USSS event and MISTI InfoSecWorld Orlando, TechnoSecurity, and multiple events for Florida security integrators such as BSI in Tampa, as well as a recent Florida Bar Association event where I spoke on eDiscovery.

Awesome, thank you for sharing so much, can you tell us just a bit about your life?

I became a Florida licensed PI in 2009 for the purposes of performing computer forensics (License number C 2800597) and have also registered my company, Forensics & Recovery LLC, as a PI Agency (License A 29004).

Beyond my work, I have been with my wife Nancy for over 22 years and have 5 children and 2 grand children.

My income has varied along with my roles throughout my career, reaching a peak at CyberGuard in a sales role managing the PAC Rim with W2 income of over $800k, and in a management role for the past few years averaging over $300k, and, yes, I have the Social Security income reports to back it up.

After my accident on my Harley in 2007, I became very health conscious and dropped from 228 to 165 pounds; by watching what I eat and hitting the gym on a regular basis, I feel like I am in better shape today at 52 then I was at 30 *wink*

Original Article: http://www.sans.edu/resources/securitylab/paul_henry_sec_hero.php