Ricky Moorhouse

Blog

 

 

 

— title: 2025 Reading Log date: 2025-01-01 —

  • Moscow X - David McCloskey
  • The Library of the Dead (Edinburgh Nights #1) - T L Huchu
  • A Reluctant Spy - David Goodman
  • A Clean Kill (Garrett Mann #1) - Steve Konkoly
  • Voyage of the Damned - Frances White
  • Shadow Game - Vin Strong

2019 - Photos of the year

 

Gunwalloe Church Cove, Cornwall

Eiger

Oxford Canal

Coastal Scenes

Cornwall

West Wittering

Dawlish

The paths that beckon

Path up from Durdle Door

Ovington

Lauterbrunen

Plants

Water Avens (Geum rivale)

Cardoon

Echium Pininana

Birds

Blue Tit

Brown Pelicans

Alpine Chough

About

 

I live in the south of England with my wife, two children and our dog Nova. I work at IBM as a Cloud Architect for API Connect cloud services. I enjoy running, walking, photography and spending time with my family.

Contact me

CV

 

As an experienced operations engineer with a focus on automation and security, I have consistently initiated and led projects resulting in a more proactive approach to potential issues and significant reductions in infrastructure costs. I have experience in cloud development and operations for large highly available SaaS offerings.

Experience

IBM United Kingdom Limited — 1999 - present

Operations Lead, API Connect on IBM Cloud
2016 - present

As Operations lead my primary aim is to ensure that the service is available and that the operations team have visibility into the system components. My team provide round the clock operations and support for the API Connect service across all IBM Cloud regions. We collaborate closely with engineers across the performance and development teams to ensure that potential issues are identified and resolved promptly.

UK Cloud Operations Lead, IBM API Management & Cast Iron Live
2013 - 2016

As UK lead I am primarily responsible for ensuring the SaaS offerings continue to operate without issue during the UK business day, leading a team providing operational and customer support and platform improvements. My other responsibilities include providing consultancy to other teams moving to the cloud and working closely with the development teams to ensure a smooth transition from development, through test to production.
Key projects in this capacity have included the migration of Cast Iron Live from Amazon Web Services to the IBM SoftLayer platform and working with the NetflixOSS components for the IBM API Management SaaS offering.

Security tooling lead, Software Group UK IT
2005 - 2011

I was responsible for leading the development and support of internal web based systems used to provide system inventory and security management for Software Group in the UK. Over this time I have gained significant experience in IT Security and System Management, specifically managing the Linux hosting platform with Apache web servers and MySQL databases.

DB2 Support
2004 - 2005

Working as part of a small team managing database servers (DB2 and IMS) on z/OS to support the needs of development and test teams across IBM Hursley.

Anglican Church of Uruguay, Salto, Uruguay
2003 - 2004

During my leave of absence from IBM, my wife and I went to Uruguay through the South American Mission Society and worked alongside the Anglican Church in there in Salto. This involved working with young people in a poor neighbourhood. As a side project while I was there I set up a website for the diocese.

Education

University of Portsmouth — BSc Computer Science (hons) 2002

Skills

  • Cloud Technologies including Amazon Web Services, IBM Cloud, NetflixOSS.
  • Linux system administration and security.
  • Working as part of an international distributed team.
  • Managing, operating and diagnosing issues within complex architectures.
  • Automation and integration tooling using Ansible, Python and other scripting languages.
  • Use of Version control systems - git, Rational Team Concert and subversion.
  • Web Application development using Python, Perl, PHP and Javascript.
  • MySQL database and server administration including replication.
  • Use of ZenHub / GitHub for workload management and planning.
  • DB2 database administration and z/OS application support.

Garden

 

Plants

  • Giant Montbretia Crocosmia masoniorum

  • Oriental Poppy 'Türkenlouis' Papaver orientale

  • Japanese spirea 'Golden Princess' Spiraea japonica

  • Hellebore

  • Penstemon 'Laura'

Pond plants

  • Cotton bud rush Scirpus cereus
  • Variegated water drop wort 'Flamingo' Oenanth fistulosa
  • Golden Japanese sweet flag Acorus gramineus Ogon
  • Yellow Water Iris Iris pseudacorus

Monitoring API Connect

 

Monitoring Metrics

We've standardised on Grafana for viewing metric data via dashboards and sending alerts to PagerDuty and Slack - here's a rough list of the different types of data we're monitoring. The majority of these are collected via collectd and stored in graphite, although some of the dashboards will use data directly from our Elasticsearch logging infrastructure.

System Metrics

Standard system metrics collected by collectd from the individual VMs and push to graphite

  • CPU usage
  • Network utilisation
    • Socket states (e.g. count of established, are processes listening)
    • Transfer rate
  • Memory usage
    • system overall
    • process size (e.g. datapower)
  • Disk usage
  • Load

Product Metrics

API Invocation

  • Transactions per second [ Count of ExtLatency logs from datapower ] l
  • Rate of errors (for stable APIs) [ Analysis of response codes from analytics ] a
  • Response time a
  • Spread of traffic across gateways a
  • Deep dive into individual APIs a

Analytics

Ingestion of Analytics Events
  • Error level for inbound data to Analytics l
  • WSM Agent lost record count from DataPower cs
  • Logstash pipeline stats s
Health of Analytics Cluster
  • Cluster health cs
  • Shard Counts cs
  • Pending Tasks cs

API Manager

  • Apache connection levels cp
  • Kibana health s

Informix

  • Tablespace usage cs
  • Bufferpool usage cs

Overall Product usage

Internal REST endpoints used to regularly generate reports for data such as

  • Number of provider orgs
  • Number of API definitions
  • Policy usage

Externals

  • Synthetic monitoring - response status and times using Hem and Uptime
    • Sample APIs - echo, proxy, etc.
    • Product APIs - LB Healthcheck, v1/me/orgs

Key to data sources:

  • cs - CollectD custom script
  • cp - CollectD plugin
  • s - Custom script
  • a - Analytics records from ELK
  • l - Log analysis in ELK

Log Analysis

We use ELK as our centralised logging infrastructure with all of our systems offloading logs via syslog.

The logs are parsed and indexed by logstash on the way into the cluster. Everything gets indexed, and some patterns are identified to raise PagerDuty alerts.

Some examples of patterns we're using to alert on include:

  • Higher than normal rate of 502 or 503 errors in the Analytics inbound logs
  • Errors for known error states - e.g. out of memory and database conditions.

Pond

 

Pond visitors

Common frog

This lovely frog actually visited the front garden first and was found by Laura whilst she was clearing the flower beds and we took the frog round to the back in a large plant pot. On seeing the pond, the frog sat for a couple of minutes and then dived straight in.

Red damselfly

On the bank holiday Monday, this damsel fly kept visiting the pond and siting enjoying the sun

Recommended Recipes

 

Here's a selection of recipes I've tried and recommend:

Mains

Sweet stuff