Deadline with legal implications Splitting company's data when merger was disolved Partitioning DBaaS between Silicon...
Vision
to reality
Lorem ipsum dolor sit amet, consectetur adipiscing elit. Ut elit tellus, luctus nec ullamcorper mattis, pulvinar dapibus leo.
Lorem ipsum dolor sit amet, consectetur adipiscing elit.
Lorem ipsum dolor sit amet, consectetur adipiscing elit.
Lorem ipsum dolor sit amet, consectetur adipiscing elit.
Search and recommendation intelligent system infrastructure for a Fortune 500 semiconductor company
Background
A data scientist program manager of a large Silicon Valley electronics equipment provider came up with a vision of creating an intelligent personnel profile search and recommendation service.
The system is required to periodically perform data extraction from various sources (databases, cloud APIs, unstructured documents,…), perform data sanitization, and data analysis. It is also required to have a user-facing application that allows users to search and find recommendations, as well as updating the system’s data source influencing subsequent search and recommendation operations.
Issues
The client’s data science team were experts mathematicians and PhDs in their respective domains and were looking for a software engineer that could both design and build the system that they had envisioned. A misstep of hiring an inexperienced engineer had been costly with no working deliverable.
Omar was brought in to remediate this issue and bring this project into reality.
Solution
Given the need for data-mining, data correlation, data crunching, user application, instrumentation and configuration – Omar designed a multi-facet microservice architecture that mainly consist of data extraction services, administrative services, and user application services.
The data extraction service framework allows for a clean abstraction for integration modules to data sources that the system depends on. The framework provides an interface for instrumentation and configuration, allowing the administrator to tune the operational behaviour of these extraction modules.
The scalable distributed architecture of data-collection nodes communicating over message bus allows for scalability and extensibility. With the clean abstraction and interface of the architecture, the data extraction modules were able to be independently developed in parallel with the framework and plugged in at deployment runtime as it becomes available
The administrative service allows the client’s administrator to create and manage data extraction jobs, view reports of job execution and configure job specific specific parameters at runtime. It provides clean interface to tune the machine learning and recommendation parameters. It also provides reporting on analysis of usage pattern by end user.
The user application service provides a comprehensive API representing and encapsulating the business logic of end user applications. The clarity of the application service API allowed the for clean web controllers and user interface to be developed.
Omar also worked closely with various IT staff of the company to deploy, secure and integrate system system with the company’s cloud ecosystem and SSO security provider.
Results
The vision became a reality.
And within budget and timeframe. Omar worked closely with the Program Manager to support the platform release and onboarding campaign.
It was successfully deployed to production serving the needs of the client.
Explore other Case Studies
Lifting the Cloud Silicon Valley multinational technology company public cloud initiative Background Top researchers at...