![]() |
Installing & Configuring Docker Engine & Docker-compose on CentOS. Source: pixabay |
Real-time Monitoring and Log Streaming in Google Cloud Platform (GCP)
May 19, 2021 / by Rathish
![]() |
Monitoring Dashboard with charts - credit pixabay |
Getting insights into performance, availability and health status of infrastructure and application is very critical for building and managing reliable systems. When we are dealing with clusters of instances, it is becoming very challenging to collect, aggregate and derive actionable insights from data in real time. There are monitoring tools available to address this challenge, both open source and commercial products, we are going to discuss about how to achieve real-time log streaming analytics and monitoring in Google Cloud Platform.
Understanding MySQL Architecture
May 18, 2021 / by Rathish
Major components:
![]() |
MySQL ARCHITECTURE |
Face Recognition - Computing Euclidean distance in PostgreSQL
May 17, 2021 / by Rathish
In this article, we are going to discuss about implementing Euclidean distance in PostgreSQL database. Before getting into actual implementation, let me give you a quick background to understand the need for writing this article. I have been working on Face Authentication system and to perform Face Verification task, we need to compute the distance between two faces. There are a lot of implementations out there to achieve this using Python, however it did not help in my case, so I have implemented the Euclidean distance computation in PostgreSQL. Let's see about the challenges and solutions in detail.
Face Recognition System:
- Face Registration
- Face Verification
Face Registration:
This part contains below steps:
- Users will register their faces with name and image
- The registered photos will be stored in folder with name as folder name
- The user images will be feed to Convolutional Neural Network (CNN) model and extract 128 measurements for each image.
- Using a K-Nearest Neighbour classifier to train a classifier with name and their corresponding 128 dimension encodings.
- Save the classifier as pickle file on application directory.
Face Verification:
This part contains below steps:
- When a new user try to login with their image, it will be feed to same Convolutional Neural Network (CNN) and extract 128 dimension number vector.
- This 128 dimension vector is passed to classifier and it will be compared against all the pre-trained face encodings.
- The classifier will return name of the encodings where distance comparison value is less than 0.3 (as I defined threshold as 0.3)
The Challenges:
- Adding new users to system is complicated as it requires training for every users.
- Each time we add new user, need to train all the existing registered user images, generating 128 dimension number vector for all the images during training is time & resource consuming process, it requires higher GPU processing and doing this for each new users is not the right way.
- Deleting existing user is complicated as it deals with classifier object as pickle file.
- Parallelising to multiple servers with this approach is not feasible, as classifier stored as pickle file, we need to keep moving this file to all the servers, every-time we make changes to training data.
- Cannot add multiple images for a person as it increases the computation.
The Solution:
After dealing with these issues for sometime, I have changed my approach to overcome above mentioned challenges. The current implementation works as follows:
![]() |
Face Recognition - Computing Euclidean distance in PostgreSQL |
SQL Server Always On Availability Group (AG) Listener: step by step guide
June 10, 2019 / by Rathish
Related articles in Always On Availability Group:
Always On Availability Group Read-Only Routing
Windows Server Failover Cluster Quorum configuration for SQL Server Always On
Adding SSISDB to Always On Availability Group
- Domain Name Systems (DNS)
- Listener Port
- One or more IP addresses (VIP)
- DNS:
salesag.mscorp.com
- Port:
1433
- IP: static or DHCP
Creating Availability Group Listener:
New Availability Group - Specify Replicas |
Step by step guide to add SSISDB to SQL Server 2016 Always On Availability Group (AG)
June 01, 2019 / by Rathish
Related articles in Always On Availability Group (AG):
How to configure Read-Only routing on SQL Server 2016 Always On Availability Group (AG)?
Step 1: Create SSIDB on primary replica:
Creating Integration Service (SSIS) Catalog |
Understanding and configuring Windows Server Failover Cluster Quorum for SQL Server Always On
May 25, 2019 / by Rathish
Related articles in Always On:
- Node Majority
- Node & Disk Majority
- Node & File Share Majority
- Node & Cloud Storage Majority
- Disk-only - No Majority.
![]() |
Failover Cluster Manager - Quorum Configuration |
How to configure Read-Only routing on SQL Server 2016 Always On Availability Group (AG)?
May 17, 2019 / by Rathish
- Read-Only Routing URL
- Read-Only Routing List
Architecting reliable backup and recovery strategy
February 22, 2019 / by Rathish
- Recovery Time Objective (RTO)
- Recovery Point Objective (RPO)
- Business Strategy
- Backup
- Backup Type
- Backup Frequency
- Backup Device
- Retention and Archival plans
- Restore
- Backup Availability
- Restoration Access
- Documentation
- Testing
- Identifying the correct backup media
- Identifying the authorized persons to perform restore
- Identifying documentation related to the restore
- How frequently backups need to occur?
- How much data is to be backed up each time?
- The type of media, that the backups will be held on?
- Retention and Archival plans for the media?
- Full backup.
- Differential backup.
- Log backup.
- Full backup – every week
- Differential backup – every day
- Log backup – every hour
Backup Device:
- Disk Files – (SAN, NAS, etc.)
- Cloud Storage (Azure Blob Storage, Amazon Cloud Storage, etc.)
Best Practice:
Summary:
[Solved] MySQL User Operation - ERROR 1396 (HY000): Operation CREATE / DROP USER failed for 'user'@'host'
October 15, 2018 / by Rathish
Operation DROP USER failed
REVOKE ALL ON *.* FROM 'user'@'host';
DROP USER 'user'@'host';
FLUSH PRIVILEGES;
mysql.user: User accounts, global privileges, and other non-privilege columns
mysql.proxies_priv: Proxy-user privilege