Hi, I’m Sinan
a Business Analyst Data Engineer Data Analyst.

2 May, 2021

ETL Data Engineering with Uber Trip Data and GCP products

Project Overview

As part of my data engineering journey, I built an end-to-end ETL (Extract, Transform, Load) pipeline using Python and SQL to process Uber trip data efficiently. The goal was to extract raw trip data, clean and transform it into a structured format, and load it into a database for further analysis. This project helped me gain hands-on experience with data ingestion, transformation, database management, and analytics..

Data Extraction and Ingestion

I sourced Uber trip data from Kaggle, consisting of millions of ride records containing timestamps, locations, fares, and trip details. Using Python and Pandas, I: Loaded large CSV datasets efficiently. Handled missing values and inconsistent formats. Standardized date-time formats for time-based analysis.

Data Transformation & Cleaning

To ensure high-quality data for analysis, I performed the following transformations: Filtered out anomalies such as negative fares and duplicate records. Geospatial Processing: Converted latitude/longitude into city-based regions. Feature Engineering: Derived new columns like trip duration, peak-hour indicators, and distance categories.

Data Storage & Loading

I structured the cleaned dataset into a relational database using PostgreSQL. The database was designed with: Fact_Trips Table for trip transactions. Dim_Date, Dim_Location, Dim_Customer tables for structured analysis. Indexing & Partitioning for query optimization.

`

Data Analysis & Visualization

Once the data was ready, I conducted exploratory analysis using SQL and Power BI: Ride demand patterns by hour, day, and city. Revenue trends & surge pricing analysis. Customer segmentation based on ride frequency and spending. Heatmaps of peak Uber locations using Power BI.

This project helped me understand real-world ETL challenges, optimize data pipelines, and create scalable solutions for business intelligence.

2 May, 2021

Text Analysis for Sentiment and Readability Leveraging NLP & Python

Project Overview

As part of my journey into Natural Language Processing (NLP), I recently completed an exciting project that focused on automating the extraction and analysis of textual data from online articles. The objective was to not just scrape the content from a set of URLs, but to deeply analyze the text for sentiment and readability using Python.

Objectives

The core goals of the project were: To extract clean article text from a list of provided URLs, specifically avoiding headers, footers, and irrelevant page elements To perform a comprehensive text analysis, computing variables such as Positive and Negative Scores, Polarity, Subjectivity, and key readability metrics like Fog Index, Word Count, and more.

Tools & Techniques Used

📌 Web Scraping: I utilized BeautifulSoup to scrape and extract meaningful content from each URL, ensuring that the textual data collected was precise and free of clutter. 📌 NLP Techniques: With the help of NLTK, I implemented several core NLP techniques including: Tokenization Stopword Removal Complex Word Identification 📌 Custom Algorithms: I also developed and implemented custom logic to calculate crucial language metrics: Polarity and Subjectivity Scores Syllable Counts Personal Pronoun Detection 📌 Structured Output: All the results were compiled into a clean and structured Excel file, complete with clickable URLs for easy reference and navigation.

Key Results & Insights

Successfully extracted and analyzed textual content across multiple web pages. Generated insightful sentiment and readability metrics that can be used for further content evaluation. Delivered a well-organized output that is ready for interpretation or presentation.

`

Key Takeaways

This project was instrumental in deepening my understanding of NLP and its practical applications in the real world. I strengthened my skills in Python, especially with NLTK and Pandas, and learned how to build modular and reusable code for processing unstructured text data. Most importantly, it showed me how powerful automation can be when it comes to analyzing large volumes of text efficiently.

2 May, 2021

End-to-End ETL Data Engineering Project with Azure & Tata Store Sales Data

Project Overview

One of the most exciting and technically enriching projects I’ve worked on recently involved building a complete ETL data pipeline using cloud-based tools from the Microsoft Azure ecosystem. I used a publicly available dataset titled Tata Store Sales Data from Kaggle, which consisted of over 541,000 rows and 9 columns of transactional retail data—perfect for simulating a real-world retail analytics scenario.

🛠️ Tools & Technologies Used:

Azure Data Factory Azure Data Lake Storage Azure Databricks (PySpark) Azure Synapse Analytics Power BI

Project Breakdown:

1. Data Ingestion & Integration

To kick off the pipeline, I first uploaded the raw dataset to GitHub and then connected it to Azure Data Lake Storage Gen2 using Azure Data Factory. This established a seamless and automated data ingestion flow from GitHub to the cloud.

2. Organizing the Data Lake

Inside the Azure Data Lake, I created a structured folder system—one directory dedicated to raw data and another to transformed data. This separation helped maintain a clean architecture and traceability across the ETL pipeline.

3. Data Cleaning & Transformation with Databricks

For the transformation stage, I used PySpark in Azure Databricks to perform a range of preprocessing tasks: cleaning nulls, standardizing values, typecasting, and feature engineering. More importantly, I converted the flat table into a star schema, which is commonly used in data warehousing to optimize query performance and support analytical use cases. The schema included: Fact_Sales: Core transactional data Dim_Date: Calendar dimension for time-based analysis Dim_Product: Product-specific details Dim_Customer: Customer information Dim_Country: Country-wise categorization

4. Data Warehousing & Analysis in Synapse

Post-transformation, I loaded the dimension and fact tables into Azure Synapse Analytics, where I wrote several T-SQL queries to analyze the data from different business angles. These included: Customer Retention Rate & Churn Rate Country-wise Sales Trends Customer Segmentation based on behavior and geography Fast-Moving Product Identification for operational optimization

5. Dashboarding with Power BI

To visualize the insights, I connected Azure Synapse directly to Power BI and built an interactive dashboard that brings the entire dataset to life. It provides marketing teams, business managers, and executives with a clear view of KPIs and patterns, enabling data-driven decision-making.

`

Key Takeaways

This project not only reinforced my knowledge of cloud-based data engineering and ETL design but also gave me practical experience with data lakes, databricks, data warehousing, and real-time business analytics. It brought together everything from raw data ingestion to dashboard reporting in one cohesive pipeline. A special shoutout to Darshil Parmar—while I used a different dataset, his video helped me structure the overall flow and approach for this project.

🔗 Kaggle Dataset

🔗 Power BI Dashboard

2 May, 2021

UAE Real Estate Price Predictor – My First Machine Learning Deployment

At the start of 2025, I took a leap into something I had been wanting to do for a while—deploying my very first machine learning model. What started as an idea to predict real estate prices in the UAE soon became a fully functional web app. This was more than just a data science project; it was the beginning of a new chapter in my journey as a builder and problem-solver.

From Raw Data to Real Insights

I began by sourcing the dataset from Kaggle, which included features like: Number of bedrooms & bathrooms Square footage Furnishing status Whether the listing was verified The raw data needed a lot of cleaning and prep. From handling missing values to encoding categorical variables, I ensured the dataset was model-ready. This phase was messy, but incredibly satisfying—every fix made the data just a bit more honest.

Model Building & a Major Breakthrough

Initially, I experimented with several regression models. After some tuning, I picked GradientBoostingRegressor. However, my first evaluation gave me an R² score below 50%—not great. Digging deeper, I realized the target variable (price) was heavily right-skewed. Applying a log transformation to it was a game-changer. The model performance surged, reaching a solid R² score of 76%. This was a big win—my first real taste of how the right transformation can unlock a model's potential. I also made a mental note to later evaluate it using MAE and RMSE to get a fuller picture of its performance beyond just R².

`

Deploying with FastAPI + Render

Once the model was trained and tested, I shifted focus to deployment. I used FastAPI to turn the model into a web service: Pickled the model using joblib Created a schema for API requests Wrote Python code to handle prediction routes After local testing, I hosted the app on Render. This was not without its hiccups—dependency issues, memory limits, runtime logs—but working through them taught me more than any tutorial could. ✅ Try the Live App – Enter the details, and it predicts the property price in real-time!

💡 What I Learned

Skewed data can kill model accuracy; transforming the target variable is sometimes essential. Deployment isn’t just about coding—it’s about debugging, optimization, and pushing through unknowns. The importance of user experience—even the best model is useless if it’s hard to use.

🔮 What’s Next?

Improving model performance with advanced techniques and regularization Integrating GenAI to generate contextual price justifications and enhance interactivity Adding new features like location coordinates, amenities, and listing descriptions using NLP This project is close to my heart—not because it’s perfect, but because it represents growth, courage, and a lot of firsts. As I continue refining this model and taking on bigger challenges, I’m excited to keep building real things that solve real problems.

🔗 Check my app here

2 May, 2021

Sibaq 2022 – Building the Digital Backbone of a National Arts Fest

Sibaq 2022 wasn’t just another web development project—it was a defining moment in my journey as a full-stack developer. Over the course of 3.5 months, our team of 8 developers (3 frontend, 5 backend) came together to build a fully dynamic platform for Sibaq 2022, the national arts fest of Darul Huda Islamic University, with participation from 30+ colleges, 2000+ contestants, and 600+ programs. From candidate registration and program scheduling to real-time result announcements and media handling, we engineered a platform that served as the operational nerve center of the entire event.

🧠 My Role: Database Design & Management

As the Database Architect and SQL Lead, I took charge of designing and managing the relational database schema that powered the entire application. My responsibilities included: Structuring core tables for candidates, colleges, programs, results, judges, and rounds Implementing efficient relationships and constraints to ensure data consistency Writing and optimizing complex SQL queries for backend integration and admin dashboards Using PhpMyAdmin for real-time database monitoring and administration Coordinating with the backend team to ensure smooth ORM implementation and API integration This role was critical in maintaining the accuracy, integrity, and performance of the platform’s data layer, especially under high concurrent usage during the live fest days.

Key Highlights:

End-to-End Full-Stack Development using 🔧 NestJS (Backend): Clean modular architecture that made collaboration intuitive 💻 Next.js (Frontend): For a performant and dynamic user interface 🎨 Tailwind CSS + ShadCN: Elevating the styling game with clean, utility-first design

Admin Portals & Core Functionalities

We built: Dynamic Admin Dashboards for event coordinators College-specific portals for managing candidates and programs Public Landing Page showcasing events, results, and updates Robust Result Management System that supported both live results and mass uploads

`

🛠️ Technical Deep Dive

This project helped us master a wide spectrum of technologies and skills: Git & GitHub Collaboration: Smooth coordination among 8 developers MySQL + ORM: Designing relational databases, using PhpMyAdmin for SQL operations TypeScript Everywhere: Strong typing from backend to frontend ensured fewer bugs Remote Hosting & Server Management: Deployed the stack initially on AWS EC2 with S3 for asset storage Later migrated to DigitalOcean, optimizing performance and cost Used SSH for server access and PM2 for process management Deployment Automation: Wrote custom BASH scripts for repeatable, smooth deployments Project Management: Managed tasks and progress with Jira, ensuring clear timelines and accountability

The Collaboration That Made It All Happen

Beyond the tech stack, what made Sibaq 2022 special was the team. We learned how to: Navigate real project pressures Manage multiple feature rollouts simultaneously Handle last-minute bugs during the live event Rely on each other’s strengths to push forward

Impact & Legacy

30+ Colleges used the platform actively throughout the fest 600+ programs were managed digitally The platform handled thousands of real-time result entries and updates First-time introduction of tech-powered operations for the arts fest, setting a new standard for future editions

Sibaq 2022 wasn’t just a successful deployment—it was a milestone that taught me leadership, technical resilience, and the power of teamwork. It proved to me that with passion, organization, and a strong tech foundation, we can build solutions that truly matter.

2 May, 2021

Empowering Scholarship Management with Looker Studio at CIGI

Project Overview

During my time at CIGI (Center for Information and Guidance India), I had the opportunity to lead a data-driven initiative that significantly improved the way the organization managed its scholarship distribution system. Using Google Looker Studio, I designed interactive dashboards that brought clarity, efficiency, and transparency to their operations.

Objective

To create a comprehensive dashboard system that allowed CIGI’s stakeholders to: Track scholarship disbursements across multiple regions and time periods Monitor application inflow, approval status, and fund utilization Identify gaps and delays in the distribution process Make data-backed decisions for future planning and outreach

My Role & Responsibilities

As the Dashboard Developer and Data Analyst, I was responsible for: Understanding User Needs: Held discussions with program coordinators to map out key performance indicators (KPIs) and pain points. Data Preparation: Cleaned and structured raw data using Google Sheets and SQL, ensuring it was ready for visualization. Dashboard Design: Built user-friendly dashboards using Looker Studio, incorporating: Filters for region, scholarship type, time period, and gender Dynamic charts and progress trackers Real-time updates from Google Sheets Optimization: Ensured the dashboard loaded quickly, even with large datasets Training: Conducted knowledge transfer sessions to help CIGI staff navigate and utilize the dashboards effectively

`

Key Insights Enabled

Regional analysis of scholarship coverage Real-time monitoring of pending applications Trends in applicant demographics and academic levels Fund utilization rates by program and year

🌟 Impact

The dashboards transformed how CIGI approached scholarship tracking and reporting. Manual reporting was replaced by automated, visual, and interactive dashboards, saving time and enabling faster, smarter decision-making. It was a proud moment to see the dashboards being used in internal reviews and planning sessions, directly contributing to better outreach and impact. This project deepened my skills in Looker Studio, data storytelling, and client-focused dashboard development—proving how effective data visualization can empower even nonprofit operations to scale their impact.

2 May, 2021

Islamic Inheritance Calculator – A Deeply Personal Coding Milestone

Project Overview

This project stands out among everything I’ve built—not just because of the logic involved, but because of how personally connected I am to it. I spent nearly three months working on this Islamic inheritance calculator, and from start to finish, it challenged me in ways that few other projects have. It was during the early days of my coding journey when I stumbled upon the idea, and something about it just pulled me in. I knew the laws around Islamic inheritance were extremely detailed and mathematically precise, and I wanted to see if I could bring them to life through code.

Bridging Two Worlds: Code Meets Classical Scholarship

What made this project unique was how it brought together my two worlds—technology and Islamic scholarship. Having completed my Master’s in Islamic Studies and Islamic Jurisprudence, I had the rare opportunity to study the intricate details of Islamic inheritance directly from authentic and classical texts. These weren't just surface-level learnings—I went into the smallest nuances, understanding every exception, condition, and legal phrasing. So when I decided to build this calculator, I wasn't just coding; I was translating sacred knowledge into a logical system that a computer could understand and implement.

What Sparked the Idea?

The complexity of the subject itself was what drew me to it. Islamic inheritance laws are one of the most mathematically and legally rigorous areas of Shariah. Every detail matters—who survives the deceased, their gender, their relationship, and the presence or absence of other heirs. The combinations are endless, and every possible variation had to be accounted for. At the time, there were no AI tools like ChatGPT or GitHub Copilot, and I couldn't rely on Stack Overflow for help either. It was just me, a blank code editor, and a mountain of logic waiting to be translated. That challenge excited me more than it scared me. `

Tools and Tech Stack

I kept things simple and fundamental—HTML and CSS for the frontend, and JavaScript for everything else. No frameworks, no libraries, just raw code. All the logic was written manually using vanilla JavaScript, and every scenario had to be tested by hand. It wasn’t glamorous, but it was pure and honest work. I wanted to make sure every bit of the code was my own, written from scratch and grounded in what I had learned.

The Core Challenge

The hardest part was figuring out how to structure the logic in a way that could handle all the complex dependencies. Islamic inheritance isn’t linear—it’s a web of conditional logic. I had to think like both a jurist and a developer. How do you translate centuries-old jurisprudence into clean, working code? I remember going over one particular scenario again and again, just to make sure the fractions added up correctly and the legal rights were respected. It was painstaking, but incredibly rewarding when it finally worked.

What I Learned

Through this project, I gained a deeper appreciation for logic and the structure of code. I learned how to take abstract legal rules and turn them into something tangible and functional. It taught me patience—testing each scenario manually was a slow process, but it also made me more thorough. It also showed me the power of pure JavaScript when used with intention. Without relying on fancy tools or external packages, I was able to build something that was both legally sound and functionally solid.

Looking Back

Looking back, this project was much more than a coding exercise. It was an exploration of my own identity as someone who loves both theology and technology. It allowed me to connect deeply with my academic roots while honing my skills as a developer. And more than anything, it taught me that coding isn't just about building apps—sometimes it's about preserving knowledge, solving centuries-old problems, and making them accessible in new and modern ways.

🔗 Live Website: https://faraid-website.onrender.com

💻 GitHub Repository: https://github.com/sinankmuriyanal/faraidh

2 May, 2025

From Logistic Regression to Random Forests: My First ML Journey on the Titanic

Over the past few weeks, I set sail on my very first machine learning competition on Kaggle: Titanic – Machine Learning from Disaster . At first glance, it looked simple—predict whether passengers survived the Titanic disaster based on details like age, gender, class, fare, and family size. But this “beginner-friendly” challenge turned into a crash course in data preprocessing, feature engineering, model building, and the harsh reality of model generalization. Here’s how the journey unfolded.

Step 1: Exploring the Data (EDA & Preprocessing)

I began with the essentials: identifying missing values, filling them using median/mode, or dropping them where necessary. Through visual and statistical exploration, I formed my first hypothesis: Gender, age, family size, and passenger class should strongly influence survival. Context matters—the Titanic story itself taught us that women and children were prioritized. This stage helped me separate the noise from the signal.

Step 2: First Models – Logistic Regression & Decision Trees

After cleaning the dataset, I split the data into 80% training and 20% validation. Logistic Regression gave me a decent start. A Decision Tree followed, giving similar results. Both hovered around 80% validation accuracy. My first Kaggle submission? A score of 0.75100. Modest, but motivating enough to keep going `

Step 3: Feature Engineering – Adding Context

Curiosity led me to engineer new features: Extracted titles from names (“Mr”, “Mrs”, “Miss”) to capture social status. Imputed missing ages based on titles and family groups. Created an “is alone” feature to highlight social dynamics. Scaled fare and age for consistency. Encoded categorical variables like sex and embarked station. These tweaks added depth to the model’s understanding of the dataset.

Step 4: Random Forest to the Rescue 🌲

Then came the turning point: Random Forest. I intentionally kept preprocessing minimal and let the model handle complexity. The result? My Kaggle score jumped to 0.77511—my personal best so far.

Step 5: Cross-Validation & Feature Importance

To make sure I wasn’t just overfitting, I introduced: GridSearchCV with 5-fold cross-validation for hyperparameter tuning. Feature importance plots to validate which variables truly mattered. This added confidence to my model-building process.

Step 6: Testing Advanced Models – XGBoost & Ensembles

Naturally, I wondered: could advanced models push the score even higher? XGBoost gave better training accuracy but surprisingly lower test performance (0.746). A Voting Classifier ensemble landed around 0.765, slightly below my Random Forest score. Lesson learned: complex ≠ better. A simpler model with good features often generalizes better.

What’s Next?

This first Kaggle project was more than just about scores. It taught me how: Feature engineering can make or break a model. Cross-validation is crucial to avoid overfitting. Sometimes, simplicity beats complexity. I plan to dive into top Kaggle solutions, analyze their approaches, and refine my own pipeline.

Useful Links

🔗Kaggle Titanic Competition: https://www.kaggle.com/competitions/titanic

💻 GitHub Repository (Code & Notebooks): https://github.com/sinankmuriyanal/Titanic-ML-Project

This project marks just the beginning of my machine learning journey. The Titanic may have sunk, but my curiosity in data science is only rising. 🚀

23 Oct, 2025

AI-Powered Chatbot for Student Support: My DSE FAQ Project

Introduction

Ever spent hours hunting for answers about DSE MBA admissions or waitlist movements? PDFs, emails, and multiple websites can make the process painfully slow for applicants. I asked myself: Can AI make this smarter and faster?

That question led me to build the DSE FAQ Chatbot — an AI assistant that provides instant, accurate answers to 150+ student queries directly on Telegram, powered entirely on free-tier infrastructure.

Project Overview

The chatbot combines Large Language Models (LLM) with RAG (Retrieval-Augmented Generation), leveraging Python, FAISS, Hugging Face Datasets, and Render for deployment. It’s designed not just to answer, but to think, learn, and improve over time. The project goal was clear: build an AI assistant that is accurate, fast, and cost-effective, while maintaining structured logs for analysis. `

How the Chatbot Thinks: Behind the Scenes

1. Understanding Queries When a student types something like: “Hey, what’s the MBA (BA) cut-off?” The bot first splits the query: “Hey” → conversational “What’s the MBA (BA) cut-off?” → factual This ensures the bot doesn’t get distracted by small talk — just like humans filter noise before answering.

2. Session Memory

Each chat gets a session ID, functioning like a notebook for the conversation. Previous questions are remembered. Follow-ups like “What about HRD?” are understood in context. Session data is stored in Render-hosted PostgreSQL — no local files, no lost data.

3. Analysis & Reasoning

For every factual query, the bot: Formats the question for clarity. Checks its cache — if answered before, it responds instantly. Uses LLM reasoning if no cached answer exists. Evaluates confidence — if below 70%, it retrieves context from the indexed 150+ FAQs using FAISS. This is RAG in action — a fact-checking layer that ensures accuracy and contextual relevance.

4. Logging & Feedback Loop

Every interaction is logged with: User query Bot response Confidence score Response time Feedback is collected through a dedicated API endpoint. These logs form the foundation for continuous improvement, allowing me to measure: Accuracy Latency User satisfaction

5. Free-Tier Engineering

Building this AI assistant on $0 infrastructure was a challenge. FastAPI handles queries and feedback. Render hosts the backend and database. FAISS provides semantic search over FAQs. Hugging Face Datasets store static content efficiently. Optimizations like warm caching, async API calls, and token management ensured fast, reliable responses despite free-tier limits.

Challenges & Learnings

Accuracy & Coverage: Prompt engineering + dataset updates improved answer quality. Latency: Free-tier sleeping servers were handled with caching tricks. Session Coherence: Building conversational context improved user experience. Budget Constraints: Demonstrated that meaningful AI can run on minimal costs. Biggest takeaway? AI is not just coding. It’s about designing systems, thinking like an analyst, and iterating quickly.

The Impact

The chatbot: Answers 150+ FAQs instantly on Telegram. Stores every interaction for data-driven improvement. Operates completely free, showing that resourceful engineering and smart design can overcome constraints.

Future Plans

Update RAG with recent waitlists, admission notices, and placement reports. Integrate user feedback analytics to track friendliness and accuracy. Build an admin dashboard to monitor response speed, coverage, and usage trends. Extend coverage to more courses and departments, supporting more students.

Technical Stack

Backend: Python + FastAPI RAG Index: FAISS Database: PostgreSQL (Render-hosted) LLM: OpenAI / Fireworks GPT-OSS-20B Data Management: Hugging Face Datasets Frontend: Telegram Bot API

Conclusion

This project was more than a chatbot. It was a lesson in applied analytics, AI engineering, and resourceful problem-solving. Every log, feedback, and confidence score is a step toward making the system smarter — and helping students faster.

🔗Telegram Bot https://telegram.me/DSE_Chat_bot

💻 GitHub Repository (Code & Notebooks): Telegram Bot Repo

💻 GitHub Repository (Code & Notebooks): LLM API Repo