[{"content":"This course is designed for complete beginners who want to build a strong foundation in Python and Data Structures. If you’ve ever felt lost while learning DSA, this course is built specifically for you.\nWhat this course covers Python fundamentals for problem solving Core data structures (arrays, strings, recursion, linked lists, stacks, queues, trees, graphs) Step-by-step algorithmic thinking Writing clean and readable code Building confidence through consistent practice You can view the detailed curriculum breakdown here:\n📖 Complete DSA with Python Curriculum → How this course is taught Beginner-first explanations Live sessions with interaction Concepts broken down using examples Focus on what and why before how Practice-driven learning (not passive watching) Who this course is for Beginners with little or no programming experience Students struggling with DSA concepts Working professionals restarting fundamentals Course format Live classes + recordings Structured modules Regular practice problems Notes and references provided Enrollment 👉 Enrollment is currently open.\nClick the button below to join the course.\nEnroll Now ","permalink":"https://blogs.codeverra.com/courses/dsa-python-zero-to-hero/","summary":"\u003cp\u003eThis course is designed for \u003cstrong\u003ecomplete beginners\u003c/strong\u003e who want to build a strong foundation in Python and Data Structures.\nIf you’ve ever felt lost while learning DSA, this course is built specifically for you.\u003c/p\u003e\n\u003chr\u003e\n\u003ch2 id=\"what-this-course-covers\"\u003eWhat this course covers\u003c/h2\u003e\n\u003cul\u003e\n\u003cli\u003ePython fundamentals for problem solving\u003c/li\u003e\n\u003cli\u003eCore data structures (arrays, strings, recursion, linked lists, stacks, queues, trees, graphs)\u003c/li\u003e\n\u003cli\u003eStep-by-step algorithmic thinking\u003c/li\u003e\n\u003cli\u003eWriting clean and readable code\u003c/li\u003e\n\u003cli\u003eBuilding confidence through consistent practice\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003eYou can view the \u003cstrong\u003edetailed curriculum breakdown here\u003c/strong\u003e:\u003cbr\u003e\n\u003ca href=\"/blog/dsa-with-python-curriculum/\"\ntarget=\"_blank\"\nrel=\"noopener noreferrer\"\u003e\n📖 Complete DSA with Python Curriculum →\n\u003c/a\u003e\u003c/p\u003e","tags":null,"title":"DSA with Python: Zero to Hero"},{"content":"DSA Live Practice Series This is a live, topic-focused practice series designed for learners who already have basic familiarity with Data Structures and Algorithms and want to strengthen their problem-solving skills through guided live sessions.\nThe sessions are not a structured DSA course from scratch. Instead, each live class will focus on one specific DSA concept or pattern such as recursion, trees, dynamic programming, graphs, sliding window, two pointers, and similar topics. The emphasis will be on intuition, patterns, common mistakes, and problem-solving approaches, rather than syllabus completion.\nEach session is standalone, so you can attend based on your interest or weak areas without worrying about missing previous classes. Live problem solving, thought process explanation, and Q\u0026amp;A will be the core format.\nThis series is also part of the initial rollout of the Codeverra platform and will evolve based on learner feedback.\nWho this is for Learners who have already studied DSA basics Learners who are looking to revise DSA concepts Learners looking for live leetcode solving sessions Students looking for focused practice and clarity on tricky topics\nWho this is not for Absolute beginners looking for a complete DSA curriculum from day one\nEnrollment Enroll Now ","permalink":"https://blogs.codeverra.com/courses/dsa-live-practice/","summary":"\u003ch2 id=\"dsa-live-practice-series\"\u003eDSA Live Practice Series\u003c/h2\u003e\n\u003cp\u003eThis is a live, topic-focused practice series designed for learners who already have basic familiarity with Data Structures and Algorithms and want to strengthen their problem-solving skills through guided live sessions.\u003c/p\u003e\n\u003cp\u003eThe sessions are not a structured DSA course from scratch. Instead, each live class will focus on one specific DSA concept or pattern such as recursion, trees, dynamic programming, graphs, sliding window, two pointers, and similar topics. The emphasis will be on intuition, patterns, common mistakes, and problem-solving approaches, rather than syllabus completion.\u003c/p\u003e","tags":null,"title":"DSA Live Practice Series"},{"content":"Advanced LLD Topics — A Deep Dive Who this is for: Anyone who has finished the SOLID principles and the three design-pattern modules (Creational, Structural, Behavioural) and is ready to learn how to use that knowledge well — in real systems, in real interviews, in real production code.\nWhat this module builds on: SOLID Principles (Module 2), Creational Patterns (Module 3), Structural Patterns (Module 4), Behavioural Patterns (Module 5). What this module feeds into: Case Studies (Module 9) and Testing (Module 8). The Dependency Injection section here is the foundation for every fake, mock, and stub you\u0026rsquo;ll write in Module 8.\nTable of Contents Why This Module Exists Topic 1 — Thread Safety in LLD 2.1 The Booking That Sold the Same Room Twice 2.2 What Is a Race Condition? 2.3 The Fix — threading.Lock 2.4 The Thread-Safe Singleton 2.5 The GIL Myth — What Python\u0026rsquo;s GIL Does and Does Not Protect 2.6 RLock — When a Lock Needs to Be Re-entrant 2.7 queue.Queue — Thread Safety for Free 2.8 LLD Scenarios Where Thread Safety Matters Topic 2 — Dependency Injection as a Practice 3.1 The Class You Can\u0026rsquo;t Test 3.2 What DI Actually Is 3.3 Constructor Injection 3.4 Setter Injection 3.5 Method Injection 3.6 Why DI Makes Testing Possible — Side by Side 3.7 The Composition Root 3.8 DI Containers — Brief Mention Topic 3 — Anti-Patterns: What Not to Build 4.1 The God Class 4.2 The Anemic Domain Model 4.3 Over-Engineering / Premature Pattern Application Topic 4 — Design Judgment: When Not to Use a Pattern 5.1 Every Pattern Has a Cost 5.2 The Worth-It Table 5.3 Three Questions Before Any Pattern 5.4 YAGNI — You Aren\u0026rsquo;t Gonna Need It 5.5 The Simplest Thing That Could Possibly Work 5.6 The Interview Signal: Patterns You Didn\u0026rsquo;t Use Summary and Key Takeaways Practice Problems with Solutions 1. Why This Module Exists You\u0026rsquo;ve made it through SOLID. You\u0026rsquo;ve worked through Creational, Structural, and Behavioural patterns. You can name Singleton, Factory, Strategy, Observer, Decorator, and Adapter, and explain when to use each.\nThat\u0026rsquo;s the vocabulary. This module is about the judgment.\nKnowing patterns and using them well are two different skills. The first is taught in books. The second is forged by working on real systems and watching them strain — or break — under load. This module collects the four topics that consistently separate developers who \u0026ldquo;know patterns\u0026rdquo; from developers who \u0026ldquo;design software\u0026rdquo;:\nThread safety. Your beautiful design works perfectly in tests. Single-threaded tests. Then production runs it with 200 concurrent users and the bugs arrive. Every senior LLD interview probes this. Dependency Injection as a daily habit. The Dependency Inversion Principle (DIP) from SOLID isn\u0026rsquo;t a thing you \u0026ldquo;use once\u0026rdquo; — it\u0026rsquo;s how you write classes. DI is the practice. Without it, your code has no seams for tests, no flexibility for change. Anti-patterns by name. Patterns get named so we can talk about them. Anti-patterns get named for the same reason. An engineer who says \u0026ldquo;this is turning into a God Class, let me extract responsibilities\u0026rdquo; demonstrates more in one sentence than a candidate who just rearranges boxes on a whiteboard. Design judgment. When not to apply a pattern. Restraint is a senior-level signal. Wrapping a 30-line script in Strategy and Observer is not mastery — it\u0026rsquo;s anxiety dressed up as architecture. Each topic follows the same arc: a real scenario that hurts, the named problem, the fix in code, and a discussion of when the fix applies (and when it doesn\u0026rsquo;t). Practice problems with full solutions live at the end.\n2. Topic 1 — Thread Safety in LLD 2.1 The Booking That Sold the Same Room Twice It\u0026rsquo;s 11:58 PM on a Friday. Two people at opposite ends of the city are both trying to book the last room in a popular hotel for the weekend. Both phones open the booking app within 50 milliseconds of each other.\nBoth apps query: \u0026ldquo;Is room 401 available this weekend?\u0026rdquo; The server replies \u0026ldquo;yes\u0026rdquo; to both — because at the moment each query arrived, the room was available. Both apps now display a confirm button. Both users tap it. Two HTTP requests reach the booking service within a millisecond of each other.\nBoth threads on the server check room.is_available. Both see True. Both set it to False. Both write a booking row. Both return \u0026ldquo;confirmed.\u0026rdquo;\nOn Saturday morning, two families show up at the front desk with confirmation emails for room 401.\nThis is a race condition — a bug whose existence depends on the precise interleaving of two threads\u0026rsquo; operations. Single-user testing will never reveal it. Load tests with sparse traffic will rarely reveal it. Production at peak traffic reveals it routinely, and the support team learns to dread Mondays.\nDesigns that ignore concurrency work perfectly in development and fail unpredictably in production. Every interviewer worth their salt will probe this. This section is how you respond.\n2.2 What Is a Race Condition? A race condition is any bug whose presence depends on the order in which two or more operations execute, when that order isn\u0026rsquo;t guaranteed.\nThe most common shape of this bug is check-then-act:\n# BROKEN — race condition on check-then-act class HotelRoom: def __init__(self, room_id: str): self.room_id = room_id self.is_available = True def book(self, guest_name: str) -\u0026gt; str: if self.is_available: # (1) check # ↑ thread A reaches here, sees True # ↑ thread B *also* reaches here at the same time, also sees True self.is_available = False # (2) act return f\u0026#34;Room {self.room_id} booked for {guest_name}\u0026#34; return \u0026#34;Room not available\u0026#34; Notice how lines (1) and (2) are conceptually one operation — \u0026ldquo;if available, claim it\u0026rdquo; — but Python doesn\u0026rsquo;t enforce them as one. Between the if and the assignment, the operating system is free to suspend thread A and run thread B. If it does so right after the if, both threads pass the check, both threads do the assignment, and both threads return a success message. One room, two bookings.\nYou can reproduce this with a simulator:\nimport threading room = HotelRoom(\u0026#34;401\u0026#34;) results = [] def attempt_to_book(guest_name): results.append(room.book(guest_name)) # 100 concurrent attempts — see how many succeed threads = [threading.Thread(target=attempt_to_book, args=(f\u0026#34;guest-{i}\u0026#34;,)) for i in range(100)] for t in threads: t.start() for t in threads: t.join() successes = [r for r in results if \u0026#34;booked\u0026#34; in r] print(f\u0026#34;Successful bookings: {len(successes)}\u0026#34;) # Expected: 1. Actual: often 2, sometimes 3 or more. Run this five times. You\u0026rsquo;ll see different numbers each time. That\u0026rsquo;s the hallmark of a race condition — non-deterministic behaviour.\n2.3 The Fix — threading.Lock The right tool is a lock. A lock is a small flag with two rules:\nOnly one thread can \u0026ldquo;hold\u0026rdquo; the lock at any moment. A thread that tries to acquire a held lock waits until it\u0026rsquo;s released. We use it to wrap \u0026ldquo;check then act\u0026rdquo; so that the two operations together become atomic — indivisible from any other thread\u0026rsquo;s point of view.\nimport threading class HotelRoom: def __init__(self, room_id: str): self.room_id = room_id self.is_available = True self._lock = threading.Lock() def book(self, guest_name: str) -\u0026gt; str: with self._lock: # acquire — others wait here if self.is_available: self.is_available = False return f\u0026#34;Room {self.room_id} booked for {guest_name}\u0026#34; return \u0026#34;Room not available\u0026#34; # lock released automatically on exit, even if we raise Now thread A enters the with block. Thread B arrives a microsecond later and blocks at the with statement. Thread A checks (is_available = True), acts (is_available = False), and returns. Then thread A releases the lock. Thread B enters, checks (is_available = False), and returns \u0026ldquo;not available.\u0026rdquo;\nRe-run the 100-concurrent-attempts simulation: exactly 1 success, every time.\nImportant details:\nThe lock is per-room, not global. Different rooms can be booked concurrently — they don\u0026rsquo;t share a lock. A single global lock would correctly prevent over-booking but at the cost of serialising every booking in the entire hotel. The with self._lock: form is the right idiom. It releases the lock even if an exception is raised inside the block. (Recall the context-manager section in the OOP module.) The lock protects the invariant — not the data. \u0026ldquo;If is_available is True at the moment of check, this thread is the one that claims it\u0026rdquo; is the invariant. Anything that participates in that invariant must run inside the lock. 2.4 The Thread-Safe Singleton The basic Singleton you learned in the Creational Patterns module looks like this:\nclass Singleton: _instance = None def __new__(cls): if cls._instance is None: cls._instance = super().__new__(cls) return cls._instance Read carefully: this is a check-then-act. Two threads can both pass the is None check, and both can create an instance. The Singleton is now a Doubleton. Subtle, hard to reproduce, and devastating in production.\nThe fix is double-checked locking:\nimport threading class Singleton: _instance = None _lock = threading.Lock() def __new__(cls): if cls._instance is None: # (1) fast path, no lock with cls._lock: if cls._instance is None: # (2) recheck inside lock cls._instance = super().__new__(cls) return cls._instance Why check twice?\nThe outer check (line 1) is for speed. Once the singleton exists, every future call hits this fast path and returns immediately — no lock acquisition needed. Locks are cheap but not free. The inner check (line 2) is for correctness. Two threads can both pass the outer check at the same time, both enter the with block (one at a time, since it\u0026rsquo;s a lock), and both try to create the instance. The inner check stops the second thread from overwriting the first thread\u0026rsquo;s instance. Both checks together are required. Drop the outer check and every access pays for a lock. Drop the inner check and you might get two singletons.\nA cleaner alternative for many practical cases is module-level instantiation — Python guarantees module imports run exactly once:\n# config.py class _Config: def __init__(self): self.db_url = \u0026#34;...\u0026#34; config = _Config() # module-level, thread-safe by virtue of import semantics If you can use this pattern, prefer it. Reach for double-checked locking when you genuinely need lazy instantiation and the singleton lives in a more dynamic context.\n2.5 The GIL Myth A common Python-specific confusion: \u0026ldquo;Python has a GIL, so threads run one at a time, so I don\u0026rsquo;t need locks.\u0026rdquo;\nThis is wrong, and the misunderstanding causes a remarkable number of production bugs.\nThe Global Interpreter Lock (GIL) means that only one Python bytecode instruction executes at a time across threads. What it does not mean is that any Python statement you write is atomic. Statements compile into multiple bytecode instructions, and the GIL can switch between threads at the boundaries.\nConsider this:\ncounter = 0 def increment(): global counter counter += 1 # looks atomic. Isn\u0026#39;t. counter += 1 compiles into roughly:\nLoad the current value of counter. Add 1. Store the result back into counter. The GIL can suspend the thread between any of those three steps. Two threads both load 5, both add 1 to get 6, both store 6. Two increments produced one effective increment. Lost update.\nDemonstrate it:\nimport threading counter = 0 N = 100_000 def increment_many(): global counter for _ in range(N): counter += 1 t1 = threading.Thread(target=increment_many) t2 = threading.Thread(target=increment_many) t1.start(); t2.start() t1.join(); t2.join() print(counter) # Expected: 200_000. Actual: typically less — sometimes much less. The fix is the same as before — a lock:\ncounter = 0 lock = threading.Lock() def increment_many(): global counter for _ in range(N): with lock: counter += 1 Or, in cases like this, use an atomic primitive — for example itertools.count() or queue.Queue — which we\u0026rsquo;ll see in a moment.\nThe takeaway: the GIL gives you instruction-level atomicity, not statement-level atomicity, and definitely not multi-statement atomicity. Any sequence longer than one bytecode instruction can be interrupted. \u0026ldquo;Check then act,\u0026rdquo; \u0026ldquo;read then modify then write,\u0026rdquo; \u0026ldquo;read field A then field B\u0026rdquo; — all interruptable, all in need of synchronisation.\n2.6 RLock — When a Lock Needs to Be Re-entrant A regular Lock can be acquired by one thread at a time. If the same thread tries to acquire it again — say, because a method that holds the lock calls another method that also acquires it — the thread deadlocks against itself.\nclass Account: def __init__(self, balance): self.balance = balance self._lock = threading.Lock() def withdraw(self, amount): with self._lock: if self.balance \u0026gt;= amount: self.balance -= amount def transfer(self, other, amount): with self._lock: self.withdraw(amount) # ← deadlocks! same lock, same thread other.balance += amount The fix is threading.RLock — a re-entrant lock. It tracks which thread holds it and a count of acquisitions. The same thread can re-acquire it freely; only when the matching number of releases happen does another thread get a chance.\nclass Account: def __init__(self, balance): self.balance = balance self._lock = threading.RLock() def withdraw(self, amount): with self._lock: if self.balance \u0026gt;= amount: self.balance -= amount def transfer(self, other, amount): with self._lock: self.withdraw(amount) # works — same thread, re-entry allowed other.balance += amount Use RLock whenever a class has methods that call other locked methods of the same class. Use Lock otherwise — it\u0026rsquo;s slightly faster.\n2.7 queue.Queue — Thread Safety for Free For producer–consumer scenarios — one or more threads pushing work onto a queue, others pulling and processing — Python\u0026rsquo;s queue.Queue is already thread-safe. You don\u0026rsquo;t need to wrap it in a lock; that work has been done.\nimport queue import threading q: queue.Queue = queue.Queue() def producer(): for i in range(5): q.put(f\u0026#34;job-{i}\u0026#34;) # thread-safe q.put(None) # sentinel — signal \u0026#34;done\u0026#34; def consumer(): while True: item = q.get() # thread-safe; blocks until something is available if item is None: break print(f\u0026#34;processing {item}\u0026#34;) q.task_done() threading.Thread(target=producer).start() threading.Thread(target=consumer).start() Reach for queue.Queue whenever the pattern is \u0026ldquo;one thread enqueues, another dequeues.\u0026rdquo; It\u0026rsquo;s significantly less error-prone than building this yourself with a list and a lock.\n2.8 LLD Scenarios To anchor this in real LLD designs, here are the most common interview problems whose answers require thread safety:\nHotel/room booking. Two users booking the last available room. (As above.) Lock per room (fine-grained) or per hotel (coarse). Parking lot. Two cars arriving simultaneously, both being assigned the same spot. Lock the spot table or the assign_spot operation. Ticket reservation. Two buyers both purchasing the last seat. Lock per seat or per show. Inventory / e-commerce checkout. Two orders consuming the last item in stock. Lock per SKU. Connection pool. Two threads both being handed the same database connection. The pool\u0026rsquo;s acquire/release methods need a lock. Singleton resources. Configuration, registries, factories. Double-checked locking. In every case, the test is the same: what compound check-then-act exists in this code? What invariant must hold across that check-then-act? Whatever participates in that invariant goes inside the lock.\nWe\u0026rsquo;re keeping the scope to Lock, RLock, and queue.Queue here. Async (asyncio) and multiprocessing are entirely different concurrency models — they belong to a different module of the course.\nCross-reference. Remember the State pattern from the Behavioural module? A vending machine moving between \u0026ldquo;ready,\u0026rdquo; \u0026ldquo;money-inserted,\u0026rdquo; \u0026ldquo;dispensing\u0026rdquo; states is exactly the kind of small state machine that needs a lock around state transitions if multiple input sources can drive it at once. Same for an elevator\u0026rsquo;s Idle → Moving → Idle cycle.\n3. Topic 2 — Dependency Injection as a Practice 3.1 The Class You Can\u0026rsquo;t Test Read this class. It works. It even looks reasonable.\nclass OrderService: def __init__(self): self.db = MySQLDatabase(host=\u0026#34;prod-db\u0026#34;, port=3306, password=\u0026#34;...\u0026#34;) self.notifier = EmailNotifier(smtp_host=\u0026#34;smtp.gmail.com\u0026#34;, port=587) def place_order(self, user_id: str, items: list): order_id = self.db.insert_order(user_id, items) self.notifier.send( to=self.db.get_user_email(user_id), subject=\u0026#34;Order confirmed\u0026#34;, body=f\u0026#34;Your order {order_id} is on the way.\u0026#34;, ) return order_id Now try to test this without a real database or a live SMTP server. You can\u0026rsquo;t. The moment you instantiate OrderService in a test, it tries to open a real MySQL connection. There\u0026rsquo;s no seam — no place to slip in a fake.\nWhich means: every test of place_order requires a database to be running. Every CI build requires SMTP credentials. Every developer needs a local DB to even start the app. The class works in production but is a millstone to maintain.\nYou already met the principle that solves this: Dependency Inversion (the D in SOLID). Depend on abstractions, not concretions. This section is how DIP becomes a daily coding habit — through a practice called Dependency Injection (DI).\n3.2 What DI Actually Is DI is not a framework. It\u0026rsquo;s not magic. It\u0026rsquo;s one idea, stated in one line:\nA class should not create its dependencies; it should receive them.\nThat\u0026rsquo;s it. No annotations, no container, no XML config. Just a different way to write __init__.\nCompare:\n# Without DI — the class creates its own dependencies class OrderService: def __init__(self): self.db = MySQLDatabase() self.notifier = EmailNotifier() # With DI — the class accepts its dependencies class OrderService: def __init__(self, db, notifier): self.db = db self.notifier = notifier The difference is two lines of code. The consequences are enormous: testability, flexibility, decoupling.\nThere are three injection styles. We\u0026rsquo;ll see each one.\n3.3 Constructor Injection The most common, most explicit, and almost always the right default. Dependencies arrive as __init__ arguments and are stored as instance attributes.\nfrom typing import Protocol # --- Define contracts for what the service needs --- class Database(Protocol): def insert_order(self, user_id: str, items: list) -\u0026gt; str: ... def get_user_email(self, user_id: str) -\u0026gt; str: ... class Notifier(Protocol): def send(self, to: str, subject: str, body: str) -\u0026gt; None: ... # --- The service depends on contracts, not concrete classes --- class OrderService: def __init__(self, db: Database, notifier: Notifier): self.db = db self.notifier = notifier def place_order(self, user_id: str, items: list) -\u0026gt; str: order_id = self.db.insert_order(user_id, items) self.notifier.send( to=self.db.get_user_email(user_id), subject=\u0026#34;Order confirmed\u0026#34;, body=f\u0026#34;Your order {order_id} is on the way.\u0026#34;, ) return order_id Constructor injection is preferred because:\nAll dependencies are visible at the top of the class — no hidden creation buried in a method. The class is immediately ready to work after construction. No \u0026ldquo;did I remember to set the notifier?\u0026rdquo; risk. Dependencies are easy to spot during code review — they\u0026rsquo;re the parameters of __init__. Notice we typed against Protocols, not concrete classes. That\u0026rsquo;s the DIP part of DI: high-level code depends on contracts, low-level code provides implementations. (Recall Section 15A in the OOP module.)\n3.4 Setter Injection For optional dependencies. The most common example is a logger — most of your code can work without one, but you\u0026rsquo;d like the option to wire one in.\nclass OrderService: def __init__(self, db: Database, notifier: Notifier): self.db = db self.notifier = notifier self.logger = None # optional, may be left unset def set_logger(self, logger): self.logger = logger def place_order(self, user_id: str, items: list): if self.logger: self.logger.info(f\u0026#34;placing order for {user_id}\u0026#34;) order_id = self.db.insert_order(user_id, items) # ... return order_id Setter injection is fine when the dependency is genuinely optional. Don\u0026rsquo;t use it for required dependencies — it lets callers forget the wiring and create half-initialised objects.\n3.5 Method Injection When a dependency is needed only for one specific operation, pass it as a method argument rather than storing it on the instance.\nclass CheckoutService: def __init__(self, db: Database): self.db = db def calculate_total(self, cart, tax_calculator) -\u0026gt; float: \u0026#34;\u0026#34;\u0026#34;Tax rules can change per call — region, season, promotion.\u0026#34;\u0026#34;\u0026#34; subtotal = sum(item.price for item in cart.items) return subtotal + tax_calculator.calculate(subtotal, cart.region) The tax_calculator isn\u0026rsquo;t a long-lived dependency of the service; it varies by call. Passing it as a method argument expresses exactly that — and avoids polluting the instance with state that doesn\u0026rsquo;t really belong to it.\n3.6 Why DI Makes Testing Possible — Side by Side The promise of DI is testability. Let\u0026rsquo;s prove it concretely.\nThe undisciplined version — untestable in any meaningful sense:\nclass OrderService: def __init__(self): self.db = MySQLDatabase() # connects to real DB self.notifier = EmailNotifier() # connects to real SMTP def place_order(self, user_id, items): order_id = self.db.insert_order(user_id, items) self.notifier.send(self.db.get_user_email(user_id), \u0026#34;...\u0026#34;, \u0026#34;...\u0026#34;) return order_id # To test this you need: # - A running MySQL with the right schema # - Valid SMTP credentials # - Maybe a mocking library to monkey-patch globals (ugly) # Tests are slow, flaky, and require infrastructure to even start. The DI version — trivially testable:\nclass OrderService: def __init__(self, db, notifier): self.db = db self.notifier = notifier def place_order(self, user_id, items): order_id = self.db.insert_order(user_id, items) self.notifier.send(self.db.get_user_email(user_id), \u0026#34;Confirmed\u0026#34;, f\u0026#34;#{order_id}\u0026#34;) return order_id # --- A real, runnable test --- class FakeDatabase: def __init__(self): self.orders = [] self.users = {\u0026#34;u-1\u0026#34;: \u0026#34;amit@example.com\u0026#34;} def insert_order(self, user_id, items): order_id = f\u0026#34;o-{len(self.orders) + 1}\u0026#34; self.orders.append({\u0026#34;id\u0026#34;: order_id, \u0026#34;user_id\u0026#34;: user_id, \u0026#34;items\u0026#34;: items}) return order_id def get_user_email(self, user_id): return self.users[user_id] class FakeNotifier: def __init__(self): self.sent = [] def send(self, to, subject, body): self.sent.append({\u0026#34;to\u0026#34;: to, \u0026#34;subject\u0026#34;: subject, \u0026#34;body\u0026#34;: body}) def test_place_order_sends_confirmation(): db = FakeDatabase() notifier = FakeNotifier() svc = OrderService(db=db, notifier=notifier) order_id = svc.place_order(user_id=\u0026#34;u-1\u0026#34;, items=[\u0026#34;headphones\u0026#34;]) assert order_id == \u0026#34;o-1\u0026#34; assert db.orders == [{\u0026#34;id\u0026#34;: \u0026#34;o-1\u0026#34;, \u0026#34;user_id\u0026#34;: \u0026#34;u-1\u0026#34;, \u0026#34;items\u0026#34;: [\u0026#34;headphones\u0026#34;]}] assert len(notifier.sent) == 1 assert notifier.sent[0][\u0026#34;to\u0026#34;] == \u0026#34;amit@example.com\u0026#34; assert \u0026#34;o-1\u0026#34; in notifier.sent[0][\u0026#34;body\u0026#34;] test_place_order_sends_confirmation() print(\u0026#34;✓ test passed\u0026#34;) Read that test. It runs in milliseconds. It needs no infrastructure. It precisely asserts what place_order does without depending on anything external. That entire test is possible only because of DI. Without it, there is no seam at which to insert FakeDatabase and FakeNotifier.\nDI creates seams. Tests live at those seams.\n3.7 The Composition Root If OrderService doesn\u0026rsquo;t create its dependencies, someone has to. That someone is the composition root — usually a single location (often main.py or an app factory function) where all concrete implementations are wired together.\n# main.py — the ONE place that knows about concrete implementations. import config def build_app(): # Infrastructure db = MySQLDatabase(connection_string=config.DB_URL) notifier = EmailNotifier(smtp_config=config.SMTP) logger = StdLogger(level=config.LOG_LEVEL) payment_gateway = StripeGateway(api_key=config.STRIPE_KEY) # Services — each gets exactly the dependencies it needs order_service = OrderService(db=db, notifier=notifier) checkout_service = CheckoutService(db=db, payment_gateway=payment_gateway) user_service = UserService(db=db, logger=logger) # The HTTP layer — gets the services app = HTTPApp(order=order_service, checkout=checkout_service, user=user_service) return app if __name__ == \u0026#34;__main__\u0026#34;: app = build_app() app.run() Outside this file, no class knows about MySQL, SMTP, or Stripe. Everyone depends on abstractions. The composition root is the only file you\u0026rsquo;d touch if you switched the database from MySQL to Postgres — every other class stays untouched because it never knew about MySQL in the first place.\nContrast with the test:\n# test_order_service.py — a completely different composition root def make_test_service(): db = FakeDatabase() notifier = FakeNotifier() return OrderService(db=db, notifier=notifier) Same class. Different composition root. Same test, no infrastructure.\n3.8 DI Containers In large systems with hundreds of classes, manually wiring everything in main.py becomes verbose. DI containers (e.g., dependency-injector, injector in Python; Spring in Java) automate this — you register a class, declare what it depends on, and the container constructs the graph for you.\nFor LLD interviews and most production Python services, you don\u0026rsquo;t need a container. Constructor injection is what matters. Containers are a production convenience for very large systems, not a precondition for doing DI properly.\nIf you do reach for one, do it because the manual wiring genuinely became painful — not because the container exists.\nCross-references.\nBack to SOLID (Module 2): DI is how you practice DIP every day. The principle says \u0026ldquo;depend on abstractions\u0026rdquo;; DI is the practical answer to \u0026ldquo;but how?\u0026rdquo; Forward to Testing (Module 8): every fake, mock, and stub in the testing module only works because the system under test was built with DI. Without seams, no doubles. 4. Topic 3 — Anti-Patterns: What Not to Build Every experienced engineer has a horror story. A codebase where fixing one bug took two weeks because touching anything broke three unrelated things. A class so big the IDE struggled to load it. A \u0026ldquo;framework\u0026rdquo; written for a six-line problem.\nAnti-patterns are the named culprits. Knowing them by name is as important as knowing patterns. The candidate who says \u0026ldquo;this is becoming a God Class — let me extract responsibilities before adding the feature\u0026rdquo; demonstrates more about their design instincts in one sentence than a candidate who simply draws bigger boxes.\nWe\u0026rsquo;ll cover three. Each follows the same arc: the definition, an analogy, painful code, the fix.\n4.1 The God Class Definition. One class that knows everything, does everything, and is imported by everything.\nAnalogy. Imagine a company where every employee reports directly to the CEO. Every decision — hiring, marketing, billing, tech, lunch orders — goes through the CEO. The CEO knows everything. The CEO does everything. What happens when the CEO is sick? The company stops. What happens when you hire someone new? They go straight to the CEO. What happens when the CEO is overworked and makes a mistake? Everything is affected.\nThe CEO is a single point of failure and a bottleneck. Now imagine that CEO is a class in your codebase.\nThe painful code:\nclass ECommerceSystem: \u0026#34;\u0026#34;\u0026#34;Does everything. Owned by everyone. Feared by all.\u0026#34;\u0026#34;\u0026#34; def __init__(self): self.db = ... # --- User auth --- def register_user(self, email, password): ... def login(self, email, password): ... def logout(self, session_id): ... def reset_password(self, email): ... # --- Catalog --- def search_products(self, query): ... def get_product(self, product_id): ... def add_product(self, product): ... def update_inventory(self, product_id, qty): ... # --- Cart --- def add_to_cart(self, user_id, product_id): ... def remove_from_cart(self, user_id, product_id): ... def get_cart(self, user_id): ... # --- Payments --- def charge_card(self, user_id, amount): ... def refund(self, transaction_id): ... # --- Notifications --- def send_email(self, to, subject, body): ... def send_sms(self, to, body): ... # --- Logging / metrics --- def log_event(self, event): ... def emit_metric(self, name, value): ... # ...18 more methods you forgot to scroll past... What\u0026rsquo;s wrong:\nMany reasons to change. Marketing changes notification copy → edit this class. Payments switches gateway → edit this class. New auth flow → edit this class. Three teams modifying one file is a merge-conflict factory. Untestable. Want to test the cart logic? You inherit the entire class — including the DB, the SMTP client, the payment gateway, all of it. Hidden coupling. A change to send_email could ripple into register_user because nothing prevents one method from quietly depending on another. Every new feature touches it. That\u0026rsquo;s a sign, not a coincidence. Signs to recognise it in the wild:\nMore than 5–7 public methods spanning unrelated concerns. Imports from 10+ modules at the top of the file. A class name that hand-waves: Manager, Helper, Utils, System, Handler. Every new feature requires touching this class. The fix — extract along responsibility lines:\nclass UserService: def register(self, email, password): ... def login(self, email, password): ... def logout(self, session_id): ... def reset_password(self, email): ... class CatalogService: def search(self, query): ... def get(self, product_id): ... class CartService: def add(self, user_id, product_id): ... def remove(self, user_id, product_id): ... def get(self, user_id): ... class PaymentProcessor: def charge(self, user_id, amount): ... def refund(self, transaction_id): ... class InventoryManager: def get_stock(self, product_id): ... def update(self, product_id, qty): ... class Notifier: def email(self, to, subject, body): ... def sms(self, to, body): ... Each class has one reason to change. Tests for CartService don\u0026rsquo;t drag in SMTP. The payments team can iterate on PaymentProcessor without coordinating with the catalog team. This is the Single Responsibility Principle (Module 2) showing up in the wild.\nA god class isn\u0026rsquo;t defeated by one refactor — it\u0026rsquo;s prevented by noticing it forming and extracting responsibilities while the cost of extracting is still small. Once a class has 40 methods and seven teams depend on it, the surgery is much harder.\n4.2 The Anemic Domain Model Definition. Domain classes that hold data but have no meaningful behaviour. All logic lives in separate \u0026ldquo;manager\u0026rdquo; or \u0026ldquo;service\u0026rdquo; classes that reach into the domain objects and manipulate them directly.\nAnalogy. Imagine a hospital where patients are just folders of paper. The patients themselves don\u0026rsquo;t speak, don\u0026rsquo;t decide, don\u0026rsquo;t act. All decisions are made by administrators who pull the folder, read the data, decide something, and edit the folder. The patients have no agency. When the rules for treating diabetic patients change, you update the administrators\u0026rsquo; procedures — not the patients\u0026rsquo; own behaviour, because they don\u0026rsquo;t have any.\nDomain models in code can devolve into this. The data lives in one place; the logic lives in another. They drift apart, and the logic loses its grip on the invariants the data is supposed to obey.\nThe painful code:\n@dataclass class BankAccount: account_id: str owner: str balance: float class BankAccountManager: \u0026#34;\u0026#34;\u0026#34;All the actual behaviour lives here, reaching into BankAccount.\u0026#34;\u0026#34;\u0026#34; def deposit(self, account: BankAccount, amount: float): account.balance += amount def withdraw(self, account: BankAccount, amount: float): if account.balance \u0026gt;= amount: account.balance -= amount else: raise ValueError(\u0026#34;Insufficient funds\u0026#34;) def transfer(self, src: BankAccount, dst: BankAccount, amount: float): if src.balance \u0026gt;= amount: src.balance -= amount dst.balance += amount # Usage acc = BankAccount(\u0026#34;a-1\u0026#34;, \u0026#34;Amit\u0026#34;, 1000.0) mgr = BankAccountManager() mgr.withdraw(acc, 200) Why this is anaemic — and why it\u0026rsquo;s actually a problem, not just a stylistic choice:\nThe invariant is unenforceable. \u0026ldquo;Balance can\u0026rsquo;t go negative\u0026rdquo; is a fact about a BankAccount. But anyone, anywhere, can write acc.balance -= 5000 and bypass withdraw entirely. The data class doesn\u0026rsquo;t protect itself. Logic gets duplicated. Different managers (or just future developers) re-implement the rules slightly differently. Now there are two definitions of \u0026ldquo;what does withdrawal mean,\u0026rdquo; and they slowly diverge. Encapsulation is fake. The data is technically inside BankAccount, but every other class freely reaches in. The boundary is ceremonial. The rule: logic belongs where the data lives.\nclass BankAccount: def __init__(self, account_id: str, owner: str, balance: float): if balance \u0026lt; 0: raise ValueError(\u0026#34;Initial balance cannot be negative\u0026#34;) self.account_id = account_id self.owner = owner self._balance = balance # private — protected by methods @property def balance(self) -\u0026gt; float: return self._balance def deposit(self, amount: float) -\u0026gt; None: if amount \u0026lt;= 0: raise ValueError(\u0026#34;Deposit must be positive\u0026#34;) self._balance += amount def withdraw(self, amount: float) -\u0026gt; None: if amount \u0026lt;= 0: raise ValueError(\u0026#34;Withdrawal must be positive\u0026#34;) if self._balance \u0026lt; amount: raise ValueError(\u0026#34;Insufficient funds\u0026#34;) self._balance -= amount def transfer(self, dst: \u0026#34;BankAccount\u0026#34;, amount: float) -\u0026gt; None: self.withdraw(amount) dst.deposit(amount) # Usage — the account guards itself acc = BankAccount(\u0026#34;a-1\u0026#34;, \u0026#34;Amit\u0026#34;, 1000.0) acc.withdraw(200) # acc._balance = -1_000_000 ← still possible in Python, but flagged by convention The invariant (\u0026ldquo;balance never goes negative\u0026rdquo;) is now intrinsic to the class. Every code path that changes the balance goes through the class\u0026rsquo;s own methods. The behaviour and the data live together.\nWhen you still want a service. Not every coordination concern fits inside one entity. Multi-account transfers with audit logging, cross-currency conversion, and atomic database updates may need to live in a TransferService — because they coordinate multiple accounts and infrastructure. But within that service, the intrinsic behaviour of an account stays on the account. The service orchestrates; the account enforces its own rules.\nAnaemic models often arise from over-applying the \u0026ldquo;DTO pattern\u0026rdquo; or from blindly mapping ORM tables to classes without adding methods. Both are warning signs.\n4.3 Over-Engineering / Premature Pattern Application Definition. Applying patterns that aren\u0026rsquo;t yet needed — creating complexity in anticipation of requirements that may never arrive.\nAnalogy. Building a 12-lane highway for a village of 200 people because \u0026ldquo;it might grow one day.\u0026rdquo; The cost — land, materials, maintenance — is real today. The benefit is hypothetical. By the time the village actually grows (if it ever does), the highway\u0026rsquo;s design might be wrong for whatever it actually grew into.\nCode has the same shape. Every pattern adds a cost — a class, a layer, a vocabulary word a new reader has to learn before they can navigate the code. That cost is paid every day by everyone who touches the file. The benefit — extensibility, swappability — is only realised when you actually have the variation the pattern enables.\nThe painful code:\nA teammate has wrapped a simple temperature converter in three patterns \u0026ldquo;for flexibility\u0026rdquo;:\nfrom abc import ABC, abstractmethod # 1. An abstract strategy class ConversionStrategy(ABC): @abstractmethod def convert(self, value: float) -\u0026gt; float: ... # 2. Three concrete strategies — for three fixed formulas class CelsiusToFahrenheit(ConversionStrategy): def convert(self, c): return c * 9 / 5 + 32 class CelsiusToKelvin(ConversionStrategy): def convert(self, c): return c + 273.15 class FahrenheitToCelsius(ConversionStrategy): def convert(self, f): return (f - 32) * 5 / 9 # 3. A factory to choose the strategy class ConverterFactory: _registry = { (\u0026#34;C\u0026#34;, \u0026#34;F\u0026#34;): CelsiusToFahrenheit, (\u0026#34;C\u0026#34;, \u0026#34;K\u0026#34;): CelsiusToKelvin, (\u0026#34;F\u0026#34;, \u0026#34;C\u0026#34;): FahrenheitToCelsius, } @classmethod def get(cls, from_unit, to_unit) -\u0026gt; ConversionStrategy: return cls._registry[(from_unit, to_unit)]() # 4. A plugin loader, in case \u0026#34;users want to add custom units\u0026#34; class PluginLoader: def load_plugins(self, directory): ... def register_strategy(self, key, strategy_class): ... # Usage strategy = ConverterFactory.get(\u0026#34;C\u0026#34;, \u0026#34;F\u0026#34;) print(strategy.convert(100)) # 212.0 What\u0026rsquo;s wrong:\nPatterns for fixed problems. There are three temperature formulas. They will not change next quarter, next year, or in the lifetime of the codebase. Strategy is overkill. A factory selecting among three things. A dict could do this in one line. A plugin loader for plugins that don\u0026rsquo;t exist. Built for a future that hasn\u0026rsquo;t arrived and probably never will. A reader can\u0026rsquo;t see the formula. They have to navigate three classes and a registry to find c * 9 / 5 + 32. The right code:\ndef celsius_to_fahrenheit(c): return c * 9 / 5 + 32 def celsius_to_kelvin(c): return c + 273.15 def fahrenheit_to_celsius(f): return (f - 32) * 5 / 9 CONVERTERS = { (\u0026#34;C\u0026#34;, \u0026#34;F\u0026#34;): celsius_to_fahrenheit, (\u0026#34;C\u0026#34;, \u0026#34;K\u0026#34;): celsius_to_kelvin, (\u0026#34;F\u0026#34;, \u0026#34;C\u0026#34;): fahrenheit_to_celsius, } def convert(value, from_unit, to_unit): return CONVERTERS[(from_unit, to_unit)](value) print(convert(100, \u0026#34;C\u0026#34;, \u0026#34;F\u0026#34;)) # 212.0 Three functions and a dict. Anyone can read this. It does exactly what the original did. It will keep working for years. If someday a real need for runtime-pluggable strategies appears — say, the team adds 30 unit types and wants users to plug in their own — you can refactor toward the original Strategy design at that point. You\u0026rsquo;ll know what shape to give it because you\u0026rsquo;ll have the actual requirement, not a guess.\nThe rule: YAGNI. You Aren\u0026rsquo;t Gonna Need It. Apply a pattern when you feel the pain it solves, not before. The best code for a feature you don\u0026rsquo;t need yet is no code at all.\nOver-engineering is the most expensive anti-pattern in this list, because unlike a God Class or an Anemic Model, it actively looks like good engineering at first glance. Junior reviewers nod approvingly at the Strategy hierarchy. Only experience teaches you to count the moving parts and ask: \u0026ldquo;do we actually have the problem these patterns solve?\u0026rdquo;\n5. Topic 4 — Design Judgment: When Not to Use a Pattern Knowing patterns is knowing tools. Wisdom is knowing which to pick up and which to leave in the drawer.\nA developer who wraps a 30-line script in Singleton, Factory, Strategy, and Observer is not demonstrating mastery. They\u0026rsquo;re demonstrating anxiety — the worry that someone might one day need flexibility they can\u0026rsquo;t yet envision, so they pre-pay for every kind of flexibility. The cost is real today; the benefit is hypothetical.\nReal mastery shows in restraint: knowing when a function is enough.\n5.1 Every Pattern Has a Cost Every pattern you apply adds, at minimum:\nOne new class or file. One new layer of indirection. One more concept a future reader must understand before the code makes sense. That cost is paid every day, by every developer who touches that code. It compounds. A codebase with 50 unnecessary abstractions takes longer to onboard onto, longer to debug, and longer to change — because changes now require navigating the abstractions to find the actual logic.\nThe benefit — extensibility, decoupling, testability — is real only when you actually have the problem the pattern solves. A Strategy pattern with two strategies that will never grow gives you zero flexibility benefit but charges you for the indirection forever.\n5.2 The Worth-It Table A rough mental model for the most common patterns:\nPattern Worth it when Not worth it when Singleton One truly global resource with shared mutable state (a connection pool, config registry, metrics collector). You just want to share an object — use DI instead. Singletons create hidden global state. Factory The concrete type varies at runtime based on input (config, user choice, data). You always create the same type. A direct constructor or class method is clearer. Strategy The algorithm genuinely varies at runtime, and the set of strategies is growing or extensible. Two variations that are unlikely to grow. An if/else or a dict-of-functions is more honest. Observer Multiple independent things must react to a change, and you don\u0026rsquo;t know who they all are. One thing reacts. Just call it directly. Decorator Behaviours compose in many combinations (logging × caching × auth × retries). One or two variations. A subclass is fine. Adapter You must integrate two existing, fixed interfaces you can\u0026rsquo;t change. You control both sides. Change one to match. Command Operations need to be queued, logged, undone, or replayed. Operations execute immediately and forever. A function call is simpler. Read this table not as a verdict but as a starting point for the conversation: am I adding this pattern because the problem demands it, or because the pattern is nearby?\n5.3 Three Questions Before Any Pattern Before applying any pattern, ask yourself:\nWhat specific problem does this pattern solve? Do I have that problem right now, in this code? What complexity does adding this pattern introduce? If the answer to question 1 doesn\u0026rsquo;t match a real pain visible in question 2, put the pattern down. If the cost in question 3 exceeds the visible benefit, put the pattern down.\nA concrete example. You\u0026rsquo;re writing a discount system. Two discount types exist today: FLAT (₹100 off) and PERCENTAGE (10% off). A teammate suggests Strategy.\nApply the three questions:\nWhat problem does Strategy solve? Allows new algorithms to be added without changing existing code. Do we have that problem? Marginally. There are exactly two types and no current plans for more. What does Strategy cost? An abstract base class, two concrete classes, a factory or registry, a context that holds the current strategy. Four to five new classes for what is currently a five-line if/else. The answer is: not yet. An if/else (or a dict-of-functions) is the right tool today. If discount rules start coming from a database, if business users start requesting custom rules, if the count grows to six and feels likely to keep growing — then the Strategy refactor has a real reason to exist. You\u0026rsquo;ll do it then, with the actual requirements as your guide, instead of guessing now.\n5.4 YAGNI YAGNI — You Aren\u0026rsquo;t Gonna Need It — is the single most useful design principle that has nothing to do with any specific pattern.\nStated plainly: build for requirements you have, not requirements you imagine.\nThe argument: imagined requirements are notoriously wrong. The abstraction you build for scenario A almost never matches what scenario B actually needs when it arrives. You end up with two costs — the abstraction you over-built for the wrong future, plus the refactor required to support the actual future. Compared to building straightforwardly today and refactoring when the real need appears, YAGNI almost always wins.\nThe best code for a feature you don\u0026rsquo;t need yet is no code at all.\nThis is not an argument for sloppy code or for ignoring the future. It is an argument for staying honest about the difference between today\u0026rsquo;s requirements and tomorrow\u0026rsquo;s speculation.\n5.5 The Simplest Thing That Could Possibly Work A useful question to ask before adding any abstraction: \u0026ldquo;What\u0026rsquo;s the simplest version that works, is readable, and passes all the tests?\u0026rdquo;\nIf that simpler version is maintainable, ship it. Add abstraction only when the simple version starts cracking under real pressure.\nReal pressure looks like:\nA third variant of the algorithm appears and the if/else is becoming a tangle. Two teams want different behaviour and the shared code path can\u0026rsquo;t accommodate both. A unit test becomes painful to write because of a hidden dependency. These are signals that the simple version has reached its useful life. Now refactor toward the pattern that addresses the real pain. The refactor will be obvious, because the pain is visible.\nRefactoring under pressure is risk-free because the pressure tells you exactly what to refactor to. Refactoring in anticipation is high-risk because you\u0026rsquo;re optimising against a future you can only guess at.\n5.6 The Interview Signal Here\u0026rsquo;s a small thing that pays disproportionate dividends in interviews: naming a pattern you chose NOT to use, and saying why.\n\u0026ldquo;I considered using Strategy here, but there are only two pricing rules and they\u0026rsquo;re unlikely to change. The class hierarchy isn\u0026rsquo;t worth it yet. If the rules start coming from configuration or the count grows past four or five, I\u0026rsquo;d refactor to Strategy then.\u0026rdquo;\nThat sentence signals:\nYou know Strategy exists. You know what problem it solves. You know how to recognise when that problem is real. You\u0026rsquo;re not adding complexity to look senior. That\u0026rsquo;s a more mature signal than someone who applies Strategy to everything. Interviewers notice.\nMake this an explicit habit during system-design discussions. When you decline a pattern, name it and explain why. Even if no one asks. It\u0026rsquo;s the cleanest way to show that you understand the patterns deeply enough to choose against them when appropriate.\n6. Summary and Key Takeaways Five points to internalise from this module:\nThread safety is not automatic. The GIL does not protect compound operations like counter += 1, if available: claim(), or read field then write another. Every check-then-act in a concurrent system is a potential race condition. Wrap them in threading.Lock (or RLock when the same thread re-acquires) — or use queue.Queue for producer/consumer flows.\nDependency Injection is how DIP becomes a daily habit. Pass dependencies into your classes; don\u0026rsquo;t construct them inside. The seams DI creates are exactly where fakes, stubs, and mocks live in tests. Without DI, there are no seams — the code is a wall.\nThree anti-patterns to recognise and name on sight.\nGod Class — one class doing everything; split along responsibility lines. Anemic Domain Model — data without behaviour; let logic live where the data lives. Over-Engineering — patterns applied before the problem exists; YAGNI. Every pattern has a real cost in complexity. Classes, indirection, vocabulary. That cost is paid every day. The benefit must be visible today, not hypothetical, before you should pay it.\nThe most important design question is not \u0026ldquo;which pattern?\u0026rdquo; but \u0026ldquo;do I even need a pattern here?\u0026rdquo; Restraint is a senior signal. Naming patterns you didn\u0026rsquo;t use — and why — communicates more than applying them indiscriminately.\n7. Practice Problems with Solutions Twelve problems across four categories. Solutions include both code and reasoning — because the reasoning is the actual lesson.\nType 1 — Spot the Issue Read the code, identify the problem, name it, explain the fix. No coding required.\nP1.1 — The Ticketing Race class TicketingSystem: def __init__(self, total_seats: int): self.available = total_seats def book_seat(self, user_id: str) -\u0026gt; bool: if self.available \u0026gt; 0: self.available -= 1 print(f\u0026#34;Seat booked for {user_id}\u0026#34;) return True return False What is the bug? Under what conditions does it manifest? What is the fix?\nSolution.\nThe bug. Classic check-then-act race condition. if self.available \u0026gt; 0 (the check) and self.available -= 1 (the act) are two separate operations. When two threads call book_seat near-simultaneously and only one seat remains, both can read available \u0026gt; 0 as True before either has decremented. Both proceed to decrement. available ends up at -1, and two bookings are confirmed for the last seat.\nWhen it manifests. Any time multiple threads call book_seat on the same instance. The narrower the window between the check and the act, the rarer the bug — but it\u0026rsquo;s always possible, and concurrent load makes it likely. Single-threaded testing will never catch it.\nThe fix. Wrap the compound operation in a lock so that no other thread can interleave between the check and the act:\nimport threading class TicketingSystem: def __init__(self, total_seats: int): self.available = total_seats self._lock = threading.Lock() def book_seat(self, user_id: str) -\u0026gt; bool: with self._lock: if self.available \u0026gt; 0: self.available -= 1 print(f\u0026#34;Seat booked for {user_id}\u0026#34;) return True return False Note: the lock must wrap both the check and the act. Wrapping only the decrement (or only the check) would still allow the race.\nP1.2 — The God Class class UserSystem: def register(self, email, password): ... def login(self, email, password): ... def send_welcome_email(self, email): ... def update_profile(self, user_id, data): ... def save_to_db(self, user): ... def generate_report(self, user_id): ... def log_activity(self, user_id, action): ... What\u0026rsquo;s wrong? Name the anti-pattern. What responsibilities can you identify? How would you split this?\nSolution.\nThe anti-pattern. God Class. UserSystem is doing too many unrelated jobs, each with its own reason to change, its own collaborators, and its own test requirements.\nResponsibilities visible in the methods:\nAuthentication. register, login. (Plus probably logout, reset_password if the class grows further.) Notifications. send_welcome_email. (And almost certainly other emails before long.) Profile management. update_profile. (User attributes — name, address, preferences.) Persistence. save_to_db. (Database concerns.) Reporting. generate_report. (Aggregations, formatting, possibly export.) Auditing. log_activity. (Cross-cutting concern for security/compliance.) The split.\nclass AuthService: def register(self, email, password): ... def login(self, email, password): ... class UserRepository: # persistence concerns def save(self, user): ... def find_by_id(self, user_id): ... class ProfileService: def update(self, user_id, data): ... class Notifier: def send_welcome_email(self, email): ... class ReportService: def generate(self, user_id): ... class ActivityLogger: def log(self, user_id, action): ... Each class now has one reason to change. The auth team can iterate on AuthService without coordinating with the reporting team. Tests for ProfileService no longer drag in SMTP credentials. New requirements (e.g., changing the auth flow) modify one class instead of one file owned by everyone.\nIn practice, an orchestrating UserService may exist on top — but its job is coordination between these smaller services, not implementing each one.\nP1.3 — The Over-Engineered Converter from abc import ABC, abstractmethod class ConverterStrategy(ABC): @abstractmethod def convert(self, amount): ... class INRtoUSD(ConverterStrategy): def convert(self, amount): return amount / 83.5 class USDtoEUR(ConverterStrategy): def convert(self, amount): return amount * 0.92 class AbstractConverterFactory(ABC): @abstractmethod def create(self, src, dst): ... class DefaultConverterFactory(AbstractConverterFactory): def create(self, src, dst): return STRATEGY_REGISTRY.get((src, dst))() class PluginLoader: def __init__(self): self.plugins = {} def load(self, name, cls): self.plugins[name] = cls def find_or_load(self, src, dst): ... STRATEGY_REGISTRY = {(\u0026#34;INR\u0026#34;, \u0026#34;USD\u0026#34;): INRtoUSD, (\u0026#34;USD\u0026#34;, \u0026#34;EUR\u0026#34;): USDtoEUR} Is this design appropriate? What is the correct implementation given that the rates are fixed in code and only two conversions are supported?\nSolution.\nThe anti-pattern. Over-engineering / premature pattern application. For two hard-coded conversions, the design uses Strategy, Abstract Factory, a registry, and a plugin loader. Four pattern-shaped pieces of infrastructure for two fixed values.\nCosts being paid for no benefit:\nSix classes plus a registry where two functions would do. A reader must understand four design patterns to find a single multiplication. Maintenance, code review, and onboarding all pay for indirection that has no current customer. The right implementation:\ndef inr_to_usd(amount): return amount / 83.5 def usd_to_eur(amount): return amount * 0.92 CONVERTERS = { (\u0026#34;INR\u0026#34;, \u0026#34;USD\u0026#34;): inr_to_usd, (\u0026#34;USD\u0026#34;, \u0026#34;EUR\u0026#34;): usd_to_eur, } def convert(amount, src, dst): return CONVERTERS[(src, dst)](amount) print(convert(1000, \u0026#34;INR\u0026#34;, \u0026#34;USD\u0026#34;)) # ~11.98 Two functions, a dict, and a wrapper. Reads in five seconds. Does exactly what the original did. If, in some future iteration, exchange rates start coming from a live API, conversions multiply to dozens, and rates change daily — then refactoring to a Strategy or to a RateProvider abstraction will be obvious and justified by the actual requirements. Until then, simpler is better.\nType 2 — Fix the Code Broken implementation provided. Write the corrected version with explanation.\nP2.1 — Make It Thread-Safe The connection pool below issues database connections to threads. It is not thread-safe. Identify the race condition. Add the right locking. Explain where the race happened and why the lock placement matters.\nclass ConnectionPool: def __init__(self, size: int): self._connections = [self._create_connection(i) for i in range(size)] self._in_use = set() def acquire(self) -\u0026gt; \u0026#34;Connection\u0026#34;: for conn in self._connections: if conn.id not in self._in_use: self._in_use.add(conn.id) return conn raise RuntimeError(\u0026#34;No connections available\u0026#34;) def release(self, conn: \u0026#34;Connection\u0026#34;) -\u0026gt; None: self._in_use.discard(conn.id) def _create_connection(self, id_: int) -\u0026gt; \u0026#34;Connection\u0026#34;: return Connection(id=id_) Solution.\nWhere the race is. Inside acquire(). Two threads can both iterate, both find the same conn not in _in_use, and both add it to _in_use and return it. Both think they have an exclusive connection — but they\u0026rsquo;re holding the same one. Database operations from the two threads would now collide on a single connection, with chaos following.\nA secondary race is release(). If release is called from one thread while acquire is iterating in another, the membership check conn.id not in self._in_use can give a stale answer.\nThe fix. A lock around the entire acquire body and around release. Both operations participate in the same invariant (\u0026quot;_in_use accurately reflects who owns what\u0026quot;), so both must take the lock.\nimport threading class ConnectionPool: def __init__(self, size: int): self._connections = [self._create_connection(i) for i in range(size)] self._in_use: set[int] = set() self._lock = threading.Lock() def acquire(self) -\u0026gt; \u0026#34;Connection\u0026#34;: with self._lock: for conn in self._connections: if conn.id not in self._in_use: self._in_use.add(conn.id) return conn raise RuntimeError(\u0026#34;No connections available\u0026#34;) def release(self, conn: \u0026#34;Connection\u0026#34;) -\u0026gt; None: with self._lock: self._in_use.discard(conn.id) def _create_connection(self, id_: int) -\u0026gt; \u0026#34;Connection\u0026#34;: return Connection(id=id_) Why the lock placement matters. A common mistake is to lock only the self._in_use.add(conn.id) line. That doesn\u0026rsquo;t help — by the time the lock is acquired, two threads have already chosen the same conn. The lock must cover the compound operation: iterate, check, claim. All three together form the atomic block.\nA more sophisticated implementation might use a queue.Queue of available connections — which is itself thread-safe — to avoid manual locking altogether:\nimport queue class ConnectionPool: def __init__(self, size: int): self._available: queue.Queue = queue.Queue() for i in range(size): self._available.put(self._create_connection(i)) def acquire(self, timeout=None): return self._available.get(timeout=timeout) # thread-safe def release(self, conn): self._available.put(conn) # thread-safe Both designs are correct. The first explicitly shows the lock; the second uses a thread-safe data structure that hides the lock. For production, the queue-based version is usually cleaner.\nP2.2 — Refactor to Use DI The InvoiceService below creates its dependencies inside its constructor. Refactor it to use constructor injection. Then write a small test using fake implementations to prove the refactor works.\nclass PDFExporter: def export(self, invoice) -\u0026gt; bytes: # imagine real PDF generation here, requiring a library and disk I/O return b\u0026#34;%PDF-...\u0026#34; class EmailSender: def send(self, to, subject, body, attachments): # imagine real SMTP work here, requiring credentials and a network call print(f\u0026#34;sent to {to}\u0026#34;) class InvoiceService: def __init__(self): self.exporter = PDFExporter() self.email = EmailSender() def send_invoice(self, invoice, to_email): pdf = self.exporter.export(invoice) self.email.send( to=to_email, subject=f\u0026#34;Invoice #{invoice.id}\u0026#34;, body=\u0026#34;Your invoice is attached.\u0026#34;, attachments=[(\u0026#34;invoice.pdf\u0026#34;, pdf)], ) Solution.\nRefactored class:\nfrom typing import Protocol # --- Contracts: what the service needs, expressed structurally --- class Exporter(Protocol): def export(self, invoice) -\u0026gt; bytes: ... class Sender(Protocol): def send(self, to: str, subject: str, body: str, attachments: list): ... class InvoiceService: def __init__(self, exporter: Exporter, sender: Sender): # Dependencies arrive as constructor arguments. # The class no longer knows about PDF generation or SMTP. self.exporter = exporter self.sender = sender def send_invoice(self, invoice, to_email): pdf = self.exporter.export(invoice) self.sender.send( to=to_email, subject=f\u0026#34;Invoice #{invoice.id}\u0026#34;, body=\u0026#34;Your invoice is attached.\u0026#34;, attachments=[(\u0026#34;invoice.pdf\u0026#34;, pdf)], ) Composition root — wires the real implementations together:\n# main.py service = InvoiceService(exporter=PDFExporter(), sender=EmailSender()) Test — uses fakes, no PDF library, no SMTP server:\nfrom dataclasses import dataclass @dataclass class FakeInvoice: id: str class FakeExporter: def __init__(self): self.calls = [] def export(self, invoice): self.calls.append(invoice) return b\u0026#34;FAKE_PDF_BYTES\u0026#34; class FakeSender: def __init__(self): self.sent = [] def send(self, to, subject, body, attachments): self.sent.append({ \u0026#34;to\u0026#34;: to, \u0026#34;subject\u0026#34;: subject, \u0026#34;body\u0026#34;: body, \u0026#34;attachments\u0026#34;: attachments, }) def test_send_invoice_attaches_pdf_and_emails_recipient(): exporter = FakeExporter() sender = FakeSender() svc = InvoiceService(exporter=exporter, sender=sender) invoice = FakeInvoice(id=\u0026#34;INV-100\u0026#34;) svc.send_invoice(invoice, to_email=\u0026#34;user@example.com\u0026#34;) # Exporter was called with the right invoice assert exporter.calls == [invoice] # Sender received exactly one email with the PDF attached assert len(sender.sent) == 1 email = sender.sent[0] assert email[\u0026#34;to\u0026#34;] == \u0026#34;user@example.com\u0026#34; assert email[\u0026#34;subject\u0026#34;] == \u0026#34;Invoice #INV-100\u0026#34; assert email[\u0026#34;attachments\u0026#34;] == [(\u0026#34;invoice.pdf\u0026#34;, b\u0026#34;FAKE_PDF_BYTES\u0026#34;)] test_send_invoice_attaches_pdf_and_emails_recipient() print(\u0026#34;✓ test passed\u0026#34;) What changed, and why:\n__init__ accepts exporter and sender instead of constructing them. The class is now decoupled from PDF and SMTP implementations. The class depends on Protocols (Exporter, Sender), not concrete classes — applying DIP from SOLID. The test runs without any infrastructure: no PDF library, no SMTP server, no network. It exercises the logic of send_invoice, which is what we actually want to test. The composition root (main.py) is now the only place where PDFExporter() and EmailSender() are mentioned. Swap one out (e.g., replace SMTP with SendGrid) by editing exactly one line. Compare this to testing the original version: you\u0026rsquo;d have needed monkey-patching, mocking libraries, or a real SMTP server. DI made the test obvious.\nP2.3 — Fix the Anemic Domain Model The code below splits a ShoppingCart into a passive data class and an active service that reaches into it. Move the cart\u0026rsquo;s intrinsic behaviour back onto the ShoppingCart itself, leaving only coordination logic in the service (if anything remains there at all).\nfrom dataclasses import dataclass, field @dataclass class ShoppingCart: user_id: str items: list = field(default_factory=list) discount_code: str | None = None class ShoppingCartService: def add_item(self, cart: ShoppingCart, item: dict): cart.items.append(item) def remove_item(self, cart: ShoppingCart, item_id: str): cart.items = [i for i in cart.items if i[\u0026#34;id\u0026#34;] != item_id] def apply_discount(self, cart: ShoppingCart, code: str): if code in {\u0026#34;WELCOME10\u0026#34;, \u0026#34;FESTIVE25\u0026#34;}: cart.discount_code = code else: raise ValueError(\u0026#34;Invalid discount code\u0026#34;) def calculate_total(self, cart: ShoppingCart) -\u0026gt; float: subtotal = sum(i[\u0026#34;price\u0026#34;] * i[\u0026#34;qty\u0026#34;] for i in cart.items) if cart.discount_code == \u0026#34;WELCOME10\u0026#34;: return subtotal * 0.9 if cart.discount_code == \u0026#34;FESTIVE25\u0026#34;: return subtotal * 0.75 return subtotal Solution.\nThe intrinsic behaviour — adding items, removing items, applying a discount, calculating the total — belongs to the cart. These operations protect the cart\u0026rsquo;s invariants (valid discount codes, never-negative quantities, etc.). Moving them onto the cart itself is what gives the class its agency back.\nclass ShoppingCart: _VALID_DISCOUNTS = {\u0026#34;WELCOME10\u0026#34;: 0.10, \u0026#34;FESTIVE25\u0026#34;: 0.25} def __init__(self, user_id: str): self.user_id = user_id self._items: list[dict] = [] self._discount_code: str | None = None def add_item(self, item: dict) -\u0026gt; None: if item.get(\u0026#34;qty\u0026#34;, 0) \u0026lt;= 0: raise ValueError(\u0026#34;Item quantity must be positive\u0026#34;) if item.get(\u0026#34;price\u0026#34;, 0) \u0026lt; 0: raise ValueError(\u0026#34;Item price cannot be negative\u0026#34;) self._items.append(item) def remove_item(self, item_id: str) -\u0026gt; None: self._items = [i for i in self._items if i[\u0026#34;id\u0026#34;] != item_id] def apply_discount(self, code: str) -\u0026gt; None: if code not in self._VALID_DISCOUNTS: raise ValueError(f\u0026#34;Invalid discount code: {code}\u0026#34;) self._discount_code = code def remove_discount(self) -\u0026gt; None: self._discount_code = None def subtotal(self) -\u0026gt; float: return sum(i[\u0026#34;price\u0026#34;] * i[\u0026#34;qty\u0026#34;] for i in self._items) def total(self) -\u0026gt; float: discount_rate = self._VALID_DISCOUNTS.get(self._discount_code, 0) return self.subtotal() * (1 - discount_rate) def __len__(self): return sum(i[\u0026#34;qty\u0026#34;] for i in self._items) The service shrinks dramatically — possibly disappears, or becomes a thin orchestrator for cross-cutting concerns (saving the cart, integrating with checkout):\nclass CheckoutService: \u0026#34;\u0026#34;\u0026#34;Coordinates a cart with payment + persistence. The cart enforces its own rules.\u0026#34;\u0026#34;\u0026#34; def __init__(self, payment_gateway, repo): self.payment_gateway = payment_gateway self.repo = repo def checkout(self, cart: ShoppingCart) -\u0026gt; str: total = cart.total() # cart computes its own total transaction_id = self.payment_gateway.charge(cart.user_id, total) self.repo.save(cart, status=\u0026#34;paid\u0026#34;, txn=transaction_id) return transaction_id What changed, and why:\nValidations (qty \u0026gt; 0, valid discount code) live inside the cart. The cart can no longer enter an invalid state from outside. total() and subtotal() are properties of the cart, not of an external service — because they\u0026rsquo;re computed from cart data. The cart\u0026rsquo;s _items and _discount_code are private. External code can no longer reach in and bypass the rules. This is the encapsulation the dataclass version pretended to have but didn\u0026rsquo;t really enforce. The remaining service (CheckoutService) is genuinely about coordination between the cart, the payment gateway, and persistence — concerns that span multiple objects. That\u0026rsquo;s a legitimate service responsibility. The rule restated: logic belongs where the data lives. Intrinsic operations (those that depend only on the object\u0026rsquo;s own state and protect its own invariants) go on the object. Coordination across multiple objects and infrastructure stays in services.\nType 3 — Design Judgment Open-ended scenarios. No single right answer — the goal is reasoning through tradeoffs.\nP3.1 — To Pattern or Not to Pattern? A teammate is reviewing your code for a discount system. There are exactly two discount types today: FLAT (e.g., ₹100 off) and PERCENTAGE (e.g., 10% off). Your current implementation:\ndef calculate_discount(subtotal: float, kind: str, value: float) -\u0026gt; float: if kind == \u0026#34;FLAT\u0026#34;: return max(subtotal - value, 0) if kind == \u0026#34;PERCENTAGE\u0026#34;: return subtotal * (1 - value / 100) raise ValueError(f\u0026#34;Unknown discount kind: {kind}\u0026#34;) Your teammate says: \u0026ldquo;This should use the Strategy pattern. Each discount type should be its own class implementing a DiscountStrategy interface. That way we can add new discount types without modifying existing code — it\u0026rsquo;s Open/Closed.\u0026rdquo;\nShould you agree? Explain your reasoning. Under what conditions would you refactor to Strategy?\nSolution.\nThe short answer: not yet. Here\u0026rsquo;s the reasoning.\nApply the three questions from Section 5.3:\nWhat problem does Strategy solve? It allows new variants of an algorithm to be added without modifying the dispatch code — solving the Open/Closed Principle when the set of variants is open-ended or extensible.\nDo we have that problem right now? No. There are exactly two discount types. There is no current plan to add more. They are stable business rules.\nWhat does adding Strategy cost? An abstract base class (or Protocol), two concrete classes, a factory or registry to pick between them, and a call site that delegates through the registry. Roughly 30–40 lines and 4 new symbols for what is currently 5 lines and 1 function.\nYour teammate is technically right that Strategy could work here. They are wrong that it should be applied here. The Open/Closed Principle is a guideline, not a moral imperative. It\u0026rsquo;s worth applying when extension is realistic and the cost of indirection is low compared to the cost of future modification. Neither is true for two fixed discount types.\nThe right answer is the dict-of-functions version, which is honest about the structure of the problem:\ndef flat_discount(subtotal, value): return max(subtotal - value, 0) def percent_discount(subtotal, value): return subtotal * (1 - value / 100) DISCOUNTS = {\u0026#34;FLAT\u0026#34;: flat_discount, \u0026#34;PERCENTAGE\u0026#34;: percent_discount} def calculate_discount(subtotal, kind, value): try: return DISCOUNTS[kind](subtotal, value) except KeyError: raise ValueError(f\u0026#34;Unknown discount kind: {kind}\u0026#34;) This costs almost nothing more than the original. It cleanly separates each formula, and registering a new kind is one line — without the ceremony of a class hierarchy.\nWhen would refactoring to Strategy be justified? The refactor earns its keep when one of these is true:\nThe count grows past five or six discount types and continues to grow. The dispatch table starts feeling cluttered. Strategies become stateful. Buy-one-get-one needs to remember which item triggered the offer. Tiered discounts have thresholds. Stateful logic doesn\u0026rsquo;t fit cleanly in a single function. Strategies come from configuration, not from code. A rules engine builds them at runtime from data. Now you need a real abstraction with a defined interface, because the strategies aren\u0026rsquo;t all known at compile time. Two implementations must coexist — e.g., the \u0026ldquo;old\u0026rdquo; discount engine and the \u0026ldquo;new\u0026rdquo; one — for an A/B test or a migration. A common interface helps. Until one of these is true, the dict version is the right code. That you can name Strategy and explain why you didn\u0026rsquo;t use it is the more senior signal here.\nP3.2 — Choose Between Three Approaches You\u0026rsquo;re designing a logging system for a medium-sized service. Three teammates propose three different designs:\nDev A: \u0026ldquo;Use a Singleton. There\u0026rsquo;s one logger for the whole app. Every module imports it and calls logger.info(...).\u0026rdquo; Dev B: \u0026ldquo;Use Observer. Each class emits events; the logging system subscribes and writes the ones it cares about.\u0026rdquo; Dev C: \u0026ldquo;Use Dependency Injection. Every service that needs logging gets a logger passed into its constructor. Wire them up in the composition root.\u0026rdquo; For each approach, list its strengths and weaknesses for this problem. Then recommend one, with justification.\nSolution.\nDev A — Singleton.\nStrengths:\nExtremely convenient. Every module can import logger; logger.info(...). Truly minimal ceremony. New code starts logging in one line. Weaknesses:\nHidden global state. Every class that uses the logger has an invisible dependency on it. You can\u0026rsquo;t see, by looking at a class\u0026rsquo;s constructor, that it logs. Untestable. In a test, you have to either let the real logger write to disk/network or monkey-patch the global — both ugly. Replacing the logger for one test affects every other test running in the same process. Configuration is awkward. What if one service should log at DEBUG and another at INFO? With a singleton, you have one configuration for everyone. Inflexible. Want a different logger for a feature flag, or a tenant-specific logger? Singletons don\u0026rsquo;t support per-call substitution. Dev B — Observer.\nStrengths:\nDecoupled. Emitters know nothing about subscribers. Open to extension — new subscribers (metrics collectors, alerters) plug in without changing emitters. Weaknesses:\nOverkill for logging. Logging isn\u0026rsquo;t \u0026ldquo;many independent things reacting to events\u0026rdquo; — it\u0026rsquo;s \u0026ldquo;one thing recording what happened.\u0026rdquo; Observer\u0026rsquo;s value is fan-out; logging is fan-in. Inverts control unnecessarily. Logging is a synchronous, well-understood operation. The class wants to say \u0026ldquo;log this.\u0026rdquo; Forcing it to \u0026ldquo;emit an event\u0026rdquo; adds indirection for no benefit. Debugging becomes harder. Tracing why a particular log line appeared requires understanding the event bus. With a direct logger call, the call site is the answer. More code, more concepts. Event types, registration, dispatch — all just to write a line of text. Observer is the right answer for use cases like \u0026ldquo;the order was placed; now notify inventory, billing, and email all at once.\u0026rdquo; It is not the right answer for \u0026ldquo;record what\u0026rsquo;s happening in this class.\u0026rdquo;\nDev C — Dependency Injection.\nStrengths:\nExplicit dependencies. A class that needs logging declares it in its constructor. Code review sees the dependency at a glance. Trivially testable. In tests, pass a FakeLogger that records calls. Assertions on logged content are clean and fast. Per-instance configuration. Different services can receive loggers configured differently (different levels, different sinks). Flexible. Future changes — adding context, switching from stdout to a service like Datadog — happen in the composition root, not throughout the codebase. Weaknesses:\nA little more ceremony than the singleton (\u0026ldquo;I have to pass it in to every class\u0026rdquo;). Many classes end up with logger as a constructor argument. Some teams find this verbose. Recommendation: Dev C — Dependency Injection, with a shared logger instance.\nThe composition root creates one configured logger (or a small number, e.g., one per major subsystem) and passes it into the services that need it:\n# composition root shared_logger = StructuredLogger(level=\u0026#34;INFO\u0026#34;, sinks=[StdoutSink(), DatadogSink()]) order_service = OrderService(db=db, notifier=notifier, logger=shared_logger) user_service = UserService(db=db, logger=shared_logger) This captures the practical benefit Dev A wanted (one logger for the app, configured once) while giving you the testability Dev C wanted (every class accepts a logger; tests pass a fake). Best of both, none of the singleton\u0026rsquo;s downsides.\nThe Singleton (Dev A) and Observer (Dev B) approaches each get one thing right but trade away properties — testability, explicitness, debuggability — that matter every day in a real codebase. DI gives them up for very little cost.\nThis problem is a useful demonstration of a broader principle: DI is not a single solution to one problem; it\u0026rsquo;s an enabling discipline. It makes other patterns optional rather than necessary.\nType 4 — Capstone Problem Synthesise everything from the module.\nP4.1 — Review and Improve a Ride-Hailing Service Below is a TripService that has been working in production but has been generating mysterious bugs and is impossible to test. Your senior engineer asks you to do three things:\nIdentify the issues in the code. Name each by its anti-pattern, principle violated, or risk category. Sketch the corrected design. What classes/responsibilities should exist? Rewrite TripService applying the fixes. Annotate the rewrite with comments explaining why each change was made. import threading import time class DatabaseConnection: def __init__(self): # imagine an expensive setup: TCP handshake, auth, schema check time.sleep(0.5) def execute(self, query, params): ... def fetch_one(self, query, params): return {\u0026#34;id\u0026#34;: 1} class EmailService: def send(self, to, subject, body): # imagine real SMTP work print(f\u0026#34;[email] sent to {to}\u0026#34;) class TripService: \u0026#34;\u0026#34;\u0026#34;Handles creating trips, assigning drivers, calculating fares, and notifying users.\u0026#34;\u0026#34;\u0026#34; def __init__(self): # Creates its own database — every TripService creation costs 500ms. self.db = DatabaseConnection() # Drivers list, kept in memory. self.available_drivers = [\u0026#34;driver-A\u0026#34;, \u0026#34;driver-B\u0026#34;, \u0026#34;driver-C\u0026#34;] def create_trip(self, rider_id: str, pickup: tuple, drop: tuple) -\u0026gt; dict: # 1. Assign a driver. No locking — two simultaneous trip creations could # both grab the same driver. if not self.available_drivers: raise RuntimeError(\u0026#34;No drivers available\u0026#34;) driver_id = self.available_drivers.pop(0) # 2. Fare calculation lives directly inside this method. # Magic numbers. Hard to test without exercising the whole flow. distance_km = ((pickup[0] - drop[0]) ** 2 + (pickup[1] - drop[1]) ** 2) ** 0.5 base_fare = 50 per_km = 15 surge = 1.2 if 17 \u0026lt;= time.localtime().tm_hour \u0026lt;= 20 else 1.0 fare = (base_fare + distance_km * per_km) * surge # 3. Persistence directly on self.db — coupled to MySQL specifics. trip_id = f\u0026#34;trip-{int(time.time())}\u0026#34; self.db.execute( \u0026#34;INSERT INTO trips VALUES (?, ?, ?, ?)\u0026#34;, (trip_id, rider_id, driver_id, fare), ) # 4. Notification with a hard-coded EmailService instantiated inline. EmailService().send( to=f\u0026#34;{rider_id}@example.com\u0026#34;, subject=\u0026#34;Trip confirmed\u0026#34;, body=f\u0026#34;Your trip {trip_id} with {driver_id} costs ₹{fare:.2f}\u0026#34;, ) return {\u0026#34;trip_id\u0026#34;: trip_id, \u0026#34;driver\u0026#34;: driver_id, \u0026#34;fare\u0026#34;: fare} Solution.\n1. Issues identified:\n# Issue Category A TripService.__init__ creates its own DatabaseConnection. The 500ms setup runs every time. Untestable without a real DB. Dependency Inversion violation; no seams for tests. B EmailService() is constructed inline inside create_trip. Each notification creates a new service instance, untestable, hard-coded to email. DIP violation; tight coupling to a concrete implementation. C available_drivers.pop(0) is a check-then-act with no locking. Two concurrent trips could both pop the same driver. Race condition / thread-safety bug. D Fare calculation is inlined inside create_trip with magic numbers and a hidden surge-pricing rule. Cannot be tested in isolation. Single Responsibility violation + Anemic Domain (fare logic belongs on a FarePolicy). E TripService knows about: drivers list management, fare formulas, database SQL, email content, and trip creation. Many reasons to change. God Class smell (forming, not yet full). F EmailService is a hard dependency. The system can never use SMS, push notifications, or no notification at all (e.g., in tests). DIP violation. G No abstraction over notifications — assumes email is the only channel. Open/Closed violation. 2. Sketch of the corrected design.\n┌──────────────────────────────────────────────────────────────────┐ │ composition root (main.py) │ │ - creates DB connection, FarePolicy, DriverPool, Notifier, │ │ and wires them into TripService │ └─────────────────────────┬────────────────────────────────────────┘ │ ▼ ┌─────────────────────────┐ │ TripService │ │ (orchestrates only) │ └─────┬───────┬────────┬──┘ │ │ │ ┌───────────▼─┐ ┌──▼─────┐ ┌▼──────────────┐ │ DriverPool │ │ Fare- │ │ Notifier │ │ (thread- │ │ Policy │ │ (Protocol) │ │ safe) │ │ │ │ │ └─────────────┘ └────────┘ └───────────────┘ ▲ ┌───────────┼───────────┐ │ EmailNotifier SMSNotifier ... Roles:\nTripService orchestrates. It does not own DB connections, drivers, or fare rules — it composes them. DriverPool owns the available-drivers list with thread-safe acquire/release. FarePolicy (or FareCalculator) is the unit-testable home of fare logic. The surge rule lives here too, with its own dependency (a Clock) so it can be tested. TripRepository wraps DB writes — the service depends on the repository, not on SQL. Notifier is a Protocol. Concrete implementations: EmailNotifier, SMSNotifier, and NullNotifier for tests. 3. Rewritten TripService:\nimport threading import time from dataclasses import dataclass from typing import Protocol # ---------- Contracts ---------- class Clock(Protocol): def hour(self) -\u0026gt; int: ... class FarePolicy(Protocol): def calculate(self, pickup: tuple, drop: tuple) -\u0026gt; float: ... class TripRepository(Protocol): def save(self, trip: \u0026#34;Trip\u0026#34;) -\u0026gt; None: ... class Notifier(Protocol): def notify(self, to: str, subject: str, body: str) -\u0026gt; None: ... # ---------- Domain object ---------- @dataclass(frozen=True) class Trip: \u0026#34;\u0026#34;\u0026#34;A trip is a value object — once issued, its facts don\u0026#39;t change.\u0026#34;\u0026#34;\u0026#34; trip_id: str rider_id: str driver_id: str fare: float # ---------- Thread-safe driver pool ---------- class DriverPool: \u0026#34;\u0026#34;\u0026#34; Owns the available-drivers list. Thread-safe acquire/release so that two concurrent trip-creations can never claim the same driver. \u0026#34;\u0026#34;\u0026#34; def __init__(self, drivers: list[str]): self._drivers = list(drivers) # defensive copy self._lock = threading.Lock() def acquire(self) -\u0026gt; str: # Compound check-then-act is wrapped in a single critical section. # Removing this lock reintroduces the original race. with self._lock: if not self._drivers: raise RuntimeError(\u0026#34;No drivers available\u0026#34;) return self._drivers.pop(0) def release(self, driver_id: str) -\u0026gt; None: with self._lock: self._drivers.append(driver_id) # ---------- Fare policy ---------- class StandardFarePolicy: \u0026#34;\u0026#34;\u0026#34; Encapsulates fare calculation. Lives outside TripService so it can be tested independently — given pickup/drop and a clock, it produces a fare. No I/O, no global state. \u0026#34;\u0026#34;\u0026#34; BASE_FARE = 50.0 PER_KM = 15.0 SURGE_MULTIPLIER = 1.2 SURGE_START_HOUR = 17 SURGE_END_HOUR = 20 def __init__(self, clock: Clock): # The clock is injected so tests can pretend it\u0026#39;s 6pm without waiting. self.clock = clock def calculate(self, pickup: tuple, drop: tuple) -\u0026gt; float: distance_km = ((pickup[0] - drop[0]) ** 2 + (pickup[1] - drop[1]) ** 2) ** 0.5 fare = self.BASE_FARE + distance_km * self.PER_KM if self.SURGE_START_HOUR \u0026lt;= self.clock.hour() \u0026lt;= self.SURGE_END_HOUR: fare *= self.SURGE_MULTIPLIER return fare # ---------- The orchestrator ---------- class TripService: \u0026#34;\u0026#34;\u0026#34; Orchestrates a trip creation. Owns no drivers, no fare formulas, no SQL. Composes collaborators that do. \u0026#34;\u0026#34;\u0026#34; def __init__( self, driver_pool: DriverPool, fare_policy: FarePolicy, trip_repo: TripRepository, notifier: Notifier, ): # All dependencies arrive via the constructor — no hidden creation. # Every collaborator can be swapped (real DB ↔ fake, email ↔ null) in tests. self.driver_pool = driver_pool self.fare_policy = fare_policy self.trip_repo = trip_repo self.notifier = notifier def create_trip(self, rider_id: str, pickup: tuple, drop: tuple) -\u0026gt; Trip: # 1. Acquire a driver atomically (race-free thanks to DriverPool\u0026#39;s lock). driver_id = self.driver_pool.acquire() try: # 2. Fare logic delegated entirely to the policy. fare = self.fare_policy.calculate(pickup, drop) # 3. Persistence delegated to the repository. trip = Trip( trip_id=f\u0026#34;trip-{int(time.time() * 1000)}\u0026#34;, rider_id=rider_id, driver_id=driver_id, fare=fare, ) self.trip_repo.save(trip) # 4. Notification through the injected Notifier — could be email, # SMS, push, or a null notifier in tests. self.notifier.notify( to=rider_id, subject=\u0026#34;Trip confirmed\u0026#34;, body=f\u0026#34;Trip {trip.trip_id} with driver {driver_id} costs ₹{fare:.2f}\u0026#34;, ) return trip except Exception: # If anything after the driver acquire fails, return the driver to the pool. # Without this, a failure during save/notify would leak the driver forever. self.driver_pool.release(driver_id) raise Composition root (production wiring):\n# main.py class SystemClock: def hour(self) -\u0026gt; int: return time.localtime().tm_hour class MySQLTripRepo: def __init__(self, conn): self.conn = conn def save(self, trip: Trip) -\u0026gt; None: self.conn.execute( \u0026#34;INSERT INTO trips (id, rider_id, driver_id, fare) VALUES (?, ?, ?, ?)\u0026#34;, (trip.trip_id, trip.rider_id, trip.driver_id, trip.fare), ) class EmailNotifier: def __init__(self, email_service): self.email = email_service def notify(self, to, subject, body): self.email.send(f\u0026#34;{to}@example.com\u0026#34;, subject, body) db = DatabaseConnection() # built once service = TripService( driver_pool=DriverPool([\u0026#34;driver-A\u0026#34;, \u0026#34;driver-B\u0026#34;, \u0026#34;driver-C\u0026#34;]), fare_policy=StandardFarePolicy(clock=SystemClock()), trip_repo=MySQLTripRepo(conn=db), notifier=EmailNotifier(email_service=EmailService()), ) Test wiring (same TripService, different collaborators):\nclass FakeClock: def __init__(self, hour): self._hour = hour def hour(self): return self._hour class InMemoryTripRepo: def __init__(self): self.saved = [] def save(self, trip): self.saved.append(trip) class FakeNotifier: def __init__(self): self.sent = [] def notify(self, to, subject, body): self.sent.append((to, subject, body)) def test_create_trip_at_surge_hour_uses_surge_pricing(): pool = DriverPool([\u0026#34;driver-1\u0026#34;]) repo = InMemoryTripRepo() notifier = FakeNotifier() service = TripService( driver_pool=pool, fare_policy=StandardFarePolicy(clock=FakeClock(hour=18)), # surge hour trip_repo=repo, notifier=notifier, ) trip = service.create_trip(\u0026#34;rider-1\u0026#34;, pickup=(0, 0), drop=(10, 0)) # 10km * 15 = 150 base + 50 = 200, * 1.2 surge = 240.0 assert trip.fare == 240.0 assert trip.driver_id == \u0026#34;driver-1\u0026#34; assert repo.saved == [trip] assert notifier.sent[0][0] == \u0026#34;rider-1\u0026#34; test_create_trip_at_surge_hour_uses_surge_pricing() print(\u0026#34;✓ test passed\u0026#34;) Summary of what was changed and why:\nDI for every dependency. The constructor declares everything TripService needs. No hidden creation. The same service can be wired with real or fake collaborators. (Fixes issues A, B, F.) Thread-safe driver pool. A dedicated DriverPool class owns the list and serialises access with a lock. The driver-assignment race is eliminated. (Fixes issue C.) Fare policy extracted. StandardFarePolicy is a small, focused class with its own injectable clock. It\u0026rsquo;s directly unit-testable — give it coordinates and a clock, get a fare. The surge rule is named and documented, not hidden inside a method. (Fixes issues D, G.) Repository pattern. TripService doesn\u0026rsquo;t write SQL. It delegates to a TripRepository. Swap MySQL for Postgres or an in-memory store without changing the service. (Decouples from infrastructure.) Notifier is a Protocol. Email is now one possible notifier. SMS, push, and a null-notifier-for-tests can be plugged in without changing the service. (Fixes issue G.) Trip is a frozen dataclass. A trip\u0026rsquo;s facts are immutable once issued. Returning a Trip is more useful than returning a dict and gives downstream code a typed object to reason about. Driver leak on error is fixed. The try/except in create_trip ensures that if the save or notification fails, the driver is released — not left \u0026ldquo;in use\u0026rdquo; forever. The refactored TripService orchestrates collaborators rather than doing all the work itself. Each collaborator has a single responsibility, a single reason to change, and a clear test surface. This is the entire module\u0026rsquo;s content — thread safety, DI, anti-pattern avoidance, and design judgment — applied together to one realistic problem.\nYou\u0026rsquo;re now equipped with the design vocabulary, the testing seams, the concurrency awareness, and (most importantly) the restraint that separates an engineer who knows patterns from one who designs systems. The remaining modules — Case Studies, Testing — build on this foundation.\nThis content is part of Codeverra — a platform for learning coding, data science, DSA, and AI from scratch. Explore more: https://codeverra.com\n","permalink":"https://blogs.codeverra.com/blog/low_level_design/07_advanced_lld_topics/","summary":"\u003ch1 id=\"advanced-lld-topics--a-deep-dive\"\u003eAdvanced LLD Topics — A Deep Dive\u003c/h1\u003e\n\u003cblockquote\u003e\n\u003cp\u003e\u003cstrong\u003eWho this is for:\u003c/strong\u003e Anyone who has finished the SOLID principles and the three design-pattern modules (Creational, Structural, Behavioural) and is ready to learn how to use that knowledge \u003cem\u003ewell\u003c/em\u003e — in real systems, in real interviews, in real production code.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cblockquote\u003e\n\u003cp\u003e\u003cstrong\u003eWhat this module builds on:\u003c/strong\u003e SOLID Principles (Module 2), Creational Patterns (Module 3), Structural Patterns (Module 4), Behavioural Patterns (Module 5).\n\u003cstrong\u003eWhat this module feeds into:\u003c/strong\u003e Case Studies (Module 9) and Testing (Module 8). The Dependency Injection section here is the foundation for every fake, mock, and stub you\u0026rsquo;ll write in Module 8.\u003c/p\u003e","tags":["lld","design-patterns","python"],"title":"Advanced LLD Topics: A Deep Dive"},{"content":"Design Patterns, Part 1: Foundations and Creational Patterns Table of Contents What a Design Pattern Is Why Design Patterns Are Studied Classification of the GoF Catalogue The Template Used for Each Pattern The Problem Space of Creational Patterns Singleton Factory Method Abstract Factory Builder Prototype Comparative Summary of the Five Patterns Key Takeaways Practice Exercises with Solutions Self-Assessment Questions 1. What a Design Pattern Is 1.1 Definition Design pattern. A named, general, reusable solution to a problem that recurs within a particular context of software design. A pattern is a description of communicating objects and classes that are customised to solve a general design problem in that context.\nThe definition has four load-bearing components, and each of them is part of the formal definition of a pattern rather than an optional extra:\nComponent Meaning Consequence if missing Name A handle that identifies the design vocabulary item The solution cannot be discussed, taught, or referenced concisely Problem The situation, including its preconditions, in which the pattern applies The pattern gets applied where it does not belong Solution The arrangement of classes and objects, their responsibilities and collaborations There is nothing to implement Consequences The results and trade-offs of applying the pattern The engineer cannot evaluate whether the cost is justified The original formulation of this idea comes from the architect Christopher Alexander, who wrote that each pattern describes a problem that occurs repeatedly in the environment, and then describes the core of the solution to that problem in such a way that the solution can be used many times over without ever being implemented the same way twice.\n1.2 What a Design Pattern Is Not Three common misconceptions are worth removing before the catalogue is studied, because each one leads to a distinct category of misuse.\nA pattern is not code. A pattern specifies a structure: which roles exist, what each role is responsible for, and how the roles collaborate. The concrete code that realises that structure differs by language, by domain, and by the constraints of the surrounding system. Two correct implementations of Observer may share no identical lines of code.\nA pattern is not a library or a framework. There is no package to install. A library provides an implementation you call; a pattern provides a design decision you make. The distinction matters because a library can be adopted incrementally and removed later, whereas a pattern applied to a system shapes the structure of that system\u0026rsquo;s source code.\nA pattern is not an architecture. Design patterns operate at the level of a few classes collaborating to solve a local design problem. Architectural styles such as layered architecture, microservices, or event-driven architecture operate at the level of entire systems. The GoF catalogue is squarely at the class and object level, which is why it belongs to low-level design rather than high-level design.\n1.3 The Gang of Four Catalogue In 1994, Erich Gamma, Richard Helm, Ralph Johnson, and John Vlissides published Design Patterns: Elements of Reusable Object-Oriented Software. The book documented 23 patterns that the authors had repeatedly observed in well-designed object-oriented systems. The four authors are collectively referred to as the Gang of Four, abbreviated GoF, and the 23 patterns are referred to as the classical patterns.\nThe catalogue is not exhaustive. Many widely used patterns postdate it or fall outside its scope, including Repository, Unit of Work, Dependency Injection, Model-View-Controller, and Publish-Subscribe. The GoF 23 remain the standard baseline because they are the set that practitioners across languages and industries assume as shared knowledge.\n1.4 The Two Structuring Mechanisms Behind Every Pattern Every pattern in the catalogue achieves its flexibility through one of two mechanisms, and identifying which one a pattern uses is the fastest way to understand it.\nInheritance (class-level reuse). A subclass overrides a method of its superclass, and the superclass invokes that method without knowing the subclass. The relationship is fixed at compile time and cannot be changed while the program runs. Factory Method and Template Method use this mechanism.\nComposition and delegation (object-level reuse). An object holds a reference to another object and forwards work to it. The relationship is established at runtime and can be changed while the program runs. Strategy, Abstract Factory, Bridge, and Decorator use this mechanism.\nThe GoF book states a principle that follows directly from this distinction: favour object composition over class inheritance. Composition preserves runtime flexibility and avoids the tight coupling that inheritance creates between a subclass and the internal implementation of its superclass.\n2. Why Design Patterns Are Studied 2.1 Design Vocabulary Communication about design becomes precise and compact when the participants share a vocabulary. The statement \u0026ldquo;the pricing rules should be Strategy objects selected by a registry\u0026rdquo; communicates a class structure, a runtime substitution mechanism, and a set of trade-offs in one sentence. Without the vocabulary, the same statement requires a diagram and several minutes of explanation, and the listener still has to verify that both parties agree on the details.\n2.2 Documented Consequences The value of a catalogued pattern is not only that the structure works, but that its failure modes have been recorded. Singleton is known to complicate testing. Abstract Factory is known to resist the addition of new product types. Visitor is known to break when the object hierarchy changes. An engineer who selects a pattern from the catalogue inherits decades of documented experience about where that structure causes pain, which is information that cannot be derived from the structure itself.\n2.3 Relationship to the SOLID Principles The SOLID principles state properties that a good design possesses. Patterns are concrete structures that produce those properties. The relationship is that of a rule to an instance of the rule.\nPrinciple Statement Patterns that realise it Single Responsibility A class should have one reason to change Builder separates construction from representation; Command separates invocation from execution Open/Closed Software entities should be open for extension, closed for modification Factory Method, Strategy, Decorator, Observer Liskov Substitution Subtypes must be usable wherever their base type is expected Every pattern that relies on an abstract product or abstract handler Interface Segregation Clients should not depend on methods they do not use Adapter, Facade Dependency Inversion High-level modules should depend on abstractions, not on concrete implementations Abstract Factory, Strategy, Bridge Studying patterns converts the principles from statements that are easy to agree with into structures that can be recognised and applied.\n2.4 Comprehension of Existing Systems Widely used Python libraries are built out of these structures. Django\u0026rsquo;s middleware stack is Chain of Responsibility. SQLAlchemy\u0026rsquo;s query construction API is Builder. The io module\u0026rsquo;s buffered and text wrappers around raw streams are Decorator. unittest.TestCase.setUp and tearDown surround runTest in a Template Method. Recognising the pattern in an unfamiliar codebase removes the need to reconstruct the designer\u0026rsquo;s intent from the code alone.\n2.5 The Cost of Misapplication Every pattern adds indirection, and indirection has a measurable cost: more classes to read, more files to open, and a longer path between the point where a behaviour is invoked and the point where it is implemented. A pattern is justified when the flexibility it provides is flexibility the system actually requires.\nThe failure mode has a name. Pattern fever is the application of patterns for their own sake, typically visible as an abstract factory producing a single product family, a strategy interface with one implementation, or a builder for a class with three fields. The corrective question, asked before any pattern is introduced, is: what specific change to the requirements does this structure make cheaper, and is that change plausible?\n3. Classification of the GoF Catalogue The catalogue is organised along two dimensions.\nPurpose describes what the pattern does.\nScope describes whether the pattern applies primarily to classes, in which case relationships are fixed at compile time through inheritance, or primarily to objects, in which case relationships are established at runtime through composition.\nPurpose Definition Count Creational Abstracts the instantiation process, so that a system is independent of how its objects are created, composed, and represented 5 Structural Concerns how classes and objects are composed to form larger structures 7 Behavioural Concerns algorithms and the assignment of responsibilities between objects, and describes the communication patterns between them 11 The complete catalogue:\nCreational Structural Behavioural Singleton Adapter Chain of Responsibility Factory Method Bridge Command Abstract Factory Composite Interpreter Builder Decorator Iterator Prototype Facade Mediator Flyweight Memento Proxy Observer State Strategy Template Method Visitor This document covers the creational family. Structural patterns are covered in Part 2 and behavioural patterns in Part 3.\n4. The Template Used for Each Pattern Each pattern below is presented using the same eight-part template. The order is deliberate: the problem is established before the solution is named, because a pattern learned without its problem will be applied without its problem.\nProblem context. Working code that exhibits the defect the pattern addresses, with the defect stated precisely. Intent. The one-sentence statement of what the pattern accomplishes, in the GoF formulation. Participants. The roles the pattern defines, each with a definition and a responsibility. Reference implementation. Complete Python code with commentary on the mechanism. Variants. Alternative implementations and the conditions under which each is preferred. Applied example. A use of the pattern in a realistic system. Consequences. Benefits and liabilities, stated as trade-offs rather than as advantages. Python-specific considerations. Language features that simplify, replace, or complicate the pattern. 5. The Problem Space of Creational Patterns 5.1 The Defect in Direct Instantiation The default way to obtain an object in Python is a direct constructor call:\nnotifier = EmailNotifier(smtp_host=\u0026#34;smtp.example.com\u0026#34;, port=587) This expression does three things simultaneously. It names a concrete class, it supplies construction arguments, and it produces an instance. The consequence is that the calling code now depends on all three. A change to any one of them requires a change to the caller.\nCoupling. The formal term for this dependency is compile-time coupling to a concrete type. The calling module cannot be compiled, imported, or tested without EmailNotifier being available and constructible. If EmailNotifier opens a network socket in its constructor, every test of the calling code requires a network.\nDuplication of the selection decision. When the class to instantiate depends on runtime data, the selection logic appears at every call site. Ten call sites means ten conditional blocks that must be kept synchronised.\nExposure of construction complexity. If constructing a valid object requires seven arguments in a specific order with cross-field validation rules, every call site must reproduce that knowledge correctly.\n5.2 The Axes of Variation Creational patterns exist because there are several independent decisions embedded in object creation, and each pattern isolates a different one so that it can be changed without disturbing the rest of the system.\nPattern Decision it isolates The question it answers Singleton How many instances may exist How is it guaranteed that exactly one instance exists and is reachable? Factory Method Which concrete class is instantiated How can a class defer the choice of concrete type to its subclasses? Abstract Factory Which family of related concrete classes is instantiated How are groups of objects created so that they are guaranteed to be mutually compatible? Builder The sequence and validation of the construction process How is an object with many optional components assembled without an unreadable constructor? Prototype Whether an object is constructed or copied How is a new object produced from an existing configured instance? 5.3 Terminology Used Throughout These terms recur in every creational pattern and are used with the following precise meanings.\nClient. The code that needs an object and consumes it. The client is the party being protected from the creation details.\nProduct. The object being created. When the product type is abstract, the client depends on the abstraction rather than on any concrete class.\nConcrete Product. A specific implementation of the product interface.\nCreator. The party responsible for producing products. Depending on the pattern this may be a method, a function, a class, or an object.\nInstantiation. The act of allocating and initialising an object. In Python this is a two-step process: __new__ allocates and returns an object, then __init__ initialises the object that __new__ returned. This distinction is directly exploited by Singleton and by Prototype.\n6. Singleton 6.1 Problem Context Certain resources in an application are shared by design, and duplicating them produces incorrect behaviour rather than merely wasteful behaviour. Three representative cases:\nA configuration registry parses a configuration file at startup. If two registries exist, one may be reloaded after a change while the other retains stale values, and two parts of the system will then disagree about the same setting.\nA connection pool manages a bounded set of database connections in order to respect a server-side connection limit. Two pools, each limited to ten connections, will together open twenty, defeating the limit that the pool exists to enforce.\nAn in-memory cache stores computed results for reuse. Two caches halve the hit rate and can hold contradictory entries for the same key.\nThe following code exhibits the defect:\nclass ConnectionPool: def __init__(self, size=10): self.connections = [f\u0026#34;conn-{i}\u0026#34; for i in range(size)] pool_in_service_a = ConnectionPool() pool_in_service_b = ConnectionPool() print(pool_in_service_a is pool_in_service_b) # False print(len(pool_in_service_a.connections) + len(pool_in_service_b.connections)) # 20 Nothing in the class prevents a second instantiation, and nothing at the call site indicates that a second instantiation is a defect. The requirement \u0026ldquo;exactly one instance exists\u0026rdquo; is stated nowhere in the code and is therefore enforced nowhere.\n6.2 Intent Ensure a class has only one instance, and provide a global point of access to it.\nThe intent contains two separate guarantees, and it is useful to note that they are separable. The first is a cardinality constraint on instances. The second is global accessibility. A shared instance created once at startup and passed explicitly to its users satisfies the first guarantee without the second, and this alternative is discussed in Section 6.8.\n6.3 Participants Participant Definition Responsibility Singleton The class whose instantiation is restricted Holds the reference to its own unique instance, controls the creation of that instance, and exposes it to clients Singleton is the only GoF pattern with a single participant, because the pattern constrains one class rather than arranging a collaboration between several.\n6.4 Reference Implementation: Overriding __new__ To restrict instantiation, the restriction must be placed at the point where instantiation occurs. In Python, the expression ClassName() invokes the metaclass call machinery, which performs two steps:\nClassName.__new__(cls, *args, **kwargs) is called. This method allocates and returns an object. If the returned object is an instance of ClassName, then __init__ is called on it with the same arguments. Because __init__ receives an object that already exists, it is too late to prevent a second object from being created there. The interception must occur in __new__.\nclass ConnectionPool: _instance = None # Class attribute: one slot shared by the class itself, # not per instance. This is where the unique instance lives. def __new__(cls, size=10): # __new__ is the allocator. Returning an already existing object from it # means no new object is allocated at all, which is exactly the constraint # the pattern requires. if cls._instance is None: cls._instance = super().__new__(cls) cls._instance._initialised = False return cls._instance def __init__(self, size=10): # __init__ runs on EVERY ConnectionPool() call, because Python calls it on # whatever __new__ returned. Without the guard below, the second call would # re-run initialisation and discard the state accumulated by the first. if self._initialised: return self.connections = [f\u0026#34;conn-{i}\u0026#34; for i in range(size)] self._initialised = True def acquire(self): return self.connections.pop() def release(self, conn): self.connections.append(conn) a = ConnectionPool(size=10) b = ConnectionPool(size=99) # size is ignored: the instance already exists print(a is b) # True print(len(a.connections)) # 10 Two properties of this implementation deserve attention.\nInitialisation runs exactly once. The _initialised flag is required because Python invokes __init__ on every call regardless of whether __new__ allocated anything. Omitting the flag produces a subtle defect: the pool appears to be a singleton by identity, yet its state resets on every call.\nLater arguments are silently discarded. The second call passes size=99 and receives a pool of size 10. This is inherent to the pattern rather than a flaw in this implementation, and it is a reason to prefer an explicit configuration step over constructor arguments on a singleton class.\n6.5 Variant: The Decorator Implementation The __new__ approach mixes the singleton constraint into the class it constrains, which violates the single responsibility principle and must be repeated in every class that needs the constraint. A decorator extracts the constraint into a reusable component.\nimport functools def singleton(cls): instances = {} # Closure variable. It survives for the lifetime of the # decorated class because get_instance references it, and # it is unreachable from outside, which makes it a private # per-class cache. @functools.wraps(cls) def get_instance(*args, **kwargs): if cls not in instances: instances[cls] = cls(*args, **kwargs) return instances[cls] return get_instance @singleton class Cache: def __init__(self): self.store = {} def put(self, key, value): self.store[key] = value def get(self, key): return self.store.get(key) a = Cache() a.put(\u0026#34;user:1\u0026#34;, {\u0026#34;name\u0026#34;: \u0026#34;Amit\u0026#34;}) b = Cache() print(a is b) # True print(b.get(\u0026#34;user:1\u0026#34;)) # {\u0026#39;name\u0026#39;: \u0026#39;Amit\u0026#39;} The mechanism is name rebinding. After decoration, the name Cache refers to the function get_instance, not to the class. Calling Cache() therefore calls the function, which consults the cache dictionary and constructs the real class only on the first call.\nThis variant has one significant limitation: because Cache is now a function, it cannot be subclassed and isinstance(x, Cache) raises a TypeError. Where either capability is required, the metaclass variant below is preferable.\n6.6 Variant: The Metaclass Implementation A metaclass is the class of a class. Just as an object\u0026rsquo;s behaviour on obj() is determined by its class\u0026rsquo;s __call__, a class\u0026rsquo;s behaviour on ClassName() is determined by its metaclass\u0026rsquo;s __call__. Overriding __call__ in a metaclass intercepts instantiation one level above __new__ and __init__ together, which removes the need for the _initialised flag.\nclass SingletonMeta(type): _instances = {} def __call__(cls, *args, **kwargs): # cls here is the class being instantiated, for example Logger. # super().__call__ is what normally runs __new__ followed by __init__. # Skipping it entirely on later calls means __init__ never re-runs. if cls not in cls._instances: cls._instances[cls] = super().__call__(*args, **kwargs) return cls._instances[cls] class Logger(metaclass=SingletonMeta): def __init__(self, log_file=\u0026#34;app.log\u0026#34;): self.log_file = log_file self.lines_written = 0 def log(self, message): self.lines_written += 1 print(f\u0026#34;[{self.log_file}] {message}\u0026#34;) a = Logger(\u0026#34;service.log\u0026#34;) a.log(\u0026#34;started\u0026#34;) b = Logger() print(a is b) # True print(b.lines_written) # 1, state preserved The metaclass variant preserves normal class semantics: Logger remains a class, subclassing works, and isinstance works. Each subclass receives its own entry in _instances, so a subclass is a separate singleton rather than a competitor for the same slot.\n6.7 Variant: Module-Level State Python modules are themselves singletons. The first import of a module executes it and stores the resulting module object in sys.modules; every later import of the same name returns the cached object without re-executing the file. A module therefore provides single-instance semantics with no additional machinery.\n# logger.py _log_file = \u0026#34;app.log\u0026#34; _lines_written = 0 def log(message): global _lines_written _lines_written += 1 print(f\u0026#34;[{_log_file}] {message}\u0026#34;) def line_count(): return _lines_written # any_other_module.py import logger logger.log(\u0026#34;started\u0026#34;) print(logger.line_count()) # 1, shared across every importer This is the idiomatic Python solution when the shared resource is a set of functions over shared state and does not require inheritance, multiple configurations, or lazy construction. The standard library uses it: random exposes module-level functions backed by a single hidden Random instance.\n6.8 The Testing Problem and Dependency Injection The liability that dominates discussion of this pattern is its effect on testing, and the mechanism is worth stating precisely.\nA singleton is global mutable state. Two consequences follow. First, tests are no longer independent: a test that writes to the singleton changes the starting conditions of every test that runs afterwards, which makes results depend on test execution order. Second, the dependency is hidden: a class that calls Logger() inside a method does not declare in its constructor that it requires a logger, so the dependency is invisible in the class\u0026rsquo;s interface and cannot be substituted by a test double without patching a global name.\nDependency injection is the standard alternative. The instance is created once, at the application\u0026rsquo;s composition root, and is passed explicitly to the objects that require it.\nclass OrderService: def __init__(self, logger, pool): # The dependencies are declared. They can be substituted in tests # by passing different objects, with no patching and no global state. self._logger = logger self._pool = pool def place(self, order): conn = self._pool.acquire() try: self._logger.log(f\u0026#34;placing order {order}\u0026#34;) finally: self._pool.release(conn) # Composition root: the single place that knows about concrete implementations. def main(): logger = Logger(\u0026#34;prod.log\u0026#34;) pool = ConnectionPool(size=10) service = OrderService(logger, pool) service.place(\u0026#34;order-1\u0026#34;) The cardinality guarantee is preserved because main creates exactly one of each. The global access point is removed, and with it the testing problem. Singleton remains appropriate when a shared instance must be reachable from code that cannot be given a constructor parameter, such as deep inside a third-party callback, and when the alternative would be threading a reference through many unrelated layers.\n6.9 Thread Safety The implementations above are not safe under concurrency. Consider two threads executing __new__ at the same time:\nThread A evaluates cls._instance is None and finds it true. The interpreter switches to thread B before A assigns. Thread B evaluates the same condition, also finds it true, and constructs an instance. Thread A resumes and constructs a second instance, overwriting B\u0026rsquo;s. Two instances now exist, and objects created before the overwrite hold a reference to the discarded one. The window is small, which makes the defect intermittent and difficult to reproduce.\nThe correction is to hold a lock across the check and the assignment, using the double-checked locking idiom so that the lock is acquired only on the uncontended first call:\nimport threading class SafeSingletonMeta(type): _instances = {} _lock = threading.Lock() def __call__(cls, *args, **kwargs): if cls not in cls._instances: # fast path, no lock with cls._lock: if cls not in cls._instances: # re-check, now under the lock cls._instances[cls] = super().__call__(*args, **kwargs) return cls._instances[cls] The second check inside the lock is not redundant. A thread that blocked on the lock while another thread was constructing the instance must observe the result of that construction rather than construct again.\n6.10 Consequences Benefit Explanation Enforced cardinality The constraint is expressed in the class rather than in a convention that call sites may violate Lazy initialisation The instance is constructed on first use, so an expensive resource is not created if it is never needed Single point of access Code in any module can reach the instance without a reference being passed to it Liability Explanation Global mutable state Test isolation is lost, and state changes propagate to unrelated code with no visible dependency Hidden dependencies A class\u0026rsquo;s interface does not disclose what it depends on, which violates the dependency inversion principle Concurrency defects Naive implementations admit a race on first construction Subclassing ambiguity Whether a subclass shares or replaces the parent\u0026rsquo;s instance must be decided explicitly, and different implementations answer it differently Lifetime is the process The instance lives until the process exits, which prevents deterministic release of the resources it holds 6.11 Related Patterns Monostate, also called Borg, achieves the observable effect of Singleton without constraining the number of instances. Every instance shares the same attribute dictionary, so instances differ in identity but not in state.\nclass Monostate: _shared_state = {} def __init__(self): self.__dict__ = self._shared_state # all instances share one namespace a = Monostate() a.region = \u0026#34;ap-south-1\u0026#34; b = Monostate() print(b.region) # ap-south-1 print(a is b) # False Monostate is preferable when subclasses should be able to add their own shared state and when client code should not need to know that sharing is occurring.\nAbstract Factory, Builder, and Prototype are frequently implemented as singletons, because a system normally requires only one instance of each.\nFacade objects are often singletons for the same reason.\n6.12 Python-Specific Considerations functools.lru_cache applied to a zero-argument factory function produces single-instance semantics with less code than any of the class-based variants, and it is thread-safe:\nfrom functools import lru_cache @lru_cache(maxsize=None) def get_pool() -\u0026gt; ConnectionPool: return ConnectionPool(size=10) The ordering of preference in Python is: a module-level object or function set where inheritance is not needed; lru_cache on a factory function where laziness is needed; the metaclass implementation where a genuine class with singleton semantics is needed; and the __new__ implementation only where a metaclass is unavailable because of a conflict with another metaclass.\n7. Factory Method 7.1 Problem Context A notification subsystem supports several delivery channels. Each channel is a separate class implementing a common operation.\nclass EmailNotifier: def send(self, message): print(f\u0026#34;Email: {message}\u0026#34;) class SMSNotifier: def send(self, message): print(f\u0026#34;SMS: {message}\u0026#34;) class PushNotifier: def send(self, message): print(f\u0026#34;Push: {message}\u0026#34;) def notify_user(user, message): if user.channel == \u0026#34;email\u0026#34;: notifier = EmailNotifier() elif user.channel == \u0026#34;sms\u0026#34;: notifier = SMSNotifier() elif user.channel == \u0026#34;push\u0026#34;: notifier = PushNotifier() else: raise ValueError(f\u0026#34;Unsupported channel: {user.channel}\u0026#34;) notifier.send(message) The function performs two distinct jobs: it selects a concrete class, and it uses the resulting object. Three defects follow.\nThe selection logic is duplicated. Any other function needing a notifier reproduces the same conditional chain. Adding a Slack channel requires locating and editing every copy.\nThe open/closed principle is violated. Extending the system with a new channel requires modifying existing, tested code rather than adding new code.\nThe module depends on every concrete class. Importing this module imports all four notifier classes and everything they import, including their transport libraries.\n7.2 Intent Define an interface for creating an object, but let subclasses decide which class to instantiate. Factory Method lets a class defer instantiation to subclasses.\nThe essential separation is between the decision that an object of some abstract type is needed, which stays in the general algorithm, and the decision about which concrete type to construct, which moves into a substitutable location.\n7.3 Participants Participant Definition Responsibility Product The interface of the objects the factory method creates Declares the operations clients invoke Concrete Product An implementation of the Product interface Provides channel-specific or type-specific behaviour Creator The class that declares the factory method and contains the algorithm that uses its result Declares the factory method, usually abstract, and implements operations that call it Concrete Creator A subclass of Creator Overrides the factory method to return a specific Concrete Product The defining structural feature is that the Creator calls its own factory method without knowing which Concrete Product it will receive. The base class therefore contains a complete algorithm with one substitutable step.\n7.4 Reference Implementation from abc import ABC, abstractmethod # ---------- Product ---------- class Notifier(ABC): @abstractmethod def send(self, message: str) -\u0026gt; None: ... # ---------- Concrete Products ---------- class EmailNotifier(Notifier): def send(self, message): print(f\u0026#34;Email: {message}\u0026#34;) class SMSNotifier(Notifier): def send(self, message): print(f\u0026#34;SMS: {message}\u0026#34;) # ---------- Creator ---------- class NotificationService(ABC): @abstractmethod def create_notifier(self) -\u0026gt; Notifier: \u0026#34;\u0026#34;\u0026#34;The factory method. The base class declares that a Notifier will be needed but refuses to decide which one, delegating that decision to subclasses.\u0026#34;\u0026#34;\u0026#34; def notify(self, user, message) -\u0026gt; None: \u0026#34;\u0026#34;\u0026#34;The template algorithm. It is written once, in terms of the abstract Product, and is inherited unchanged by every subclass. Retry policy, auditing, and formatting live here and are never duplicated.\u0026#34;\u0026#34;\u0026#34; notifier = self.create_notifier() formatted = f\u0026#34;Dear {user}, {message}\u0026#34; notifier.send(formatted) # ---------- Concrete Creators ---------- class EmailService(NotificationService): def create_notifier(self) -\u0026gt; Notifier: return EmailNotifier() class SMSService(NotificationService): def create_notifier(self) -\u0026gt; Notifier: return SMSNotifier() service: NotificationService = EmailService() service.notify(\u0026#34;Amit\u0026#34;, \u0026#34;your order has shipped\u0026#34;) # Email: Dear Amit, your order has shipped The value of the structure is visible in notify. Any behaviour that must apply to every channel is written once. Adding a Slack channel requires two new classes and no modification to NotificationService or to any client that holds a NotificationService reference.\n7.5 Variant: Parameterised Factory Where the variation is a data value rather than a subclass of a larger algorithm, a single function that maps an identifier to a class is sufficient. This variant is often called Simple Factory and is not, strictly, the GoF pattern, because no subclassing is involved. It solves the same problem and is far more common in Python.\n_REGISTRY: dict[str, type[Notifier]] = { \u0026#34;email\u0026#34;: EmailNotifier, \u0026#34;sms\u0026#34;: SMSNotifier, } def create_notifier(channel: str) -\u0026gt; Notifier: try: cls = _REGISTRY[channel] except KeyError: raise ValueError(f\u0026#34;Unsupported channel: {channel}\u0026#34;) from None return cls() create_notifier(\u0026#34;sms\u0026#34;).send(\u0026#34;hello\u0026#34;) Classes are first-class objects in Python, so the dictionary stores the class itself and calls it. The conditional chain is replaced by a lookup table, which is a data structure that can be extended at runtime.\n7.6 Variant: Self-Registering Subclasses The registry above must still be edited when a product is added. __init_subclass__ is a hook that Python calls automatically whenever a subclass of a class is defined, which allows each product to register itself.\nclass Notifier(ABC): registry: dict[str, type[\u0026#34;Notifier\u0026#34;]] = {} def __init_subclass__(cls, channel: str = None, **kwargs): # Called at class-definition time for every subclass. The `channel` # keyword comes from the class statement\u0026#39;s argument list. super().__init_subclass__(**kwargs) if channel: Notifier.registry[channel] = cls @abstractmethod def send(self, message: str) -\u0026gt; None: ... class EmailNotifier(Notifier, channel=\u0026#34;email\u0026#34;): def send(self, message): print(f\u0026#34;Email: {message}\u0026#34;) class SlackNotifier(Notifier, channel=\u0026#34;slack\u0026#34;): def send(self, message): print(f\u0026#34;Slack: {message}\u0026#34;) def create_notifier(channel: str) -\u0026gt; Notifier: return Notifier.registry[channel]() print(sorted(Notifier.registry)) # [\u0026#39;email\u0026#39;, \u0026#39;slack\u0026#39;] create_notifier(\u0026#34;slack\u0026#34;).send(\u0026#34;deploy finished\u0026#34;) Adding a channel now requires exactly one new class in one new file, with no edit to any existing file. This is the strongest realisation of the open/closed principle among the variants, and it is the structure used by plugin systems.\n7.7 Applied Example: Document Parsing from abc import ABC, abstractmethod class Parser(ABC): @abstractmethod def parse(self, raw: bytes) -\u0026gt; str: ... class PDFParser(Parser): def parse(self, raw): return f\u0026#34;PDF text extracted from {len(raw)} bytes\u0026#34; class DocxParser(Parser): def parse(self, raw): return f\u0026#34;DOCX text extracted from {len(raw)} bytes\u0026#34; class CSVParser(Parser): def parse(self, raw): return f\u0026#34;CSV rows extracted from {len(raw)} bytes\u0026#34; _PARSERS = {\u0026#34;pdf\u0026#34;: PDFParser, \u0026#34;docx\u0026#34;: DocxParser, \u0026#34;csv\u0026#34;: CSVParser} def parser_for(extension: str) -\u0026gt; Parser: if extension not in _PARSERS: raise ValueError(f\u0026#34;No parser registered for \u0026#39;{extension}\u0026#39;\u0026#34;) return _PARSERS[extension]() def ingest(filename: str, raw: bytes) -\u0026gt; str: extension = filename.rsplit(\u0026#34;.\u0026#34;, 1)[-1].lower() return parser_for(extension).parse(raw) # client knows only Parser print(ingest(\u0026#34;report.pdf\u0026#34;, b\u0026#34;x\u0026#34; * 1024)) The ingest function contains the ingestion workflow and no knowledge of parser classes. A new format is supported by adding a class and one registry entry.\n7.8 Consequences Benefit Explanation Client decoupled from concrete classes The client depends only on the Product interface, satisfying the dependency inversion principle Extension without modification New products are added by writing new code, satisfying the open/closed principle Creation logic localised Argument defaults, validation, and configuration lookups for construction live in one place Substitutability in tests A test can supply a factory that returns doubles, without patching Liability Explanation Additional indirection Reading the code requires following the factory to learn which class is actually used Class proliferation in the subclassing variant Each product may require a corresponding creator subclass Registry drift With a mapping-based registry, an unregistered product fails at runtime rather than at import time 7.9 Related Patterns Abstract Factory is a set of factory methods on one object, each producing a member of a related family. Factory Method produces one product; Abstract Factory produces several that must be mutually compatible.\nTemplate Method is the same inheritance mechanism applied to behaviour rather than creation. A factory method is frequently one step inside a template method.\nPrototype is an alternative that does not require a Creator subclass hierarchy: instead of a subclass deciding which class to instantiate, an object is cloned.\n7.10 Python-Specific Considerations Because classes are first-class objects and functions can be passed as arguments, the full GoF Creator hierarchy is rarely necessary in Python. A dictionary of callables, a function parameter with a class as its default, or __init_subclass__ registration achieves the same decoupling with fewer classes.\nThe Creator subclass hierarchy earns its cost only when the Creator contains substantial shared behaviour, as NotificationService.notify did in Section 7.4. If the Creator contains nothing but the factory method, the hierarchy is ceremony and a function is preferable.\n8. Abstract Factory 8.1 Problem Context A reporting subsystem must support two storage backends, PostgreSQL and MySQL. Each backend requires three collaborating objects: a connection, a dialect-specific query builder, and a transaction manager. The objects are not independent, because a query built in PostgreSQL dialect cannot execute on a MySQL connection.\nApplying Factory Method separately to each object type produces the following:\nconnection = connection_factory(\u0026#34;postgres\u0026#34;) query_builder = query_builder_factory(\u0026#34;mysql\u0026#34;) # accepted, and wrong transaction = transaction_factory(\u0026#34;postgres\u0026#34;) Nothing detects the inconsistency. The system compiles, imports, starts, and fails at runtime with a syntax error from the database when a MySQL-dialect query reaches a PostgreSQL server. The defect is that the constraint \u0026ldquo;these three objects must come from the same family\u0026rdquo; is not expressed anywhere in the type system or in the construction code.\n8.2 Intent Provide an interface for creating families of related or dependent objects without specifying their concrete classes.\nThe word family is the operative term. A family is a set of Concrete Products that are designed to work together and that must not be mixed with members of another family.\n8.3 Participants Participant Definition Responsibility Abstract Factory An interface declaring one creation operation per abstract product Defines what the family consists of Concrete Factory An implementation of Abstract Factory Implements every creation operation to return products of one specific family Abstract Product The interface for one kind of product Declares operations clients invoke on that kind Concrete Product An implementation of an Abstract Product belonging to one family Provides family-specific behaviour Client Code parameterised by an Abstract Factory Creates and uses products exclusively through the abstract interfaces The consistency guarantee arises from a structural property: a Concrete Factory implements every creation operation, and it implements all of them for the same family. Since the client obtains all products from one factory object, mixing families becomes impossible.\n8.4 Reference Implementation from abc import ABC, abstractmethod # ---------- Abstract Products ---------- class Connection(ABC): @abstractmethod def execute(self, sql: str) -\u0026gt; None: ... class QueryBuilder(ABC): @abstractmethod def limit_clause(self, n: int) -\u0026gt; str: ... # ---------- Concrete Products: PostgreSQL family ---------- class PostgresConnection(Connection): def execute(self, sql): print(f\u0026#34;[postgres] {sql}\u0026#34;) class PostgresQueryBuilder(QueryBuilder): def limit_clause(self, n): return f\u0026#34;LIMIT {n}\u0026#34; # ---------- Concrete Products: MySQL family ---------- class MySQLConnection(Connection): def execute(self, sql): print(f\u0026#34;[mysql] {sql}\u0026#34;) class MySQLQueryBuilder(QueryBuilder): def limit_clause(self, n): return f\u0026#34;LIMIT 0, {n}\u0026#34; # ---------- Abstract Factory ---------- class StorageFactory(ABC): \u0026#34;\u0026#34;\u0026#34;One creation operation per abstract product. The set of operations defines the composition of the family.\u0026#34;\u0026#34;\u0026#34; @abstractmethod def create_connection(self) -\u0026gt; Connection: ... @abstractmethod def create_query_builder(self) -\u0026gt; QueryBuilder: ... # ---------- Concrete Factories ---------- class PostgresFactory(StorageFactory): def create_connection(self) -\u0026gt; Connection: return PostgresConnection() def create_query_builder(self) -\u0026gt; QueryBuilder: return PostgresQueryBuilder() class MySQLFactory(StorageFactory): def create_connection(self) -\u0026gt; Connection: return MySQLConnection() def create_query_builder(self) -\u0026gt; QueryBuilder: return MySQLQueryBuilder() # ---------- Client ---------- def fetch_recent_orders(factory: StorageFactory, n: int) -\u0026gt; None: \u0026#34;\u0026#34;\u0026#34;Parameterised by the factory. It names no concrete class, and it cannot construct an inconsistent pair because both products come from one factory.\u0026#34;\u0026#34;\u0026#34; connection = factory.create_connection() builder = factory.create_query_builder() connection.execute(f\u0026#34;SELECT * FROM orders ORDER BY id DESC {builder.limit_clause(n)}\u0026#34;) fetch_recent_orders(PostgresFactory(), 10) # [postgres] SELECT * FROM orders ORDER BY id DESC LIMIT 10 fetch_recent_orders(MySQLFactory(), 10) # [mysql] SELECT * FROM orders ORDER BY id DESC LIMIT 0, 10 Switching the entire system to a different backend requires changing one expression, the argument passed to fetch_recent_orders. In a real application that argument is supplied once, at the composition root, from configuration.\n8.5 The Two Dimensions of Extension Abstract Factory\u0026rsquo;s benefits and its principal liability both derive from the same structural fact: the design has two dimensions of variation, and it treats them very differently.\nDimension What changes Cost of extension Adding a family A new backend, for example SQLite One new Concrete Factory plus one Concrete Product per product kind. No existing file is modified. Adding a product kind A new member of every family, for example TransactionManager The Abstract Factory interface changes, which forces every existing Concrete Factory to implement the new operation. Every family must be modified. The design is open for extension along the family dimension and closed along the product dimension. This asymmetry is the standard reason Abstract Factory is rejected in systems where the set of product kinds is still unstable, and it is the correct choice when the set of product kinds is fixed and the set of families is expected to grow.\n8.6 Variant: Factory as a Dictionary of Callables Where the products are simple and no shared factory behaviour exists, the family can be expressed as a mapping instead of a class hierarchy. Type safety is weaker, and the volume of code is much smaller.\nPOSTGRES = {\u0026#34;connection\u0026#34;: PostgresConnection, \u0026#34;query_builder\u0026#34;: PostgresQueryBuilder} MYSQL = {\u0026#34;connection\u0026#34;: MySQLConnection, \u0026#34;query_builder\u0026#34;: MySQLQueryBuilder} def fetch_recent_orders(family: dict, n: int) -\u0026gt; None: connection = family[\u0026#34;connection\u0026#34;]() builder = family[\u0026#34;query_builder\u0026#34;]() connection.execute(f\u0026#34;SELECT * FROM orders {builder.limit_clause(n)}\u0026#34;) The consistency guarantee still holds, because all products are drawn from one dictionary. What is lost is the compile-time check that a family is complete: a missing key surfaces as a KeyError at runtime rather than as an instantiation error at import time, which is what an unimplemented abstract method would produce.\n8.7 Applied Examples Cross-platform user interface toolkits. A MacFactory produces MacButton, MacCheckbox, and MacMenu; a WindowsFactory produces the Windows equivalents. This is the example used in the original GoF text.\nTheming. A light-theme factory and a dark-theme factory each produce a matched set of colours, fonts, and icon sets, which prevents a dark-theme icon appearing on a light-theme surface.\nMulti-cloud infrastructure clients. An AWS factory produces an object-storage client, a queue client, and a secrets client that all use one credential provider and one region; a GCP factory produces the equivalents.\nTest doubles. An in-memory factory produces fake implementations of an entire family, allowing a full system test with no external infrastructure.\n8.8 Factory Method Compared with Abstract Factory Criterion Factory Method Abstract Factory Number of products created One Several, forming a family Primary mechanism Inheritance, a subclass overrides a method Composition, the client holds a factory object Unit of substitution A Creator subclass A whole Concrete Factory object Time of binding Fixed when the Creator subclass is chosen Changeable at runtime by passing a different factory Guarantee provided The correct concrete type is produced The products produced are mutually compatible Typical size One abstract method One abstract method per product kind Abstract Factory is commonly implemented using Factory Method for each of its creation operations, so the patterns compose rather than compete.\n8.9 Consequences Benefit Explanation Enforced family consistency Incompatible products cannot be combined, because they cannot be obtained from the same factory Concrete classes isolated Concrete product names appear only inside Concrete Factories Families are interchangeable at runtime The factory is an ordinary object and can be replaced by assignment Supports the dependency inversion principle Clients depend only on abstract products and one abstract factory Liability Explanation Adding a product kind is expensive Every Concrete Factory must be modified, which violates the open/closed principle along that dimension Class count grows multiplicatively Product kinds multiplied by families, plus the abstract interfaces Indirection cost Two levels of abstraction separate the client from the object it actually uses Premature use is common With one family, the entire structure is overhead with no benefit 8.10 Related Patterns Factory Method implements the individual creation operations of a Concrete Factory.\nPrototype offers an alternative implementation: a Concrete Factory can hold a configured prototypical instance of each product and return clones instead of constructing new objects.\nSingleton is frequently applied to Concrete Factories, since one instance per family is sufficient.\nBuilder is used when an individual product within the family is itself complex to construct.\n9. Builder 9.1 Problem Context An HTTP request object has one required component, the URL, and several optional ones: method, headers, query parameters, body, timeout, retry count, and TLS verification. Expressing this through a constructor produces the following call site:\nrequest = HttpRequest(\u0026#34;https://api.example.com/users\u0026#34;, \u0026#34;POST\u0026#34;, {\u0026#34;Authorization\u0026#34;: \u0026#34;Bearer xyz\u0026#34;}, {\u0026#34;active\u0026#34;: \u0026#34;true\u0026#34;}, \u0026#39;{\u0026#34;name\u0026#34;: \u0026#34;Amit\u0026#34;}\u0026#39;, 60, 3, True) Three defects are present.\nThe telescoping constructor problem. A telescoping constructor is a constructor with a long positional parameter list, typically accompanied by a series of overloads or defaults that allow trailing parameters to be omitted. The call site is unreadable because the meaning of each argument depends on its position, and transposing two arguments of the same type is a defect the language cannot detect.\nConstruction is not atomic with respect to validation. Some rules span several fields: a body is meaningful only with methods that accept one, and a retry count is meaningful only with a positive timeout. Checking these rules inside the constructor is possible but places validation logic in the same place as field assignment, and it cannot express rules that depend on a component supplied later.\nThe object cannot be assembled incrementally. When components are gathered across several steps, for instance headers added by different middleware layers, a single constructor call is the wrong shape for the code that produces the arguments.\nKeyword arguments remove the first defect but not the second or third:\nrequest = HttpRequest(url=\u0026#34;https://api.example.com/users\u0026#34;, method=\u0026#34;POST\u0026#34;, headers={\u0026#34;Authorization\u0026#34;: \u0026#34;Bearer xyz\u0026#34;}, timeout=60) 9.2 Intent Separate the construction of a complex object from its representation, so that the same construction process can create different representations.\nThe intent has two halves that are worth separating, because in practice most uses exercise only the first. The first half, separating construction from representation, is the common case: a dedicated object accumulates components and produces the finished product. The second half, one construction process producing different representations, is the case where the same sequence of steps is directed at different Builder implementations, for example to produce an HTML document and a PDF document from one traversal of the same source.\n9.3 Participants Participant Definition Responsibility Builder The interface declaring one operation per component of the product Defines the construction steps available Concrete Builder An implementation of Builder Accumulates the components, tracks construction state, validates, and provides an operation to retrieve the finished product Product The complex object under construction Holds the assembled components; often has no public construction logic of its own Director An object that executes a fixed sequence of Builder operations Encodes a reusable recipe, so that a common configuration is not re-specified at every call site The Director is optional and is frequently omitted. It is worth introducing when the same sequence of construction steps is required in several places.\n9.4 Reference Implementation class HttpRequest: \u0026#34;\u0026#34;\u0026#34;The Product. It is deliberately passive: it holds components and does not know how they were gathered.\u0026#34;\u0026#34;\u0026#34; def __init__(self, url, method, headers, query, body, timeout): self.url = url self.method = method self.headers = headers self.query = query self.body = body self.timeout = timeout def __repr__(self): return (f\u0026#34;HttpRequest({self.method} {self.url} headers={self.headers} \u0026#34; f\u0026#34;query={self.query} body={self.body} timeout={self.timeout})\u0026#34;) class HttpRequestBuilder: \u0026#34;\u0026#34;\u0026#34;The Concrete Builder. It holds partial state, which is what makes incremental assembly and deferred validation possible.\u0026#34;\u0026#34;\u0026#34; def __init__(self): self._url = None self._method = \u0026#34;GET\u0026#34; self._headers = {} self._query = {} self._body = None self._timeout = 30 # Each configuration operation returns self. This is what permits chained # calls: the value of one call is the receiver of the next. Without the # return, each call evaluates to None and the second attribute access fails. def url(self, value: str) -\u0026gt; \u0026#34;HttpRequestBuilder\u0026#34;: self._url = value return self def method(self, value: str) -\u0026gt; \u0026#34;HttpRequestBuilder\u0026#34;: self._method = value.upper() return self def header(self, key: str, value: str) -\u0026gt; \u0026#34;HttpRequestBuilder\u0026#34;: self._headers[key] = value # additive: callable repeatedly return self def query_param(self, key: str, value: str) -\u0026gt; \u0026#34;HttpRequestBuilder\u0026#34;: self._query[key] = value return self def body(self, value: str) -\u0026gt; \u0026#34;HttpRequestBuilder\u0026#34;: self._body = value return self def timeout(self, seconds: int) -\u0026gt; \u0026#34;HttpRequestBuilder\u0026#34;: self._timeout = seconds return self def build(self) -\u0026gt; HttpRequest: \u0026#34;\u0026#34;\u0026#34;The single point at which the accumulated state is validated as a whole and converted into a Product. Cross-field rules that no individual setter could check are enforced here.\u0026#34;\u0026#34;\u0026#34; if self._url is None: raise ValueError(\u0026#34;url is required\u0026#34;) if self._body is not None and self._method in (\u0026#34;GET\u0026#34;, \u0026#34;HEAD\u0026#34;): raise ValueError(f\u0026#34;{self._method} requests must not carry a body\u0026#34;) if self._timeout \u0026lt;= 0: raise ValueError(\u0026#34;timeout must be positive\u0026#34;) return HttpRequest(self._url, self._method, dict(self._headers), dict(self._query), self._body, self._timeout) request = (HttpRequestBuilder() .method(\u0026#34;post\u0026#34;) .url(\u0026#34;https://api.example.com/users\u0026#34;) .header(\u0026#34;Authorization\u0026#34;, \u0026#34;Bearer xyz\u0026#34;) .header(\u0026#34;Content-Type\u0026#34;, \u0026#34;application/json\u0026#34;) .query_param(\u0026#34;active\u0026#34;, \u0026#34;true\u0026#34;) .body(\u0026#39;{\u0026#34;name\u0026#34;: \u0026#34;Amit\u0026#34;}\u0026#39;) .timeout(60) .build()) print(request) Three design decisions in this implementation are worth stating explicitly.\nThe setters return self. This produces a fluent interface, defined as an API in which method calls are chained so that the code reads as a sequence of configuration statements. The mechanism is nothing more than returning the receiver.\nValidation is concentrated in build. The setters accept values without judging them, because a value that is invalid in combination with a component not yet supplied cannot be judged in isolation. The rule that a GET request must not carry a body is checkable only when both the method and the body are known.\nThe product receives copies of the mutable components. dict(self._headers) prevents the builder from retaining a reference into the finished product. Without the copy, reusing the builder to construct a second request would mutate the headers of the first.\n9.5 Variant: The Director class RequestDirector: \u0026#34;\u0026#34;\u0026#34;Holds reusable construction recipes. The Director knows the sequence of steps; the Builder knows how to perform each step.\u0026#34;\u0026#34;\u0026#34; @staticmethod def json_post(builder: HttpRequestBuilder, url: str, payload: str) -\u0026gt; HttpRequest: return (builder .method(\u0026#34;POST\u0026#34;) .url(url) .header(\u0026#34;Content-Type\u0026#34;, \u0026#34;application/json\u0026#34;) .header(\u0026#34;Accept\u0026#34;, \u0026#34;application/json\u0026#34;) .body(payload) .timeout(60) .build()) @staticmethod def health_check(builder: HttpRequestBuilder, url: str) -\u0026gt; HttpRequest: return builder.method(\u0026#34;GET\u0026#34;).url(url).timeout(5).build() print(RequestDirector.json_post(HttpRequestBuilder(), \u0026#34;https://api.example.com/users\u0026#34;, \u0026#39;{\u0026#34;name\u0026#34;: \u0026#34;Amit\u0026#34;}\u0026#39;)) print(RequestDirector.health_check(HttpRequestBuilder(), \u0026#34;https://api.example.com/health\u0026#34;)) The separation matters when several Builder implementations exist. The same Director recipe applied to a CurlCommandBuilder instead of an HttpRequestBuilder would emit a shell command rather than a request object, which is the \u0026ldquo;same construction process, different representations\u0026rdquo; half of the intent.\n9.6 Variant: Immutable Product Where the product must be immutable after construction, the builder is the natural place to hold the mutable partial state. The product is then created in one step at the end.\nfrom dataclasses import dataclass, field from types import MappingProxyType @dataclass(frozen=True) class ImmutableRequest: url: str method: str = \u0026#34;GET\u0026#34; headers: MappingProxyType = field(default_factory=lambda: MappingProxyType({})) timeout: int = 30 class ImmutableRequestBuilder: def __init__(self): self._headers = {} self._url = None self._method = \u0026#34;GET\u0026#34; self._timeout = 30 def url(self, v): self._url = v; return self def method(self, v): self._method = v.upper(); return self def header(self, k, v): self._headers[k] = v; return self def timeout(self, v): self._timeout = v; return self def build(self) -\u0026gt; ImmutableRequest: if self._url is None: raise ValueError(\u0026#34;url is required\u0026#34;) return ImmutableRequest(self._url, self._method, MappingProxyType(dict(self._headers)), self._timeout) req = ImmutableRequestBuilder().url(\u0026#34;https://x.dev\u0026#34;).header(\u0026#34;A\u0026#34;, \u0026#34;1\u0026#34;).build() # req.url = \u0026#34;other\u0026#34; raises FrozenInstanceError frozen=True blocks attribute assignment on the dataclass, and MappingProxyType provides a read-only view over the header dictionary, which prevents mutation through the attribute.\n9.7 Applied Examples Query construction. SQLAlchemy\u0026rsquo;s select(...).where(...).order_by(...).limit(...) accumulates clauses and compiles them into SQL only when the statement is executed. Each call returns a new object, which is an immutable variant of the pattern.\nTest data construction. A UserBuilder with sensible defaults allows a test to specify only the field under test: UserBuilder().with_status(\u0026quot;suspended\u0026quot;).build(). This keeps tests readable and insulates them from changes to unrelated fields.\nConfiguration objects. Application configuration assembled from defaults, then a file, then environment variables, then command-line flags, is an incremental assembly problem with a validation step at the end.\nDocument generation. A traversal of a document tree calls add_heading, add_paragraph, and add_table on a Builder. Substituting an HtmlBuilder for a PdfBuilder changes the output format without changing the traversal.\n9.8 Consequences Benefit Explanation Readable construction Each component is named at the call site, which removes positional ambiguity Incremental assembly Components can be supplied across several statements and by several collaborators Cross-field validation build sees the complete configuration and can enforce rules that no individual setter can check Multiple representations One construction sequence can drive different Builder implementations Supports immutable products The mutable partial state resides in the builder rather than in the product Liability Explanation Duplication of the product\u0026rsquo;s shape Each component of the product usually corresponds to a builder operation, so a field added to the product requires a method added to the builder Additional class per product The class count doubles for every product that gets a builder Deferred error detection A missing required component is reported at build time rather than at the point where it was omitted Unnecessary for simple products For objects with a few fields, keyword arguments are clearer and shorter 9.9 Python-Specific Considerations Python\u0026rsquo;s keyword arguments with defaults, combined with @dataclass, cover a large fraction of the cases that require a Builder in languages without them:\nfrom dataclasses import dataclass, field @dataclass class HttpRequest: url: str method: str = \u0026#34;GET\u0026#34; headers: dict = field(default_factory=dict) timeout: int = 30 request = HttpRequest(url=\u0026#34;https://api.example.com\u0026#34;, method=\u0026#34;POST\u0026#34;, timeout=60) field(default_factory=dict) is required rather than = {} because a mutable default evaluated once at function-definition time would be shared by every instance, which is a well-known defect class in Python.\nA Builder is justified over keyword arguments when at least one of the following holds:\nValidation rules span multiple fields and must be enforced as a unit. Components are contributed incrementally by different parts of the system. The same construction sequence must produce more than one representation. A fluent interface materially improves readability at many call sites. The product must be immutable while its construction is inherently multi-step. If none of these hold, a dataclass with keyword arguments is the better design.\n9.10 Related Patterns Abstract Factory also constructs objects without exposing concrete classes. The distinction is that Abstract Factory returns a product immediately from each call, whereas Builder accumulates state across calls and returns the product only at the end. Abstract Factory emphasises families of products; Builder emphasises the construction process of one complex product.\nComposite is frequently the product a Builder assembles, since tree structures are built incrementally.\nFluent Interface is the general API style of which chained builders are the most common instance.\n10. Prototype 10.1 Problem Context A simulation spawns several hundred enemy entities. Each entity carries statistics, an equipment loadout, an ability list, and a behaviour profile that is derived at startup by evaluating configuration and a difficulty curve. Constructing one entity from raw configuration takes measurable time and requires access to the configuration subsystem.\ndef spawn_guard(config, difficulty_curve): stats = derive_stats(config, difficulty_curve) # expensive loadout = resolve_equipment(config[\u0026#34;guard\u0026#34;][\u0026#34;items\u0026#34;]) # expensive abilities = compile_abilities(config[\u0026#34;guard\u0026#34;][\u0026#34;skills\u0026#34;])# expensive return Enemy(\u0026#34;Guard\u0026#34;, stats, loadout, abilities) guards = [spawn_guard(config, curve) for _ in range(500)] # 500 identical derivations The defects are that identical derivation work is repeated for every instance, and that the spawning code must retain access to the configuration subsystem in order to create an object it has already created once.\nThere is a second, structurally distinct case. When an object has accumulated state at runtime, for example a document that has been edited or an entity that has been buffed, no constructor can reproduce that state, because the state is the result of a history rather than of a set of parameters.\n10.2 Intent Specify the kinds of objects to create using a prototypical instance, and create new objects by copying this prototype.\nThe pattern replaces construction with copying. The consequence is that the specification of what to create is an object rather than a class, which means that the specification can be assembled at runtime, altered at runtime, and stored in a data structure.\n10.3 Participants Participant Definition Responsibility Prototype An interface declaring a cloning operation Declares clone Concrete Prototype A class implementing the cloning operation Returns a copy of itself, with a correct policy for which parts are copied and which are shared Client Code that requests new objects Calls clone on a prototype instead of invoking a constructor 10.4 Reference Implementation import copy class Enemy: def __init__(self, name, stats, loadout, abilities): self.name = name self.stats = stats # dict, mutable self.loadout = loadout # list, mutable self.abilities = abilities # list, mutable def clone(self) -\u0026gt; \u0026#34;Enemy\u0026#34;: return copy.deepcopy(self) def __repr__(self): return f\u0026#34;Enemy({self.name}, stats={self.stats}, abilities={self.abilities})\u0026#34; # The prototype is derived once, using the expensive path. guard_prototype = Enemy(\u0026#34;Guard\u0026#34;, {\u0026#34;hp\u0026#34;: 100, \u0026#34;armour\u0026#34;: 20}, [\u0026#34;sword\u0026#34;, \u0026#34;shield\u0026#34;], [\u0026#34;block\u0026#34;, \u0026#34;slash\u0026#34;]) # Every subsequent instance is a copy, with only the differences applied. squad = [] for i in range(3): guard = guard_prototype.clone() guard.name = f\u0026#34;Guard-{i}\u0026#34; squad.append(guard) squad[0].abilities.append(\u0026#34;shield_bash\u0026#34;) # affects only this clone print(guard_prototype) # abilities=[\u0026#39;block\u0026#39;, \u0026#39;slash\u0026#39;], unchanged print(squad[0]) # abilities=[\u0026#39;block\u0026#39;, \u0026#39;slash\u0026#39;, \u0026#39;shield_bash\u0026#39;] print(squad[1]) # abilities=[\u0026#39;block\u0026#39;, \u0026#39;slash\u0026#39;] 10.5 Shallow and Deep Copying The correctness of every Prototype implementation rests on this distinction, and getting it wrong produces defects in which mutating one object silently alters another.\nShallow copy. A new outer object is allocated, and its attributes are bound to the same objects the original\u0026rsquo;s attributes were bound to. The copy is independent with respect to rebinding an attribute, and shared with respect to mutating the object an attribute refers to.\nDeep copy. A new outer object is allocated, and each attribute is bound to a recursively produced copy of what the original referred to. The result is fully independent.\nimport copy original = {\u0026#34;scores\u0026#34;: [1, 2, 3]} shallow = copy.copy(original) deep = copy.deepcopy(original) shallow[\u0026#34;scores\u0026#34;].append(4) # mutates the list both dicts refer to print(original[\u0026#34;scores\u0026#34;]) # [1, 2, 3, 4] print(deep[\u0026#34;scores\u0026#34;]) # [1, 2, 3] shallow[\u0026#34;scores\u0026#34;] = [9] # rebinding, not mutation print(original[\u0026#34;scores\u0026#34;]) # [1, 2, 3, 4], unaffected by rebinding The last two lines illustrate why the distinction is stated in terms of mutation rather than assignment. Assigning a new object to an attribute of the copy never affects the original; mutating an object reachable from both does.\nDeep copy is the default choice for Prototype whenever the object has mutable attributes. Shallow copy is correct only where the shared parts are genuinely immutable or are intended to be shared, such as a texture atlas or a lookup table.\nDeep copying has three failure modes worth knowing. Objects holding file handles, sockets, threads, or database connections cannot be meaningfully deep-copied, since the underlying operating system resource is not duplicated. Very large object graphs make deep copying more expensive than construction, which removes the pattern\u0026rsquo;s justification. Cyclic references are handled correctly by copy.deepcopy through its memoisation table, described next.\n10.6 Customising the Copy: __copy__ and __deepcopy__ copy.deepcopy calls an object\u0026rsquo;s __deepcopy__ method if the class defines one. Overriding it is required whenever \u0026ldquo;copy every attribute verbatim\u0026rdquo; is the wrong policy for at least one attribute.\nimport copy from datetime import datetime class Enemy: def __init__(self, name, stats, abilities, spawned_at=None): self.name = name self.stats = stats self.abilities = abilities self.spawned_at = spawned_at or datetime.now() self.entity_id = id(self) def __deepcopy__(self, memo): # `memo` is deepcopy\u0026#39;s bookkeeping dictionary. It maps the id() of each # object already copied to its copy. Threading it through nested # deepcopy calls is what prevents infinite recursion on cyclic # references and preserves sharing: an object referenced twice in the # original graph is copied once and referenced twice in the result. clone = Enemy( self.name, copy.deepcopy(self.stats, memo), copy.deepcopy(self.abilities, memo), ) # Fields whose value must be unique to each instance are recomputed # rather than copied. A spawn timestamp describes this object\u0026#39;s own # creation, so copying the prototype\u0026#39;s timestamp would be incorrect. clone.spawned_at = datetime.now() clone.entity_id = id(clone) memo[id(self)] = clone return clone The rule for deciding which attributes to copy and which to recompute is: copy the attributes that describe what the object is, and recompute the attributes that describe this particular instance\u0026rsquo;s identity or history.\n10.7 Variant: Prototype Registry Prototypes are commonly held in a registry keyed by name, which turns the set of available object templates into runtime data that can be loaded from configuration.\nclass PrototypeRegistry: def __init__(self): self._prototypes: dict[str, Enemy] = {} def register(self, key: str, prototype: Enemy) -\u0026gt; None: self._prototypes[key] = prototype def create(self, key: str) -\u0026gt; Enemy: if key not in self._prototypes: raise KeyError(f\u0026#34;No prototype registered under \u0026#39;{key}\u0026#39;\u0026#34;) return copy.deepcopy(self._prototypes[key]) registry = PrototypeRegistry() registry.register(\u0026#34;guard\u0026#34;, Enemy(\u0026#34;Guard\u0026#34;, {\u0026#34;hp\u0026#34;: 100}, [\u0026#34;block\u0026#34;])) registry.register(\u0026#34;archer\u0026#34;, Enemy(\u0026#34;Archer\u0026#34;, {\u0026#34;hp\u0026#34;: 70}, [\u0026#34;shoot\u0026#34;, \u0026#34;evade\u0026#34;])) wave = [registry.create(\u0026#34;guard\u0026#34;), registry.create(\u0026#34;archer\u0026#34;), registry.create(\u0026#34;guard\u0026#34;)] The registry looks similar to a parameterised factory, and the distinction is precise: a factory registry maps a key to a class and calls it, whereas a prototype registry maps a key to a configured instance and copies it. The prototype registry can therefore offer variants that no class distinguishes, such as \u0026ldquo;guard with a full loadout\u0026rdquo; and \u0026ldquo;guard with a damaged shield\u0026rdquo;, without defining a class for each.\n10.8 Applied Examples Entity spawning in simulations and games, as above.\nTest fixtures. A fully populated base object is constructed once for a test module, and each test clones it and modifies the single field under examination. This gives isolation between tests without paying the construction cost repeatedly.\nConfiguration templates. A default configuration object is cloned per environment and the differing keys are overridden.\nEditor state snapshots. Cloning an object graph at a point in time captures state that no constructor could reproduce. This overlaps with the Memento pattern, which addresses the same need with an emphasis on encapsulation of the saved state.\nDocument duplication. Duplicating a slide, a layer, or a shape in an editing application is exactly the pattern, with the prototype being whatever the user selected.\n10.9 Consequences Benefit Explanation Avoids repeated expensive construction Derivation work is performed once for the prototype Captures runtime-accumulated state State produced by a history, not by parameters, is reproducible only by copying Adds and removes object kinds at runtime Prototypes are objects, so the catalogue of available kinds is data rather than code Reduces subclass count Variants that differ only in field values need no class of their own Liability Explanation Copy semantics are error-prone An incorrect shallow copy produces defects that appear far from their cause Some objects are not copyable Objects owning operating system resources require a custom __deepcopy__ or cannot use the pattern Deep copy may be slower than construction For large graphs of cheap objects, the pattern can be a pessimisation The prototype must remain valid Accidental mutation of the prototype corrupts every object cloned afterwards 10.10 Related Patterns Abstract Factory may hold prototypes internally and implement its creation operations by cloning them rather than by instantiating classes.\nComposite and Decorator structures benefit from Prototype, because copying a configured tree of objects is otherwise tedious to write.\nMemento stores a snapshot of an object\u0026rsquo;s state for later restoration. Prototype produces an independent object intended for use alongside the original; Memento produces an opaque record intended to replace the original\u0026rsquo;s state later.\nSingleton constrains a class to one instance and is, in that sense, the opposite constraint.\n10.11 Python-Specific Considerations copy.deepcopy provides the pattern\u0026rsquo;s mechanism without any explicit Prototype interface, so the pattern in Python usually reduces to a clone method one line long, or to calling copy.deepcopy directly.\nFor dataclasses, dataclasses.replace produces a copy with selected fields changed, which expresses \u0026ldquo;clone with modifications\u0026rdquo; directly:\nfrom dataclasses import dataclass, replace @dataclass class Config: host: str port: int debug: bool = False base = Config(host=\u0026#34;localhost\u0026#34;, port=5432) staging = replace(base, host=\u0026#34;staging.internal\u0026#34;) replace performs a shallow copy, so the caveat about shared mutable fields applies to it as well.\n11. Comparative Summary of the Five Patterns 11.1 Intent and Mechanism Pattern Intent in one line Primary mechanism What varies Singleton Ensure one instance exists and is globally reachable Interception of instantiation in __new__, a metaclass __call__, or a module The number of instances, fixed at one Factory Method Defer the choice of concrete class to a subclass or a lookup Inheritance, or a mapping from key to class Which concrete product is produced Abstract Factory Create families of related products without naming concrete classes Composition, the client holds a factory object Which entire family is produced Builder Separate the construction of a complex object from its representation An object that accumulates partial state and validates on completion The construction process Prototype Create objects by copying a configured instance copy.deepcopy with an optional __deepcopy__ policy Whether an object is built or copied 11.2 Cost and Applicability Pattern Classes added Runtime substitutable Chief liability Apply when Singleton None, one class is constrained No Global mutable state complicates testing A shared resource must be unique and reference passing is impractical Factory Method One product hierarchy, optionally one creator hierarchy Yes, by replacing the factory Indirection between call site and concrete class The concrete type depends on runtime data or configuration Abstract Factory Product kinds multiplied by families Yes, by passing a different factory Adding a product kind modifies every factory Several objects must be mutually compatible Builder One builder per product Yes, by using a different builder Duplicates the product\u0026rsquo;s field list Many optional components or cross-field validation Prototype None, one method is added Yes, prototypes are data Copy semantics errors and non-copyable resources Construction is expensive or state is history-dependent 11.3 Selection Procedure The following sequence resolves most cases. Each question should be answered against the requirements as they exist, not against requirements that are anticipated.\nIs a direct constructor call adequate, meaning the concrete class is fixed, construction is simple, and no cardinality constraint applies? If yes, use it and stop. This is the correct answer more often than any pattern. Must exactly one instance exist? Prefer creating it once at the composition root and injecting it. Apply Singleton only if that is impractical, and prefer a module-level object or lru_cache over a class-based implementation. Does the concrete class depend on runtime data? Use a parameterised factory function or a registry. Introduce the Creator subclass hierarchy only if the Creator also holds substantial shared behaviour. Must several created objects be mutually compatible? Use Abstract Factory, provided the set of product kinds is stable. Does the object have many optional components, or validation rules that span fields, or is it assembled incrementally? Use Builder, after confirming that keyword arguments with a dataclass are insufficient. Is construction expensive, or does the object carry state that a constructor cannot reproduce? Use Prototype, and decide the shallow or deep copy policy explicitly. 12. Key Takeaways Creational patterns separate the knowledge of what is created from the code that uses it. Every one of the five achieves this separation along a different axis: cardinality, concrete type, family, construction process, or origin.\nA pattern is a problem, a solution, and a set of consequences. Learning the solution structure without the problem produces misapplication, and learning it without the consequences produces designs whose costs were never weighed.\nThe dependency inversion and open/closed principles are visible in four of the five patterns. Clients depend on abstract products, and new concrete types are added without modifying existing code. Singleton is the exception and works against dependency inversion, which is why it attracts more criticism than the others.\nPython\u0026rsquo;s language features reduce the code required for several of these patterns. Modules are singletons, classes are first-class objects that can be stored in dictionaries, keyword arguments with dataclasses cover many builder cases, and copy.deepcopy supplies Prototype\u0026rsquo;s mechanism. Recognise the pattern, then implement it in the idiom of the language rather than transcribing a Java structure.\nIndirection is a cost paid on every reading of the code and repaid only when the anticipated change occurs. Introduce a creational pattern when the variation it isolates is variation the system actually has.\nThe patterns compose. Abstract Factory is normally implemented with Factory Methods, Concrete Factories are frequently Singletons, and either may use a Builder or a Prototype to produce an individual product.\n13. Practice Exercises with Solutions Exercise 1: Configuration Registry with Correct Initialisation Implement a ConfigManager such that repeated instantiation returns the same object, configuration loaded through one reference is visible through every other, and initialisation does not reset state on subsequent calls. Then state why a module-level dictionary would be a reasonable alternative and when it would not be.\nSolution\nclass ConfigManager: _instance = None def __new__(cls): if cls._instance is None: cls._instance = super().__new__(cls) cls._instance._config = {} # set here, so it runs exactly once return cls._instance def load(self, mapping: dict) -\u0026gt; None: self._config.update(mapping) # update, not reassign, to preserve # any values loaded earlier def get(self, key, default=None): return self._config.get(key, default) def set(self, key, value) -\u0026gt; None: self._config[key] = value a = ConfigManager() a.load({\u0026#34;db_host\u0026#34;: \u0026#34;localhost\u0026#34;, \u0026#34;port\u0026#34;: 5432}) b = ConfigManager() print(b.get(\u0026#34;db_host\u0026#34;)) # localhost b.set(\u0026#34;port\u0026#34;, 6000) print(a.get(\u0026#34;port\u0026#34;)) # 6000 print(a is b) # True The initialisation of _config is placed inside __new__ under the is None guard. Placing it in __init__ without a guard would clear the configuration on every ConfigManager() call, which is the most common defect in hand-written singletons.\nA module-level dictionary in a config.py file provides the same guarantees with less machinery, because module objects are cached in sys.modules. It is the better choice unless the configuration must support subclassing, multiple named configurations, or lazy construction that depends on runtime arguments.\nExercise 2: Extensible Shape Factory Build a shape-drawing subsystem in which client code obtains a shape by name and calls draw on it. Adding a new shape must not require modifying any existing file. Implement it twice: once with an explicit registry dictionary, and once with __init_subclass__ registration. State the difference in extension cost.\nSolution\nfrom abc import ABC, abstractmethod # ---------- Version A: explicit registry ---------- class Shape(ABC): @abstractmethod def draw(self) -\u0026gt; None: ... class Circle(Shape): def draw(self): print(\u0026#34;Drawing circle\u0026#34;) class Square(Shape): def draw(self): print(\u0026#34;Drawing square\u0026#34;) _SHAPES: dict[str, type[Shape]] = {\u0026#34;circle\u0026#34;: Circle, \u0026#34;square\u0026#34;: Square} def shape_factory(kind: str) -\u0026gt; Shape: if kind not in _SHAPES: raise ValueError(f\u0026#34;Unknown shape: {kind}\u0026#34;) return _SHAPES[kind]() # ---------- Version B: self-registering subclasses ---------- class AutoShape(ABC): registry: dict[str, type[\u0026#34;AutoShape\u0026#34;]] = {} def __init_subclass__(cls, name: str = None, **kwargs): super().__init_subclass__(**kwargs) if name: AutoShape.registry[name] = cls @abstractmethod def draw(self) -\u0026gt; None: ... class AutoCircle(AutoShape, name=\u0026#34;circle\u0026#34;): def draw(self): print(\u0026#34;Drawing circle\u0026#34;) class AutoHexagon(AutoShape, name=\u0026#34;hexagon\u0026#34;): def draw(self): print(\u0026#34;Drawing hexagon\u0026#34;) def auto_shape_factory(kind: str) -\u0026gt; AutoShape: return AutoShape.registry[kind]() for k in sorted(AutoShape.registry): auto_shape_factory(k).draw() In version A, adding a shape requires a new class plus an edit to the _SHAPES dictionary, so one existing file is modified. In version B, adding a shape requires only the new class, and registration occurs as a side effect of the class definition, provided the module defining it is imported. The trade-off is that version B\u0026rsquo;s registration depends on import order, so a shape defined in a module that is never imported will be absent from the registry with no error.\nExercise 3: Abstract Factory for a Cross-Platform Form Design an Abstract Factory producing two product kinds, TextInput and SubmitButton, in two families, Material and iOS. Write a render_form client that is parameterised by the factory. Then explain what would change if a third product kind, Checkbox, were required.\nSolution\nfrom abc import ABC, abstractmethod # ---------- Abstract Products ---------- class TextInput(ABC): @abstractmethod def render(self) -\u0026gt; None: ... class SubmitButton(ABC): @abstractmethod def render(self) -\u0026gt; None: ... # ---------- Material family ---------- class MaterialTextInput(TextInput): def render(self): print(\u0026#34;[Material text input with floating label]\u0026#34;) class MaterialSubmitButton(SubmitButton): def render(self): print(\u0026#34;[Material raised button]\u0026#34;) # ---------- iOS family ---------- class IOSTextInput(TextInput): def render(self): print(\u0026#34;(iOS rounded text input)\u0026#34;) class IOSSubmitButton(SubmitButton): def render(self): print(\u0026#34;(iOS filled button)\u0026#34;) # ---------- Abstract Factory ---------- class FormUIFactory(ABC): @abstractmethod def create_text_input(self) -\u0026gt; TextInput: ... @abstractmethod def create_submit_button(self) -\u0026gt; SubmitButton: ... class MaterialFactory(FormUIFactory): def create_text_input(self): return MaterialTextInput() def create_submit_button(self): return MaterialSubmitButton() class IOSFactory(FormUIFactory): def create_text_input(self): return IOSTextInput() def create_submit_button(self): return IOSSubmitButton() # ---------- Client ---------- def render_form(factory: FormUIFactory) -\u0026gt; None: factory.create_text_input().render() factory.create_submit_button().render() render_form(MaterialFactory()) render_form(IOSFactory()) Adding a Checkbox product kind requires: one new abstract product class, one concrete class per family, a new abstract method on FormUIFactory, and an implementation of that method in every existing Concrete Factory. The number of files modified grows with the number of families, which is the asymmetry described in Section 8.5. Adding a third family, in contrast, requires only new files.\nExercise 4: Builder with Cross-Field Validation Implement a builder for a DatabaseConnectionConfig with the fields host, port, database, user, password, ssl_enabled, ssl_cert_path, pool_size, and timeout. Enforce these rules at build time: host and database are required; pool_size must be between 1 and 100; if ssl_enabled is true then ssl_cert_path is required; and supplying password without user is an error. Explain why each rule cannot be enforced in an individual setter.\nSolution\nclass DatabaseConnectionConfig: def __init__(self, host, port, database, user, password, ssl_enabled, ssl_cert_path, pool_size, timeout): self.host = host self.port = port self.database = database self.user = user self.password = password self.ssl_enabled = ssl_enabled self.ssl_cert_path = ssl_cert_path self.pool_size = pool_size self.timeout = timeout def __repr__(self): return (f\u0026#34;DatabaseConnectionConfig({self.user}@{self.host}:{self.port}\u0026#34; f\u0026#34;/{self.database} ssl={self.ssl_enabled} pool={self.pool_size})\u0026#34;) class DatabaseConfigBuilder: def __init__(self): self._host = None self._port = 5432 self._database = None self._user = None self._password = None self._ssl_enabled = False self._ssl_cert_path = None self._pool_size = 10 self._timeout = 30 def host(self, v): self._host = v; return self def port(self, v): self._port = v; return self def database(self, v): self._database = v; return self def credentials(self, user, password): self._user, self._password = user, password return self def with_ssl(self, cert_path): self._ssl_enabled = True self._ssl_cert_path = cert_path return self def pool_size(self, v): self._pool_size = v; return self def timeout(self, v): self._timeout = v; return self def build(self) -\u0026gt; DatabaseConnectionConfig: errors = [] if self._host is None: errors.append(\u0026#34;host is required\u0026#34;) if self._database is None: errors.append(\u0026#34;database is required\u0026#34;) if not 1 \u0026lt;= self._pool_size \u0026lt;= 100: errors.append(\u0026#34;pool_size must be between 1 and 100\u0026#34;) if self._ssl_enabled and not self._ssl_cert_path: errors.append(\u0026#34;ssl_cert_path is required when ssl is enabled\u0026#34;) if self._password is not None and self._user is None: errors.append(\u0026#34;password supplied without user\u0026#34;) if errors: raise ValueError(\u0026#34;Invalid configuration: \u0026#34; + \u0026#34;; \u0026#34;.join(errors)) return DatabaseConnectionConfig( self._host, self._port, self._database, self._user, self._password, self._ssl_enabled, self._ssl_cert_path, self._pool_size, self._timeout) config = (DatabaseConfigBuilder() .host(\u0026#34;db.internal\u0026#34;) .database(\u0026#34;orders\u0026#34;) .credentials(\u0026#34;app_user\u0026#34;, \u0026#34;s3cret\u0026#34;) .with_ssl(\u0026#34;/etc/ssl/client.pem\u0026#34;) .pool_size(25) .build()) print(config) The requiredness rules for host and database cannot be enforced in a setter because a setter that is never called cannot raise. The SSL rule and the credentials rule relate two fields, and at the time either setter runs the other field may not yet have been supplied, so neither setter has enough information to decide. Only pool_size could be validated in its setter, since it depends on nothing else; validating it in build alongside the others keeps all rules in one place and allows every violation to be reported together rather than one per exception.\nThe credentials and with_ssl methods set two related fields in one call, which is a common builder technique for making an invalid intermediate state unreachable rather than merely detectable.\nExercise 5: Prototype with Correct Copy Semantics Implement a Document class with title, body, tags as a list, and metadata as a dictionary. Provide both a deep and a shallow cloning operation, demonstrate the difference in behaviour, and then override __deepcopy__ so that a created_at timestamp is set to the time of cloning rather than copied from the prototype.\nSolution\nimport copy from datetime import datetime class Document: def __init__(self, title, body, tags=None, metadata=None): self.title = title self.body = body self.tags = tags if tags is not None else [] self.metadata = metadata if metadata is not None else {} self.created_at = datetime.now() def clone(self) -\u0026gt; \u0026#34;Document\u0026#34;: return copy.deepcopy(self) def shallow_clone(self) -\u0026gt; \u0026#34;Document\u0026#34;: return copy.copy(self) def __deepcopy__(self, memo): clone = Document( self.title, self.body, copy.deepcopy(self.tags, memo), copy.deepcopy(self.metadata, memo), ) clone.created_at = datetime.now() # identity of this copy, not copied memo[id(self)] = clone return clone def __repr__(self): return f\u0026#34;Document({self.title!r}, tags={self.tags}, metadata={self.metadata})\u0026#34; prototype = Document(\u0026#34;Blog Template\u0026#34;, \u0026#34;Introduction...\u0026#34;, tags=[\u0026#34;draft\u0026#34;], metadata={\u0026#34;author\u0026#34;: \u0026#34;Amit\u0026#34;}) deep = prototype.clone() deep.tags.append(\u0026#34;published\u0026#34;) deep.metadata[\u0026#34;reviewed\u0026#34;] = True print(\u0026#34;prototype after deep clone mutation:\u0026#34;, prototype) # tags=[\u0026#39;draft\u0026#39;], metadata={\u0026#39;author\u0026#39;: \u0026#39;Amit\u0026#39;} shallow = prototype.shallow_clone() shallow.tags.append(\u0026#34;leaked\u0026#34;) print(\u0026#34;prototype after shallow clone mutation:\u0026#34;, prototype) # tags=[\u0026#39;draft\u0026#39;, \u0026#39;leaked\u0026#39;], the mutation propagated print(prototype.created_at == deep.created_at) # False The shallow clone shares the tags list object with the prototype, so appending through either name mutates the single list both refer to. The deep clone binds tags to a new list, so the two are independent. The __deepcopy__ override demonstrates the general rule: attributes describing what the object is are copied, and attributes describing this instance\u0026rsquo;s own identity or creation are recomputed.\nExercise 6: Capstone, Combining Four Patterns Design a data access layer that supports two database backends, exposes read and write connections, ensures a single connection pool exists per process, and constructs its configuration through a fluent interface. Identify which pattern plays which role, and justify each choice.\nSolution\nfrom abc import ABC, abstractmethod import threading # ---------- Abstract Products ---------- class Connection(ABC): @abstractmethod def execute(self, sql: str) -\u0026gt; None: ... # ---------- Concrete Products ---------- class MySQLReadConnection(Connection): def execute(self, sql): print(f\u0026#34;[mysql-read] {sql}\u0026#34;) class MySQLWriteConnection(Connection): def execute(self, sql): print(f\u0026#34;[mysql-write] {sql}\u0026#34;) class PostgresReadConnection(Connection): def execute(self, sql): print(f\u0026#34;[pg-read] {sql}\u0026#34;) class PostgresWriteConnection(Connection): def execute(self, sql): print(f\u0026#34;[pg-write] {sql}\u0026#34;) # ---------- Abstract Factory: one family per backend ---------- class ConnectionFactory(ABC): @abstractmethod def create_read(self) -\u0026gt; Connection: ... @abstractmethod def create_write(self) -\u0026gt; Connection: ... class MySQLFactory(ConnectionFactory): def create_read(self): return MySQLReadConnection() def create_write(self): return MySQLWriteConnection() class PostgresFactory(ConnectionFactory): def create_read(self): return PostgresReadConnection() def create_write(self): return PostgresWriteConnection() # ---------- Builder for the pool configuration ---------- class PoolConfig: def __init__(self, factory, max_read, max_write, timeout): self.factory = factory self.max_read = max_read self.max_write = max_write self.timeout = timeout class PoolConfigBuilder: def __init__(self): self._factory = None self._max_read = 10 self._max_write = 5 self._timeout = 30 def backend(self, factory: ConnectionFactory): self._factory = factory; return self def max_read(self, n): self._max_read = n; return self def max_write(self, n): self._max_write = n; return self def timeout(self, s): self._timeout = s; return self def build(self) -\u0026gt; PoolConfig: if self._factory is None: raise ValueError(\u0026#34;backend factory is required\u0026#34;) if self._max_write \u0026gt; self._max_read: raise ValueError(\u0026#34;max_write must not exceed max_read\u0026#34;) return PoolConfig(self._factory, self._max_read, self._max_write, self._timeout) # ---------- Singleton pool, with a Factory Method for role selection ---------- class SingletonMeta(type): _instances = {} _lock = threading.Lock() def __call__(cls, *args, **kwargs): if cls not in cls._instances: with cls._lock: if cls not in cls._instances: cls._instances[cls] = super().__call__(*args, **kwargs) return cls._instances[cls] class ConnectionPool(metaclass=SingletonMeta): def __init__(self, config: PoolConfig): self._config = config def get_connection(self, role: str) -\u0026gt; Connection: \u0026#34;\u0026#34;\u0026#34;Factory Method in its parameterised form: the role selects which concrete connection is produced, and the caller never names a class.\u0026#34;\u0026#34;\u0026#34; creators = { \u0026#34;read\u0026#34;: self._config.factory.create_read, \u0026#34;write\u0026#34;: self._config.factory.create_write, } if role not in creators: raise ValueError(f\u0026#34;Unknown role: {role}\u0026#34;) return creators[role]() # ---------- Composition root ---------- config = (PoolConfigBuilder() .backend(PostgresFactory()) .max_read(20) .max_write(5) .timeout(15) .build()) pool = ConnectionPool(config) assert ConnectionPool(config) is pool # singleton guarantee pool.get_connection(\u0026#34;read\u0026#34;).execute(\u0026#34;SELECT * FROM orders\u0026#34;) pool.get_connection(\u0026#34;write\u0026#34;).execute(\u0026#34;INSERT INTO orders VALUES (...)\u0026#34;) Role in the design Pattern Justification ConnectionPool Singleton The pool enforces a bound on concurrent connections; a second pool would defeat that bound get_connection(role) Factory Method, parameterised form The concrete connection class depends on a runtime value, and callers must not name concrete classes ConnectionFactory and its subclasses Abstract Factory Read and write connections must belong to the same backend, and the factory makes mixing impossible PoolConfigBuilder Builder The configuration has several optional fields and a cross-field rule relating max_write to max_read The composition root is the only location naming a concrete factory, so switching backends is a one-line change confined to that location.\n14. Self-Assessment Questions Attempt each question before consulting the answer.\nQ1. Why must the singleton constraint be enforced in __new__ or in a metaclass rather than in __init__? Because __init__ receives an object that has already been allocated. __new__ is the allocator, and returning an existing object from it means no allocation occurs. A metaclass __call__ intercepts at a level above both, which additionally prevents __init__ from re-running and resetting the instance\u0026rsquo;s state.\nQ2. A singleton implemented by overriding __new__ loses its state on every instantiation call. What is the defect? Initialisation was placed in an unguarded __init__. Python calls __init__ on whatever __new__ returns, on every call, so the second ClassName() re-initialises the existing instance. The corrections are to place initialisation inside the is None branch of __new__, to guard __init__ with a flag, or to use the metaclass implementation, which skips both methods after the first call.\nQ3. State the precise difference between Factory Method and Abstract Factory. Factory Method produces one product and varies the concrete class through subclassing or a parameter. Abstract Factory produces a family of several related products and varies the entire family by substituting the factory object. The distinguishing question is whether the requirement is \u0026ldquo;which concrete X\u0026rdquo; or \u0026ldquo;which mutually compatible set of X, Y, and Z\u0026rdquo;.\nQ4. Why is Abstract Factory described as open for extension in one dimension and closed in the other? Adding a new family requires only new classes, because the abstract interfaces are unchanged. Adding a new product kind changes the Abstract Factory interface and therefore forces every existing Concrete Factory to implement the new operation. The pattern is therefore appropriate when the set of product kinds is stable and the set of families is expected to grow.\nQ5. Under what conditions is a Builder justified over a dataclass with keyword arguments? When validation rules span multiple fields and must be enforced as a unit; when components are contributed incrementally by different collaborators; when the same construction sequence must produce more than one representation; when a fluent interface materially improves readability across many call sites; or when the product must be immutable but its construction is inherently multi-step. If none of these apply, a dataclass is the better design.\nQ6. Why does a builder's build method usually copy mutable components into the product? Without the copy, the product holds a reference to a container the builder still owns. Reusing the builder for a second product would then mutate the first product\u0026rsquo;s state. Copying at build time makes each product independent of the builder\u0026rsquo;s subsequent use.\nQ7. Explain why shallow copying is unsafe for Prototype in most cases, and identify the case where it is correct. A shallow copy binds the copy\u0026rsquo;s attributes to the same objects the original\u0026rsquo;s attributes refer to. Mutating any of those objects through either reference affects both, which produces defects far from their cause. Shallow copying is correct when the shared attributes are immutable, or when sharing is intended, for example a texture atlas or a read-only lookup table shared by every instance for memory reasons.\nQ8. What is the purpose of the memo argument in __deepcopy__? It maps the identity of each object already copied to its copy. Passing it into nested deepcopy calls prevents infinite recursion on cyclic references, and preserves the sharing structure of the original graph: an object referenced from two places in the original is copied once and referenced from two places in the result.\nQ9. Distinguish a factory registry from a prototype registry. A factory registry maps a key to a class and calls that class to produce an instance. A prototype registry maps a key to a configured instance and copies it. The prototype registry can distinguish variants that no class distinguishes, since the variation is in field values rather than in type, and its contents can be assembled from configuration at runtime.\nQ10. A codebase wraps every class in a factory. What is wrong with this, and what is the corrective question? Factories are justified when the concrete class varies, when construction logic is complex, or when callers must be decoupled from concrete types. Where none of these hold, the factory adds a file to read and a step to trace while removing nothing. The corrective question before introducing any pattern is: what specific change to the requirements does this structure make cheaper, and is that change plausible?\nQ11. Why is Singleton considered to work against the dependency inversion principle? A class that obtains the shared instance by calling the singleton\u0026rsquo;s access point inside its own methods does not declare that dependency in its interface. The dependency is therefore invisible to readers and unsubstitutable by callers, which is the opposite of depending on an abstraction supplied from outside. Dependency injection restores the declaration by making the shared instance a constructor parameter.\nQ12. Identify the pattern for each description. A class holding a reference to its own unique instance and controlling access to it: Singleton. An object accumulating components across several calls and producing the finished object on a final call: Builder. An interface with one creation operation per product kind, implemented once per family: Abstract Factory. A method on a base class that subclasses override to determine which concrete class the base class\u0026rsquo;s algorithm will use: Factory Method. A configured instance copied to produce new objects rather than being reconstructed: Prototype. The structural patterns, covered in Part 2, address a different question. Creational patterns govern how objects come into existence; structural patterns govern how existing objects are composed into larger assemblies, and include Adapter, Bridge, Composite, Decorator, Facade, Flyweight, and Proxy.\nThis content is part of Codeverra, a platform for learning coding, data science, DSA, and AI from scratch. Explore more: https://codeverra.com\n","permalink":"https://blogs.codeverra.com/blog/low_level_design/03_design_patterns_creational/","summary":"\u003ch1 id=\"design-patterns-part-1-foundations-and-creational-patterns\"\u003eDesign Patterns, Part 1: Foundations and Creational Patterns\u003c/h1\u003e\n\u003ch2 id=\"table-of-contents\"\u003eTable of Contents\u003c/h2\u003e\n\u003col\u003e\n\u003cli\u003e\u003ca href=\"#1-what-a-design-pattern-is\"\u003eWhat a Design Pattern Is\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#2-why-design-patterns-are-studied\"\u003eWhy Design Patterns Are Studied\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#3-classification-of-the-gof-catalogue\"\u003eClassification of the GoF Catalogue\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#4-the-template-used-for-each-pattern\"\u003eThe Template Used for Each Pattern\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#5-the-problem-space-of-creational-patterns\"\u003eThe Problem Space of Creational Patterns\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#6-singleton\"\u003eSingleton\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#7-factory-method\"\u003eFactory Method\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#8-abstract-factory\"\u003eAbstract Factory\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#9-builder\"\u003eBuilder\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#10-prototype\"\u003ePrototype\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#11-comparative-summary-of-the-five-patterns\"\u003eComparative Summary of the Five Patterns\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#12-key-takeaways\"\u003eKey Takeaways\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#13-practice-exercises-with-solutions\"\u003ePractice Exercises with Solutions\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#14-self-assessment-questions\"\u003eSelf-Assessment Questions\u003c/a\u003e\u003c/li\u003e\n\u003c/ol\u003e\n\u003chr\u003e\n\u003ch2 id=\"1-what-a-design-pattern-is\"\u003e1. What a Design Pattern Is\u003c/h2\u003e\n\u003ch3 id=\"11-definition\"\u003e1.1 Definition\u003c/h3\u003e\n\u003cp\u003e\u003cstrong\u003eDesign pattern.\u003c/strong\u003e A named, general, reusable solution to a problem that recurs within a particular context of software design. A pattern is a description of communicating objects and classes that are customised to solve a general design problem in that context.\u003c/p\u003e","tags":["lld","design-patterns","python"],"title":"Design Patterns, Part 1: Foundations and Creational Patterns"},{"content":"Design Patterns — Part 2: Structural Patterns Who this is for: Anyone who has worked through the Creational patterns and is ready to learn how objects fit together — how to combine classes and objects into larger structures that stay flexible and maintainable.\nTable of Contents Structural Patterns — What and Why Adapter Bridge Composite Decorator Facade Flyweight Proxy Summary of Structural Patterns Practice Exercises with Solutions Self-Assessment Test 1. Structural Patterns Creational patterns answered: \u0026ldquo;how do we make objects flexibly?\u0026rdquo; Structural patterns answer: \u0026ldquo;how do we combine objects into larger structures flexibly?\u0026rdquo;\nIn any non-trivial system, you have many objects, and they have to fit together. Naively, you hardcode those connections — class A owns a class B which calls class C. Then C\u0026rsquo;s interface changes, or you need to plug in an alternative D, or you need to wrap B with extra behavior. The tight connections resist change.\nStructural patterns are recipes for gluing objects together in flexible ways:\nAdapter — lets incompatible interfaces work together. Bridge — decouples abstraction from implementation so they can vary independently. Composite — treats individual objects and groups of objects uniformly. Decorator — adds responsibilities to an object dynamically, without changing its class. Facade — provides a simplified interface to a complex subsystem. Flyweight — shares objects to support huge numbers of similar instances efficiently. Proxy — provides a placeholder or surrogate for another object. Each one is a specific answer to a specific integration problem. Let\u0026rsquo;s meet them.\n2. Adapter The Pain You\u0026rsquo;re building an app. You have this interface that your existing code expects:\nclass PaymentProcessor: def pay(self, amount): ... A new requirement: integrate with a third-party payment library — say, StripeAPI — that your company just licensed. But StripeAPI has its own interface:\nclass StripeAPI: def make_payment(self, cents): ... # Takes cents, not rupees/dollars! You can\u0026rsquo;t modify StripeAPI (it\u0026rsquo;s a third-party library). You can\u0026rsquo;t easily change every caller of PaymentProcessor.pay(amount) either — that\u0026rsquo;s your entire codebase. You need a translator between the two.\nIntent Convert the interface of a class into another interface clients expect. Adapter lets classes work together that couldn\u0026rsquo;t otherwise because of incompatible interfaces.\nAnalogy The classic analogy: a travel plug adapter.\nYour Indian laptop charger has round pins. The wall socket in the US has flat pins. Both work electrically — they just don\u0026rsquo;t fit. A travel adapter sits in between: flat pins on one side (to fit the wall), round holes on the other (for your charger). Neither device changes; the adapter bridges the interface mismatch.\nAdapter in software plays exactly the same role.\nStructure Target — the interface your client code expects (PaymentProcessor). Adaptee — the existing class with a different interface (StripeAPI). Adapter — a class that implements Target and delegates to Adaptee, translating as needed. Python Implementation # Target — what our code expects class PaymentProcessor: def pay(self, amount_in_rupees): raise NotImplementedError # Adaptee — third-party, can\u0026#39;t modify class StripeAPI: def make_payment(self, cents): print(f\u0026#34;Stripe charged {cents} cents\u0026#34;) # Adapter — bridges the two class StripeAdapter(PaymentProcessor): def __init__(self, stripe_api: StripeAPI): self.stripe_api = stripe_api def pay(self, amount_in_rupees): # Translate rupees → cents (hypothetical: ₹1 = 100 paise-equivalent) cents = int(amount_in_rupees * 100) self.stripe_api.make_payment(cents) # Client code — only knows about PaymentProcessor def checkout(processor: PaymentProcessor, amount): processor.pay(amount) stripe = StripeAPI() adapter = StripeAdapter(stripe) checkout(adapter, 500) # Stripe charged 50000 cents The client (checkout) remains oblivious to Stripe. Tomorrow, if a new vendor arrives with their own quirky API, you write a new adapter — the rest of the code stays untouched.\nA Subtle Variant — Object Adapter vs Class Adapter Object Adapter (shown above) — the adapter holds the adaptee as a field, delegating via composition. Preferred. Class Adapter — the adapter inherits from both Target and Adaptee, wiring them through multiple inheritance. Less flexible; only works in languages/situations that support multiple inheritance. In Python, Object Adapter is almost always the right choice.\nRealistic Use Cases Integrating third-party libraries that don\u0026rsquo;t match your domain\u0026rsquo;s vocabulary. Wrapping legacy code so new code can consume it cleanly. Testing — wrapping a complex dependency behind a simple adapter you control. Bridging sync and async APIs (a classic modern use). Tradeoffs Pros:\nLets you keep using third-party code without distorting your own design. Isolates translation logic in one place. Respects OCP — you can plug in new vendors behind new adapters. Cons:\nAdds a layer of indirection. If you need to adapt many methods with lots of translation, the adapter itself becomes complex. Related Patterns Facade — also simplifies an interface but typically wraps an entire subsystem, not translate a single class\u0026rsquo;s interface. Decorator — adds responsibilities while preserving interface. Adapter changes interface. Bridge — intentionally designed upfront; Adapter is retrofit. 3. Bridge The Pain Suppose you\u0026rsquo;re designing a UI framework with Shape types (Circle, Square) and multiple rendering backends (RasterRenderer, VectorRenderer). Naively:\nclass RasterCircle: ... class VectorCircle: ... class RasterSquare: ... class VectorSquare: ... Already 4 classes. Add Triangle → 6 classes. Add a third renderer → 9 classes. This is a combinatorial explosion. Every new shape OR renderer multiplies your class count.\nThe root cause: you\u0026rsquo;ve merged two independent axes of variation (shape × renderer) into a single class hierarchy.\nIntent Decouple an abstraction from its implementation so that the two can vary independently.\nThe idea: separate the two axes. Shapes on one side, renderers on the other, connected via a reference (\u0026ldquo;bridge\u0026rdquo;).\nAnalogy Think of TV remotes and TVs. You have different remotes (basic, smart, universal) and different TV models (LG, Sony, Samsung). You don\u0026rsquo;t build 3 × 3 = 9 specific remote-TV products. You build remotes that speak a common protocol (IR signals), and TVs that understand that protocol. Any remote can pair with any TV.\nOr: drivers and cars. A driver (abstraction) operates any car (implementation). You don\u0026rsquo;t design a \u0026ldquo;Honda-driver\u0026rdquo; separately from a \u0026ldquo;Toyota-driver\u0026rdquo; — the driving behavior is one axis, the car is another axis.\nStructure Abstraction — the high-level layer (e.g., Shape). Refined Abstraction — subclasses of Abstraction (Circle, Square). Implementor — the interface for the low-level layer (Renderer). Concrete Implementors — implementations of Implementor (RasterRenderer, VectorRenderer). Bridge — the Abstraction holds a reference to an Implementor, delegating low-level work to it. Python Implementation from abc import ABC, abstractmethod # Implementor class Renderer(ABC): @abstractmethod def render_circle(self, radius): pass @abstractmethod def render_square(self, side): pass # Concrete Implementors class RasterRenderer(Renderer): def render_circle(self, radius): print(f\u0026#34;Drawing circle of radius {radius} pixel-by-pixel\u0026#34;) def render_square(self, side): print(f\u0026#34;Drawing square of side {side} pixel-by-pixel\u0026#34;) class VectorRenderer(Renderer): def render_circle(self, radius): print(f\u0026#34;Drawing vector circle radius={radius}\u0026#34;) def render_square(self, side): print(f\u0026#34;Drawing vector square side={side}\u0026#34;) # Abstraction (holds a bridge to Implementor) class Shape(ABC): def __init__(self, renderer: Renderer): # THIS is the \u0026#34;bridge\u0026#34;: instead of Shape inheriting rendering behavior # (which would force one subclass per shape×renderer combination), Shape # *holds a reference* to a Renderer via composition. The two hierarchies # — what to draw (Shape) and how to draw it (Renderer) — now vary # independently and are connected only by this single attribute. self.renderer = renderer @abstractmethod def draw(self): pass # Refined Abstractions class Circle(Shape): def __init__(self, renderer, radius): super().__init__(renderer) self.radius = radius def draw(self): # Circle knows WHAT it is (a circle of some radius) but delegates HOW to # render across the bridge. It never asks whether the renderer is raster # or vector — that\u0026#39;s the decoupling the Bridge buys us. self.renderer.render_circle(self.radius) class Square(Shape): def __init__(self, renderer, side): super().__init__(renderer) self.side = side def draw(self): self.renderer.render_square(self.side) # Client code — mix and match freely raster = RasterRenderer() vector = VectorRenderer() Circle(raster, 5).draw() # Drawing circle of radius 5 pixel-by-pixel Circle(vector, 5).draw() # Drawing vector circle radius=5 Square(vector, 3).draw() # Drawing vector square side=3 Now:\nAdding Triangle → one new class in the Abstraction hierarchy. Adding a new renderer (SVGRenderer) → one new class in the Implementor hierarchy. No combinatorial explosion. Any shape can pair with any renderer. To see why this matters concretely: with inheritance you\u0026rsquo;d need a class per combination — RasterCircle, VectorCircle, RasterSquare, VectorSquare… that\u0026rsquo;s N shapes × M renderers = N×M classes, and adding one renderer means adding N new classes. With the Bridge, you have N shape classes plus M renderer classes = N+M, and adding a renderer costs exactly one class. Five shapes and four renderers is 20 classes the naive way versus 9 with the Bridge — and the gap widens fast.\nAdapter vs Bridge — A Common Confusion They look structurally similar but serve different purposes:\nAdapter Bridge Retrofit: makes two incompatible interfaces work together after the fact Upfront design: lets two independent hierarchies evolve separately Usually between third-party code and your code Usually within your own design Fixes a mismatch Prevents combinatorial explosion You reach for Adapter because you\u0026rsquo;re forced to. You reach for Bridge because you planned ahead.\nRealistic Use Cases Cross-platform abstractions — e.g., a Window abstraction with WindowsImpl / MacImpl / LinuxImpl implementations. Rendering engines that support multiple backends (as shown). Database-agnostic ORMs — a query abstraction that can delegate to different SQL dialects. Notification systems where messages (sms, email, push) are the abstraction and channels (Twilio, SendGrid) are the implementors. Tradeoffs Pros:\nPrevents combinatorial class explosion. Abstraction and Implementation evolve independently — different teams can own each axis. Supports OCP on both axes. Cons:\nUpfront complexity — two hierarchies and wiring between them. Overkill if you really only have one implementation, or only one abstraction. Related Patterns Adapter — similar structure, different intent (retrofit vs upfront). Strategy — also delegates work to an injected object, but Strategy is about swappable algorithms, Bridge about separating abstraction from implementation hierarchies. 4. Composite The Pain Consider building a file system. You have Files (leaf nodes, contain data) and Directories (can contain Files and other Directories). You want to compute the total size of a thing — whether it\u0026rsquo;s a single File or a whole Directory tree.\nNaively:\ndef total_size(thing): if isinstance(thing, File): return thing.size elif isinstance(thing, Directory): total = 0 for child in thing.children: if isinstance(child, File): total += child.size elif isinstance(child, Directory): total += total_size(child) # Recurse return total The isinstance checks sprawl everywhere. Adding Symlink means updating every such function. The structure\u0026rsquo;s recursion logic is tangled with its type logic.\nIntent Compose objects into tree structures to represent part-whole hierarchies. Composite lets clients treat individual objects and compositions of objects uniformly.\nThe word \u0026ldquo;uniformly\u0026rdquo; is the key. A client can call .size() on anything — a leaf or a branch — and get the right answer, without caring which it is.\nAnalogy Think of a folder on your computer. You right-click → Properties. It tells you \u0026ldquo;Size: 2.4 GB.\u0026rdquo; Whether that folder contains 3 files or 3,000 files in 50 subfolders, you asked one thing and got one answer. You don\u0026rsquo;t manually sum up file sizes.\nOr: an organization chart. \u0026ldquo;How many people report to the CTO?\u0026rdquo; You want a single number. The CTO might have direct reports who are managers, and those managers have their own reports, etc. You don\u0026rsquo;t build different logic per layer — you ask every node, \u0026ldquo;how many under you?\u0026rdquo; and sum up.\nStructure Component — common interface for both leaves and composites (defines .size(), .list(), or whatever the common operation is). Leaf — the individual items (e.g., File). Composite — contains children (Leaves or other Composites) and implements the operation by delegating to children. Python Implementation from abc import ABC, abstractmethod # Component class FileSystemNode(ABC): @abstractmethod def size(self) -\u0026gt; int: pass @abstractmethod def show(self, indent=0): pass # Leaf class File(FileSystemNode): def __init__(self, name, size): self.name = name self._size = size def size(self): # A File is a LEAF: it knows its own size directly. This is the base # case that stops the recursion — no children to descend into. return self._size def show(self, indent=0): print(\u0026#34; \u0026#34; * indent + f\u0026#34;- {self.name} ({self._size} bytes)\u0026#34;) # Composite class Directory(FileSystemNode): def __init__(self, name): self.name = name self.children = [] def add(self, child: FileSystemNode): # `child` is typed as FileSystemNode, so a Directory can hold Files AND # other Directories interchangeably — that uniformity is the pattern. self.children.append(child) return self # chainable def size(self): # The heart of Composite. Directory doesn\u0026#39;t check whether each child is # a File or a Directory — it just calls child.size(). Polymorphism # routes that call to File.size() (returns a number, recursion stops) or # to another Directory.size() (which sums ITS children, recursing # deeper). One line handles a tree of arbitrary depth because both node # types honor the same FileSystemNode.size() contract. return sum(c.size() for c in self.children) def show(self, indent=0): print(\u0026#34; \u0026#34; * indent + f\u0026#34;[DIR] {self.name}/\u0026#34;) for c in self.children: c.show(indent + 2) # Client code — treats everything uniformly root = (Directory(\u0026#34;root\u0026#34;) .add(File(\u0026#34;readme.txt\u0026#34;, 100)) .add(Directory(\u0026#34;src\u0026#34;) .add(File(\u0026#34;main.py\u0026#34;, 500)) .add(File(\u0026#34;utils.py\u0026#34;, 300))) .add(Directory(\u0026#34;docs\u0026#34;) .add(File(\u0026#34;guide.md\u0026#34;, 1000)))) root.show() print(f\u0026#34;Total size: {root.size()} bytes\u0026#34;) Output:\n[DIR] root/ - readme.txt (100 bytes) [DIR] src/ - main.py (500 bytes) - utils.py (300 bytes) [DIR] docs/ - guide.md (1000 bytes) Total size: 1900 bytes Notice: root.size() recursively works through the whole tree. No isinstance, no branching. Each node knows how to compute its own size, and Directory simply sums its children\u0026rsquo;s results.\nRealistic Use Cases File systems. UI component trees — a Panel contains Buttons and other Panels. render() recurses naturally. Menus with submenus — every menu item has a click() or render() that works uniformly. Organization/team hierarchies — query \u0026ldquo;total headcount\u0026rdquo; at any level. Parsed expression trees (2 + (3 * 4)) — leaves are numbers, composites are operators. Bill-of-materials — a product made of sub-products made of parts; total_cost() recursively. Tradeoffs Pros:\nUniform treatment of leaves and composites — simplifies client code enormously. New leaf or composite types plug in cleanly (OCP). Recursion becomes natural and encapsulated. Cons:\nCommon interface may have to include operations irrelevant to some types (e.g., .add(child) on a File — throws, returns nothing, or is absent by design — tension with LSP/ISP). Can make the design \u0026ldquo;too permissive\u0026rdquo; — it\u0026rsquo;s harder to constrain what kinds of things can go where. Related Patterns Decorator — also has a recursive structure (decorator wrapping decorator), but its intent is adding responsibilities, not representing part-whole relationships. Iterator — often combined with Composite to traverse the tree. Visitor — used to add new operations to a Composite structure without modifying the node classes. 5. Decorator The Pain You have a Coffee class. Price: ₹100. Now marketing wants to sell coffee with milk (+₹20), coffee with sugar (+₹10), coffee with milk and sugar, coffee with milk and chocolate, coffee with cream and cinnamon and\u0026hellip;\nThe combinatorics are brutal. You could try inheritance:\nclass Coffee: ... class CoffeeWithMilk(Coffee): ... class CoffeeWithSugar(Coffee): ... class CoffeeWithMilkAndSugar(Coffee): ... Five add-ons = 2^5 = 32 subclasses. This is insane.\nAlternatively, you could add every possible flag to one class:\nclass Coffee: def __init__(self, milk=False, sugar=False, chocolate=False, ...): ... Works but becomes a god class; adding a new add-on means modifying Coffee itself (OCP violation).\nYou need a way to compose responsibilities dynamically at runtime.\nIntent Attach additional responsibilities to an object dynamically. Decorators provide a flexible alternative to subclassing for extending functionality.\nAnalogy Think of wrapping a gift. You have a present (the core object). You wrap it in one paper (one decorator). Then put it in a bag (another decorator). Then attach a bow (another decorator). Each layer adds something (appearance, cost, complexity) but the underlying gift is unchanged. And you can unwrap, rewrap, rearrange freely.\nOr: putting toppings on a pizza. You start with a plain base. Add cheese. Add olives. Add mushrooms. Each topping is a wrapper; you can combine them in any order.\nStructure Component — the common interface (e.g., Beverage with cost()). ConcreteComponent — the core object (Coffee). Decorator — a class that implements Component AND holds a reference to another Component. It forwards requests and adds behavior before/after. ConcreteDecorators — specific decorators (MilkDecorator, SugarDecorator). Python Implementation — Classical Approach from abc import ABC, abstractmethod # Component class Beverage(ABC): @abstractmethod def cost(self): pass @abstractmethod def description(self): pass # ConcreteComponent class Coffee(Beverage): def cost(self): return 100 def description(self): return \u0026#34;Coffee\u0026#34; # Decorator base class BeverageDecorator(Beverage): def __init__(self, wrapped: Beverage): self.wrapped = wrapped # Concrete Decorators class Milk(BeverageDecorator): def cost(self): return self.wrapped.cost() + 20 def description(self): return self.wrapped.description() + \u0026#34;, milk\u0026#34; class Sugar(BeverageDecorator): def cost(self): return self.wrapped.cost() + 10 def description(self): return self.wrapped.description() + \u0026#34;, sugar\u0026#34; class Chocolate(BeverageDecorator): def cost(self): return self.wrapped.cost() + 30 def description(self): return self.wrapped.description() + \u0026#34;, chocolate\u0026#34; # Usage — compose at runtime order = Chocolate(Milk(Sugar(Coffee()))) print(order.description()) # Coffee, sugar, milk, chocolate print(order.cost()) # 100 + 10 + 20 + 30 = 160 # How cost() cascades: you call cost() on the OUTERMOST wrapper first, but each # decorator must ask its inner object before it can add its own price. So the # call travels INWARD to the core, then accumulates on the way back OUT: # Chocolate.cost() # -\u0026gt; Milk.cost() # -\u0026gt; Sugar.cost() # -\u0026gt; Coffee.cost() = 100 (innermost: the real base) # -\u0026gt; 100 + 10 = 110 (Sugar adds on the way back) # -\u0026gt; 110 + 20 = 130 (Milk adds) # -\u0026gt; 130 + 30 = 160 (Chocolate adds last) Each decorator wraps its subordinate, adds its bit to cost() and description(), and forwards to the inner object. The composition is built fresh at runtime — different orders produce different chains.\nAdding a new add-on (say, Cinnamon) requires one new class and zero changes anywhere else.\nPython-Native Decorators Python has @decorator syntax for decorating functions and methods — conceptually related to this pattern, but not identical. The pattern applies to objects (wrapping instances); Python\u0026rsquo;s @decorator applies to functions.\ndef with_logging(func): def wrapper(*args, **kwargs): print(f\u0026#34;Calling {func.__name__}\u0026#34;) result = func(*args, **kwargs) print(f\u0026#34;Done {func.__name__}\u0026#34;) return result return wrapper @with_logging def greet(name): print(f\u0026#34;Hello {name}\u0026#34;) greet(\u0026#34;Amit\u0026#34;) # Calling greet # Hello Amit # Done greet Same idea — wrapping to add behavior — applied at the function level.\nRealistic Use Cases I/O streams — Java\u0026rsquo;s InputStream → BufferedInputStream → GzipInputStream → CipherInputStream is textbook Decorator. Middleware in web frameworks — a request is wrapped by authentication, logging, caching middlewares. Each is a decorator. UI widgets — a TextView decorated with ScrollableDecorator, BorderDecorator, etc. Permissions / access control layers — a PaymentService wrapped by AuditedPaymentService, which wraps FraudCheckingPaymentService. Feature toggles — wrap a service with a decorator that applies behavior only when a flag is on. Tradeoffs Pros:\nDynamic, composable responsibilities at runtime. Far better than subclass explosion. Strongly OCP — new decorators add functionality without touching existing code. Single Responsibility — each decorator does one thing well. Cons:\nDebugging a long chain can be confusing (\u0026ldquo;who added this cost?\u0026rdquo;). Many small decorator classes can fragment logic. Order can matter and be tricky — Milk(Chocolate(Coffee())) vs Chocolate(Milk(Coffee())) may differ subtly in some designs. Related Patterns Composite — also has wrapping structure, but Composite is part-whole, Decorator is adding responsibilities. Strategy — swaps algorithms; Decorator stacks behaviors. Chain of Responsibility — also involves a chain of objects, but each CoR link may stop the chain; decorators always pass through. 6. Facade The Pain You\u0026rsquo;re integrating a home theater system. To watch a movie, you have to:\nprojector.turn_on() projector.set_input(\u0026#34;hdmi-2\u0026#34;) screen.lower() amplifier.turn_on() amplifier.set_volume(25) amplifier.set_source(\u0026#34;dvd\u0026#34;) dvd_player.turn_on() dvd_player.play(\u0026#34;Interstellar\u0026#34;) lights.dim(10) Nine lines of intricate coordination just to watch a movie. Every time. Every caller. And if the vendor updates one device, every caller has to update their 9-line dance.\nWhat you want is a button: \u0026ldquo;Watch Movie.\u0026rdquo; A single entry point.\nIntent Provide a unified, simplified interface to a set of interfaces in a subsystem. Facade defines a higher-level interface that makes the subsystem easier to use.\nAnalogy A hotel concierge. Instead of you calling the kitchen, the transport desk, the ticket office, and housekeeping separately, you call the concierge. They coordinate behind the scenes. You say \u0026ldquo;please arrange a dinner and movie tickets\u0026rdquo;; they handle it.\nOr: car ignition. You turn one key. Behind the scenes: starter motor, fuel pump, ignition coil, ECU, alternator — a whole orchestra. You only see the key.\nStructure Subsystem classes — the existing, complex set of classes. Facade — a new class that exposes a small, convenient API and orchestrates the subsystem internally. Python Implementation # Subsystem classes — complex but unchanged class Projector: def turn_on(self): print(\u0026#34;Projector ON\u0026#34;) def set_input(self, src): print(f\u0026#34;Projector input: {src}\u0026#34;) class Screen: def lower(self): print(\u0026#34;Screen lowered\u0026#34;) def raise_(self): print(\u0026#34;Screen raised\u0026#34;) class Amplifier: def turn_on(self): print(\u0026#34;Amp ON\u0026#34;) def set_volume(self, v): print(f\u0026#34;Volume: {v}\u0026#34;) def set_source(self, src): print(f\u0026#34;Amp source: {src}\u0026#34;) class DVDPlayer: def turn_on(self): print(\u0026#34;DVD ON\u0026#34;) def play(self, movie): print(f\u0026#34;Playing \u0026#39;{movie}\u0026#39;\u0026#34;) def stop(self): print(\u0026#34;DVD stopped\u0026#34;) class Lights: def dim(self, pct): print(f\u0026#34;Lights at {pct}%\u0026#34;) def on(self): print(\u0026#34;Lights ON\u0026#34;) # Facade class HomeTheater: def __init__(self): self.projector = Projector() self.screen = Screen() self.amp = Amplifier() self.dvd = DVDPlayer() self.lights = Lights() def watch_movie(self, movie): self.lights.dim(10) self.screen.lower() self.projector.turn_on() self.projector.set_input(\u0026#34;hdmi-2\u0026#34;) self.amp.turn_on() self.amp.set_source(\u0026#34;dvd\u0026#34;) self.amp.set_volume(25) self.dvd.turn_on() self.dvd.play(movie) def end_movie(self): self.dvd.stop() self.screen.raise_() self.lights.on() # Client code — beautifully simple theater = HomeTheater() theater.watch_movie(\u0026#34;Interstellar\u0026#34;) # ... theater.end_movie() The client\u0026rsquo;s nine-line ritual has become theater.watch_movie(...). The facade owns the complexity.\nFacade Is Not About Hiding — It\u0026rsquo;s About Convenience A subtle point: the subsystem classes are still accessible. A power user can still do theater.amp.set_volume(40) if they need low-level control. Facade isn\u0026rsquo;t a restriction; it\u0026rsquo;s a convenience layer.\nRealistic Use Cases SDKs / libraries — a library\u0026rsquo;s top-level API is usually a Facade over its internal modules. Microservice clients — a UserServiceClient Facade hiding HTTP requests, retries, auth, serialization. Compiler toolchains — compile_file(path) hides lexing + parsing + codegen + linking. Frameworks — Flask\u0026rsquo;s Flask() hides request parsing, routing, response generation. Database ORMs — session.query(User).filter(...).all() is a facade over SQL + connection management. Facade vs Adapter vs Mediator Facade Adapter Mediator Simplify a subsystem Translate one class\u0026rsquo;s interface Coordinate communication between peers New, convenient API Matches an existing expected API Centralizes otherwise tangled peer-to-peer links Tradeoffs Pros:\nMassive simplification for common operations. Decouples callers from subsystem internals. The subsystem can evolve freely as long as the facade\u0026rsquo;s API stays stable. Cons:\nRisk of \u0026ldquo;god facade\u0026rdquo; — it grows to wrap too much, becoming a dumping ground. May hide useful complexity (users might need finer control than the facade exposes). Related Patterns Abstract Factory — often used inside a facade to create subsystem objects. Singleton — a facade is often a singleton, since one per application usually suffices. 7. Flyweight The Pain You\u0026rsquo;re building a text editor. Each character on screen is an object with font, size, color, position. A 100,000-character document = 100,000 character objects. Each one stores its own font metadata. Memory balloons. Performance dies.\nBut notice: most of those characters share the exact same font data. There\u0026rsquo;s no reason each A needs its own copy of \u0026ldquo;Arial, 12pt, black.\u0026rdquo; The variable parts are the position and specific character. The invariant parts (font properties) can be shared.\nYou need a way to share the bulk of data across many objects.\nIntent Use sharing to support large numbers of fine-grained objects efficiently.\nAnalogy Think of a library. Thousands of readers want to read Harry Potter. The library doesn\u0026rsquo;t create a new book for every reader — there\u0026rsquo;s one book that many readers access (perhaps reserving it in turn). The book\u0026rsquo;s content is the shared state; each reader has their own bookmark (their own position).\nOr: playing cards. In 100 games, the rules of the game (shared) don\u0026rsquo;t change; only the specific sequence of cards dealt (extrinsic, per-game). You don\u0026rsquo;t ship a new rulebook per game.\nIntrinsic vs Extrinsic State — The Key Distinction The Flyweight pattern splits object state into two parts:\nIntrinsic state — shared, immutable, context-independent. (Font \u0026ldquo;Arial, 12pt, black\u0026rdquo;.) Extrinsic state — unique per use, passed in from outside when needed. (Position, specific character.) The intrinsic state lives inside a Flyweight object (shared among many contexts). The extrinsic state is not stored — it\u0026rsquo;s passed in as arguments when operations are called.\nStructure Flyweight — object that stores intrinsic state and accepts extrinsic state as parameters. Flyweight Factory — caches flyweights; returns existing ones when possible, creates new ones otherwise. Client — holds extrinsic state and asks the factory for flyweights. Python Implementation class TreeType: \u0026#34;\u0026#34;\u0026#34;Flyweight — shared across many trees in a forest.\u0026#34;\u0026#34;\u0026#34; def __init__(self, name, color, texture): # INTRINSIC state: identical for every tree of this species, so it\u0026#39;s # stored ONCE here and shared. This is the heavy data we refuse to # duplicate millions of times — that\u0026#39;s the entire point of the pattern. self.name = name self.color = color self.texture = texture # Imagine this is 5 MB of image data def render(self, x, y): # x, y are EXTRINSIC — they differ per tree, so they\u0026#39;re NOT stored on # the flyweight. The caller passes them in at render time instead. print(f\u0026#34;Rendering {self.name} ({self.color}) at ({x}, {y})\u0026#34;) class TreeTypeFactory: _types = {} @classmethod def get_tree_type(cls, name, color, texture): key = (name, color, texture) if key not in cls._types: cls._types[key] = TreeType(name, color, texture) return cls._types[key] class Tree: \u0026#34;\u0026#34;\u0026#34;Context — holds extrinsic state (position) + reference to flyweight.\u0026#34;\u0026#34;\u0026#34; def __init__(self, x, y, tree_type: TreeType): # EXTRINSIC state: unique to THIS tree, so it lives here in the context. self.x = x self.y = y # A reference (not a copy) to the shared flyweight. A thousand oaks all # point at the SAME TreeType object — only x/y differ between them. self.type = tree_type def render(self): # Recombine the two halves: pass our extrinsic position into the shared # flyweight, which supplies the intrinsic appearance. self.type.render(self.x, self.y) class Forest: def __init__(self): self.trees = [] def plant_tree(self, x, y, name, color, texture): tree_type = TreeTypeFactory.get_tree_type(name, color, texture) self.trees.append(Tree(x, y, tree_type)) def render(self): for t in self.trees: t.render() forest = Forest() for i in range(1_000_000): forest.plant_tree(i, i, \u0026#34;Oak\u0026#34;, \u0026#34;green\u0026#34;, \u0026#34;oak_texture.png\u0026#34;) for i in range(1_000_000): forest.plant_tree(i, -i, \u0026#34;Pine\u0026#34;, \u0026#34;dark green\u0026#34;, \u0026#34;pine_texture.png\u0026#34;) # Millions of trees, but only TWO TreeType objects in memory print(len(TreeTypeFactory._types)) # 2 Two tree types in memory serve two million trees. The heavy texture data is shared. Positions are stored per-tree because they vary.\nRealistic Use Cases Game engines — particle systems, vegetation, enemy units (thousands of sprites sharing textures). Text editors — characters sharing font data. Chess/Go engines — many pieces sharing piece-type data. Icons in file explorers — one icon object per file type, shared across all files. String interning — Python itself uses flyweight for short strings and small integers internally. Python\u0026rsquo;s Built-in Flyweight-Like Optimizations Python interns small integers and some strings automatically:\na = 5 b = 5 print(a is b) # True — same object, shared x = \u0026#34;hello\u0026#34; y = \u0026#34;hello\u0026#34; print(x is y) # Often True in CPython due to string interning This is the Flyweight idea applied by the runtime.\nTradeoffs Pros:\nMassive memory savings for very large numbers of similar objects. Enables rendering/processing that would otherwise OOM. Cons:\nComplicates the design — splitting intrinsic vs extrinsic state requires care. Flyweights must be immutable (or carefully thread-safe), else shared state becomes a bug. Passing extrinsic state everywhere can be cumbersome. Premature optimization for small datasets — adds complexity without payoff. Only apply when you have genuine scale (thousands+ of similar objects) and memory pressure. For hundreds of objects, don\u0026rsquo;t bother.\nRelated Patterns Singleton — also about sharing, but Singleton = \u0026ldquo;exactly one\u0026rdquo;; Flyweight = \u0026ldquo;share many instances that happen to be identical.\u0026rdquo; Factory — Flyweight Factory is a specific factory design. Composite — often Flyweights are leaves in Composite structures (like characters in a document tree). 8. Proxy The Pain You have an Image class that loads a large file from disk and renders it. In your UI, you may create 50 image references but most are off-screen — the user scrolls past them. Loading all 50 eagerly wastes memory and time.\nOr: you have a RemoteService class that calls a slow network API. You\u0026rsquo;d like to cache results, or check permissions, or log calls — without bloating RemoteService itself.\nYou need something that looks like the real object to the client but interposes extra logic before (or instead of) delegating to the real object.\nIntent Provide a surrogate or placeholder for another object to control access to it.\nAnalogy A debit card is a proxy for your bank account. You hand it to the shopkeeper. They treat it like cash. Behind the scenes: the card communicates with the bank, checks your balance (access control), logs the transaction (logging), maybe converts currency (virtual work). Your account isn\u0026rsquo;t directly accessed, but the final effect is the same.\nOr: a receptionist. You want to see the CEO. The receptionist is the proxy — filters unauthorized visitors, schedules meetings, sometimes answers simple questions without bothering the CEO.\nTypes of Proxy The classical pattern has several flavors:\nVirtual Proxy — defers expensive object creation until actually needed (lazy loading). Protection Proxy — enforces access control. Remote Proxy — represents an object in a different address space (e.g., on another server). Logging/Smart Proxy — adds housekeeping like logging, caching, reference counting. Structure Subject — the common interface the proxy and real object share. RealSubject — the actual object. Proxy — same interface as RealSubject; holds reference to RealSubject; adds pre/post logic. Python Implementation — Virtual Proxy (Lazy Loading) from abc import ABC, abstractmethod class Image(ABC): @abstractmethod def display(self): pass class RealImage(Image): def __init__(self, filename): self.filename = filename self._load_from_disk() def _load_from_disk(self): print(f\u0026#34;Loading {self.filename} from disk... (expensive)\u0026#34;) def display(self): print(f\u0026#34;Displaying {self.filename}\u0026#34;) class ImageProxy(Image): def __init__(self, filename): self.filename = filename self._real_image = None # Not loaded yet def display(self): if self._real_image is None: self._real_image = RealImage(self.filename) # Lazy load self._real_image.display() # Usage img = ImageProxy(\u0026#34;giant_photo.jpg\u0026#34;) # Cheap — no disk I/O yet print(\u0026#34;Proxy created; nothing loaded\u0026#34;) img.display() # NOW loads from disk img.display() # Uses cached RealImage — no reload Python Implementation — Protection Proxy class Document: def __init__(self, content): self.content = content def read(self): return self.content def write(self, content): self.content = content class ProtectedDocument: def __init__(self, document: Document, user_role: str): self._doc = document self._role = user_role def read(self): return self._doc.read() def write(self, content): if self._role != \u0026#34;admin\u0026#34;: raise PermissionError(\u0026#34;Only admins can write\u0026#34;) self._doc.write(content) doc = Document(\u0026#34;Sensitive data\u0026#34;) admin_view = ProtectedDocument(doc, \u0026#34;admin\u0026#34;) guest_view = ProtectedDocument(doc, \u0026#34;guest\u0026#34;) print(guest_view.read()) # OK admin_view.write(\u0026#34;Updated\u0026#34;) # OK # guest_view.write(\u0026#34;Hack\u0026#34;) # PermissionError! Python Implementation — Logging Proxy class LoggingProxy: def __init__(self, wrapped): self._wrapped = wrapped def __getattr__(self, name): \u0026#34;\u0026#34;\u0026#34;Forward any attribute/method access, with logging.\u0026#34;\u0026#34;\u0026#34; # __getattr__ is a Python fallback hook: it runs ONLY when normal # attribute lookup fails — i.e., when `name` isn\u0026#39;t found on the proxy # instance or its class. Since LoggingProxy defines no add/multiply of # its own, EVERY call like calc.add lands here. That\u0026#39;s what lets one # method intercept an unbounded set of calls without us listing them. # (Note: _wrapped IS found normally in __init__, so it never recurses # here — only the wrapped object\u0026#39;s own methods fall through.) attr = getattr(self._wrapped, name) if callable(attr): # If the looked-up attribute is a method, we don\u0026#39;t return it raw — # we return a wrapper closure that logs, calls through, logs again, # and passes the result back. The closure captures `name` and `attr`. def logged(*args, **kwargs): print(f\u0026#34;Calling {name}({args}, {kwargs})\u0026#34;) result = attr(*args, **kwargs) print(f\u0026#34;{name} returned {result!r}\u0026#34;) return result return logged return attr # Plain (non-callable) attributes pass straight through class Calculator: def add(self, a, b): return a + b def multiply(self, a, b): return a * b calc = LoggingProxy(Calculator()) calc.add(2, 3) # Logs call + result calc.multiply(4, 5) # Same This is a dynamic proxy — Python\u0026rsquo;s __getattr__ makes it trivial to intercept all method calls without listing them manually. Very Pythonic. The mechanism is worth internalizing: __getattr__(self, name) is invoked by Python only as a last resort, when an attribute can\u0026rsquo;t be found through the usual channels (the instance dict, then the class, then base classes). Because our proxy deliberately implements none of the wrapped object\u0026rsquo;s methods, they all \u0026ldquo;miss\u0026rdquo; and route through __getattr__, where we forward them. Contrast this with __getattribute__, which intercepts every access (even ones that would succeed) and is far easier to break with infinite recursion — __getattr__ is the safer, more idiomatic choice for a forwarding proxy.\nRealistic Use Cases ORM lazy loading — user.orders doesn\u0026rsquo;t hit the DB until you actually iterate. Remote object access — client-side proxy for a server object (RPC frameworks use this pattern heavily). Access control layers — wrap a service with a permission-checking proxy. Caching proxy — wrap a slow service to cache responses. Mock objects in tests — test doubles are proxies that record calls or return fake responses. Proxy vs Decorator — Close Cousins Both wrap an object with the same interface. The difference is intent:\nDecorator — adds new responsibilities (behavioral extension). Proxy — controls access to the subject (lazy, remote, protected, logged). Structurally they\u0026rsquo;re twins; philosophically they\u0026rsquo;re cousins.\nTradeoffs Pros:\nAdds cross-cutting concerns (logging, caching, security) without bloating the real object. Enables lazy initialization of expensive objects. Respects SRP — the real object focuses on its job; the proxy focuses on control. Cons:\nAdds indirection (may surprise users if proxy silently triggers loading). Multiple layered proxies can obscure behavior (which proxy caught this call?). Can complicate debugging (stack traces go through proxy layers). Related Patterns Decorator — same structure, different intent (extension vs control). Adapter — same structure, different intent (interface change). Facade — simplifies a subsystem, not a single object. 9. Summary All seven structural patterns address the same underlying question: how do we compose objects into flexible structures? But each attacks a distinct problem:\nPattern Problem It Solves Core Mechanism Adapter Interface mismatch between existing classes Translate one interface to another Bridge Combinatorial explosion from two independent axes of variation Separate abstraction from implementation; compose at runtime Composite Treating individual objects and trees of objects uniformly Common interface for leaves and composites Decorator Adding responsibilities dynamically without subclass explosion Wrap an object in layers, each adding behavior Facade Complex subsystem with many moving parts; clients want simple usage A new class exposing a simple API, orchestrating the subsystem Flyweight Too many similar objects consuming too much memory Share intrinsic state; pass extrinsic state as arguments Proxy Need to control access to an object (lazy, remote, secure, logged) Same-interface wrapper that adds control logic Decision Flow Are two components speaking different \u0026ldquo;languages\u0026rdquo;? → Adapter. Am I staring at a class explosion from two independent axes? → Bridge. Do I need to treat single objects and groups the same way? → Composite. Do I want to compose optional behaviors flexibly at runtime? → Decorator. Am I forcing clients to do complex orchestration of multiple classes? → Facade. Do I have millions of similar objects eating memory? → Flyweight. Do I want to control, defer, or monitor access to an object? → Proxy. Key Takeaways Structural patterns are about connecting objects, not creating them. They give you vocabulary for composition. Adapter, Decorator, and Proxy are structural twins with different intents. Adapter changes interface; Decorator extends responsibilities; Proxy controls access. Learn to distinguish by intent, not structure. Composite and Decorator both use recursion. Composite for part-whole; Decorator for wrapping. Facade is often the first pattern you reach for — it\u0026rsquo;s the simplest and addresses one of the most common pains (complex subsystems). Flyweight is niche but crucial when you actually face it. Most systems don\u0026rsquo;t need it; games and editors do. Bridge is preventive, Adapter is corrective. If you\u0026rsquo;re designing from scratch and see two independent axes of variation — use Bridge. If you\u0026rsquo;re integrating legacy or third-party code — Adapter. Python often simplifies these patterns. __getattr__ for Proxy, duck typing for Adapter-lite, decorators (the syntax) relating to Decorator (the pattern), @dataclass for trivial data — all make Python implementations lighter than textbook versions. 10. Practice Exercises Exercise 1 — Adapter You have an existing TemperatureSensor interface used throughout your code:\nclass TemperatureSensor: def get_celsius(self) -\u0026gt; float: ... A new third-party sensor library arrives with this interface (you cannot modify it):\nclass ExternalFahrenheitSensor: def read_fahrenheit(self) -\u0026gt; float: return 98.6 # example Write an adapter that lets you use ExternalFahrenheitSensor wherever TemperatureSensor is expected.\nSolution:\nclass FahrenheitSensorAdapter(TemperatureSensor): def __init__(self, external: ExternalFahrenheitSensor): self._external = external def get_celsius(self) -\u0026gt; float: f = self._external.read_fahrenheit() return (f - 32) * 5 / 9 def display_room_temp(sensor: TemperatureSensor): print(f\u0026#34;Room temperature: {sensor.get_celsius():.1f}°C\u0026#34;) external = ExternalFahrenheitSensor() adapter = FahrenheitSensorAdapter(external) display_room_temp(adapter) # Works transparently The rest of your codebase — any function expecting TemperatureSensor — works unchanged.\nExercise 2 — Bridge Design a Message abstraction that can be sent through different channels. Support these message types: TextMessage, UrgentMessage. Support these channels: EmailChannel, SMSChannel, SlackChannel. Show that any message × any channel combination works without creating N × M classes.\nSolution:\nfrom abc import ABC, abstractmethod # Implementor class Channel(ABC): @abstractmethod def send(self, content: str): pass # Concrete Implementors class EmailChannel(Channel): def send(self, content): print(f\u0026#34;[Email] {content}\u0026#34;) class SMSChannel(Channel): def send(self, content): print(f\u0026#34;[SMS] {content}\u0026#34;) class SlackChannel(Channel): def send(self, content): print(f\u0026#34;[Slack] {content}\u0026#34;) # Abstraction class Message(ABC): def __init__(self, channel: Channel): self.channel = channel @abstractmethod def send(self, text: str): pass # Refined Abstractions class TextMessage(Message): def send(self, text): self.channel.send(text) class UrgentMessage(Message): def send(self, text): self.channel.send(f\u0026#34;🚨 URGENT: {text.upper()} 🚨\u0026#34;) # Mix and match TextMessage(EmailChannel()).send(\u0026#34;Hello there\u0026#34;) UrgentMessage(SMSChannel()).send(\u0026#34;Server is down\u0026#34;) UrgentMessage(SlackChannel()).send(\u0026#34;Page the oncall\u0026#34;) 2 message types × 3 channels = 6 combinations, with only 5 classes in total (plus base abstractions). New channel or new message type = one new class each.\nExercise 3 — Composite Model a company\u0026rsquo;s organizational hierarchy. An Employee has a name and salary. A Manager is an Employee who also manages other employees (or managers). Implement a total_salary_cost() method that works uniformly on any node.\nSolution:\nfrom abc import ABC, abstractmethod class OrgNode(ABC): @abstractmethod def total_salary_cost(self): pass @abstractmethod def describe(self, indent=0): pass class Employee(OrgNode): def __init__(self, name, salary): self.name = name self.salary = salary def total_salary_cost(self): return self.salary def describe(self, indent=0): print(\u0026#34; \u0026#34; * indent + f\u0026#34;- {self.name}: {self.salary}\u0026#34;) class Manager(OrgNode): def __init__(self, name, salary): self.name = name self.salary = salary self.reports = [] def add(self, node: OrgNode): self.reports.append(node) return self def total_salary_cost(self): return self.salary + sum(r.total_salary_cost() for r in self.reports) def describe(self, indent=0): print(\u0026#34; \u0026#34; * indent + f\u0026#34;[MGR] {self.name}: {self.salary}\u0026#34;) for r in self.reports: r.describe(indent + 2) # Build hierarchy cto = (Manager(\u0026#34;CTO\u0026#34;, 500_000) .add(Manager(\u0026#34;Eng Manager\u0026#34;, 200_000) .add(Employee(\u0026#34;Dev 1\u0026#34;, 100_000)) .add(Employee(\u0026#34;Dev 2\u0026#34;, 110_000))) .add(Manager(\u0026#34;QA Manager\u0026#34;, 180_000) .add(Employee(\u0026#34;Tester 1\u0026#34;, 80_000)))) cto.describe() print(f\u0026#34;\\nTotal salary cost under CTO: {cto.total_salary_cost()}\u0026#34;) Same total_salary_cost() call works for an individual employee or an entire subtree.\nExercise 4 — Decorator Design a Notification interface with a send(msg) method. Implement a base EmailNotification. Then add decorators: WithLogging (logs every send), WithRetry (retries up to 3 times on failure), WithRateLimit (adds delay between sends). Compose them.\nSolution:\nimport time from abc import ABC, abstractmethod import random class Notification(ABC): @abstractmethod def send(self, msg): pass class EmailNotification(Notification): def send(self, msg): # Simulate 30% chance of failure if random.random() \u0026lt; 0.3: raise RuntimeError(\u0026#34;Network glitch\u0026#34;) print(f\u0026#34;Email sent: {msg}\u0026#34;) class NotificationDecorator(Notification): def __init__(self, wrapped: Notification): self.wrapped = wrapped class WithLogging(NotificationDecorator): def send(self, msg): print(f\u0026#34;[LOG] Sending: {msg}\u0026#34;) try: self.wrapped.send(msg) print(f\u0026#34;[LOG] Sent successfully\u0026#34;) except Exception as e: print(f\u0026#34;[LOG] Failed: {e}\u0026#34;) raise class WithRetry(NotificationDecorator): def __init__(self, wrapped, attempts=3): super().__init__(wrapped) self.attempts = attempts def send(self, msg): last_exc = None for i in range(self.attempts): try: self.wrapped.send(msg) return except Exception as e: print(f\u0026#34;[RETRY] Attempt {i+1} failed: {e}\u0026#34;) last_exc = e raise last_exc class WithRateLimit(NotificationDecorator): def __init__(self, wrapped, delay_sec=0.1): super().__init__(wrapped) self.delay = delay_sec def send(self, msg): time.sleep(self.delay) self.wrapped.send(msg) # Compose random.seed(7) notifier = WithLogging(WithRetry(WithRateLimit(EmailNotification()), attempts=5)) notifier.send(\u0026#34;Your order has shipped\u0026#34;) Each concern (logging, retry, rate-limiting) lives in its own class, can be stacked in any order, and none of them modify EmailNotification.\nExercise 5 — Facade You have a VideoDownloader, VideoTranscoder, ThumbnailGenerator, S3Uploader, and CDNRegistrar. A user uploading a video needs all five orchestrated in sequence. Write a VideoUploadService facade that exposes a single upload(file_path, title) method.\nSolution:\n# Subsystem class VideoDownloader: def download(self, path): print(f\u0026#34;Reading {path}\u0026#34;); return b\u0026#34;raw_video_bytes\u0026#34; class VideoTranscoder: def transcode(self, raw): print(\u0026#34;Transcoding to H.264...\u0026#34;); return b\u0026#34;transcoded\u0026#34; class ThumbnailGenerator: def generate(self, video): print(\u0026#34;Generating thumbnail...\u0026#34;); return b\u0026#34;thumbnail\u0026#34; class S3Uploader: def upload(self, data, key): print(f\u0026#34;Uploading {key} to S3\u0026#34;) return f\u0026#34;https://s3.example.com/{key}\u0026#34; class CDNRegistrar: def register(self, url, title): print(f\u0026#34;Registered \u0026#39;{title}\u0026#39; → {url} on CDN\u0026#34;) # Facade class VideoUploadService: def __init__(self): self.downloader = VideoDownloader() self.transcoder = VideoTranscoder() self.thumbnailer = ThumbnailGenerator() self.uploader = S3Uploader() self.cdn = CDNRegistrar() def upload(self, file_path, title): raw = self.downloader.download(file_path) transcoded = self.transcoder.transcode(raw) thumb = self.thumbnailer.generate(transcoded) video_url = self.uploader.upload(transcoded, f\u0026#34;videos/{title}.mp4\u0026#34;) self.uploader.upload(thumb, f\u0026#34;thumbs/{title}.jpg\u0026#34;) self.cdn.register(video_url, title) print(f\u0026#34;✓ Upload complete: {title}\u0026#34;) # Client service = VideoUploadService() service.upload(\u0026#34;/tmp/my_video.mov\u0026#34;, \u0026#34;my_first_video\u0026#34;) One-call API for callers; complexity absorbed by the facade.\nExercise 6 — Flyweight Model a forest simulation. You want to render 100,000 trees, but there are only 3 species (Oak, Pine, Birch), each with their own name, color, and texture (imagine heavy data). Position varies per tree. Implement Flyweight so that only 3 species objects live in memory regardless of tree count.\nSolution: (builds on the TreeType example; this exercise asks the reader to extend it)\nclass TreeSpecies: \u0026#34;\u0026#34;\u0026#34;Flyweight — heavy intrinsic state shared across many trees.\u0026#34;\u0026#34;\u0026#34; def __init__(self, name, color, texture_data): self.name = name self.color = color self.texture_data = texture_data # Imagine 10 MB def render(self, x, y, height): print(f\u0026#34;Rendering {self.name} (h={height}) at ({x}, {y})\u0026#34;) class TreeSpeciesFactory: _cache = {} @classmethod def get(cls, name, color, texture_data): if name not in cls._cache: cls._cache[name] = TreeSpecies(name, color, texture_data) return cls._cache[name] @classmethod def count(cls): return len(cls._cache) class Tree: def __init__(self, x, y, height, species: TreeSpecies): self.x = x self.y = y self.height = height self.species = species def render(self): self.species.render(self.x, self.y, self.height) class Forest: def __init__(self): self.trees = [] def plant(self, x, y, height, species_name, color, texture_data): species = TreeSpeciesFactory.get(species_name, color, texture_data) self.trees.append(Tree(x, y, height, species)) # Plant 100,000 trees import random random.seed(1) species_defs = [ (\u0026#34;Oak\u0026#34;, \u0026#34;green\u0026#34;, b\u0026#34;oak_texture\u0026#34;), (\u0026#34;Pine\u0026#34;, \u0026#34;dark green\u0026#34;, b\u0026#34;pine_texture\u0026#34;), (\u0026#34;Birch\u0026#34;, \u0026#34;light yellow\u0026#34;, b\u0026#34;birch_texture\u0026#34;), ] forest = Forest() for _ in range(100_000): name, color, tex = random.choice(species_defs) forest.plant(random.randint(0, 1000), random.randint(0, 1000), random.randint(10, 50), name, color, tex) print(f\u0026#34;Trees planted: {len(forest.trees)}\u0026#34;) print(f\u0026#34;Species objects in memory: {TreeSpeciesFactory.count()}\u0026#34;) # Trees planted: 100000 # Species objects in memory: 3 100,000 trees, 3 species objects. Memory stays flat.\nExercise 7 — Proxy Create a RemoteUserAPI with a get_user(user_id) method that (simulates) a 1-second network call. Write a CachingProxy that caches results for 10 seconds, and a LoggingProxy that logs each call. Stack them.\nSolution:\nimport time from abc import ABC, abstractmethod class UserAPI(ABC): @abstractmethod def get_user(self, user_id): pass class RemoteUserAPI(UserAPI): def get_user(self, user_id): time.sleep(1) # Simulated network return {\u0026#34;id\u0026#34;: user_id, \u0026#34;name\u0026#34;: f\u0026#34;User-{user_id}\u0026#34;} class CachingProxy(UserAPI): def __init__(self, wrapped: UserAPI, ttl_sec=10): self.wrapped = wrapped self.ttl = ttl_sec self.cache = {} # user_id -\u0026gt; (user, timestamp) def get_user(self, user_id): now = time.time() if user_id in self.cache: user, ts = self.cache[user_id] if now - ts \u0026lt; self.ttl: return user user = self.wrapped.get_user(user_id) self.cache[user_id] = (user, now) return user class LoggingProxy(UserAPI): def __init__(self, wrapped: UserAPI): self.wrapped = wrapped def get_user(self, user_id): print(f\u0026#34;[LOG] get_user({user_id}) called\u0026#34;) start = time.time() result = self.wrapped.get_user(user_id) dur = time.time() - start print(f\u0026#34;[LOG] returned in {dur:.3f}s\u0026#34;) return result # Usage — stack the proxies api = LoggingProxy(CachingProxy(RemoteUserAPI(), ttl_sec=10)) api.get_user(42) # Slow (1s); caches; logs api.get_user(42) # Fast (cached); logs api.get_user(99) # Slow (1s); new user; caches; logs Caching and logging are both cross-cutting concerns added as proxies without touching RemoteUserAPI.\nExercise 8 — Capstone (Multi-Pattern) Design an image-processing pipeline that:\nUses a Facade (ImageProcessor.process(file)) to orchestrate the pipeline. Internally uses a Composite of filters (a pipeline containing individual filters or sub-pipelines). Each filter is wrapped in a Decorator that logs execution time. The image loader is a Proxy that caches already-loaded images by path. Outline the classes and show one working example.\nSolution (sketch):\nimport time from abc import ABC, abstractmethod # --- Image and filters --- class Image: def __init__(self, path): self.path = path self.data = f\u0026#34;\u0026lt;pixels of {path}\u0026gt;\u0026#34; def __repr__(self): return f\u0026#34;Image({self.path}, data={self.data})\u0026#34; class Filter(ABC): @abstractmethod def apply(self, image: Image) -\u0026gt; Image: pass class GrayscaleFilter(Filter): def apply(self, image): image.data = f\u0026#34;gray({image.data})\u0026#34; return image class BlurFilter(Filter): def apply(self, image): image.data = f\u0026#34;blur({image.data})\u0026#34; return image # --- Composite (pipeline of filters) --- class FilterPipeline(Filter): def __init__(self): self.filters = [] def add(self, f: Filter): self.filters.append(f) return self def apply(self, image): for f in self.filters: image = f.apply(image) return image # --- Decorator (timing) --- class TimedFilter(Filter): def __init__(self, wrapped: Filter): self.wrapped = wrapped def apply(self, image): start = time.time() result = self.wrapped.apply(image) dur = (time.time() - start) * 1000 print(f\u0026#34;{self.wrapped.__class__.__name__} took {dur:.2f}ms\u0026#34;) return result # --- Proxy (image loader with caching) --- class ImageLoader(ABC): @abstractmethod def load(self, path) -\u0026gt; Image: pass class RealImageLoader(ImageLoader): def load(self, path): print(f\u0026#34;Loading {path} from disk...\u0026#34;) time.sleep(0.1) # Simulate I/O return Image(path) class CachingImageLoader(ImageLoader): def __init__(self, wrapped: ImageLoader): self.wrapped = wrapped self.cache = {} def load(self, path): if path not in self.cache: self.cache[path] = self.wrapped.load(path) else: print(f\u0026#34;Cache hit for {path}\u0026#34;) return self.cache[path] # --- Facade --- class ImageProcessor: def __init__(self): self.loader = CachingImageLoader(RealImageLoader()) self.pipeline = (FilterPipeline() .add(TimedFilter(GrayscaleFilter())) .add(TimedFilter(BlurFilter()))) def process(self, path): image = self.loader.load(path) return self.pipeline.apply(image) # --- Usage --- processor = ImageProcessor() result1 = processor.process(\u0026#34;photo.jpg\u0026#34;) # Loads + filters result2 = processor.process(\u0026#34;photo.jpg\u0026#34;) # Cache hit print(result2) Patterns identified:\nImageProcessor → Facade. FilterPipeline → Composite (itself a Filter; contains Filters). TimedFilter → Decorator. CachingImageLoader → Proxy (virtual/caching). 11. Self-Assessment Test Attempt each before revealing the answer.\nQ1. Adapter and Bridge look structurally similar. What\u0026rsquo;s the intent-level difference?\nAnswer: Adapter is retrofit — you use it when two existing components have incompatible interfaces and you need them to work together. Bridge is upfront design — you use it when you anticipate two independent axes of variation and want to prevent a combinatorial class explosion. Adapter fixes a mismatch; Bridge prevents one.\nQ2. What\u0026rsquo;s the key property that Composite provides to clients?\nAnswer: Uniform treatment of leaf nodes and composite nodes. Clients can call the same method on either without needing to know which they have. This dramatically simplifies code that traverses or operates on tree-like structures.\nQ3. Without using the word \u0026ldquo;Decorator,\u0026rdquo; describe why the pattern exists.\nAnswer: To let you add responsibilities to an object dynamically, at runtime, in any combination — without exploding into 2^N subclasses for every combination of optional behaviors. It replaces inheritance-based extension with composition-based extension.\nQ4. What\u0026rsquo;s the difference between Decorator and Proxy, given they have nearly identical structure?\nAnswer: Decorator adds behavior (extends responsibilities); Proxy controls access (lazy loading, remote access, permission checks, caching). Decorator is \u0026ldquo;and also does X\u0026rdquo;; Proxy is \u0026ldquo;is the gatekeeper to X.\u0026rdquo; Same scaffolding, different intent.\nQ5. Under what circumstances is Flyweight worth the complexity?\nAnswer: When you have a very large number of similar objects (thousands to millions) and memory is a bottleneck, AND those objects share a substantial amount of state that can be separated into intrinsic (shared) and extrinsic (per-context) parts. For hundreds of objects, the pattern is usually overkill.\nQ6. What does Facade not do?\nAnswer: Facade doesn\u0026rsquo;t hide the subsystem or restrict access to it. The subsystem classes remain usable by anyone who wants fine-grained control. Facade is a convenience layer — a simple default for common operations — not an enforcement mechanism.\nQ7. A Tree class has a method add_child(c) and a Leaf class in your Composite doesn\u0026rsquo;t support it. How do you handle this?\nAnswer: Options:\nInclude add_child in the Component interface but have Leaf throw an exception (or silently ignore). Simpler client code, but weakens LSP. Keep add_child only on Composite (not in the Component interface). Stronger design; client code may need to check before calling — but often this check isn\u0026rsquo;t needed because client code operates through the common interface. The \u0026ldquo;right\u0026rdquo; choice is design-dependent. The GoF book explicitly notes this is a trade-off between safety and transparency.\nQ8. Does the @decorator syntax in Python implement the Decorator pattern?\nAnswer: Related but not identical. The pattern applies to objects (wrapping instances). Python\u0026rsquo;s @decorator syntax wraps functions. The underlying idea — wrap to add behavior — is the same, but the pattern as GoF describes it is about object composition, while Python\u0026rsquo;s @decorator operates at function level. Both are legitimate uses of the word \u0026ldquo;decorator.\u0026rdquo;\nQ9. You have three layered proxies: LoggingProxy(CachingProxy(RateLimitProxy(Real()))). A call is made. In what order do the proxies execute?\nAnswer: Outside-in on the way in; inside-out on the way out.\nCall: Logging → Caching → RateLimit → Real. Return: Real → RateLimit → Caching → Logging. Each proxy runs its pre-logic, delegates inward, then runs its post-logic.\nQ10. Facade, Adapter, and Proxy all wrap things. Tell them apart in one sentence each.\nAnswer:\nFacade — wraps an entire subsystem to simplify it. Adapter — wraps one object to make its interface compatible with something else. Proxy — wraps one object to control access to it (lazy, remote, secure, logged). Q11. Why is it important that Flyweight objects be immutable?\nAnswer: Because Flyweights are shared across many contexts. If one client mutates a Flyweight, every other client sharing it sees the change — leading to very subtle, very hard-to-debug bugs. Immutability eliminates this hazard. Practical implementation: make intrinsic state read-only after construction.\nQ12. A junior dev has written a \u0026ldquo;Facade\u0026rdquo; that now has 50 methods and is called by 30 modules. Is this healthy?\nAnswer: Probably not. A Facade that grows too large becomes a god object — a dumping ground of \u0026ldquo;every convenience we ever wanted.\u0026rdquo; Signs it\u0026rsquo;s gone wrong: methods serve very different purposes, many clients use only tiny subsets, and changes ripple through many consumers. Consider splitting it into multiple focused facades by domain (e.g., UserFacade, OrderFacade, ReportingFacade).\nYou\u0026rsquo;ve now mastered the Structural patterns. The common thread: how do we connect objects without locking ourselves into rigid, inflexible structures? Each pattern offers a different composition technique — translation, bridging, trees, wrapping, simplification, sharing, and control.\nNext up: Behavioral Patterns — the final and largest family. These patterns focus on how objects interact and distribute responsibility: Chain of Responsibility, Command, Interpreter, Iterator, Mediator, Memento, Observer, State, Strategy, Template Method, and Visitor.\nThis content is part of Codeverra — a platform for learning coding, data science, DSA, and AI from scratch. Explore more: https://codeverra.com\n","permalink":"https://blogs.codeverra.com/blog/low_level_design/04_design_patterns_structural/","summary":"\u003ch1 id=\"design-patterns--part-2-structural-patterns\"\u003eDesign Patterns — Part 2: Structural Patterns\u003c/h1\u003e\n\u003cblockquote\u003e\n\u003cp\u003e\u003cstrong\u003eWho this is for:\u003c/strong\u003e Anyone who has worked through the Creational patterns and is ready to learn how objects fit together — how to combine classes and objects into larger structures that stay flexible and maintainable.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003chr\u003e\n\u003ch2 id=\"table-of-contents\"\u003eTable of Contents\u003c/h2\u003e\n\u003col\u003e\n\u003cli\u003e\u003ca href=\"#1-structural-patterns\"\u003eStructural Patterns — What and Why\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#2-adapter\"\u003eAdapter\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#3-bridge\"\u003eBridge\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#4-composite\"\u003eComposite\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#5-decorator\"\u003eDecorator\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#6-facade\"\u003eFacade\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#7-flyweight\"\u003eFlyweight\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#8-proxy\"\u003eProxy\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#9-summary\"\u003eSummary of Structural Patterns\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#10-practice-exercises\"\u003ePractice Exercises with Solutions\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#11-self-assessment-test\"\u003eSelf-Assessment Test\u003c/a\u003e\u003c/li\u003e\n\u003c/ol\u003e\n\u003chr\u003e\n\u003ch2 id=\"1-structural-patterns\"\u003e1. Structural Patterns\u003c/h2\u003e\n\u003cp\u003eCreational patterns answered: \u003cem\u003e\u0026ldquo;how do we make objects flexibly?\u0026rdquo;\u003c/em\u003e\nStructural patterns answer: \u003cem\u003e\u0026ldquo;how do we combine objects into larger structures flexibly?\u0026rdquo;\u003c/em\u003e\u003c/p\u003e","tags":["lld","design-patterns","python"],"title":"Design Patterns, Part 2: Structural Patterns"},{"content":"Design Patterns — Part 3: Behavioral Patterns Who this is for: Anyone who has worked through Creational and Structural patterns and is ready for the final, largest, and arguably most important family — the patterns that govern how objects communicate and divide responsibility.\nTable of Contents Behavioral Patterns — What and Why Chain of Responsibility Command Interpreter Iterator Mediator Memento Observer State Strategy Template Method Visitor Summary of Behavioral Patterns Practice Exercises with Solutions Self-Assessment Test Final Words — All 23 Patterns at a Glance 1. Behavioral Patterns Creational patterns answered \u0026ldquo;how do we make objects?\u0026rdquo; Structural patterns answered \u0026ldquo;how do we glue objects together?\u0026rdquo; Behavioral patterns answer: \u0026ldquo;how do objects talk to each other, and who is responsible for what?\u0026rdquo;\nIn any system of more than a few classes, objects collaborate constantly. Poor collaboration shows up as:\nTangled, bidirectional references (class A knows about B, which knows about C, which calls back into A). Long if/elif chains selecting behavior based on state or type. Duplicate algorithms that differ only in small steps. Tight coupling between \u0026ldquo;who triggers what\u0026rdquo; and \u0026ldquo;what gets done.\u0026rdquo; Behavioral patterns give us refined vocabulary for these collaborations:\nChain of Responsibility — pass a request along a chain of handlers until one handles it. Command — encapsulate a request as an object (so you can queue, log, undo, or schedule it). Interpreter — define a grammar and evaluate sentences in it. Iterator — access elements of a collection sequentially without exposing its internals. Mediator — let objects communicate through a central mediator instead of directly. Memento — capture an object\u0026rsquo;s internal state so it can be restored later. Observer — notify interested parties when something changes (pub/sub). State — change an object\u0026rsquo;s behavior when its internal state changes. Strategy — make an algorithm interchangeable with others of the same kind. Template Method — define an algorithm\u0026rsquo;s skeleton; let subclasses fill in steps. Visitor — add new operations to an object structure without modifying the objects. Eleven patterns. Each is a specific answer to a specific collaboration pain. Let\u0026rsquo;s dive in.\n2. Chain of Responsibility The Pain Imagine processing a customer support ticket. It needs to pass through:\nAuto-responder — can it handle common FAQs? Level 1 support — basic troubleshooting. Level 2 support — deeper technical issues. Engineering — bugs requiring code changes. Naive implementation:\ndef handle_ticket(ticket): if can_auto_respond(ticket): auto_respond(ticket) elif can_L1_handle(ticket): L1_handle(ticket) elif can_L2_handle(ticket): L2_handle(ticket) else: escalate_to_engineering(ticket) Problems:\nAdding a new handler (say, a chatbot between auto-responder and L1) requires modifying this function. The order of handlers is hardcoded. The function \u0026ldquo;knows\u0026rdquo; about all handlers — a central bottleneck. Testing requires mocking the entire chain. Intent Avoid coupling the sender of a request to its receiver by giving more than one object a chance to handle the request. Chain the receiving objects and pass the request along the chain until an object handles it.\nAnalogy Think of a customer complaint at a retail store. You start with the cashier. Can\u0026rsquo;t help? Shift lead. Can\u0026rsquo;t help? Manager. Still stuck? Regional manager. Each person either resolves it or passes it up. You (the complainer) don\u0026rsquo;t need to know the org chart — you just hand your complaint to the next person in the chain.\nOr: air traffic control handoffs. An aircraft gets passed from ground control → tower → departure → center → approach → tower → ground. Each controller handles their segment or hands off.\nStructure Handler — interface declaring handle(request) and optionally a link to the next handler. ConcreteHandlers — each decides whether to handle or forward. Client — hands the request to the first handler; doesn\u0026rsquo;t know the chain\u0026rsquo;s length. Python Implementation from abc import ABC, abstractmethod class SupportHandler(ABC): def __init__(self): self._next = None def set_next(self, handler): self._next = handler return handler # Allow chaining: h1.set_next(h2).set_next(h3) def handle(self, ticket): if self._next: return self._next.handle(ticket) print(f\u0026#34;No handler for ticket: {ticket}\u0026#34;) class AutoResponder(SupportHandler): FAQ = {\u0026#34;reset password\u0026#34;, \u0026#34;pricing\u0026#34;, \u0026#34;hours\u0026#34;} def handle(self, ticket): if ticket[\u0026#34;issue\u0026#34;].lower() in self.FAQ: print(f\u0026#34;Auto-responder handled: {ticket[\u0026#39;issue\u0026#39;]}\u0026#34;) else: super().handle(ticket) class L1Support(SupportHandler): def handle(self, ticket): if ticket[\u0026#34;priority\u0026#34;] \u0026lt;= 2: print(f\u0026#34;L1 handled: {ticket[\u0026#39;issue\u0026#39;]}\u0026#34;) else: super().handle(ticket) class L2Support(SupportHandler): def handle(self, ticket): if ticket[\u0026#34;priority\u0026#34;] \u0026lt;= 4: print(f\u0026#34;L2 handled: {ticket[\u0026#39;issue\u0026#39;]}\u0026#34;) else: super().handle(ticket) class Engineering(SupportHandler): def handle(self, ticket): print(f\u0026#34;Engineering escalated: {ticket[\u0026#39;issue\u0026#39;]}\u0026#34;) # Build the chain auto = AutoResponder() l1 = L1Support() l2 = L2Support() eng = Engineering() auto.set_next(l1).set_next(l2).set_next(eng) # Send requests auto.handle({\u0026#34;issue\u0026#34;: \u0026#34;reset password\u0026#34;, \u0026#34;priority\u0026#34;: 1}) auto.handle({\u0026#34;issue\u0026#34;: \u0026#34;login broken\u0026#34;, \u0026#34;priority\u0026#34;: 2}) auto.handle({\u0026#34;issue\u0026#34;: \u0026#34;data corruption bug\u0026#34;, \u0026#34;priority\u0026#34;: 5}) Output:\nAuto-responder handled: reset password L1 handled: login broken Engineering escalated: data corruption bug Key property: the client just calls auto.handle(ticket). The client doesn\u0026rsquo;t know what the chain looks like or who will handle it. Reordering, inserting, or removing handlers changes nothing for the client.\nRealistic Use Cases Web middleware (authentication → authorization → logging → rate limiting → route handler). Event handling in UI frameworks — events bubble up the widget tree; each widget can handle or forward. Logging frameworks — handlers of different severities in a chain. Approval workflows — expense approval (manager → director → VP → CFO). Validation pipelines — each validator checks its concern and passes the request on. Tradeoffs Pros:\nDecouples sender from receivers — sender doesn\u0026rsquo;t know who handles what. OCP-friendly — add new handlers without touching existing ones. Dynamic — you can reconfigure the chain at runtime. Cons:\nNo guarantee any handler handles the request — it may fall off the end. Debugging can be tricky — who handled this? Why didn\u0026rsquo;t handler X catch it? Performance overhead if the chain is long and the winning handler is near the end. Related Patterns Decorator — also chains, but every decorator handles the request; in CoR, exactly one handles (typically). Command — often used together; Commands can be passed through a CoR. Composite — handlers themselves may form a tree structure. 3. Command The Pain You\u0026rsquo;re building a text editor. A user can cut, copy, paste, bold, italic, etc. Naively, every button calls the corresponding method directly:\ncut_button.on_click = lambda: editor.cut() paste_button.on_click = lambda: editor.paste() Now the product team wants:\nUndo (with history). Redo. Keyboard shortcut customization (user remaps Ctrl+X to \u0026ldquo;bold\u0026rdquo;). Macro recording — replay a sequence of actions. Queued operations — run this later / in background. Each of these requires treating operations as first-class things you can store, inspect, reverse, and re-execute. A direct method call is not a thing — it\u0026rsquo;s a transient event. You need to turn actions into objects.\nIntent Encapsulate a request as an object, thereby letting you parameterize clients with different requests, queue or log requests, and support undoable operations.\nAnalogy A restaurant order ticket. When you order at a restaurant, the waiter writes your order on a ticket. That ticket is a physical object. It can be passed to the kitchen, queued, logged, re-read, retried, even canceled. The waiter isn\u0026rsquo;t directly yelling \u0026ldquo;make a pizza!\u0026rdquo; to the chef — there\u0026rsquo;s an intermediating object (the ticket) that carries the request.\nOr: a TV remote. Each button holds a command. The remote doesn\u0026rsquo;t know how the TV changes channels; it just issues the \u0026ldquo;channel up\u0026rdquo; command. The TV executes it.\nStructure Command — interface with execute() (and optionally undo()). ConcreteCommand — binds a specific action to a receiver, with any needed parameters stored. Receiver — the object that actually does the work. Invoker — holds commands and triggers execute() (e.g., a button, a queue, a script). Client — creates ConcreteCommands and assigns them to Invokers. Python Implementation from abc import ABC, abstractmethod class Command(ABC): @abstractmethod def execute(self): pass @abstractmethod def undo(self): pass class Editor: def __init__(self): self.text = \u0026#34;\u0026#34; def write(self, s): self.text += s def delete_last(self, n): self.text = self.text[:-n] class WriteCommand(Command): def __init__(self, editor: Editor, text: str): self.editor = editor self.text = text def execute(self): self.editor.write(self.text) def undo(self): self.editor.delete_last(len(self.text)) class EditorApp: def __init__(self): self.editor = Editor() self.history = [] self.redo_stack = [] def run(self, command: Command): command.execute() self.history.append(command) self.redo_stack.clear() # New action invalidates redo def undo(self): if not self.history: return cmd = self.history.pop() cmd.undo() self.redo_stack.append(cmd) def redo(self): if not self.redo_stack: return cmd = self.redo_stack.pop() cmd.execute() self.history.append(cmd) # Usage app = EditorApp() app.run(WriteCommand(app.editor, \u0026#34;Hello \u0026#34;)) app.run(WriteCommand(app.editor, \u0026#34;World\u0026#34;)) print(app.editor.text) # Hello World app.undo() print(app.editor.text) # Hello app.undo() print(app.editor.text) # (empty) app.redo() print(app.editor.text) # Hello Each user action becomes a Command object. The app keeps a history list. Undo pops the last command and calls undo(). Redo pushes it back.\nRealistic Use Cases Undo/redo systems (editors, IDEs, graphic tools). Transaction systems — each transaction is a command, replayable, rollbackable. Task queues (Celery, Sidekiq) — jobs are commands serialized and executed asynchronously. Macro recording — record a sequence of commands, replay later. Smart home automation — \u0026ldquo;when I say \u0026lsquo;movie night,\u0026rsquo; run these 5 commands.\u0026rdquo; GUI frameworks — menu items, buttons, and keyboard shortcuts all bound to Command objects. Tradeoffs Pros:\nUndo/redo becomes trivial (well-designed commands are reversible). Actions become storable, queueable, schedulable, loggable. Decouples invoker (button) from receiver (business logic). Macros/compositions of commands = composite Commands. Cons:\nMore classes — a command per action. Can feel heavy for simple apps. Undo logic is sometimes genuinely hard (think: undoing a DB DELETE that also cascaded). Memory usage — history lists can grow large; often need pruning. Related Patterns Composite — MacroCommand is a Composite of Commands. Memento — often used inside Command to save state for undo. Chain of Responsibility — Commands can be passed along CoR handlers. 4. Interpreter The Pain You have a domain-specific language — maybe a filter expression like \u0026quot;age \u0026gt; 30 AND country == 'IN'\u0026quot;, or a math formula, or a search query syntax. You need to evaluate these expressions repeatedly for different inputs.\nNaive approach: write a huge parser + evaluator tangled with if/elif on every token. The code becomes an unreadable mess, and every new grammar rule requires invasive changes.\nIntent Given a language, define a representation for its grammar along with an interpreter that uses the representation to interpret sentences in the language.\nThe idea: model each grammar rule as a class. Build an expression tree (AST) of these classes. Evaluate recursively.\nAnalogy Think of sentence diagramming in English class. A sentence like \u0026ldquo;The cat sat on the mat\u0026rdquo; is broken down: \u0026lt;sentence\u0026gt; = \u0026lt;subject\u0026gt; \u0026lt;verb\u0026gt; \u0026lt;prepositional_phrase\u0026gt;. Each piece is a rule; together they form a tree. To \u0026ldquo;understand\u0026rdquo; the sentence, you interpret each piece.\nThe Interpreter pattern encodes such grammar-as-classes literally.\nStructure AbstractExpression — interface with interpret(context). TerminalExpression — leaf; represents a literal or variable. NonterminalExpression — represents grammar rules composed of other expressions (And, Or, Greater, etc.). Context — holds the data against which expressions are evaluated. Python Implementation — A Tiny Boolean Filter Language Let\u0026rsquo;s build interpreter for: (age \u0026gt; 18) AND (country == \u0026quot;IN\u0026quot;)\nThe single most important idea in this pattern: each production rule of the grammar becomes a class. A grammar for our filter language might read:\nexpression := variable | literal | comparison | conjunction comparison := expression \u0026#34;\u0026gt;\u0026#34; expression | expression \u0026#34;==\u0026#34; expression conjunction := expression \u0026#34;AND\u0026#34; expression Map that directly onto classes. Terminals (Variable, Literal) are the leaves of the tree — they resolve to a value with no further sub-expressions. Non-terminals (GreaterThan, Equals, And) are the internal nodes — they combine sub-expressions and recurse into them. Every class implements the same interpret(context) method, so the whole tree evaluates by uniform recursion (this is Composite quietly doing the structural work underneath Interpreter).\nfrom abc import ABC, abstractmethod class Expression(ABC): # The single operation every grammar element must support. Note interpret() # takes the `context` (the data we\u0026#39;re evaluating against) — the SAME tree is # reused across many contexts, which is the payoff of building it once. @abstractmethod def interpret(self, context: dict): pass # Terminals — leaves of the AST. They bottom out the recursion: no children. class Variable(Expression): # Terminal: looks its name up in the context. The \u0026#34;age\u0026#34; in (age \u0026gt; 18). def __init__(self, name): self.name = name def interpret(self, context): return context[self.name] class Literal(Expression): # Terminal: a constant value baked into the rule. The 18 in (age \u0026gt; 18). def __init__(self, value): self.value = value def interpret(self, context): return self.value # Nonterminals — internal nodes. Each holds sub-expressions and recurses. class GreaterThan(Expression): # Nonterminal for the \u0026#34;\u0026gt;\u0026#34; production. It doesn\u0026#39;t know if left/right are # variables, literals, or whole sub-trees — it just calls interpret() on # them and compares. That uniformity is what makes the grammar composable. def __init__(self, left: Expression, right: Expression): self.left = left self.right = right def interpret(self, ctx): return self.left.interpret(ctx) \u0026gt; self.right.interpret(ctx) class Equals(Expression): # Nonterminal for the \u0026#34;==\u0026#34; production — same recursive shape as GreaterThan. def __init__(self, left, right): self.left = left self.right = right def interpret(self, ctx): return self.left.interpret(ctx) == self.right.interpret(ctx) class And(Expression): # Nonterminal for \u0026#34;AND\u0026#34; — combines two boolean sub-expressions. Because each # side is itself an Expression, And can sit atop arbitrarily deep trees. def __init__(self, left, right): self.left = left self.right = right def interpret(self, ctx): return self.left.interpret(ctx) and self.right.interpret(ctx) # Build: (age \u0026gt; 18) AND (country == \u0026#34;IN\u0026#34;) # This nested construction IS the abstract syntax tree (AST). The shape of the # object graph mirrors the shape of the expression exactly. expr = And( GreaterThan(Variable(\u0026#34;age\u0026#34;), Literal(18)), Equals(Variable(\u0026#34;country\u0026#34;), Literal(\u0026#34;IN\u0026#34;)) ) # Evaluate for different contexts print(expr.interpret({\u0026#34;age\u0026#34;: 25, \u0026#34;country\u0026#34;: \u0026#34;IN\u0026#34;})) # True print(expr.interpret({\u0026#34;age\u0026#34;: 15, \u0026#34;country\u0026#34;: \u0026#34;IN\u0026#34;})) # False print(expr.interpret({\u0026#34;age\u0026#34;: 40, \u0026#34;country\u0026#34;: \u0026#34;US\u0026#34;})) # False Each grammar rule is a small class. The expression tree is built once and evaluated many times for different data contexts.\nRealistic Use Cases Rule engines — business rules expressed as parse trees. SQL query planners — SQL statements become expression trees internally. Regular expression engines — a regex is compiled into an expression tree, interpreted against strings. Template engines — parse templates into trees of nodes (text, variables, loops), interpret against data. Scripting languages embedded in applications (e.g., formula cells in spreadsheets). Tradeoffs Pros:\nExtending the grammar is easy — one new class per rule. Reusability — the same AST can be evaluated many times, cached, optimized. Clean mapping between grammar and code structure. Cons:\nGrammars of even moderate size balloon into many classes. Performance — pure interpretation is slow compared to compiled code. Often, real systems use Interpreter for prototyping then compile to bytecode/native. Parsing (turning text into an expression tree) is a separate problem; Interpreter only addresses evaluation of an already-built tree. Python-Specific Notes Python\u0026rsquo;s ast module and eval() can sometimes replace Interpreter entirely for simple cases — but at the cost of security (eval runs arbitrary code). Libraries like lark, pyparsing, and sympy often give you parser + interpreter functionality far beyond what you\u0026rsquo;d hand-roll.\nInterpreter is the least-used GoF pattern in modern Python code — but understanding it helps you read and build grammar-driven systems.\nRelated Patterns Composite — the AST is a Composite tree. Visitor — often used to traverse the AST and perform operations like type-checking, optimization, or compilation. Iterator — used to traverse AST nodes. 5. Iterator The Pain You have various collections — a list, a tree, a graph, a paginated API, a streaming file. You want to traverse each one. But each has its own internal structure.\nIf clients have to know the internal structure of every collection to iterate it — tree traversal algorithm for trees, cursor API for DB, batch pagination for APIs — your code is a mess.\nYou need a uniform way to step through elements, regardless of what\u0026rsquo;s inside.\nIntent Provide a way to access the elements of an aggregate object sequentially without exposing its underlying representation.\nAnalogy A book. You read page 1, then page 2, then page 3. You don\u0026rsquo;t need to know whether pages are printed with offset lithography, stored in an ebook\u0026rsquo;s EPUB file, or rendered from a PDF. You just turn_page(). The iterator (your thumb, or the \u0026ldquo;next page\u0026rdquo; button) abstracts the traversal.\nOr: a TV remote\u0026rsquo;s channel up/down button. The TV has hundreds of channels stored internally; you don\u0026rsquo;t care how. You press \u0026ldquo;up,\u0026rdquo; it gives you the next one.\nStructure Iterator — interface with __next__ and __iter__ in Python (or hasNext() / next() in other languages). ConcreteIterator — implementation that traverses a specific collection. Aggregate — the collection that returns an Iterator. Python Implementation Python has first-class built-in iterator support. Any object implementing __iter__ (returning self or another iterator) and __next__ is iterable.\nclass TreeNode: def __init__(self, value, children=None): self.value = value self.children = children or [] class PreorderIterator: def __init__(self, root: TreeNode): self.stack = [root] def __iter__(self): return self def __next__(self): if not self.stack: raise StopIteration node = self.stack.pop() # Push children in reverse so leftmost is processed first self.stack.extend(reversed(node.children)) return node.value class Tree: def __init__(self, root): self.root = root def __iter__(self): return PreorderIterator(self.root) tree = Tree(TreeNode(\u0026#34;A\u0026#34;, [ TreeNode(\u0026#34;B\u0026#34;, [TreeNode(\u0026#34;D\u0026#34;), TreeNode(\u0026#34;E\u0026#34;)]), TreeNode(\u0026#34;C\u0026#34;, [TreeNode(\u0026#34;F\u0026#34;)]), ])) for v in tree: print(v, end=\u0026#34; \u0026#34;) # A B D E C F The client does for v in tree — no idea whether the tree is stored flat, recursively, on disk, or streamed.\nGenerator Functions — Python\u0026rsquo;s Iterator Superpower Python\u0026rsquo;s yield keyword makes writing iterators dramatically simpler:\ndef preorder(node): yield node.value for child in node.children: yield from preorder(child) tree_root = TreeNode(\u0026#34;A\u0026#34;, [ TreeNode(\u0026#34;B\u0026#34;, [TreeNode(\u0026#34;D\u0026#34;), TreeNode(\u0026#34;E\u0026#34;)]), TreeNode(\u0026#34;C\u0026#34;, [TreeNode(\u0026#34;F\u0026#34;)]), ]) for v in preorder(tree_root): print(v, end=\u0026#34; \u0026#34;) # A B D E C F Four lines. No iterator class. No state machine. yield handles everything. This is the Pythonic way to implement the Iterator pattern.\nExternal vs Internal Iteration External (what we\u0026rsquo;ve shown) — caller drives the iteration (for x in ...). Flexible: can skip, break, filter. Internal — collection controls iteration, client supplies a callback (tree.for_each(lambda v: ...)). Simpler for callers, less flexible. Most Pythonic code uses external iteration via generators.\nRealistic Use Cases Any collection-like class you write. Database cursor traversal — don\u0026rsquo;t load 10 million rows; iterate them. Paginated API consumption — each next() fetches the next page. Streaming file processing — iterate lines without loading the full file. Lazy computation — generators produce values on demand. Tradeoffs Pros:\nUniform API for traversal across many different collection types. Decouples collection implementation from traversal logic. Supports lazy evaluation — generate values only as needed. Cons:\nSimple collections might not need a formal iterator — for x in list already works. Iteration state is stored in the iterator, so you can\u0026rsquo;t rewind without re-creating. Python-Specific Notes Iterator is so deeply woven into Python that you almost never implement it \u0026ldquo;classically.\u0026rdquo; Whenever a class represents a collection, just define __iter__ (usually as a generator method with yield) and you\u0026rsquo;re done. The GoF\u0026rsquo;s elaborate iterator class hierarchy collapses into a few yield statements.\nRelated Patterns Composite — iterators often traverse Composite structures. Factory Method — aggregates often use a factory method to return their iterator (__iter__). Visitor — often combined with Iterator to process each element. 6. Mediator The Pain Imagine a chatroom with 20 users. Naively, each user has direct references to all others — when Alice sends a message, her code notifies each of the other 19. Adding/removing users means updating references everywhere. Users know each other. Knowledge is scattered.\nOr: a complex UI form. When the \u0026ldquo;country\u0026rdquo; dropdown changes, the \u0026ldquo;state\u0026rdquo; dropdown should repopulate, the \u0026ldquo;tax\u0026rdquo; field should recalculate, the \u0026ldquo;submit\u0026rdquo; button might enable/disable. If each widget directly calls methods on others, you get a web of references: country → state, country → tax, country → submit, state → tax, etc. A few widgets → n² references.\nYou need a central coordinator that absorbs peer-to-peer communication.\nIntent Define an object that encapsulates how a set of objects interact. Mediator promotes loose coupling by keeping objects from referring to each other explicitly.\nAnalogy An air traffic control tower. Planes don\u0026rsquo;t coordinate with each other directly (\u0026ldquo;you land, I\u0026rsquo;ll wait\u0026rdquo;). They talk to the tower. The tower coordinates — \u0026ldquo;Flight 101, cleared to land. Flight 202, hold.\u0026rdquo; If planes talked to each other, chaos.\nOr: a project manager. Developers don\u0026rsquo;t directly task each other — the manager routes work, resolves conflicts, and communicates across the team. One central role mediates.\nStructure Mediator — interface for communication between colleagues. ConcreteMediator — knows the colleagues; routes messages. Colleagues — each knows only the mediator, not other colleagues. Python Implementation — A Chatroom class ChatroomMediator: def __init__(self): self._users = [] def register(self, user): self._users.append(user) user.mediator = self def send(self, sender, message): for user in self._users: if user is not sender: user.receive(sender.name, message) class User: def __init__(self, name): self.name = name self.mediator = None def send(self, message): print(f\u0026#34;[{self.name} sends] {message}\u0026#34;) self.mediator.send(self, message) def receive(self, sender_name, message): print(f\u0026#34; [{self.name} got from {sender_name}]: {message}\u0026#34;) chat = ChatroomMediator() alice = User(\u0026#34;Alice\u0026#34;) bob = User(\u0026#34;Bob\u0026#34;) carol = User(\u0026#34;Carol\u0026#34;) for user in (alice, bob, carol): chat.register(user) alice.send(\u0026#34;Hi everyone!\u0026#34;) bob.send(\u0026#34;Hello Alice!\u0026#34;) Output:\n[Alice sends] Hi everyone! [Bob got from Alice]: Hi everyone! [Carol got from Alice]: Hi everyone! [Bob sends] Hello Alice! [Alice got from Bob]: Hello Alice! [Carol got from Bob]: Hello Alice! Each User knows only about the mediator — not about other users. Adding a fourth user is one register call; no other user changes.\nRealistic Use Cases UI frameworks — a form with interdependent widgets, coordinated by a mediator. Chatrooms and messaging systems — users communicate via a server, not peer-to-peer. Air traffic / traffic-light coordination — central controller. Complex business workflows where multiple services must stay coordinated. Observer vs Mediator — A Classic Confusion Observer — a one-to-many notification from a subject to its observers. Observers are passive receivers. Mediator — a many-to-many coordination. Colleagues actively send messages through the mediator. The cleanest way to feel the difference is to see what the same chatroom scenario would look like under Observer, then contrast it with the Mediator version above.\nIf we modeled the chatroom with Observer, each User would be a subject that other users subscribe to. When Alice posts, she notifies her subscribers — but here\u0026rsquo;s the key: the subject (Alice) has no idea what observers will do with the notification, and there\u0026rsquo;s no central place coordinating the group. Adding \u0026ldquo;Bob can\u0026rsquo;t see messages from people he\u0026rsquo;s blocked\u0026rdquo; means stuffing that logic into either every subject or every observer:\n# OBSERVER-style sketch — note the coupling creeping back in class User: def __init__(self, name): self.name = name self._subscribers = [] # everyone who follows ME def subscribe(self, other): self._subscribers.append(other) # to hear Alice, you subscribe to Alice def post(self, message): # Alice fires-and-forgets to her subscribers. She does NOT coordinate # the room; she has no notion of \u0026#34;the group\u0026#34; — only her own listeners. for sub in self._subscribers: sub.on_message(self.name, message) def on_message(self, sender, message): print(f\u0026#34;[{self.name}] heard from {sender}: {message}\u0026#34;) To make a group chat work this way, every user must subscribe to every other user — that\u0026rsquo;s the n-to-n web (the exact coupling Mediator exists to kill). And cross-cutting rules (blocking, rate limits, message history) have nowhere natural to live.\nIn the Mediator version (the chatroom above), the ChatroomMediator is the single place that knows the whole roster and decides who receives what. Users know only the mediator. A new rule like blocking is a one-line change inside send — the colleagues stay dumb and reusable.\nSo the deepest distinction is about knowledge and coordination, not just cardinality:\nIn Observer, the subject broadcasts blindly — it doesn\u0026rsquo;t know or care what observers do, and no one orchestrates the participants. In Mediator, the mediator actively coordinates the actions of multiple peers — it holds the interaction rules centrally and decides how peers affect each other. They overlap in practice; mediators often use observer internally (the mediator observes its colleagues, then coordinates a response).\nTradeoffs Pros:\nReduces n-to-n coupling to n-to-1. Coordination logic centralized — easier to maintain. Colleagues are simpler and more reusable. Cons:\nThe mediator itself can grow complex — a \u0026ldquo;god object\u0026rdquo; that knows too much. Performance overhead — every interaction routes through the mediator. Debugging: \u0026ldquo;who triggered this?\u0026rdquo; trails may go through the mediator. Related Patterns Facade — also centralizes, but for a subsystem viewed from outside; Mediator is internal peer coordination. Observer — frequently used to implement the mediator\u0026rsquo;s notification mechanism. 7. Memento The Pain You have an object with rich internal state — say, a TextEditor with current text, cursor position, selection, undo history. A user says \u0026ldquo;save this state; I want to come back to it.\u0026rdquo; Or: you implement undo — you need to capture state before each change so you can restore it.\nNaive approach: expose all internal fields so external code can copy them. But this violates encapsulation — everyone now knows the editor\u0026rsquo;s internals, and the editor can\u0026rsquo;t evolve its state representation without breaking external savers.\nIntent Without violating encapsulation, capture and externalize an object\u0026rsquo;s internal state so that the object can be restored to this state later.\nThe object produces a \u0026ldquo;snapshot\u0026rdquo; (the Memento) — an opaque token that external code can hold and later hand back. Only the original object can read the memento\u0026rsquo;s contents.\nAnalogy A game save file. You play for hours, accumulating inventory, quests, XP, map exploration. You save → the game writes a file. You load → the file is given back, game restored. The save file is opaque to you; only the game engine knows how to interpret it.\nOr: a photograph of a scene. The photo captures the state at that moment. You can look at it later; you can try to recreate the scene. But the photo isn\u0026rsquo;t the same as being there — it\u0026rsquo;s a self-contained, opaque snapshot.\nStructure Originator — the object whose state you want to save (e.g., Editor). Memento — a snapshot of Originator\u0026rsquo;s state. Typically has a narrow interface visible to outside code and a wide interface visible only to Originator. Caretaker — manages mementos (stores, orders them). Does NOT inspect their contents. Python Implementation class EditorMemento: \u0026#34;\u0026#34;\u0026#34;Opaque snapshot. Only Editor should read its fields.\u0026#34;\u0026#34;\u0026#34; def __init__(self, text, cursor): self._text = text self._cursor = cursor class Editor: def __init__(self): self._text = \u0026#34;\u0026#34; self._cursor = 0 def type(self, s): self._text = self._text[:self._cursor] + s + self._text[self._cursor:] self._cursor += len(s) def move_cursor(self, pos): self._cursor = pos def save(self) -\u0026gt; EditorMemento: return EditorMemento(self._text, self._cursor) def restore(self, m: EditorMemento): self._text = m._text self._cursor = m._cursor def __repr__(self): return f\u0026#34;Editor(text={self._text!r}, cursor={self._cursor})\u0026#34; class History: \u0026#34;\u0026#34;\u0026#34;Caretaker — stores mementos but doesn\u0026#39;t peek inside.\u0026#34;\u0026#34;\u0026#34; def __init__(self): self._mementos = [] def push(self, m): self._mementos.append(m) def pop(self): return self._mementos.pop() if self._mementos else None editor = Editor() history = History() editor.type(\u0026#34;Hello\u0026#34;) history.push(editor.save()) editor.type(\u0026#34; World\u0026#34;) history.push(editor.save()) editor.type(\u0026#34;!\u0026#34;) print(editor) # Editor(text=\u0026#39;Hello World!\u0026#39;, cursor=12) # Undo editor.restore(history.pop()) print(editor) # Editor(text=\u0026#39;Hello World\u0026#39;, cursor=11) editor.restore(history.pop()) print(editor) # Editor(text=\u0026#39;Hello\u0026#39;, cursor=5) History stores mementos but never reads their internals. Editor is the only class that understands the memento\u0026rsquo;s contents. Encapsulation preserved.\nRealistic Use Cases Undo/redo in editors, drawing tools, etc. Save/load game state. Transaction rollback — capture state before a transaction; restore on abort. Snapshot testing — capture application state, compare across runs. Checkpointing long-running computations. Tradeoffs Pros:\nPreserves encapsulation — external code can save/restore without inspecting internals. Separates state-management concerns (in Caretaker) from state-having concerns (in Originator). Cons:\nMementos can be memory-heavy if state is large; may need pruning. If the state has external references (files, sockets), snapshotting is nontrivial. Python\u0026rsquo;s dynamic nature makes the \u0026ldquo;encapsulation\u0026rdquo; weaker — everything\u0026rsquo;s accessible with underscore conventions rather than language enforcement. Related Patterns Command — commands often store mementos internally for undo(). Iterator — iterator state itself is a memento-like object in some implementations. Prototype — clone() is often used to implement memento creation. 8. Observer The Pain You have a WeatherStation that measures temperature. Several components should react when temperature changes: a Display, a Logger, a MobileAlertService. Naively:\nclass WeatherStation: def set_temperature(self, t): self.temperature = t display.update(t) logger.log(t) mobile_alert.check_and_send(t) Every new subscriber forces a change to WeatherStation. The station knows too much about who consumes its data. OCP, DIP, SRP all unhappy.\nYou need a way for interested parties to subscribe to events, and for the source to broadcast changes — without knowing or caring who\u0026rsquo;s listening.\nIntent Define a one-to-many dependency between objects so that when one object changes state, all its dependents are notified and updated automatically.\nThis is the pattern behind publish/subscribe systems, event buses, reactive programming, and countless UI frameworks.\nAnalogy Think of a YouTube subscription. When a creator uploads a video, YouTube notifies all subscribers. The creator doesn\u0026rsquo;t know every subscriber personally, nor do they manually message each one — they just upload, and the system broadcasts to subscribers. Subscribing and unsubscribing is entirely the viewer\u0026rsquo;s choice.\nOr: a newspaper subscription. The publisher prints. They don\u0026rsquo;t care who reads. Subscribers pay to receive. You can cancel anytime; the publisher carries on unaffected.\nStructure Subject (aka Observable, Publisher) — maintains a list of Observers; provides attach and detach; notifies them on change. Observer (aka Subscriber) — interface with update(...). ConcreteObservers — react to updates specifically. Python Implementation from abc import ABC, abstractmethod class Observer(ABC): @abstractmethod def update(self, subject): pass class Subject: def __init__(self): self._observers = [] def attach(self, observer: Observer): self._observers.append(observer) def detach(self, observer: Observer): self._observers.remove(observer) def notify(self): for obs in self._observers: obs.update(self) class WeatherStation(Subject): def __init__(self): super().__init__() self._temperature = 0 @property def temperature(self): return self._temperature @temperature.setter def temperature(self, value): self._temperature = value self.notify() class Display(Observer): def update(self, subject): print(f\u0026#34;[Display] Temperature is now {subject.temperature}°C\u0026#34;) class Logger(Observer): def update(self, subject): print(f\u0026#34;[Log] temp={subject.temperature}\u0026#34;) class MobileAlert(Observer): def update(self, subject): if subject.temperature \u0026gt; 40: print(f\u0026#34;[Mobile] ALERT! Too hot: {subject.temperature}°C\u0026#34;) station = WeatherStation() station.attach(Display()) station.attach(Logger()) station.attach(MobileAlert()) station.temperature = 30 station.temperature = 42 Output:\n[Display] Temperature is now 30°C [Log] temp=30 [Display] Temperature is now 42°C [Log] temp=42 [Mobile] ALERT! Too hot: 42°C Adding a EmailAlert subscriber? One new class. WeatherStation doesn\u0026rsquo;t change.\nPythonic Observer — Callables as Observers In Python, you often don\u0026rsquo;t need formal Observer classes. Functions are first-class:\nclass EventEmitter: def __init__(self): self._listeners = [] def on(self, callback): self._listeners.append(callback) def emit(self, *args, **kwargs): for cb in self._listeners: cb(*args, **kwargs) bell = EventEmitter() bell.on(lambda: print(\u0026#34;Ding!\u0026#34;)) bell.on(lambda: print(\u0026#34;Logging the ding...\u0026#34;)) bell.emit() This is closer to Node.js\u0026rsquo;s EventEmitter or JavaScript\u0026rsquo;s DOM events — same pattern, lighter syntax.\nPush vs Pull Push: Subject sends the changed data as argument to update(data). Observers get everything they need directly. Pull: Subject calls update(self); Observers call back into Subject to fetch what they want. Our implementation is pull. Push is more efficient but couples Subject more tightly to what Observers need.\nRealistic Use Cases UI event systems — buttons, form field changes. Model-View architectures — views observe model changes. Message queues / pub-sub (Kafka, RabbitMQ, Redis pub/sub, at larger scales). Reactive programming — RxPy, reactive streams, observables. Monitoring and alerting — metric collectors, log aggregators. Django signals, Flask signals, framework lifecycle events. Tradeoffs Pros:\nStrong decoupling between publisher and subscribers. OCP — add new subscribers without touching the publisher. Works well with dynamic subscription (attach/detach at runtime). Cons:\nObservation order isn\u0026rsquo;t guaranteed (unless you manage it). Unexpected cascades — one update triggers many observers, which may trigger their own updates. Debugging flow gets hard. Memory leaks — observers hold references, preventing garbage collection if not detached properly. For high-frequency events, notifying all observers can be a performance hotspot. Related Patterns Mediator — Observer is often used inside a Mediator. Command — the notification carries a Command to execute. Publish/Subscribe systems (Kafka, etc.) — distributed versions of Observer. 9. State The Pain You\u0026rsquo;re building a document approval workflow. A document goes through Draft → PendingReview → Approved (or Rejected). Each state has its own allowed actions: you can only submit_for_review from Draft; you can only approve or reject from PendingReview; nothing\u0026rsquo;s allowed from Approved.\nNaive implementation — a state field with if/elif everywhere:\nclass Document: def __init__(self): self.state = \u0026#34;draft\u0026#34; def submit(self): if self.state == \u0026#34;draft\u0026#34;: self.state = \u0026#34;pending_review\u0026#34; elif self.state == \u0026#34;pending_review\u0026#34;: print(\u0026#34;Already submitted\u0026#34;) else: print(\u0026#34;Can\u0026#39;t submit\u0026#34;) def approve(self): if self.state == \u0026#34;pending_review\u0026#34;: self.state = \u0026#34;approved\u0026#34; elif self.state == \u0026#34;draft\u0026#34;: print(\u0026#34;Submit first!\u0026#34;) # ... and so on Every method is an if/elif ladder over states. Adding a new state (say, NeedsRevision) requires updating every method. Behavior lives in branches, not in states.\nIntent Allow an object to alter its behavior when its internal state changes. The object will appear to change its class.\nInstead of one class branching on state, have one class per state, each encapsulating that state\u0026rsquo;s behavior.\nAnalogy Think of traffic lights. Red → Green → Yellow → Red. In each color\u0026rsquo;s regime, the rules are different (\u0026ldquo;stop,\u0026rdquo; \u0026ldquo;go,\u0026rdquo; \u0026ldquo;slow down\u0026rdquo;). The \u0026ldquo;behavior\u0026rdquo; of the traffic light is entirely different per state — and the light transitions from one state to another.\nOr: a vending machine. States: Idle → HasMoney → DispensingItem → ReturningChange. Each state accepts different inputs and transitions differently.\nStructure Context — the object whose behavior changes (holds a reference to the current State). State — interface declaring state-specific operations. ConcreteStates — implement behavior for each specific state; decide transitions. Python Implementation from abc import ABC, abstractmethod class DocumentState(ABC): @abstractmethod def submit(self, doc): pass @abstractmethod def approve(self, doc): pass @abstractmethod def reject(self, doc): pass class Draft(DocumentState): def submit(self, doc): print(\u0026#34;Submitted for review.\u0026#34;) doc.state = PendingReview() def approve(self, doc): print(\u0026#34;Cannot approve — document is still a draft.\u0026#34;) def reject(self, doc): print(\u0026#34;Cannot reject — document is still a draft.\u0026#34;) class PendingReview(DocumentState): def submit(self, doc): print(\u0026#34;Already submitted.\u0026#34;) def approve(self, doc): print(\u0026#34;Approved!\u0026#34;) doc.state = Approved() def reject(self, doc): print(\u0026#34;Rejected; back to draft.\u0026#34;) doc.state = Draft() class Approved(DocumentState): def submit(self, doc): print(\u0026#34;Already approved; no need to submit.\u0026#34;) def approve(self, doc): print(\u0026#34;Already approved.\u0026#34;) def reject(self, doc): print(\u0026#34;Cannot reject an approved document.\u0026#34;) class Document: def __init__(self): self.state = Draft() def submit(self): self.state.submit(self) def approve(self): self.state.approve(self) def reject(self): self.state.reject(self) doc = Document() doc.approve() # Cannot approve — document is still a draft. doc.submit() # Submitted for review. doc.reject() # Rejected; back to draft. doc.submit() # Submitted for review. doc.approve() # Approved! doc.submit() # Already approved; no need to submit. Each state is a class. Each class owns the behavior for that state. Transitions are explicit in the state classes. Adding a new state (NeedsRevision) is a new class; existing states unchanged except possibly for transitions.\nState vs Strategy — They Look Similar Both inject behavior via composition. The difference is who controls what:\nState Strategy Who initiates the change? The object transitions itself in response to events at runtime Who initiates the change? The client picks the strategy upfront, and it usually stays put States know about each other; they transition Strategies are independent; they don\u0026rsquo;t know one another exists Behavior changes over time, driven by events Behavior is chosen upfront by the client Models finite state machines Models interchangeable algorithms That first row is the deepest difference and the one most often missed. In State, a VendingMachine sitting in NoCoinState flips itself to HasCoinState when a coin event arrives — the transition is internal, automatic, and event-driven; the client just feeds events and never names a state. In Strategy, the client says \u0026ldquo;use ExpressShipping\u0026rdquo; and that choice persists until the client deliberately swaps it — the strategy never decides on its own to become a different strategy. Put bluntly: a State changes itself; a Strategy is changed by someone else.\nExample: document workflow is State (it transitions). Discount calculation is Strategy (you pick premium vs regular).\nRealistic Use Cases Workflow engines — documents, orders, tickets passing through states. Game character states — idle, walking, jumping, attacking. TCP connection state machines — LISTEN, SYN_SENT, ESTABLISHED, etc. Media players — stopped, playing, paused, buffering. Transaction processing — initiated, authorized, captured, refunded. Tradeoffs Pros:\nEliminates large if/elif chains on state. Each state class is cohesive and self-contained. Adding new states is a new class — strong OCP. Cons:\nMore classes (one per state). State transition logic is distributed — to understand the full flow, you read multiple classes. Sometimes a simple state enum + dispatch table is plenty; a full State pattern is overkill. Related Patterns Strategy — same scaffolding, different intent. Singleton — concrete State objects are often stateless and can be singletons, shared across all contexts. Memento — to save/restore state during transitions. 10. Strategy The Pain Your e-commerce app calculates shipping cost. Currently it\u0026rsquo;s a single method:\ndef calculate_shipping(order): # Complex rules for domestic shipping if order.is_domestic: return 50 + order.weight * 10 else: # international return 500 + order.weight * 20 + customs_fee(order) Today, the business wants another shipping strategy: express (flat ₹1000), economy (slow but ₹30), pickup (₹0). Each season, new options pop up. The method becomes a branching monster.\nThe core insight: shipping cost computation is an algorithm, and there are multiple alternative algorithms. You want to select one at runtime, and you want adding a new one to be painless.\nIntent Define a family of algorithms, encapsulate each one, and make them interchangeable. Strategy lets the algorithm vary independently from clients that use it.\nAnalogy Navigation apps. You enter \u0026ldquo;home to office.\u0026rdquo; Google Maps gives you multiple strategies: fastest, shortest, avoid tolls, walking, public transit. Each is an algorithm. Each produces a route. The user picks which strategy. The overall \u0026ldquo;plan a route\u0026rdquo; behavior doesn\u0026rsquo;t change — only the algorithm does.\nOr: paying for a meal. The action is \u0026ldquo;pay ₹500.\u0026rdquo; The strategy is cash, card, UPI, wallet, or IOU. Same outcome; different mechanism.\nStructure Context — the object that uses a strategy (e.g., Order). Strategy — interface for interchangeable algorithms. ConcreteStrategies — each implements a specific algorithm. Python Implementation from abc import ABC, abstractmethod class ShippingStrategy(ABC): @abstractmethod def calculate(self, order) -\u0026gt; float: pass class StandardShipping(ShippingStrategy): def calculate(self, order): return 50 + order.weight * 10 class ExpressShipping(ShippingStrategy): def calculate(self, order): return 1000 # Flat rate class EconomyShipping(ShippingStrategy): def calculate(self, order): return 30 + order.weight * 5 class PickupShipping(ShippingStrategy): def calculate(self, order): return 0 class Order: def __init__(self, items, weight, shipping: ShippingStrategy): self.items = items self.weight = weight self.shipping_strategy = shipping def shipping_cost(self): return self.shipping_strategy.calculate(self) order = Order([\u0026#34;book\u0026#34;], weight=2, shipping=ExpressShipping()) print(order.shipping_cost()) # 1000 order.shipping_strategy = EconomyShipping() print(order.shipping_cost()) # 40 New strategy (FreeShippingForPremium)? One new class. No modification to Order or existing strategies. Textbook OCP.\nPythonic Strategy — Functions as Strategies In Python, functions are first-class. You don\u0026rsquo;t always need ABCs:\n# Each strategy is just a function taking an order and returning a cost. No ABC, # no class hierarchy — the \u0026#34;interface\u0026#34; is the informal contract \u0026#34;callable(order) # -\u0026gt; number\u0026#34;. This is lighter, but the contract is now UNENFORCED: nothing stops # someone passing a function with the wrong signature, and there\u0026#39;s no shared base # advertising what a valid strategy looks like. With an ABC, abstractmethod would # catch a malformed strategy at definition time; here a mismatch only surfaces at # the call site, at runtime. Trade formality for brevity with eyes open. def standard(order): return 50 + order.weight * 10 def express(order): return 1000 def economy(order): return 30 + order.weight * 5 def pickup(order): return 0 class Order: def __init__(self, items, weight, shipping_strategy): self.items = items self.weight = weight self.shipping_strategy = shipping_strategy # Just a callable def shipping_cost(self): return self.shipping_strategy(self) order = Order([\u0026#34;book\u0026#34;], 2, express) print(order.shipping_cost()) # 1000 Lighter weight. Just as flexible. Loses some structure (no common interface enforced) but gains brevity. Rule of thumb: reach for functions when strategies are simple and stateless; reach for ABC-backed classes when strategies carry their own state, share helper methods, or you want the type system to document and enforce the contract.\nRealistic Use Cases Sorting algorithms — sorted(list, key=func) is strategy (the key is the strategy). Discount strategies, tax strategies, pricing strategies in e-commerce. Compression strategies in file handling. Authentication strategies (username/password, OAuth, SSO). Payment strategies (card, UPI, wallet, cash). Strategy vs State — Revisited Strategy: client chooses the algorithm; it doesn\u0026rsquo;t change based on internal events. State: the object transitions between states based on its behavior; the client just calls methods and states change automatically.\nTradeoffs Pros:\nOCP: new algorithms plug in without modifying context. Algorithms are isolated and individually testable. Runtime swapping is trivial. Eliminates if/elif ladders on algorithm type. Cons:\nMore classes (if using full ABC approach). Client has to understand available strategies to pick. Overkill for tiny decisions — a single if may be clearer. Related Patterns State — same shape, different intent. Factory — often used to create strategies based on config. Template Method — opposite in a sense: Template Method fixes the algorithm, varying only steps; Strategy varies the whole algorithm. 11. Template Method The Pain You have several \u0026ldquo;algorithms\u0026rdquo; that share the same overall structure but differ in some specific steps. For example, making different hot beverages:\nTea: boil water → steep tea → pour → add lemon. Coffee: boil water → brew grounds → pour → add sugar \u0026amp; milk. Structurally: boil water, brew, pour, add condiments. Steps 1 and 3 are identical. Steps 2 and 4 differ. If you write two independent classes, you duplicate boil+pour. If you tangle them, you get conditionals.\nYou want to: define the skeleton once, let subclasses fill in the varying parts.\nIntent Define the skeleton of an algorithm in an operation, deferring some steps to subclasses. Template Method lets subclasses redefine certain steps of an algorithm without changing the algorithm\u0026rsquo;s structure.\nAnalogy A cooking recipe template. \u0026ldquo;Preheat oven. Prepare ingredients (varies). Bake at X°C for Y minutes (varies). Let cool. Serve.\u0026rdquo; The top-level structure is fixed; the specifics (what to prep, temperature, time) vary per dish.\nOr: standardized onboarding processes at a company. \u0026ldquo;Sign offer letter. [Team-specific orientation]. HR training. [Role-specific training]. First week wrap-up.\u0026rdquo; Fixed skeleton, varying team/role parts.\nStructure AbstractClass — defines the template method (which is concrete and calls other methods in order) and declares the step methods (some abstract, some with default implementations). ConcreteClasses — override the abstract steps. Python Implementation from abc import ABC, abstractmethod class Beverage(ABC): # The template method — final structure of the algorithm def prepare(self): self.boil_water() self.brew() self.pour_in_cup() self.add_condiments() def boil_water(self): print(\u0026#34;Boiling water\u0026#34;) def pour_in_cup(self): print(\u0026#34;Pouring into cup\u0026#34;) @abstractmethod def brew(self): pass @abstractmethod def add_condiments(self): pass class Tea(Beverage): def brew(self): print(\u0026#34;Steeping tea bag\u0026#34;) def add_condiments(self): print(\u0026#34;Adding lemon\u0026#34;) class Coffee(Beverage): def brew(self): print(\u0026#34;Brewing coffee grounds\u0026#34;) def add_condiments(self): print(\u0026#34;Adding sugar and milk\u0026#34;) Tea().prepare() # Boiling water # Steeping tea bag # Pouring into cup # Adding lemon Coffee().prepare() # Boiling water # Brewing coffee grounds # Pouring into cup # Adding sugar and milk The algorithm shape (boil → brew → pour → condiments) is fixed in prepare(). Subclasses plug in their specifics.\nThis is the clearest illustration of the Hollywood Principle: \u0026ldquo;Don\u0026rsquo;t call us, we\u0026rsquo;ll call you.\u0026rdquo; In ordinary code, your class calls into a library. Here it\u0026rsquo;s inverted — the parent Beverage.prepare() is in charge of the control flow, and it calls down into the subclass\u0026rsquo;s hooks (brew, add_condiments) at the moments it decides. The subclass never invokes prepare\u0026rsquo;s steps in order itself; it just supplies the pieces and waits to be called. That inversion of control is exactly what keeps the algorithm\u0026rsquo;s skeleton tamper-proof: a subclass can change what brewing means, but it cannot reorder boil/brew/pour or skip a step, because it isn\u0026rsquo;t the one driving.\nHooks — Optional Steps You can also provide default implementations that subclasses may override but don\u0026rsquo;t have to:\nclass Beverage(ABC): def prepare(self): self.boil_water() self.brew() self.pour_in_cup() if self.customer_wants_condiments(): # Hook self.add_condiments() def customer_wants_condiments(self): return True # Default Subclasses override customer_wants_condiments only if they need different logic.\nTemplate Method vs Strategy Both solve \u0026ldquo;variation in behavior.\u0026rdquo; The key difference:\nTemplate Method Strategy Uses inheritance — subclass overrides steps Uses composition — inject a whole algorithm Fixes algorithm structure, varies steps Varies the entire algorithm \u0026ldquo;Hollywood principle\u0026rdquo;: don\u0026rsquo;t call us, we\u0026rsquo;ll call you (parent calls subclass\u0026rsquo;s hooks) Client picks a strategy object Fewer runtime decisions More runtime flexibility Both are valid for different situations. Prefer Strategy when you want more flexibility and runtime swapping; prefer Template Method when the skeleton truly is fixed and only steps vary.\nRealistic Use Cases Framework extension points — Django\u0026rsquo;s class-based views, Flask\u0026rsquo;s view functions, unittest\u0026rsquo;s setUp/tearDown. Data processing pipelines — fixed \u0026ldquo;extract → transform → load\u0026rdquo; skeleton; subclasses customize each step. Lifecycle hooks in libraries (e.g., React\u0026rsquo;s lifecycle methods). Game loops — fixed structure (input → update → render), specific games customize. Tradeoffs Pros:\nEliminates duplication of algorithm structure. Enforces consistency — every subclass follows the same sequence. Great for \u0026ldquo;framework\u0026rdquo; code — you provide the skeleton, users fill in the blanks. Cons:\nInheritance-based — tight coupling between parent and child. Subclasses may be tempted to override more than intended. Harder to test individual steps in isolation. Can lead to deep inheritance hierarchies. Related Patterns Strategy — composition-based alternative. Factory Method — often itself a Template Method (create the product, then customize it). Hollywood Principle — the underlying philosophy (\u0026ldquo;don\u0026rsquo;t call us, we\u0026rsquo;ll call you\u0026rdquo;). 12. Visitor The Pain You have a document object model — paragraphs, images, tables, lists. On this model you need to perform various operations: spell check, word count, HTML export, PDF export, translation, accessibility audit, etc.\nYou could add a method for each operation to every node class (spell_check(), word_count(), export_html(), \u0026hellip;). But:\nEvery new operation bloats every class. Classes grow unboundedly. The classes stop having a single responsibility — they accumulate unrelated operations. If you can\u0026rsquo;t modify the classes (e.g., third-party library), you\u0026rsquo;re stuck. You want: define new operations on an object structure without modifying the object classes.\nIntent Represent an operation to be performed on the elements of an object structure. Visitor lets you define a new operation without changing the classes of the elements on which it operates.\nThe trick: each element accepts a Visitor. The Visitor has a method per element type. The element dispatches to the right method.\nAnalogy A tax auditor visiting different businesses. A restaurant, a factory, a retailer — each has different books, inventory, and concerns. The auditor (visitor) has specialized routines: how to audit a restaurant, how to audit a factory, how to audit a retailer. The businesses don\u0026rsquo;t change; the auditor brings the operation.\nNext year, there\u0026rsquo;s a \u0026ldquo;health inspector\u0026rdquo; visitor — different routines, same businesses, no modifications to the businesses.\nStructure Visitor — interface declaring visit_X(x) for each element type X. ConcreteVisitors — specific operations. Element — interface with accept(visitor) method. ConcreteElements — implement accept by calling visitor.visit_Self(self) (double dispatch). Python Implementation from abc import ABC, abstractmethod # Elements class DocElement(ABC): @abstractmethod def accept(self, visitor): pass class Paragraph(DocElement): def __init__(self, text): self.text = text def accept(self, visitor): # FIRST DISPATCH — selected by THIS element\u0026#39;s type. We\u0026#39;re inside # Paragraph.accept, so Python already resolved \u0026#34;which element am I?\u0026#34; # That\u0026#39;s why we can hard-code the call to visit_paragraph below. visitor.visit_paragraph(self) # SECOND DISPATCH — selected by the # visitor\u0026#39;s type: WordCountVisitor.visit_paragraph vs # HTMLExportVisitor.visit_paragraph resolve differently. Element type × # visitor type = the right method. Together: double dispatch. class Image(DocElement): def __init__(self, url, alt): self.url = url self.alt = alt def accept(self, visitor): visitor.visit_image(self) class Table(DocElement): def __init__(self, rows): self.rows = rows def accept(self, visitor): visitor.visit_table(self) # Visitors class Visitor(ABC): @abstractmethod def visit_paragraph(self, p): pass @abstractmethod def visit_image(self, i): pass @abstractmethod def visit_table(self, t): pass class WordCountVisitor(Visitor): def __init__(self): self.count = 0 def visit_paragraph(self, p): self.count += len(p.text.split()) def visit_image(self, i): self.count += len(i.alt.split()) # count alt-text words def visit_table(self, t): for row in t.rows: for cell in row: self.count += len(cell.split()) class HTMLExportVisitor(Visitor): def __init__(self): self.html = [] def visit_paragraph(self, p): self.html.append(f\u0026#34;\u0026lt;p\u0026gt;{p.text}\u0026lt;/p\u0026gt;\u0026#34;) def visit_image(self, i): self.html.append(f\u0026#39;\u0026lt;img src=\u0026#34;{i.url}\u0026#34; alt=\u0026#34;{i.alt}\u0026#34;\u0026gt;\u0026#39;) def visit_table(self, t): rows_html = \u0026#34;\u0026#34;.join( \u0026#34;\u0026lt;tr\u0026gt;\u0026#34; + \u0026#34;\u0026#34;.join(f\u0026#34;\u0026lt;td\u0026gt;{c}\u0026lt;/td\u0026gt;\u0026#34; for c in row) + \u0026#34;\u0026lt;/tr\u0026gt;\u0026#34; for row in t.rows ) self.html.append(f\u0026#34;\u0026lt;table\u0026gt;{rows_html}\u0026lt;/table\u0026gt;\u0026#34;) # A document document = [ Paragraph(\u0026#34;The quick brown fox jumps\u0026#34;), Image(\u0026#34;fox.jpg\u0026#34;, \u0026#34;A red fox\u0026#34;), Table([[\u0026#34;Name\u0026#34;, \u0026#34;Age\u0026#34;], [\u0026#34;Amit\u0026#34;, \u0026#34;30\u0026#34;]]), Paragraph(\u0026#34;The end.\u0026#34;), ] # Operation 1: word count wc = WordCountVisitor() for elem in document: elem.accept(wc) print(f\u0026#34;Total words: {wc.count}\u0026#34;) # Operation 2: HTML export — no changes to element classes html = HTMLExportVisitor() for elem in document: elem.accept(html) print(\u0026#34;\\n\u0026#34;.join(html.html)) Adding a third operation — say, SpellCheckVisitor — is a new class. The document element classes (Paragraph, Image, Table) don\u0026rsquo;t change. OCP for operations, at the cost of structural stability for element types.\nDouble Dispatch — The Clever Bit When you call elem.accept(visitor):\nelem knows its type (say, Paragraph). It calls visitor.visit_paragraph(self) — the exact right method on the visitor. This is double dispatch: the method resolved depends on both the element type AND the visitor type. Most OOP languages (including Python) have single dispatch (on the method\u0026rsquo;s receiver only); accept manually fakes the second dispatch.\nWhy can\u0026rsquo;t we just call visitor.visit(element) directly and skip the accept dance? Because Python (like Java, C#, C++) dispatches on exactly one type — the object before the dot. If we wrote a single visit(self, element) method, the visitor would receive an element whose concrete type (Paragraph? Image? Table?) it can only discover by writing isinstance checks — the very if/elif ladder Visitor exists to abolish. The two-step accept → visit_xxx handshake launders the missing dispatch through the type system: step one is a normal method call on the element (Python resolves which element), and from inside that element\u0026rsquo;s accept we now statically know the type, so we can call the correctly-named visit_paragraph/visit_image/visit_table — and that call resolves on the visitor\u0026rsquo;s type. Two single dispatches, chained, give us the double dispatch the language won\u0026rsquo;t give us directly.\nVisitor\u0026rsquo;s Tradeoff — Operations vs Elements Visitor makes it easy to add operations (new visitors = no change to elements), but hard to add element types (a new element class forces every existing visitor to implement visit_new_type). This is often called the Expression Problem.\nBefore using Visitor, ask: do element types change less than operations? If yes → Visitor. If operations are stable but element types evolve → inheritance methods on elements.\nRealistic Use Cases Compilers — AST nodes visited by type checkers, optimizers, code generators. Document formats — nodes in a parsed doc visited by exporters (HTML, PDF, plain text, accessibility). Object databases — visitors for serialization, validation, indexing. Static analysis tools — visit AST nodes to flag code smells. Tradeoffs Pros:\nAdds new operations without touching element classes. Keeps element classes focused (SRP). Works well on stable hierarchies with many operations. Cons:\nAdding a new element type is painful — every existing visitor must change. Double-dispatch ceremony (accept methods) feels heavy. Visitor methods may need access to internal state, forcing wider interfaces on elements. Python-Specific Notes Python\u0026rsquo;s duck typing and functools.singledispatch can achieve similar effects without the formal Visitor machinery:\nfrom functools import singledispatch @singledispatch def word_count(elem): raise NotImplementedError @word_count.register def _(p: Paragraph): return len(p.text.split()) @word_count.register def _(i: Image): return len(i.alt.split()) Not always better, but often simpler when formal Visitor is overkill.\nRelated Patterns Composite — Visitors typically traverse Composite structures. Interpreter — ASTs built via Interpreter are often operated on with Visitor. Iterator — used to walk through elements during a visit. 13. Summary Eleven patterns. Each answers a different question about how objects collaborate:\nPattern Question It Answers Core Mechanism Chain of Responsibility Who should handle this request? Pass through a chain until someone handles it Command How do I make actions first-class (store, queue, undo)? Encapsulate a request as an object Interpreter How do I evaluate sentences in a mini-language? Each grammar rule as a class; build a tree; recurse Iterator How do I traverse a collection uniformly? An object that yields elements one at a time Mediator How do I coordinate many peers without n² connections? A central hub they all talk through Memento How do I save/restore state without breaking encapsulation? An opaque snapshot only the owner can read Observer How do I notify many parties of a change? Subscribers register; subject broadcasts State How does an object change behavior as it changes state? One class per state; state objects handle transitions Strategy How do I make algorithms interchangeable? Inject a strategy object; swap at will Template Method How do I fix an algorithm\u0026rsquo;s structure but vary its steps? Parent defines skeleton; children override steps Visitor How do I add operations to a structure without modifying it? Double dispatch: element accepts visitor, visitor handles element Decision Flow Need to decouple sender from receiver? → Chain of Responsibility. Need actions as objects (undo, queue, log)? → Command. Need to evaluate a DSL or grammar? → Interpreter. Need uniform traversal? → Iterator. Need to coordinate complex peer interactions? → Mediator. Need to save/restore state? → Memento. Need to notify many listeners of changes? → Observer. Object behavior depends on a state that transitions? → State. Need interchangeable algorithms? → Strategy. Need a fixed process with variable steps? → Template Method. Need to add operations to a stable structure? → Visitor. Recognizing Patterns in Your Own Code — A Symptom-Driven Tree The list above works once you already know which pattern you want. In practice you usually start from a smell in your own code, not from a pattern name. This tree walks from the symptom you can actually observe to the pattern that addresses it. Read it as \u0026ldquo;if you catch yourself writing X, reach for Y.\u0026rdquo;\nWhat is the code smell you\u0026#39;re staring at? │ ├─ A big if/elif (or switch) ladder that keeps growing... │ │ │ ├─ ...branching on a \u0026#34;type\u0026#34; or \u0026#34;kind\u0026#34; field, and you keep ADDING operations │ │ over a FIXED set of types? → VISITOR │ │ │ ├─ ...branching on the object\u0026#39;s current \u0026#34;mode/phase/status,\u0026#34; and the object │ │ moves between those modes over time? → STATE │ │ │ └─ ...branching to pick one of several algorithms the caller chose? │ → STRATEGY │ ├─ Two methods share the same skeleton but differ in a few steps │ (copy-pasted with small edits)... → TEMPLATE METHOD │ ├─ Objects are wired directly to each other and the web of references is │ exploding (n×n)... → MEDIATOR │ ├─ \u0026#34;When X happens, several unrelated things must react,\u0026#34; and you don\u0026#39;t want │ X to know about all of them... → OBSERVER │ ├─ You need to treat an action as a *thing* — to queue it, log it, retry it, │ or undo it... → COMMAND │ └─ ...and undo needs to restore prior state cleanly? │ → COMMAND + MEMENTO │ ├─ A request might be handled by one of several handlers and you don\u0026#39;t know │ which upfront (and want to add/reorder handlers freely)? → CHAIN OF RESPONSIBILITY │ ├─ You\u0026#39;re parsing/evaluating little expressions in a mini-language │ (filters, rules, formulas)... → INTERPRETER │ └─ You want callers to walk a collection without exposing its internals (or you wrote a custom data structure)... → ITERATOR The decisive questions, distilled:\nWho chooses the behavior, and when? The client up front → Strategy. The object itself, reacting to events → State. What\u0026rsquo;s stable and what changes? Operations grow over fixed types → Visitor. Types grow over fixed operations → keep methods on the elements (don\u0026rsquo;t use Visitor). Is the structure fixed and only steps vary? → Template Method. Is the whole algorithm swappable? → Strategy. Is it one-to-many notification? → Observer. Many-to-many coordination with central rules? → Mediator. When two candidates still feel plausible, you\u0026rsquo;re usually choosing between a static shape (inheritance, decided at definition: Template Method) and a dynamic one (composition, decided at runtime: Strategy, State). Default to the composition-based option unless the skeleton genuinely never varies — it ages better.\nKey Takeaways Behavioral patterns are the richest and most varied family. Each one addresses a genuinely distinct collaboration problem. Many pairs look similar structurally; tell them apart by intent. State vs Strategy, Template Method vs Strategy, Mediator vs Observer, Command vs Chain of Responsibility — all are disambiguated by why you use them, not just what they look like. Python simplifies several of these patterns significantly. Generators collapse Iterator. First-class functions simplify Strategy, Command, and Observer. functools.singledispatch can replace Visitor in some cases. Always recognize the pattern; choose the most Pythonic implementation. Behavior + SOLID = much of good OO design. Observer embodies OCP. Strategy embodies OCP and DIP. State embodies OCP by splitting state-dependent behavior across classes. Command embodies SRP by turning actions into first-class objects. Patterns can combine. A Command often holds a Memento. A Mediator often uses Observer. A Visitor traverses a Composite. Real systems layer patterns naturally. 14. Practice Exercises Exercise 1 — Chain of Responsibility You\u0026rsquo;re building a purchase approval system. An employee submits a purchase request. Approval rules:\nUp to ₹10,000: Team Lead. Up to ₹1,00,000: Manager. Up to ₹10,00,000: Director. Above: CFO. Design a chain. Show what happens for requests of ₹5,000, ₹80,000, ₹5,00,000, and ₹2,00,00,000.\nSolution:\nfrom abc import ABC, abstractmethod class Approver(ABC): def __init__(self): self._next = None def set_next(self, next_approver): self._next = next_approver return next_approver def approve(self, amount): if self._can_approve(amount): self._handle(amount) elif self._next: self._next.approve(amount) else: print(f\u0026#34;Cannot approve ₹{amount:,}\u0026#34;) @abstractmethod def _can_approve(self, amount): pass @abstractmethod def _handle(self, amount): pass class TeamLead(Approver): def _can_approve(self, amount): return amount \u0026lt;= 10_000 def _handle(self, amount): print(f\u0026#34;Team Lead approved ₹{amount:,}\u0026#34;) class Manager(Approver): def _can_approve(self, amount): return amount \u0026lt;= 1_00_000 def _handle(self, amount): print(f\u0026#34;Manager approved ₹{amount:,}\u0026#34;) class Director(Approver): def _can_approve(self, amount): return amount \u0026lt;= 10_00_000 def _handle(self, amount): print(f\u0026#34;Director approved ₹{amount:,}\u0026#34;) class CFO(Approver): def _can_approve(self, amount): return True # CFO handles everything above def _handle(self, amount): print(f\u0026#34;CFO approved ₹{amount:,}\u0026#34;) # Build chain lead = TeamLead() lead.set_next(Manager()).set_next(Director()).set_next(CFO()) for amt in [5_000, 80_000, 5_00_000, 2_00_00_000]: lead.approve(amt) Output:\nTeam Lead approved ₹5,000 Manager approved ₹80,000 Director approved ₹5,00,000 CFO approved ₹2,00,00,000 Exercise 2 — Command with Undo Implement a simple calculator with Add, Subtract, and Multiply commands. Support undo. Demonstrate.\nSolution:\nfrom abc import ABC, abstractmethod class Command(ABC): @abstractmethod def execute(self, calc): pass @abstractmethod def undo(self, calc): pass class Calculator: def __init__(self): self.value = 0 self.history = [] def run(self, cmd): cmd.execute(self) self.history.append(cmd) def undo(self): if self.history: self.history.pop().undo(self) class Add(Command): def __init__(self, n): self.n = n def execute(self, calc): calc.value += self.n def undo(self, calc): calc.value -= self.n class Subtract(Command): def __init__(self, n): self.n = n def execute(self, calc): calc.value -= self.n def undo(self, calc): calc.value += self.n class Multiply(Command): def __init__(self, n): self.n = n self._previous = None def execute(self, calc): self._previous = calc.value calc.value *= self.n def undo(self, calc): calc.value = self._previous calc = Calculator() calc.run(Add(10)) # 10 calc.run(Multiply(3)) # 30 calc.run(Subtract(5)) # 25 print(calc.value) # 25 calc.undo(); print(calc.value) # 30 calc.undo(); print(calc.value) # 10 calc.undo(); print(calc.value) # 0 Note: Multiply stores _previous because undoing multiplication by zero can\u0026rsquo;t be done via division. Undo design requires care.\nExercise 3 — Iterator Build a class FibonacciSequence(n) that iterates over the first n Fibonacci numbers. Use Python\u0026rsquo;s generator style.\nSolution:\nclass FibonacciSequence: def __init__(self, n): self.n = n def __iter__(self): a, b = 0, 1 for _ in range(self.n): yield a a, b = b, a + b for f in FibonacciSequence(10): print(f, end=\u0026#34; \u0026#34;) # 0 1 1 2 3 5 8 13 21 34 Short, lazy, and fully Pythonic. No explicit __next__ needed.\nExercise 4 — Mediator Model a smart home. A motion sensor triggers: lights turn on, camera records, phone gets a notification. A door sensor triggers: lights turn on, security alarm starts, phone gets a notification. Implement via Mediator so that sensors don\u0026rsquo;t directly reference all consumers.\nSolution:\nclass SmartHomeHub: def __init__(self): self.lights = None self.camera = None self.alarm = None self.phone = None def register(self, lights=None, camera=None, alarm=None, phone=None): if lights: self.lights = lights if camera: self.camera = camera if alarm: self.alarm = alarm if phone: self.phone = phone def handle(self, event): if event == \u0026#34;motion\u0026#34;: self.lights.on() self.camera.record() self.phone.notify(\u0026#34;Motion detected\u0026#34;) elif event == \u0026#34;door\u0026#34;: self.lights.on() self.alarm.sound() self.phone.notify(\u0026#34;Door opened!\u0026#34;) class Lights: def on(self): print(\u0026#34;Lights ON\u0026#34;) class Camera: def record(self): print(\u0026#34;Camera recording\u0026#34;) class Alarm: def sound(self): print(\u0026#34;Alarm SOUNDING\u0026#34;) class Phone: def notify(self, msg): print(f\u0026#34;Phone notified: {msg}\u0026#34;) class MotionSensor: def __init__(self, hub): self.hub = hub def trigger(self): self.hub.handle(\u0026#34;motion\u0026#34;) class DoorSensor: def __init__(self, hub): self.hub = hub def trigger(self): self.hub.handle(\u0026#34;door\u0026#34;) hub = SmartHomeHub() hub.register(lights=Lights(), camera=Camera(), alarm=Alarm(), phone=Phone()) motion = MotionSensor(hub) door = DoorSensor(hub) motion.trigger() print(\u0026#34;---\u0026#34;) door.trigger() Sensors only know the hub. Consumers are registered with the hub. Wiring changes are a one-line registration update.\nExercise 5 — Observer Build a simple stock ticker. A Stock has a price. Multiple observers — a PriceDisplay, an AlertSystem (fires if price drops \u0026gt; 5%), and an AutoTrader (buys if price drops \u0026gt; 10%) — should react when price changes.\nSolution:\nclass Stock: def __init__(self, symbol, price): self.symbol = symbol self._price = price self._previous = price self._observers = [] def attach(self, obs): self._observers.append(obs) def set_price(self, price): self._previous = self._price self._price = price self._notify() @property def price(self): return self._price @property def previous(self): return self._previous @property def change_pct(self): return (self._price - self._previous) / self._previous * 100 def _notify(self): for obs in self._observers: obs.update(self) class PriceDisplay: def update(self, stock): print(f\u0026#34;[Display] {stock.symbol}: ₹{stock.price} ({stock.change_pct:+.2f}%)\u0026#34;) class AlertSystem: def update(self, stock): if stock.change_pct \u0026lt; -5: print(f\u0026#34;[Alert] {stock.symbol} DROPPED {stock.change_pct:.2f}%\u0026#34;) class AutoTrader: def update(self, stock): if stock.change_pct \u0026lt; -10: print(f\u0026#34;[Trader] BUYING {stock.symbol} at ₹{stock.price}\u0026#34;) stock = Stock(\u0026#34;INFY\u0026#34;, 1500) stock.attach(PriceDisplay()) stock.attach(AlertSystem()) stock.attach(AutoTrader()) stock.set_price(1450) # -3.33% — only display fires stock.set_price(1350) # Bigger drop from 1450 — alert fires stock.set_price(1200) # Big drop from 1350 — alert + trader Exercise 6 — State Implement a vending machine with states: Idle, HasMoney, Dispensing. Actions: insert_money(), select_item(), dispense(). Valid transitions:\nIdle + insert_money → HasMoney. HasMoney + select_item → Dispensing. Dispensing + dispense → Idle. Other combinations print an error. Solution:\nfrom abc import ABC, abstractmethod class State(ABC): @abstractmethod def insert_money(self, machine): pass @abstractmethod def select_item(self, machine): pass @abstractmethod def dispense(self, machine): pass class Idle(State): def insert_money(self, m): print(\u0026#34;Money inserted.\u0026#34;); m.state = HasMoney() def select_item(self, m): print(\u0026#34;Insert money first.\u0026#34;) def dispense(self, m): print(\u0026#34;Nothing to dispense.\u0026#34;) class HasMoney(State): def insert_money(self, m): print(\u0026#34;Already have money.\u0026#34;) def select_item(self, m): print(\u0026#34;Item selected.\u0026#34;); m.state = Dispensing() def dispense(self, m): print(\u0026#34;Select an item first.\u0026#34;) class Dispensing(State): def insert_money(self, m): print(\u0026#34;Busy dispensing.\u0026#34;) def select_item(self, m): print(\u0026#34;Already selected.\u0026#34;) def dispense(self, m): print(\u0026#34;Dispensing... enjoy!\u0026#34;); m.state = Idle() class VendingMachine: def __init__(self): self.state = Idle() def insert_money(self): self.state.insert_money(self) def select_item(self): self.state.select_item(self) def dispense(self): self.state.dispense(self) m = VendingMachine() m.select_item() # Insert money first. m.insert_money() # Money inserted. m.insert_money() # Already have money. m.select_item() # Item selected. m.dispense() # Dispensing... enjoy! m.dispense() # Nothing to dispense. Exercise 7 — Strategy You have a list of numbers. Implement three sorting strategies: ascending, descending, and alphabetical (by string representation). Use Strategy. Then show the Pythonic version using functions.\nSolution:\nfrom abc import ABC, abstractmethod # Classical Strategy class SortStrategy(ABC): @abstractmethod def sort(self, data): pass class Ascending(SortStrategy): def sort(self, data): return sorted(data) class Descending(SortStrategy): def sort(self, data): return sorted(data, reverse=True) class AlphabeticalStr(SortStrategy): def sort(self, data): return sorted(data, key=str) class Sorter: def __init__(self, strategy: SortStrategy): self.strategy = strategy def sort(self, data): return self.strategy.sort(data) data = [10, 2, 33, 4] print(Sorter(Ascending()).sort(data)) # [2, 4, 10, 33] print(Sorter(Descending()).sort(data)) # [33, 10, 4, 2] print(Sorter(AlphabeticalStr()).sort(data)) # [10, 2, 33, 4] (because \u0026#34;10\u0026#34; \u0026lt; \u0026#34;2\u0026#34; \u0026lt; \u0026#34;33\u0026#34; \u0026lt; \u0026#34;4\u0026#34;) # Pythonic version — functions as strategies asc = lambda d: sorted(d) desc = lambda d: sorted(d, reverse=True) alpha = lambda d: sorted(d, key=str) for strat in (asc, desc, alpha): print(strat(data)) The second form uses Python\u0026rsquo;s key argument in sorted — a Strategy-pattern application built right into the language.\nExercise 8 — Template Method Build a DataExporter abstract class. The template method export() has steps: fetch(), transform(), format(), save(). fetch and save have defaults. Concrete classes CSVExporter and JSONExporter override transform and format.\nSolution:\nfrom abc import ABC, abstractmethod import json class DataExporter(ABC): def export(self): data = self.fetch() transformed = self.transform(data) output = self.format(transformed) self.save(output) def fetch(self): print(\u0026#34;Fetching data from DB...\u0026#34;) return [{\u0026#34;name\u0026#34;: \u0026#34;Amit\u0026#34;, \u0026#34;age\u0026#34;: 30}, {\u0026#34;name\u0026#34;: \u0026#34;Priya\u0026#34;, \u0026#34;age\u0026#34;: 25}] def save(self, output): print(f\u0026#34;Saving:\\n{output}\u0026#34;) @abstractmethod def transform(self, data): pass @abstractmethod def format(self, data): pass class CSVExporter(DataExporter): def transform(self, data): print(\u0026#34;Transforming for CSV...\u0026#34;) return data def format(self, data): keys = data[0].keys() lines = [\u0026#34;,\u0026#34;.join(keys)] for d in data: lines.append(\u0026#34;,\u0026#34;.join(str(d[k]) for k in keys)) return \u0026#34;\\n\u0026#34;.join(lines) class JSONExporter(DataExporter): def transform(self, data): print(\u0026#34;Transforming for JSON...\u0026#34;) return data def format(self, data): return json.dumps(data, indent=2) print(\u0026#34;=== CSV ===\u0026#34;) CSVExporter().export() print(\u0026#34;\\n=== JSON ===\u0026#34;) JSONExporter().export() Algorithm structure (fetch → transform → format → save) is fixed in export. Subclasses plug in the varying steps.\nExercise 9 — Visitor You have shapes: Circle, Rectangle, Triangle (with base and height). Implement two visitors: AreaVisitor and PerimeterVisitor. Don\u0026rsquo;t modify the shape classes when adding a new visitor.\nSolution:\nfrom abc import ABC, abstractmethod import math class Shape(ABC): @abstractmethod def accept(self, visitor): pass class Circle(Shape): def __init__(self, radius): self.radius = radius def accept(self, v): return v.visit_circle(self) class Rectangle(Shape): def __init__(self, width, height): self.width, self.height = width, height def accept(self, v): return v.visit_rectangle(self) class Triangle(Shape): def __init__(self, base, height, side_a, side_b): self.base, self.height = base, height self.side_a, self.side_b = side_a, side_b def accept(self, v): return v.visit_triangle(self) class Visitor(ABC): @abstractmethod def visit_circle(self, c): pass @abstractmethod def visit_rectangle(self, r): pass @abstractmethod def visit_triangle(self, t): pass class AreaVisitor(Visitor): def visit_circle(self, c): return math.pi * c.radius ** 2 def visit_rectangle(self, r): return r.width * r.height def visit_triangle(self, t): return 0.5 * t.base * t.height class PerimeterVisitor(Visitor): def visit_circle(self, c): return 2 * math.pi * c.radius def visit_rectangle(self, r): return 2 * (r.width + r.height) def visit_triangle(self, t): return t.base + t.side_a + t.side_b shapes = [Circle(5), Rectangle(4, 6), Triangle(3, 4, 5, 4)] area = AreaVisitor() perim = PerimeterVisitor() for s in shapes: print(f\u0026#34;Area: {s.accept(area):.2f}, Perimeter: {s.accept(perim):.2f}\u0026#34;) Adding a BoundingBoxVisitor = one new class. Shape classes unchanged.\nExercise 10 — Capstone (Multi-Pattern) Design a game character that combines several behavioral patterns:\nState — character has states (Idle, Walking, Jumping, Falling). Command — player input (MoveLeftCommand, JumpCommand) is dispatched as commands. Observer — game UI observes character\u0026rsquo;s health and updates display on change. Strategy — different attack strategies (melee, ranged, magic). Outline the class structure.\nSolution (outline with working code):\nfrom abc import ABC, abstractmethod # --- State --- class CharacterState(ABC): @abstractmethod def update(self, char): pass @abstractmethod def name(self): pass class Idle(CharacterState): def update(self, char): pass def name(self): return \u0026#34;Idle\u0026#34; class Walking(CharacterState): def update(self, char): pass def name(self): return \u0026#34;Walking\u0026#34; class Jumping(CharacterState): def update(self, char): char.y += 5 if char.y \u0026gt;= 10: char.state = Falling() def name(self): return \u0026#34;Jumping\u0026#34; class Falling(CharacterState): def update(self, char): char.y -= 5 if char.y \u0026lt;= 0: char.y = 0 char.state = Idle() def name(self): return \u0026#34;Falling\u0026#34; # --- Strategy --- class AttackStrategy(ABC): @abstractmethod def attack(self, char, target): pass class MeleeAttack(AttackStrategy): def attack(self, char, target): print(f\u0026#34;{char.name} slashes {target} for 10 damage\u0026#34;) class RangedAttack(AttackStrategy): def attack(self, char, target): print(f\u0026#34;{char.name} shoots {target} for 7 damage\u0026#34;) class MagicAttack(AttackStrategy): def attack(self, char, target): print(f\u0026#34;{char.name} casts fireball on {target} for 15 damage\u0026#34;) # --- Observer --- class UIObserver: def update(self, character): print(f\u0026#34;[UI] {character.name}: HP={character.hp}, state={character.state.name()}\u0026#34;) # --- Character --- class Character: def __init__(self, name, attack: AttackStrategy): self.name = name self.hp = 100 self.x = 0 self.y = 0 self.state = Idle() self.attack_strategy = attack self._observers = [] def attach(self, obs): self._observers.append(obs) def _notify(self): for o in self._observers: o.update(self) def take_damage(self, amt): self.hp -= amt self._notify() def tick(self): self.state.update(self) def attack(self, target): self.attack_strategy.attack(self, target) # --- Command --- class Command(ABC): @abstractmethod def execute(self, char): pass class MoveLeft(Command): def execute(self, char): char.x -= 1 char.state = Walking() class Jump(Command): def execute(self, char): if isinstance(char.state, Idle): char.state = Jumping() # --- Game loop --- hero = Character(\u0026#34;Hero\u0026#34;, MeleeAttack()) hero.attach(UIObserver()) hero.take_damage(20) # UI reacts MoveLeft().execute(hero) # State → Walking Jump().execute(hero) # (ignored — not in Idle) hero.state = Idle() # reset for demo Jump().execute(hero) # State → Jumping for _ in range(5): hero.tick() # animate jump/fall hero.attack(\u0026#34;goblin\u0026#34;) hero.attack_strategy = MagicAttack() hero.attack(\u0026#34;dragon\u0026#34;) Each pattern solves a distinct concern: State handles behavior-per-state, Command encodes actions, Observer informs the UI, Strategy swaps attacks. Together, they form a clean architecture for a game character.\n15. Self-Assessment Test Q1. When would you pick Chain of Responsibility over a giant if/elif dispatch?\nAnswer: When (a) the set of handlers is dynamic or needs to be reconfigured at runtime, (b) handlers represent a logical hierarchy (escalation, priority), (c) a request may be handled by any one handler but you want to decouple the sender from knowing who, or (d) new handlers will be added over time. A static if/elif is fine for stable, small dispatches.\nQ2. What problem does Command solve that a regular method call cannot?\nAnswer: A method call is transient — it happens and disappears. A Command is an object: it can be stored in a list (queuing), serialized (persisting), inspected (logging), executed later (scheduling), or reversed (undo). Essentially, Command turns invocations into first-class, manipulable entities.\nQ3. What\u0026rsquo;s the difference between Observer and Mediator?\nAnswer: Observer is about one-to-many notifications — a subject broadcasts changes to passive subscribers. Mediator is about coordinating many peers — objects interact through a central hub rather than directly with each other. Mediator often uses Observer internally, but its intent is about peer coordination, not change propagation.\nQ4. When do you use State and when do you use Strategy?\nAnswer:\nState — behavior changes automatically based on an object\u0026rsquo;s internal state transitions driven by events. The object \u0026ldquo;becomes\u0026rdquo; different over time. Strategy — behavior is chosen by an external client upfront; the object doesn\u0026rsquo;t transition by itself. Both look structurally similar (inject an object that encapsulates behavior). The intent distinguishes them. Q5. What\u0026rsquo;s \u0026ldquo;double dispatch\u0026rdquo; in Visitor, and why is it necessary?\nAnswer: Regular method dispatch in most OOP languages (Python included) is single dispatch — the method resolved depends only on the receiver\u0026rsquo;s type. Visitor needs dispatch on two types (the element AND the visitor). The element.accept(visitor) → visitor.visit_X(element) indirection achieves double dispatch manually, ensuring the right visitor method is called for the right element type.\nQ6. Why is Memento superior to just copying an object\u0026rsquo;s state?\nAnswer: Memento preserves encapsulation. External code doesn\u0026rsquo;t read the object\u0026rsquo;s internal fields directly — it just holds an opaque memento. The original object controls what\u0026rsquo;s captured and how it\u0026rsquo;s restored, so implementation details can change without breaking external code. Plain state copying tends to leak internals.\nQ7. Identify the pattern: \u0026ldquo;Subscribers register with a publisher; when the publisher changes, all subscribers are notified automatically.\u0026rdquo;\nAnswer: Observer.\nQ8. Identify the pattern: \u0026ldquo;Different algorithms for the same task, each wrapped in a class, selectable at runtime.\u0026rdquo;\nAnswer: Strategy.\nQ9. Why would you use Template Method instead of Strategy?\nAnswer: When the algorithm\u0026rsquo;s overall structure is truly fixed and only a few specific steps vary. Template Method uses inheritance — the parent owns the skeleton, children fill in blanks. Strategy uses composition — the client swaps out the whole algorithm. If most of the algorithm repeats and only small parts differ, Template Method avoids duplication. If entire algorithms vary freely, Strategy is more flexible.\nQ10. What\u0026rsquo;s a major downside of Visitor?\nAnswer: Adding a new element type to the structure forces every existing visitor to implement a method for it. This is the \u0026ldquo;Expression Problem\u0026rdquo;: Visitor makes adding operations cheap but adding element types expensive. Use Visitor when your element set is stable and operations evolve; use plain methods-on-elements when operations are stable and elements evolve.\nQ11. Why does Python often make Iterator almost invisible?\nAnswer: Because Python bakes iteration into the language: __iter__ + __next__ are first-class protocols, and yield (generators) lets you write an iterator in a few lines without a formal class. The GoF\u0026rsquo;s full Iterator class hierarchy is replaced by def method(self): yield .... The pattern is everywhere in Python, just not visible as a \u0026ldquo;pattern.\u0026rdquo;\nQ12. A colleague replaces Observer with a direct method call \u0026ldquo;for simplicity.\u0026rdquo; When is this actually fine?\nAnswer: When (a) there\u0026rsquo;s exactly one consumer of the event, (b) that consumer is stable (won\u0026rsquo;t change), (c) the consumer is intimately related to the publisher (not independent), and (d) there\u0026rsquo;s no plausible future need for additional consumers. If any of these might not hold, Observer\u0026rsquo;s decoupling is worth the small overhead.\nQ13. Can Command be used without undo? Give an example.\nAnswer: Yes. Undo is one common reason for Command, but not the only one. Commands can be used purely for queuing (background job systems like Celery), logging (audit trails), macro recording, remote execution (RPC), or simply to decouple the invoker from the receiver. Many Command implementations don\u0026rsquo;t have undo() at all.\nQ14. Why are Flyweight objects usually Singletons in practice?\nAnswer: (Cross-family question.) Flyweight shares intrinsic state. A factory typically creates one Flyweight per unique intrinsic configuration and reuses it. That \u0026ldquo;one per config\u0026rdquo; is singleton-like. They\u0026rsquo;re not identical patterns — Flyweight is about many shared instances across configs, Singleton is about exactly one instance — but concrete Flyweights often behave like singletons in practice.\nQ15. You have a 5-state workflow with complex transition rules. Should you use State, or a dictionary-based state machine?\nAnswer: Depends on complexity. For a simple state machine with straightforward transitions, a dict of {(current_state, event): new_state} is fine — and often clearer than a dozen State classes. For complex per-state behavior (different operations, entry/exit actions, rich logic within each state), the State pattern\u0026rsquo;s cohesion pays off. Use judgment: prefer simpler approaches when sufficient.\n16. Final Words — All 23 Patterns at a Glance Congratulations — you\u0026rsquo;ve now studied all 23 classical GoF patterns across three families.\nThe Complete Roster Creational (5) — how we create objects\nSingleton Factory Method Abstract Factory Builder Prototype Structural (7) — how we compose objects 6. Adapter 7. Bridge 8. Composite 9. Decorator 10. Facade 11. Flyweight 12. Proxy\nBehavioral (11) — how objects collaborate and distribute responsibility 13. Chain of Responsibility 14. Command 15. Interpreter 16. Iterator 17. Mediator 18. Memento 19. Observer 20. State 21. Strategy 22. Template Method 23. Visitor\nThe Arc You\u0026rsquo;ve Traveled OOP gave you the raw materials: classes, inheritance, polymorphism, abstraction. SOLID gave you the architectural principles: how to arrange those materials so the system welcomes change. Design Patterns gave you the vocabulary of recurring solutions: recognizable shapes that apply SOLID concretely. Notice how it all flows together. Nearly every pattern is a concrete embodiment of SOLID:\nStrategy, Observer, Chain, State, Decorator — all textbook OCP + DIP. Visitor — strong OCP for operations. Adapter, Facade, Bridge, Proxy — DIP through careful abstraction. Template Method — SRP + OCP through the Hollywood principle. Builder, Factory — SRP for construction logic. Final Principles for Pattern Mastery Patterns are a vocabulary, not a toolkit. You don\u0026rsquo;t go hunting for chances to \u0026ldquo;use a pattern.\u0026rdquo; You recognize a design pain, recall a pattern that addresses it, and apply it judiciously.\nPattern names are communication shortcuts. Saying \u0026ldquo;let\u0026rsquo;s use Observer here\u0026rdquo; conveys structure, intent, and tradeoffs in two words. Learn the names so you can communicate with other designers.\nImplementation varies wildly by language and situation. The GoF book\u0026rsquo;s Java-centric implementations are one of many. Python\u0026rsquo;s generators make Iterator trivial. First-class functions simplify Strategy and Command. Decorators-the-syntax relate to Decorator-the-pattern. Always ask: what\u0026rsquo;s the most idiomatic expression of this pattern in my language?\nRecognize when you\u0026rsquo;re using patterns without naming them. Django middleware is Chain of Responsibility. Flask\u0026rsquo;s route decorators are Decorator (the pattern) + Decorator (the syntax). SQLAlchemy sessions are a Unit of Work + Identity Map. Python context managers are Template Method. Once you know the names, you see them everywhere.\nCombine patterns freely. Real systems are rarely \u0026ldquo;just one pattern.\u0026rdquo; A web app uses Facade (request handlers) + Chain of Responsibility (middleware) + Strategy (auth, serialization) + Command (background jobs) + Observer (signals) + Iterator (querysets) + all woven together.\nNot every problem needs a pattern. Simple code is often just simple code. Patterns shine when you feel genuine pain from change, rigidity, or coupling. Premature pattern application is its own anti-pattern.\nWhat\u0026rsquo;s Next You now have a strong foundation in low-level design. From here, you can explore:\nArchitectural patterns (MVC, MVVM, Clean Architecture, Hexagonal Architecture) — larger-scale structure. Concurrency patterns (Actor, Reactor, Producer-Consumer, Future/Promise). Distributed system patterns (Saga, CQRS, Event Sourcing, Circuit Breaker). Domain-Driven Design — modeling business problems in code. Refactoring catalogs (Fowler\u0026rsquo;s Refactoring) — how to evolve designs toward patterns over time. These build on the same foundation: classes, SOLID, and patterns.\nGreat design isn\u0026rsquo;t about knowing every pattern. It\u0026rsquo;s about understanding the forces that patterns respond to — coupling, duplication, rigidity, change — and using the right tool, at the right moment, with the right judgment.\nYou now have that judgment. Go build.\nThis content is part of Codeverra — a platform for learning coding, data science, DSA, and AI from scratch. Explore more: https://codeverra.com\n","permalink":"https://blogs.codeverra.com/blog/low_level_design/05_design_patterns_behavioral/","summary":"\u003ch1 id=\"design-patterns--part-3-behavioral-patterns\"\u003eDesign Patterns — Part 3: Behavioral Patterns\u003c/h1\u003e\n\u003cblockquote\u003e\n\u003cp\u003e\u003cstrong\u003eWho this is for:\u003c/strong\u003e Anyone who has worked through Creational and Structural patterns and is ready for the final, largest, and arguably most important family — the patterns that govern how objects \u003cem\u003ecommunicate\u003c/em\u003e and \u003cem\u003edivide responsibility\u003c/em\u003e.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003chr\u003e\n\u003ch2 id=\"table-of-contents\"\u003eTable of Contents\u003c/h2\u003e\n\u003col\u003e\n\u003cli\u003e\u003ca href=\"#1-behavioral-patterns\"\u003eBehavioral Patterns — What and Why\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#2-chain-of-responsibility\"\u003eChain of Responsibility\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#3-command\"\u003eCommand\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#4-interpreter\"\u003eInterpreter\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#5-iterator\"\u003eIterator\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#6-mediator\"\u003eMediator\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#7-memento\"\u003eMemento\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#8-observer\"\u003eObserver\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#9-state\"\u003eState\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#10-strategy\"\u003eStrategy\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#11-template-method\"\u003eTemplate Method\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#12-visitor\"\u003eVisitor\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#13-summary\"\u003eSummary of Behavioral Patterns\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#14-practice-exercises\"\u003ePractice Exercises with Solutions\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#15-self-assessment-test\"\u003eSelf-Assessment Test\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#16-final-words\"\u003eFinal Words — All 23 Patterns at a Glance\u003c/a\u003e\u003c/li\u003e\n\u003c/ol\u003e\n\u003chr\u003e\n\u003ch2 id=\"1-behavioral-patterns\"\u003e1. Behavioral Patterns\u003c/h2\u003e\n\u003cp\u003eCreational patterns answered \u003cem\u003e\u0026ldquo;how do we make objects?\u0026rdquo;\u003c/em\u003e\nStructural patterns answered \u003cem\u003e\u0026ldquo;how do we glue objects together?\u0026rdquo;\u003c/em\u003e\nBehavioral patterns answer: \u003cstrong\u003e\u0026ldquo;how do objects \u003cem\u003etalk to each other\u003c/em\u003e, and \u003cem\u003ewho is responsible for what\u003c/em\u003e?\u0026rdquo;\u003c/strong\u003e\u003c/p\u003e","tags":["lld","design-patterns","python"],"title":"Design Patterns, Part 3: Behavioral Patterns"},{"content":"Design Patterns — Practice Workbook Who this is for: Anyone who has worked through the three teaching files — 03_design_patterns_creational.md, 04_design_patterns_structural.md, and 05_design_patterns_behavioral.md — and now wants to prove they actually understand by solving problems independently.\nTable of Contents Preface — How to Use This Workbook Section 1 — Creational Patterns Practice Problem 1.1 — Application Configuration (Level 1) Problem 1.2 — Document Export Flow (Level 1) Problem 1.3 — Food Delivery Order (Level 2) Problem 1.4 — Game Enemy Spawning (Level 2) Problem 1.5 — Cloud Environment Configurations (Level 3) Section 2 — Structural Patterns Practice Problem 2.1 — Third-Party Payment SDK (Level 1) Problem 2.2 — Coffee Shop Pricing (Level 2) Problem 2.3 — Smart Home Scenes (Level 2) Problem 2.4 — Library Book Database (Level 2) Problem 2.5 — File System with Virtual Files (Level 3) Problem 2.6 — UI Rendering Across Platforms (Level 3) Section 3 — Behavioural Patterns Practice Problem 3.1 — Stock Price Monitor (Level 1) Problem 3.2 — E-commerce Shipping (Level 1) Problem 3.3 — Text Editor Undo/Redo (Level 2) Problem 3.4 — Vending Machine (Level 2) Problem 3.5 — Order Validation Pipeline (Level 2) Problem 3.6 — News Aggregator (Level 3) Problem 3.7 — Chess Engine (Level 3) Section 4 — Combination Problems: Multiple Patterns Together Problem 4.1 — Ride-Sharing App (Level 3) Problem 4.2 — Hotel Booking Platform (Level 3) Problem 4.3 — Notification Service (Level 3) Problem 4.4 — Document Management System (Level 3 — Hardest) Section 5 — Anti-Pattern Traps: When NOT to Use a Pattern Problem 5.1 — A Simple CSV Script Problem 5.2 — TemperatureConverter Problem 5.3 — Internal Bulk-Rename Tool Summary — Pattern Selection Reference Card Preface You\u0026rsquo;ve read the three teaching files. Each one gave you the pattern: \u0026ldquo;Here is Observer, here is what it solves, here is the example.\u0026rdquo; Reading those files makes you familiar with patterns. It does not make you good at using them.\nThis workbook is different. Every problem here is a scenario without a hint. The pattern is not named. You have to read the requirements, decide which pattern fits (if any), design the classes, justify your decisions, and implement the solution. That is the actual skill — the one tested in interviews and the one used at work.\nHow to use this workbook — the contract Read the scenario. Stop. Do not scroll to the solution. Spend 2–3 minutes thinking: Which pattern do you think applies? What are the key classes and interfaces? In two sentences: why? Sketch your design — on paper or a whiteboard. Even a quick list of class names is enough. Then read the solution. Compare your reasoning to the walkthrough. The goal is not to match exactly. The goal is to understand where your reasoning differed and why. Run the extensibility check at the end of every solution. It is the proof. Mentally ask: \u0026ldquo;if I add X, what changes?\u0026rdquo; That answer is how you know whether the design is good. What every Level 2/3 solution contains Recognition — what pain in the scenario points to this pattern? What would go wrong without it? The wrong approach first — the naive design you\u0026rsquo;d reach for. Why it falls apart. Design Decisions — why these classes, why these interfaces, why these relationships. What were the alternatives and why were they rejected? Full Python implementation — complete, runnable, commented code. Extensibility check — \u0026ldquo;now add a new variant. How many existing files change?\u0026rdquo; Pattern reveal — the pattern name and a one-paragraph note on when not to use it. A learner who works every problem in this file — covering the solution, attempting it, comparing — finishes interview-ready. There are no shortcuts. The reps are the point.\nSection 1 — Creational Patterns Practice Five problems. Problems 1.1–1.3 are single-pattern. Problem 1.4 touches one pattern with subtleties around shallow vs. deep copying. Problem 1.5 combines two creational patterns.\nProblem 1.1 — Application Configuration (Level 1) Scenario. A Python web application needs a configuration system. The app reads from a JSON file at startup. Various modules — the database layer, the email service, the feature-flag service — all need access to the same configuration. When the app runs in production, the config should be loaded exactly once and never reloaded. When the app is tested, a different config (pointing to test infrastructure) should be loadable.\nWhat you must produce:\nThe class design with reasoning. Full implementation. A test showing that two different modules get the same config object. An explanation of why this beats a plain module-level variable. Solution Recognition The pain points spelled out:\n\u0026ldquo;All modules need access to the same config\u0026rdquo; → there must be one canonical instance. \u0026ldquo;Loaded exactly once\u0026rdquo; → not re-loaded on each access; cached after first load. \u0026ldquo;Replaceable in tests\u0026rdquo; → there must be a way to reset or override it. This is the Singleton pattern. One instance, globally accessible, created lazily, controllable in tests.\nThe naive alternative — and why it\u0026rsquo;s not quite enough You might say: \u0026ldquo;Just use a module-level variable.\u0026rdquo;\n# config.py import json config = json.load(open(\u0026#34;settings.json\u0026#34;)) # loaded once at import That works… until it doesn\u0026rsquo;t:\nReset between tests is awkward. You\u0026rsquo;d have to reload the module or reach into its globals. Lazy loading is impossible. The JSON file is read at import time, even if the test never touches the config. No interface. You can\u0026rsquo;t add validation, defaulting, or environment overlays without it getting messy. Singleton with a class gives you all of these for very little code.\nThe implementation — thread-safe, lazy, testable import json import threading from typing import Any, Optional class Config: \u0026#34;\u0026#34;\u0026#34; Application-wide configuration. Loaded once on first access. Thread-safe via double-checked locking. \u0026#34;\u0026#34;\u0026#34; _instance: Optional[\u0026#34;Config\u0026#34;] = None _lock = threading.Lock() def __init__(self, source_path: str): # NOTE: This constructor is only invoked by `get_instance`. # Calling Config(...) directly skips the singleton check. # We allow it so tests can build isolated configs. with open(source_path) as f: self._values: dict = json.load(f) # ----------------------------------------------------------- # The canonical accessor # ----------------------------------------------------------- @classmethod def get_instance(cls, source_path: str = \u0026#34;settings.json\u0026#34;) -\u0026gt; \u0026#34;Config\u0026#34;: if cls._instance is None: # 1st check (cheap, no lock) with cls._lock: if cls._instance is None: # 2nd check (under lock) cls._instance = cls(source_path) return cls._instance # ----------------------------------------------------------- # Reading values # ----------------------------------------------------------- def get(self, key: str, default: Any = None) -\u0026gt; Any: return self._values.get(key, default) # ----------------------------------------------------------- # Test hooks (deliberately exposed) # ----------------------------------------------------------- @classmethod def reset_for_tests(cls) -\u0026gt; None: \u0026#34;\u0026#34;\u0026#34;Clear the cached instance so tests can install a fresh one.\u0026#34;\u0026#34;\u0026#34; with cls._lock: cls._instance = None @classmethod def install_for_tests(cls, values: dict) -\u0026gt; \u0026#34;Config\u0026#34;: \u0026#34;\u0026#34;\u0026#34;Inject a pre-built config without touching disk.\u0026#34;\u0026#34;\u0026#34; with cls._lock: cls._instance = cls.__new__(cls) cls._instance._values = values return cls._instance Why double-checked locking? A single check (if cls._instance is None: cls._instance = cls(...)) is racy: two threads can both pass the check and both create instances. Wrapping the whole accessor in a lock is correct but slow — every read goes through the lock. Double-checked locking is the standard middle ground: the cheap check skips the lock 99% of the time; the lock catches the race only on the very first hit.\nProving it works # In two different modules: def database_module(): cfg = Config.get_instance() return cfg.get(\u0026#34;database_url\u0026#34;) def email_module(): cfg = Config.get_instance() return cfg.get(\u0026#34;smtp_host\u0026#34;) def test_modules_see_same_object(): Config.reset_for_tests() Config.install_for_tests({ \u0026#34;database_url\u0026#34;: \u0026#34;sqlite:///:memory:\u0026#34;, \u0026#34;smtp_host\u0026#34;: \u0026#34;localhost\u0026#34;, }) assert Config.get_instance() is Config.get_instance() # same object assert database_module() == \u0026#34;sqlite:///:memory:\u0026#34; assert email_module() == \u0026#34;localhost\u0026#34; The is check (not ==) is what proves singleton-ness.\nExtensibility check Add support for environment-specific configs (dev/staging/prod). Add a classmethod:\n@classmethod def load_for_env(cls, env: str) -\u0026gt; \u0026#34;Config\u0026#34;: return cls.get_instance(f\u0026#34;settings.{env}.json\u0026#34;) No existing line of Config changes. The singleton guarantee still holds (one instance once chosen).\nPattern reveal — and when not to use it Singleton. Use it when:\nThere is genuinely one instance for the whole process (config, logger, connection pool). Tests can swap or reset it cleanly. Don\u0026rsquo;t use it when:\nYou\u0026rsquo;re using it to share state between unrelated parts of the code that \u0026ldquo;happen\u0026rdquo; to want the same data. That\u0026rsquo;s hidden global state with a polite costume. Pass things through constructors (Dependency Injection) instead. You think you need one instance \u0026ldquo;for now\u0026rdquo; but might need many later. Singletons are very hard to un-Singleton once code depends on the global access pattern. Problem 1.2 — Document Export Flow (Level 1) Scenario. A document editor exports to multiple formats: PDF, DOCX, and HTML. The export process always follows the same three steps in the same order:\nPrepare the content (e.g., flatten nested formatting). Format it (apply the format-specific layout). Write the output file. Each format performs each step differently, but the order never changes. The editor should be able to support a new format (say, EPUB) without touching the core export flow.\nWhat you must produce:\nThe class hierarchy. Full implementation of the export flow and at least two concrete exporters. Proof that adding EPUB requires zero changes to existing classes. Solution Recognition Two phrases in the scenario are the clue:\n\u0026ldquo;The export process always follows the same steps in the same order\u0026rdquo; → the algorithm skeleton is fixed. \u0026ldquo;Each format does each step differently\u0026rdquo; → the steps vary by subclass. When the skeleton is fixed but the steps vary, the pattern is Template Method. The base class implements the unchangeable algorithm (the export() method that calls prepare → format → write); subclasses implement the variable steps.\nNote that this overlaps with Factory Method when one of the \u0026ldquo;steps\u0026rdquo; is \u0026ldquo;create the right object\u0026rdquo; — many sources show this scenario as Factory Method. The two are siblings: Template Method controls the flow, Factory Method controls which object the flow uses. We\u0026rsquo;ll see both interpretations and pick Template Method because the brief emphasises the flow itself being fixed, not the object that does the work.\nThe naive alternative — and why it fails class Exporter: def export(self, doc, format_type: str, output_path: str): if format_type == \u0026#34;pdf\u0026#34;: content = self._prepare_pdf(doc) formatted = self._format_pdf(content) self._write_pdf(formatted, output_path) elif format_type == \u0026#34;docx\u0026#34;: content = self._prepare_docx(doc) formatted = self._format_docx(content) self._write_docx(formatted, output_path) elif format_type == \u0026#34;html\u0026#34;: ... The problems:\nEvery new format = an elif branch + three new methods on the same class. The \u0026ldquo;flow\u0026rdquo; is duplicated three times. A change to the flow (say, adding a validation step) must be made in three places. Adding EPUB requires editing this class — OCP violation. The Template Method implementation from abc import ABC, abstractmethod class DocumentExporter(ABC): \u0026#34;\u0026#34;\u0026#34; Fixed export algorithm. Subclasses fill in the variable steps. \u0026#34;\u0026#34;\u0026#34; def export(self, document, output_path: str) -\u0026gt; None: \u0026#34;\u0026#34;\u0026#34; Template method. NEVER override this in subclasses. It defines the algorithm; subclasses only customise the steps. \u0026#34;\u0026#34;\u0026#34; prepared = self._prepare(document) formatted = self._format(prepared) self._write(formatted, output_path) # If we later add an audit-log step or a pre-validation step, # it goes here, and EVERY format gets it for free. # ----------------------------------------------------------- # The \u0026#34;primitive operations\u0026#34; — abstract steps # ----------------------------------------------------------- @abstractmethod def _prepare(self, document): \u0026#34;\u0026#34;\u0026#34;Flatten / normalise / pre-process the input.\u0026#34;\u0026#34;\u0026#34; ... @abstractmethod def _format(self, prepared): \u0026#34;\u0026#34;\u0026#34;Apply this format\u0026#39;s specific layout.\u0026#34;\u0026#34;\u0026#34; ... @abstractmethod def _write(self, formatted, output_path: str) -\u0026gt; None: \u0026#34;\u0026#34;\u0026#34;Persist to disk in this format.\u0026#34;\u0026#34;\u0026#34; ... # --------------------------------------------------------------- # Two concrete exporters # --------------------------------------------------------------- class PdfExporter(DocumentExporter): def _prepare(self, document): # Strip features PDF can\u0026#39;t represent (e.g., live form fields). return {\u0026#34;text\u0026#34;: document.text, \u0026#34;images\u0026#34;: document.images} def _format(self, prepared): # Pretend we lay out pages here using reportlab or similar. return f\u0026#34;%PDF-1.7\\n[pages of {prepared}]\u0026#34; def _write(self, formatted, output_path): with open(output_path, \u0026#34;wb\u0026#34;) as f: f.write(formatted.encode()) class HtmlExporter(DocumentExporter): def _prepare(self, document): # HTML can keep nested structure, but escape special chars. return document.text.replace(\u0026#34;\u0026lt;\u0026#34;, \u0026#34;\u0026amp;lt;\u0026#34;) def _format(self, prepared): return f\u0026#34;\u0026lt;html\u0026gt;\u0026lt;body\u0026gt;{prepared}\u0026lt;/body\u0026gt;\u0026lt;/html\u0026gt;\u0026#34; def _write(self, formatted, output_path): with open(output_path, \u0026#34;w\u0026#34;, encoding=\u0026#34;utf-8\u0026#34;) as f: f.write(formatted) Adding EPUB — the extensibility check class EpubExporter(DocumentExporter): def _prepare(self, document): # Chunk into chapters. return document.split_into_chapters() def _format(self, prepared): # Build OPF + spine XML. return _build_epub_package(prepared) def _write(self, formatted, output_path): # Write zip archive with the right MIME header. _zip_epub(formatted, output_path) Lines changed in existing files: zero. The flow in DocumentExporter.export() is unchanged. PdfExporter and HtmlExporter are unchanged. The system is open for extension, closed for modification.\nPattern reveal — and the Factory Method distinction Template Method. Use it when:\nA fixed sequence of steps must run for every variant. The variation is at the step level, not the result level. Why this is Template Method, not Abstract Factory. Abstract Factory creates families of related products that must be compatible (e.g., a PDF formatter AND a PDF font loader AND a PDF metadata handler that all must agree on PDF conventions). Here we have one thing — an exporter — that internally varies its steps. That is one type of thing, not a family of related types. Template Method is the right tool.\nIf the brief had said \u0026ldquo;each export format needs a Formatter, a FontLoader, and a MetadataWriter that must all match — PDF formatter cannot mix with HTML font loader\u0026rdquo; — then Abstract Factory would be the right answer. We\u0026rsquo;re not there.\nProblem 1.3 — Food Delivery Order (Level 2) Scenario. A food delivery platform creates delivery orders. An order has:\nRequired: customer details, delivery address, list of items. Optional: payment method (defaults to cash on delivery), special instructions, scheduled time, priority level (defaults to \u0026ldquo;normal\u0026rdquo;), corporate-invoice flag. The order construction also runs validations:\nDelivery address must lie inside the delivery zone. Items list cannot be empty. If priority is \u0026quot;express\u0026quot;, scheduled time must be within 2 hours from now. Developers have complained that Order(customer, address, items, None, None, \u0026quot;normal\u0026quot;, True, False, ...) is unreadable and error-prone.\nWhat you must produce:\nClass design with a builder. Full implementation with validation that runs at build() time. Three different order-creation examples showing readability. A comparison with the obvious alternative — \u0026ldquo;why not just use keyword arguments?\u0026rdquo; Solution Recognition A constructor with seven arguments, half of them optional, is fragile. Two things are going wrong at once:\nReadability collapses as positional arguments grow. Reading Order(c, a, i, None, None, \u0026quot;normal\u0026quot;, True, False) requires the reader to memorise the argument order. Cross-field validation (\u0026ldquo;if priority is express, scheduled_time must be within 2h\u0026rdquo;) doesn\u0026rsquo;t fit inside __init__ cleanly — at construction, half the fields might not be set yet. This is the Builder pattern. A builder is a small companion class that accumulates configuration through method calls and then produces the final object — usually with full validation — in a build() step.\nWhy not just keyword arguments? Order(customer=c, address=a, items=i, scheduled_time=t) is readable. So why a builder?\nThe keyword-argument approach works for simple construction. It falls short when:\nCross-field validation is non-trivial — __post_init__ works but mixes validation rules into the dataclass. Conditional configuration is needed — \u0026ldquo;if corporate invoice, GST number is required\u0026rdquo; reads naturally as .with_corporate_invoice(gst) and clumsily as constructor logic. A fluent, readable API is wanted at the call site, especially when most callers use only 2–3 of the optional fields and writing field4=None, field5=None, field6=None to skip to field7 is awkward. Stepwise construction — adding items one at a time during a checkout UI flow — is natural with a builder, awkward with kwargs. For a 3-field object: use kwargs. For a 7-field object with cross-field validation and stepwise construction: use a builder.\nThe implementation from dataclasses import dataclass, field from datetime import datetime, timedelta from typing import List, Optional # --------------------------------------------------------------- # The product: a frozen dataclass. # Frozen makes it immutable once built. # --------------------------------------------------------------- @dataclass(frozen=True) class Order: customer: str address: str items: List[str] payment_method: str = \u0026#34;cash\u0026#34; special_instructions: Optional[str] = None scheduled_time: Optional[datetime] = None priority: str = \u0026#34;normal\u0026#34; corporate_gst: Optional[str] = None # --------------------------------------------------------------- # The builder: accumulates state, validates on build(). # --------------------------------------------------------------- class OrderBuilder: \u0026#34;\u0026#34;\u0026#34; Fluent construction for Order. Required fields are constructor arguments of the builder itself, so you cannot forget them. Optional fields are setter methods. \u0026#34;\u0026#34;\u0026#34; DELIVERY_ZONES = {\u0026#34;Patna\u0026#34;, \u0026#34;Delhi\u0026#34;, \u0026#34;Mumbai\u0026#34;} # demo data def __init__(self, customer: str, address: str, items: List[str]): # Required fields are HERE — the builder cannot be instantiated # without them. This is the type system enforcing requiredness. self._customer = customer self._address = address self._items = list(items) # Optional state — sensible defaults. self._payment = \u0026#34;cash\u0026#34; self._instructions: Optional[str] = None self._scheduled: Optional[datetime] = None self._priority = \u0026#34;normal\u0026#34; self._corporate_gst: Optional[str] = None # ---------- Fluent setters ---------- def with_payment(self, method: str) -\u0026gt; \u0026#34;OrderBuilder\u0026#34;: self._payment = method return self # ← enables chaining def with_special_instructions(self, text: str) -\u0026gt; \u0026#34;OrderBuilder\u0026#34;: self._instructions = text return self def with_scheduled_time(self, when: datetime) -\u0026gt; \u0026#34;OrderBuilder\u0026#34;: self._scheduled = when return self def with_priority(self, level: str) -\u0026gt; \u0026#34;OrderBuilder\u0026#34;: if level not in {\u0026#34;normal\u0026#34;, \u0026#34;express\u0026#34;}: raise ValueError(f\u0026#34;Unknown priority: {level}\u0026#34;) self._priority = level return self def with_corporate_invoice(self, gst_number: str) -\u0026gt; \u0026#34;OrderBuilder\u0026#34;: if not gst_number or len(gst_number) != 15: raise ValueError(\u0026#34;GST number must be 15 chars\u0026#34;) self._corporate_gst = gst_number return self # ---------- The validation gate ---------- def build(self) -\u0026gt; Order: # 1. Single-field validations if not self._items: raise ValueError(\u0026#34;Order must have at least one item\u0026#34;) if self._address.split(\u0026#34;,\u0026#34;)[-1].strip() not in self.DELIVERY_ZONES: raise ValueError(f\u0026#34;Address not in delivery zone: {self._address}\u0026#34;) # 2. Cross-field validations if self._priority == \u0026#34;express\u0026#34;: if self._scheduled is None: raise ValueError(\u0026#34;Express orders need a scheduled_time\u0026#34;) if self._scheduled - datetime.now() \u0026gt; timedelta(hours=2): raise ValueError(\u0026#34;Express orders must be within 2 hours\u0026#34;) # 3. Construct the immutable Order return Order( customer=self._customer, address=self._address, items=self._items, payment_method=self._payment, special_instructions=self._instructions, scheduled_time=self._scheduled, priority=self._priority, corporate_gst=self._corporate_gst, ) Three call sites — see how readable each becomes # 1. The simplest possible order order = OrderBuilder( customer=\u0026#34;C001\u0026#34;, address=\u0026#34;42 MG Road, Patna\u0026#34;, items=[\u0026#34;Pizza\u0026#34;, \u0026#34;Coke\u0026#34;], ).build() # 2. A scheduled order with instructions order = ( OrderBuilder(\u0026#34;C001\u0026#34;, \u0026#34;42 MG Road, Patna\u0026#34;, [\u0026#34;Pizza\u0026#34;]) .with_scheduled_time(datetime.now() + timedelta(hours=1)) .with_special_instructions(\u0026#34;Ring doorbell twice\u0026#34;) .build() ) # 3. An express order paid by UPI order = ( OrderBuilder(\u0026#34;C002\u0026#34;, \u0026#34;5 Park St, Mumbai\u0026#34;, [\u0026#34;Burger\u0026#34;, \u0026#34;Fries\u0026#34;]) .with_priority(\u0026#34;express\u0026#34;) .with_scheduled_time(datetime.now() + timedelta(minutes=30)) .with_payment(\u0026#34;upi\u0026#34;) .build() ) Each with_* method\u0026rsquo;s name tells you what\u0026rsquo;s being set. Compare that to Order(\u0026quot;C002\u0026quot;, \u0026quot;5 Park St\u0026quot;, [\u0026quot;Burger\u0026quot;], None, None, datetime.now()+..., \u0026quot;express\u0026quot;, None) — six positional Nones and you\u0026rsquo;ve lost the will to live.\nExtensibility check Marketing wants a corporate invoice feature with a GST number. We already added with_corporate_invoice(gst) above — adding it was one method on the builder. No existing call site breaks (the builder method is optional). No other class touches it. That\u0026rsquo;s the win.\nPattern reveal — Builder, and the alternatives Builder. Use it when:\nThe object has many fields, several optional. Construction needs validation that crosses field boundaries. Stepwise / fluent construction reads more naturally than a giant constructor call. Don\u0026rsquo;t use Builder for 2- or 3-field objects. A dataclass with keyword arguments is simpler. The Builder pattern is a response to construction complexity; without that complexity, it is overhead.\nProblem 1.4 — Game Enemy Spawning (Level 2) Scenario. A game has enemy units: Warriors, Archers, Mages. Each enemy type has a base configuration — health, attack, speed, list of abilities. During gameplay, the engine spawns enemies very quickly — often by tweaking a pre-configured template enemy (e.g., a slightly stronger Warrior for a boss room). Constructing each enemy from scratch is slow because it involves loading textures, computing initial stats, allocating ability objects.\nWhat you must produce:\nDesign supporting templates and fast cloning. Python implementation using the copy module — and explaining shallow vs. deep copy. A scenario where shallow copy creates a bug, and the fix. A registry that stores named templates. Solution Recognition Two signals:\n\u0026ldquo;Creating an enemy from scratch is slow\u0026rdquo; — performance pressure on construction. \u0026ldquo;Often by tweaking a pre-configured template\u0026rdquo; — the new object is almost the same as an existing one. When new objects are nearly identical to existing ones, Prototype is the pattern. You build expensive templates once. Each spawn clones the template and tweaks a few fields.\nThe naive alternative def spawn_warrior(): w = Warrior() w.load_textures() # slow w.compute_initial_stats() # slow w.attach_abilities() # slow return w Calling this in a tight loop costs you a frame budget. Prototype trades that for a one-time setup cost and then very cheap clones.\nThe implementation import copy from dataclasses import dataclass, field from typing import Dict, List # --------------------------------------------------------------- # A simple enemy. In a real game these would have heavy state. # --------------------------------------------------------------- @dataclass class Enemy: name: str health: int attack: int speed: int abilities: List[str] = field(default_factory=list) def clone(self) -\u0026gt; \u0026#34;Enemy\u0026#34;: \u0026#34;\u0026#34;\u0026#34; Deep copy is mandatory here. `abilities` is a list — a mutable object. A shallow copy would share the SAME list between every clone, which is the bug we walk through next. \u0026#34;\u0026#34;\u0026#34; return copy.deepcopy(self) # --------------------------------------------------------------- # The registry — keeps named templates ready to clone # --------------------------------------------------------------- class EnemyRegistry: def __init__(self): self._templates: Dict[str, Enemy] = {} def register(self, name: str, template: Enemy) -\u0026gt; None: self._templates[name] = template def spawn(self, name: str) -\u0026gt; Enemy: if name not in self._templates: raise KeyError(f\u0026#34;Unknown enemy template: {name}\u0026#34;) # Cloning is the fast path. The template\u0026#39;s expensive setup # cost was paid once when register() was called. return self._templates[name].clone() # --------------------------------------------------------------- # Loading the registry once at game start # --------------------------------------------------------------- def build_registry() -\u0026gt; EnemyRegistry: reg = EnemyRegistry() reg.register(\u0026#34;basic_warrior\u0026#34;, Enemy( name=\u0026#34;Warrior\u0026#34;, health=100, attack=15, speed=5, abilities=[\u0026#34;slash\u0026#34;], )) reg.register(\u0026#34;basic_archer\u0026#34;, Enemy( name=\u0026#34;Archer\u0026#34;, health=70, attack=12, speed=8, abilities=[\u0026#34;arrow_shot\u0026#34;], )) reg.register(\u0026#34;basic_mage\u0026#34;, Enemy( name=\u0026#34;Mage\u0026#34;, health=60, attack=20, speed=4, abilities=[\u0026#34;fireball\u0026#34;], )) return reg The shallow-vs-deep copy bug — explicitly Suppose we\u0026rsquo;d used copy.copy (shallow) instead of copy.deepcopy:\nimport copy class BuggyEnemy: def __init__(self, name, abilities): self.name = name self.abilities = abilities def clone(self): return copy.copy(self) # ← shallow # Demo: template = BuggyEnemy(\u0026#34;Warrior\u0026#34;, [\u0026#34;slash\u0026#34;]) clone1 = template.clone() clone2 = template.clone() clone1.abilities.append(\u0026#34;counter_attack\u0026#34;) print(clone2.abilities) # Output: [\u0026#39;slash\u0026#39;, \u0026#39;counter_attack\u0026#39;] ← !! print(template.abilities) # Output: [\u0026#39;slash\u0026#39;, \u0026#39;counter_attack\u0026#39;] ← template corrupted too All three objects share the same abilities list. Modifying one mutates them all. In a game, this surfaces as \u0026ldquo;I added counter_attack to my boss Warrior — now every Warrior in every room has it.\u0026rdquo;\nThe fix is copy.deepcopy, which recursively duplicates all nested mutable objects. The cost is a little extra time per clone, but it\u0026rsquo;s still much faster than constructing from scratch (no asset loading, no stat computation, no ability-system registration).\nUsing the registry — and tweaking registry = build_registry() # Standard spawn goblin = registry.spawn(\u0026#34;basic_warrior\u0026#34;) print(goblin.health) # 100 # Boss spawn — clone and tweak boss = registry.spawn(\u0026#34;basic_warrior\u0026#34;) boss.name = \u0026#34;Warrior King\u0026#34; boss.health = 500 boss.abilities.append(\u0026#34;ground_slam\u0026#34;) # Confirm the template is untouched template_warrior = registry.spawn(\u0026#34;basic_warrior\u0026#34;) assert template_warrior.health == 100 assert \u0026#34;ground_slam\u0026#34; not in template_warrior.abilities Extensibility check A new enemy type — a Necromancer — is one new Enemy(...) registration. The registry, the clone method, every existing enemy: unchanged.\nPattern reveal — Prototype, and an aside Prototype. Use it when:\nConstruction is expensive (asset loading, stat computation, network calls). New instances are nearly identical to existing ones. You want to configure templates declaratively (in data) rather than imperatively (in code). An aside on Python. Prototype is less common in Python than in Java or C++, because most Python objects are cheap to construct. The pattern remains valuable specifically when \u0026ldquo;cheap construction\u0026rdquo; is a lie — heavy I/O, expensive computation, or pre-loaded asset references hidden in the constructor.\nProblem 1.5 — Cloud Environment Configurations (Level 3) Scenario. A cloud infrastructure tool creates environment configurations for dev, staging, and production. Each environment has:\nA compute cluster (CPU cores, memory, instance type) A database cluster (engine, replica count, storage size) A monitoring setup (alert thresholds, log retention) The dev environment is minimal. Staging mirrors prod at a smaller scale. Prod is full-scale.\nThe company runs on both AWS and GCP. An AWS dev environment is not the same as a GCP dev environment — different instance types, different DB engines, different monitoring stacks. All four combinations must be supported. Tomorrow Azure will join.\nWhat you must produce:\nIdentify the two patterns needed and why each one alone is insufficient. Full class hierarchy and implementation. Show that adding Azure or a new environment type (\u0026ldquo;QA\u0026rdquo;) requires only new classes — no edits. Solution Recognition Two distinct needs collide:\nFamily consistency. Every component in an AWS environment must be AWS-flavoured. You can\u0026rsquo;t mix an AWS compute cluster with a GCP database. The set of components for a given cloud provider must travel together as a family. This is the Abstract Factory signal.\nComplex construction. Each component itself is a multi-step build — compute has CPU, memory, instance type, autoscaling rules; the database has engine, replicas, storage, backup policy. A Compute(cpu, mem, type, autoscale, ...) constructor is fragile, and the values differ by environment type (dev vs prod). This is the Builder signal.\nEither pattern alone is insufficient:\nAbstract Factory alone gives you family consistency but forces each \u0026ldquo;create method\u0026rdquo; to do complex construction in one shot. Builder alone gives you readable construction but does not enforce that compute, db, and monitoring all match the same cloud provider. Together: Abstract Factory creates the right family of builders; each builder produces one complex component step-by-step.\nThe implementation We\u0026rsquo;ll walk through the hierarchy in stages so it\u0026rsquo;s not overwhelming.\nStage 1 — Domain (the products) from abc import ABC, abstractmethod from dataclasses import dataclass from typing import List @dataclass class ComputeCluster: provider: str instance_type: str cpu_cores: int memory_gb: int autoscale_min: int autoscale_max: int @dataclass class DatabaseCluster: provider: str engine: str replicas: int storage_gb: int backup_retention_days: int @dataclass class MonitoringSetup: provider: str log_retention_days: int alert_thresholds: dict @dataclass class Environment: name: str compute: ComputeCluster database: DatabaseCluster monitoring: MonitoringSetup Stage 2 — Builders (one per product) Each builder constructs one product step-by-step. The subclasses differ in their defaults and their provider-specific knowledge.\nclass ComputeBuilder(ABC): \u0026#34;\u0026#34;\u0026#34;Stepwise builder for a ComputeCluster.\u0026#34;\u0026#34;\u0026#34; def __init__(self): self._instance_type = \u0026#34;\u0026#34; self._cpu = 1 self._memory = 1 self._scale_min = 1 self._scale_max = 1 @abstractmethod def for_dev(self) -\u0026gt; \u0026#34;ComputeBuilder\u0026#34;: ... @abstractmethod def for_prod(self) -\u0026gt; \u0026#34;ComputeBuilder\u0026#34;: ... def build(self) -\u0026gt; ComputeCluster: return ComputeCluster( provider=self._provider_name(), instance_type=self._instance_type, cpu_cores=self._cpu, memory_gb=self._memory, autoscale_min=self._scale_min, autoscale_max=self._scale_max, ) @abstractmethod def _provider_name(self) -\u0026gt; str: ... class AwsComputeBuilder(ComputeBuilder): def _provider_name(self): return \u0026#34;aws\u0026#34; def for_dev(self): self._instance_type = \u0026#34;t3.small\u0026#34; self._cpu, self._memory = 2, 2 self._scale_min, self._scale_max = 1, 1 return self def for_prod(self): self._instance_type = \u0026#34;m5.2xlarge\u0026#34; self._cpu, self._memory = 8, 32 self._scale_min, self._scale_max = 3, 20 return self class GcpComputeBuilder(ComputeBuilder): def _provider_name(self): return \u0026#34;gcp\u0026#34; def for_dev(self): self._instance_type = \u0026#34;e2-small\u0026#34; self._cpu, self._memory = 2, 2 self._scale_min, self._scale_max = 1, 1 return self def for_prod(self): self._instance_type = \u0026#34;n2-standard-8\u0026#34; self._cpu, self._memory = 8, 32 self._scale_min, self._scale_max = 3, 20 return self # ----- Database builders ----- class DatabaseBuilder(ABC): def __init__(self): self._engine = \u0026#34;\u0026#34; self._replicas = 1 self._storage = 10 self._backup_days = 7 @abstractmethod def for_dev(self) -\u0026gt; \u0026#34;DatabaseBuilder\u0026#34;: ... @abstractmethod def for_prod(self) -\u0026gt; \u0026#34;DatabaseBuilder\u0026#34;: ... @abstractmethod def _provider_name(self) -\u0026gt; str: ... def build(self) -\u0026gt; DatabaseCluster: return DatabaseCluster( provider=self._provider_name(), engine=self._engine, replicas=self._replicas, storage_gb=self._storage, backup_retention_days=self._backup_days, ) class AwsDatabaseBuilder(DatabaseBuilder): def _provider_name(self): return \u0026#34;aws\u0026#34; def for_dev(self): self._engine, self._replicas, self._storage = \u0026#34;rds-postgres\u0026#34;, 1, 20 self._backup_days = 1 return self def for_prod(self): self._engine, self._replicas, self._storage = \u0026#34;aurora-postgres\u0026#34;, 3, 500 self._backup_days = 30 return self class GcpDatabaseBuilder(DatabaseBuilder): def _provider_name(self): return \u0026#34;gcp\u0026#34; def for_dev(self): self._engine, self._replicas, self._storage = \u0026#34;cloudsql-postgres\u0026#34;, 1, 20 self._backup_days = 1 return self def for_prod(self): self._engine, self._replicas, self._storage = \u0026#34;cloudsql-postgres-ha\u0026#34;, 3, 500 self._backup_days = 30 return self # ----- Monitoring builders (analogous) ----- class MonitoringBuilder(ABC): def __init__(self): self._retention = 7 self._thresholds = {} @abstractmethod def for_dev(self) -\u0026gt; \u0026#34;MonitoringBuilder\u0026#34;: ... @abstractmethod def for_prod(self) -\u0026gt; \u0026#34;MonitoringBuilder\u0026#34;: ... @abstractmethod def _provider_name(self) -\u0026gt; str: ... def build(self) -\u0026gt; MonitoringSetup: return MonitoringSetup( provider=self._provider_name(), log_retention_days=self._retention, alert_thresholds=self._thresholds, ) class AwsMonitoringBuilder(MonitoringBuilder): def _provider_name(self): return \u0026#34;aws\u0026#34; def for_dev(self): self._retention = 7 self._thresholds = {\u0026#34;cpu_pct\u0026#34;: 90} return self def for_prod(self): self._retention = 90 self._thresholds = {\u0026#34;cpu_pct\u0026#34;: 80, \u0026#34;memory_pct\u0026#34;: 85, \u0026#34;5xx_pct\u0026#34;: 1} return self class GcpMonitoringBuilder(MonitoringBuilder): def _provider_name(self): return \u0026#34;gcp\u0026#34; def for_dev(self): self._retention = 7 self._thresholds = {\u0026#34;cpu_pct\u0026#34;: 90} return self def for_prod(self): self._retention = 90 self._thresholds = {\u0026#34;cpu_pct\u0026#34;: 80, \u0026#34;memory_pct\u0026#34;: 85, \u0026#34;5xx_pct\u0026#34;: 1} return self Stage 3 — Abstract Factory (one per cloud provider) class CloudProviderFactory(ABC): \u0026#34;\u0026#34;\u0026#34;Creates a CONSISTENT family of builders. All AWS or all GCP.\u0026#34;\u0026#34;\u0026#34; @abstractmethod def compute(self) -\u0026gt; ComputeBuilder: ... @abstractmethod def database(self) -\u0026gt; DatabaseBuilder: ... @abstractmethod def monitoring(self) -\u0026gt; MonitoringBuilder: ... class AwsFactory(CloudProviderFactory): def compute(self): return AwsComputeBuilder() def database(self): return AwsDatabaseBuilder() def monitoring(self): return AwsMonitoringBuilder() class GcpFactory(CloudProviderFactory): def compute(self): return GcpComputeBuilder() def database(self): return GcpDatabaseBuilder() def monitoring(self): return GcpMonitoringBuilder() Stage 4 — The orchestrator class EnvironmentBuilder: \u0026#34;\u0026#34;\u0026#34; Knows the environment recipe (dev / staging / prod). Doesn\u0026#39;t know AWS from GCP — receives a factory. \u0026#34;\u0026#34;\u0026#34; def __init__(self, factory: CloudProviderFactory): self.factory = factory def dev(self) -\u0026gt; Environment: return Environment( name=\u0026#34;dev\u0026#34;, compute=self.factory.compute().for_dev().build(), database=self.factory.database().for_dev().build(), monitoring=self.factory.monitoring().for_dev().build(), ) def prod(self) -\u0026gt; Environment: return Environment( name=\u0026#34;prod\u0026#34;, compute=self.factory.compute().for_prod().build(), database=self.factory.database().for_prod().build(), monitoring=self.factory.monitoring().for_prod().build(), ) Using it aws_dev = EnvironmentBuilder(AwsFactory()).dev() gcp_prod = EnvironmentBuilder(GcpFactory()).prod() assert aws_dev.compute.instance_type == \u0026#34;t3.small\u0026#34; assert gcp_prod.database.engine == \u0026#34;cloudsql-postgres-ha\u0026#34; Extensibility check Add Azure support. Write three new classes — AzureComputeBuilder, AzureDatabaseBuilder, AzureMonitoringBuilder — and one factory AzureFactory that wires them. Zero changes to EnvironmentBuilder, the abstract bases, or AWS/GCP code.\nAdd a new environment type \u0026ldquo;QA\u0026rdquo;. Add qa() to EnvironmentBuilder and a for_qa() to each builder. The factory hierarchy is unchanged. (Or, even better, define environment recipes as data and remove dev/prod methods entirely — but that\u0026rsquo;s an optimisation; the pattern remains.)\nPattern reveal — Abstract Factory + Builder Two patterns, one job:\nAbstract Factory ensures that all components within one environment come from the same family (no AWS compute mixed with GCP database). Builder ensures that each individual component is constructed via a readable, validated, stepwise process — with different defaults for different environment sizes. This combination is common when you have (family of products) × (complex construction per product). Either pattern alone misses half the problem; together they fit the brief exactly.\nSection 2 — Structural Patterns Practice Six problems. The first four single-pattern. Problems 2.5 and 2.6 combine patterns.\nProblem 2.1 — Third-Party Payment SDK (Level 1) Scenario. Your team has a third-party payment library:\nclass ThirdPartyPaymentSDK: def execute_transaction( self, account_num: str, amount_cents: int, currency_code: str ) -\u0026gt; str: \u0026#34;\u0026#34;\u0026#34;Returns a transaction reference.\u0026#34;\u0026#34;\u0026#34; ... Your system uses an internal interface:\nclass PaymentGateway: def charge(self, user_id: str, amount_rupees: float) -\u0026gt; str: ... You cannot modify either side: the SDK is a vendor library, and PaymentGateway is used in 47 places across the codebase. You need them to work together.\nWhat you must produce:\nThe class that makes this work. A short note on why modifying either side would have been wrong. A demonstration that a second vendor SDK could be plugged in tomorrow with no further changes. Solution Recognition The phrase that gives it away: \u0026ldquo;You cannot modify either.\u0026rdquo;\nTwo interfaces have a structural mismatch:\nMethod names differ: charge vs. execute_transaction. Units differ: rupees vs. cents. Identifier types differ: user_id (your domain) vs. account_num (the vendor\u0026rsquo;s domain). You can\u0026rsquo;t reach into the SDK or into the existing interface — but you can build a translator between them. That translator is the Adapter pattern.\nThe naive alternatives — and why they\u0026rsquo;re wrong \u0026ldquo;Modify PaymentGateway to look like the SDK.\u0026rdquo; Forty-seven call sites break. Months of work, plenty of regressions. \u0026ldquo;Wrap the SDK and add a charge method on it.\u0026rdquo; You\u0026rsquo;d have to subclass or monkey-patch a third-party class. Fragile, and breaks on the next vendor upgrade. The Adapter sits between the two — a small class you own, that implements PaymentGateway (because that\u0026rsquo;s what the rest of your code expects) and internally calls the SDK.\nThe implementation from abc import ABC, abstractmethod # --------------------------------------------------------------- # Your existing interface — DO NOT TOUCH. # (47 callers depend on it.) # --------------------------------------------------------------- class PaymentGateway(ABC): @abstractmethod def charge(self, user_id: str, amount_rupees: float) -\u0026gt; str: ... # --------------------------------------------------------------- # The third-party SDK — DO NOT TOUCH. # (Vendor library; we don\u0026#39;t own its source.) # --------------------------------------------------------------- class ThirdPartyPaymentSDK: def execute_transaction(self, account_num, amount_cents, currency_code): print(f\u0026#34;[SDK] {account_num} -\u0026gt; {amount_cents}{currency_code}\u0026#34;) return f\u0026#34;TXN-{account_num}-{amount_cents}\u0026#34; # --------------------------------------------------------------- # The Adapter — the new class we own. # Implements our interface; delegates to the SDK. # --------------------------------------------------------------- class PaymentSDKAdapter(PaymentGateway): def __init__(self, sdk: ThirdPartyPaymentSDK, account_lookup): # `account_lookup` is a callable that maps user_id → account_num. # Injecting it keeps the adapter testable. self.sdk = sdk self.account_lookup = account_lookup def charge(self, user_id: str, amount_rupees: float) -\u0026gt; str: # Translate: user_id → account_num account = self.account_lookup(user_id) # Translate: rupees → paise (cents). # Cast to int because the SDK wants integer minor units. amount_paise = int(round(amount_rupees * 100)) # Translate: implicit \u0026#34;INR\u0026#34; → explicit currency code return self.sdk.execute_transaction( account_num=account, amount_cents=amount_paise, currency_code=\u0026#34;INR\u0026#34;, ) # --------------------------------------------------------------- # Use site — the rest of the codebase doesn\u0026#39;t see the SDK at all. # --------------------------------------------------------------- def lookup_account(user_id: str) -\u0026gt; str: # Real implementation would hit a DB. Demo only. return f\u0026#34;ACC-{user_id}\u0026#34; gateway: PaymentGateway = PaymentSDKAdapter( sdk=ThirdPartyPaymentSDK(), account_lookup=lookup_account, ) ref = gateway.charge(user_id=\u0026#34;U001\u0026#34;, amount_rupees=499.99) # 47 call sites need no change. They keep using PaymentGateway. Extensibility check — a second vendor SDK Tomorrow, you migrate to Razorpay (different method name, different unit conventions). Add a new adapter:\nclass RazorpayAdapter(PaymentGateway): def __init__(self, razorpay_client, account_lookup): self.client = razorpay_client self.account_lookup = account_lookup def charge(self, user_id, amount_rupees): account = self.account_lookup(user_id) return self.client.payments.capture( account=account, amount_paise=int(round(amount_rupees * 100)), ) The 47 call sites still don\u0026rsquo;t change. The migration is a one-line swap at the composition root:\n# Was: # gateway = PaymentSDKAdapter(ThirdPartyPaymentSDK(), lookup_account) # Now: gateway = RazorpayAdapter(razorpay_client, lookup_account) Pattern reveal — Adapter, and when not to use it Adapter. Use it when:\nTwo interfaces are structurally incompatible. One or both sides cannot be modified (legacy code, third-party libraries). The translation logic is mechanical enough to live in one class. Don\u0026rsquo;t use Adapter when:\nYou can modify both sides. Just make them match directly. The \u0026ldquo;translation\u0026rdquo; is so complex it\u0026rsquo;s really business logic in disguise — that work belongs in its own service, not buried in an adapter. Problem 2.2 — Coffee Shop Pricing (Level 2) Scenario. A coffee shop priced its drinks like this:\nBasic espresso = ₹100 Extras: extra shot (+₹30), oat milk (+₹50), caramel syrup (+₹20), whipped cream (+₹25) A customer can add multiple extras. The drink description and total price must reflect every extra. New extras are added to the menu regularly (seasonal: pumpkin spice, lavender syrup).\nWhat you must produce:\nThe naive design, with a count of how many subclasses it produces. The pattern-based design, with full implementation. A demonstration with one drink that has three extras stacked. A short explanation of the call chain that builds the cost. Solution Recognition The signal is combinatorial explosion. With 4 add-ons, there are 2^4 = 16 possible combinations. With 5 add-ons, 32. With 6, 64. A subclass per combination is unmaintainable. A single class with flags is rigid. We need a way to stack behaviour dynamically.\nThat\u0026rsquo;s the Decorator pattern: wrap an object in another object that \u0026ldquo;decorates\u0026rdquo; it with extra behaviour, and recurse — wrapping the wrapped, and the wrapped-wrapped, and so on.\nThe naive subclass-explosion design class Espresso: ... class EspressoWithExtraShot: ... class EspressoWithOatMilk: ... class EspressoWithCaramel: ... class EspressoWithWhippedCream: ... class EspressoWithExtraShotAndOatMilk: ... class EspressoWithOatMilkAndCaramel: ... # ... and 9 more ... For 4 extras: 16 classes. For 5: 32. Now add Latte and Cappuccino — and we have 48 and 96 classes respectively. Nobody is reviewing 96 classes for a coffee shop.\nThe Decorator implementation from abc import ABC, abstractmethod # --------------------------------------------------------------- # The interface every \u0026#34;drink-like thing\u0026#34; must satisfy. # Both base beverages and decorators implement this. # --------------------------------------------------------------- class Beverage(ABC): @abstractmethod def cost(self) -\u0026gt; float: ... @abstractmethod def description(self) -\u0026gt; str: ... # --------------------------------------------------------------- # Base beverages — the \u0026#34;leaf\u0026#34; objects # --------------------------------------------------------------- class Espresso(Beverage): def cost(self): return 100.0 def description(self): return \u0026#34;Espresso\u0026#34; class Latte(Beverage): def cost(self): return 120.0 def description(self): return \u0026#34;Latte\u0026#34; # --------------------------------------------------------------- # The decorator base — wraps another Beverage. # Every concrete decorator subclasses this. # --------------------------------------------------------------- class AddOnDecorator(Beverage): def __init__(self, beverage: Beverage): self._beverage = beverage # Default delegate behaviour — subclasses override with extras. def cost(self): return self._beverage.cost() def description(self): return self._beverage.description() # --------------------------------------------------------------- # Concrete decorators — each adds one specific extra # --------------------------------------------------------------- class ExtraShot(AddOnDecorator): def cost(self): return self._beverage.cost() + 30.0 def description(self): return self._beverage.description() + \u0026#34;, Extra Shot\u0026#34; class OatMilk(AddOnDecorator): def cost(self): return self._beverage.cost() + 50.0 def description(self): return self._beverage.description() + \u0026#34;, Oat Milk\u0026#34; class Caramel(AddOnDecorator): def cost(self): return self._beverage.cost() + 20.0 def description(self): return self._beverage.description() + \u0026#34;, Caramel\u0026#34; class WhippedCream(AddOnDecorator): def cost(self): return self._beverage.cost() + 25.0 def description(self): return self._beverage.description() + \u0026#34;, Whipped Cream\u0026#34; Stacking and the call chain drink: Beverage = Espresso() # cost 100 drink = OatMilk(drink) # cost 100 + 50 = 150 drink = Caramel(drink) # cost 150 + 20 = 170 drink = WhippedCream(drink) # cost 170 + 25 = 195 print(drink.description()) # → \u0026#34;Espresso, Oat Milk, Caramel, Whipped Cream\u0026#34; print(drink.cost()) # → 195.0 Tracing drink.cost() to see how Decorator actually works WhippedCream.cost() → returns self._beverage.cost() + 25 Caramel.cost() → returns self._beverage.cost() + 20 OatMilk.cost() → returns self._beverage.cost() + 50 Espresso.cost() → returns 100 ← 150 ← 170 ← 195 Each decorator asks the inner object for its cost, then adds its own. The recursion unwinds from the deepest layer outward.\nThe description() call works identically — each layer asks the inner layer for its description, then appends its own name.\nExtensibility check — pumpkin spice arrives class PumpkinSpice(AddOnDecorator): def cost(self): return self._beverage.cost() + 35.0 def description(self): return self._beverage.description() + \u0026#34;, Pumpkin Spice\u0026#34; One new class. Zero changes to existing classes. Customers can stack it with anything: PumpkinSpice(WhippedCream(Latte())).\nPattern reveal — Decorator vs. inheritance vs. flags Decorator is the right call when:\nA base object can be enhanced in many independent ways. Combinations are runtime-decided (the user picks at checkout). Adding a new way to enhance the object should not require touching old code. Don\u0026rsquo;t use Decorator when the variants are mutually exclusive (you can only pick one) — a Strategy pattern is cleaner. And don\u0026rsquo;t use it when there are 2–3 fixed combinations — subclasses are simpler.\nProblem 2.3 — Smart Home Scenes (Level 2) Scenario. A smart home system has many subsystems:\nLights — on/off, dim level Thermostat — temperature control Security — arm/disarm Media — play / pause / volume Door locks — lock / unlock A resident defines two scenes:\n\u0026ldquo;Good Morning\u0026rdquo; — lights at 70%, thermostat to 22°C, media plays morning playlist, doors unlock. \u0026ldquo;Good Night\u0026rdquo; — lights off, thermostat to 18°C, security armed, doors locked. The resident\u0026rsquo;s mobile app should trigger a scene with one tap. The app should not need to know about all the subsystems.\nWhat you must produce:\nThe pattern that fits. Full implementation of the subsystems and the simplifying entry point. A note on the important difference between this pattern and Adapter / Proxy. Solution Recognition The key requirement: \u0026ldquo;The app should not need to know about all the subsystems.\u0026rdquo; The app should not be coupled to five different APIs in a specific call order.\nWe need a front door — a single class that exposes \u0026ldquo;Good Morning\u0026rdquo; and \u0026ldquo;Good Night\u0026rdquo; as one-line operations and hides the orchestration inside. That\u0026rsquo;s the Facade pattern.\nThe naive design — app calls subsystems directly def good_morning(app): app.lights.set_brightness(70) app.thermostat.set_temperature(22) app.media.play_playlist(\u0026#34;Morning\u0026#34;) app.locks.unlock_all() Now the app must:\nKnow all five subsystems exist Know their method names Know the right call order (security must be disarmed before doors unlock) Handle errors from each one That\u0026rsquo;s tight coupling. Five subsystems × N scenes × M app screens = a lot of places to break when one subsystem changes.\nThe Facade implementation # --------------------------------------------------------------- # Subsystems — each owns its own behaviour # --------------------------------------------------------------- class Lights: def set_brightness(self, pct: int): print(f\u0026#34;Lights at {pct}%\u0026#34;) def off(self): print(\u0026#34;Lights off\u0026#34;) class Thermostat: def set_temperature(self, celsius: float): print(f\u0026#34;Thermostat -\u0026gt; {celsius}°C\u0026#34;) class Security: def arm(self): print(\u0026#34;Security armed\u0026#34;) def disarm(self): print(\u0026#34;Security disarmed\u0026#34;) class Media: def play(self, playlist: str): print(f\u0026#34;Playing playlist: {playlist}\u0026#34;) def stop(self): print(\u0026#34;Media stopped\u0026#34;) class DoorLocks: def lock_all(self): print(\u0026#34;All doors locked\u0026#34;) def unlock_all(self): print(\u0026#34;All doors unlocked\u0026#34;) # --------------------------------------------------------------- # The Facade — the only thing the app imports # --------------------------------------------------------------- class SmartHomeFacade: def __init__( self, lights: Lights, thermostat: Thermostat, security: Security, media: Media, locks: DoorLocks, ): # Subsystems are still injectable for testing — they didn\u0026#39;t # become hidden; they\u0026#39;re just routed through one entry point. self.lights = lights self.thermostat = thermostat self.security = security self.media = media self.locks = locks # ----------------------------------------------------------- # Scenes — one method per high-level \u0026#34;user intent\u0026#34; # ----------------------------------------------------------- def good_morning(self) -\u0026gt; None: # The Facade owns the ORCHESTRATION ORDER, so nobody else # has to remember it. Security disarmed BEFORE doors unlock — # because otherwise the alarm would trigger. self.security.disarm() self.locks.unlock_all() self.lights.set_brightness(70) self.thermostat.set_temperature(22) self.media.play(\u0026#34;Morning\u0026#34;) def good_night(self) -\u0026gt; None: self.media.stop() self.lights.off() self.thermostat.set_temperature(18) self.locks.lock_all() self.security.arm() # --------------------------------------------------------------- # Use site — one import, one call per scene # --------------------------------------------------------------- home = SmartHomeFacade( Lights(), Thermostat(), Security(), Media(), DoorLocks() ) home.good_morning() Important — Facade does not hide the subsystems A common misconception is that Facade replaces the underlying APIs. It doesn\u0026rsquo;t. The subsystems are still fully usable for advanced users:\n# An automation script needs custom behaviour — it can still reach # into the subsystems directly. The Facade is an alternative # entry point, not a wall. home.thermostat.set_temperature(25) home.lights.set_brightness(30) This is the important distinction between Facade and patterns like Proxy or Adapter:\nAdapter — converts an interface so two parties that couldn\u0026rsquo;t talk can talk. Proxy — controls access to an object (caching, security, lazy loading). Facade — provides a simpler entry point to a complex subsystem, without preventing direct access. Extensibility check — a new \u0026ldquo;Movie Night\u0026rdquo; scene def movie_night(self) -\u0026gt; None: self.lights.set_brightness(15) self.media.play(\u0026#34;Cinema\u0026#34;) self.thermostat.set_temperature(20) One new method on the Facade. Zero changes to subsystems. The app gains a new one-tap action by importing nothing new.\nProblem 2.4 — Library Book Database (Level 2) Scenario. A library system has a BookDatabase whose queries are very expensive — each call to get_book_details(isbn) takes 2–3 seconds. The UI layer calls this method frequently, often requesting the same book multiple times as a user browses. Additionally, some book records are restricted: only librarians can see the full details; regular users see a limited view. You cannot modify BookDatabase.\nWhat you must produce:\nThe pattern (or patterns) that solve both problems. Full implementation showing the two concerns separated. Demonstration that the client cannot tell the difference between the real database and the wrapped version. Solution Recognition Two distinct concerns:\nPerformance — repeated identical calls should be cached. Access control — restricted records should be filtered based on user role. Both fit the Proxy pattern. A Proxy implements the same interface as the real object and stands in for it, controlling access (caching, security, lazy loading, logging, etc.) without callers knowing they\u0026rsquo;re talking to a proxy.\nBecause we have two concerns, we\u0026rsquo;ll stack two proxies: one for caching, one for protection. Both implement the same BookService interface and can be composed in either order — though the typical order is protection-outside, caching-inside.\nThe implementation from abc import ABC, abstractmethod from dataclasses import dataclass from typing import Dict, Optional import time # --------------------------------------------------------------- # The interface every \u0026#34;book service\u0026#34; implements # --------------------------------------------------------------- class BookService(ABC): @abstractmethod def get_book_details(self, isbn: str) -\u0026gt; \u0026#34;Book\u0026#34;: ... @dataclass class Book: isbn: str title: str author: str is_restricted: bool full_details: str # only visible to librarians when restricted # --------------------------------------------------------------- # The REAL service — slow, but we own none of its source # --------------------------------------------------------------- class RealBookDatabase(BookService): def get_book_details(self, isbn): # Pretend this is a 2-second SQL query against a slow DB. time.sleep(2) # Demo data. return Book( isbn=isbn, title=f\u0026#34;Book {isbn}\u0026#34;, author=\u0026#34;Some Author\u0026#34;, is_restricted=(isbn.startswith(\u0026#34;R\u0026#34;)), full_details=f\u0026#34;Detailed contents of {isbn}\u0026#34;, ) # --------------------------------------------------------------- # Proxy #1 — Caching # --------------------------------------------------------------- class CachingBookProxy(BookService): def __init__(self, real: BookService): self._real = real self._cache: Dict[str, Book] = {} def get_book_details(self, isbn): if isbn in self._cache: # Cache hit — instant. No 2-second delay. return self._cache[isbn] # Cache miss — go to the real service, store, return. book = self._real.get_book_details(isbn) self._cache[isbn] = book return book # --------------------------------------------------------------- # Proxy #2 — Protection (access control) # --------------------------------------------------------------- @dataclass class User: user_id: str is_librarian: bool class ProtectionBookProxy(BookService): def __init__(self, inner: BookService, current_user: User): self._inner = inner self._user = current_user def get_book_details(self, isbn): book = self._inner.get_book_details(isbn) if book.is_restricted and not self._user.is_librarian: # Return a \u0026#34;redacted\u0026#34; copy. Don\u0026#39;t leak the full details. return Book( isbn=book.isbn, title=book.title, author=book.author, is_restricted=True, full_details=\u0026#34;[restricted — librarian access only]\u0026#34;, ) return book Stacking — protection wraps caching wraps the real DB # Composition order matters: protection outside, caching inside. # That way the cache stores the REAL data once; protection filters # per-request based on the user. db = RealBookDatabase() cached = CachingBookProxy(db) service: BookService = ProtectionBookProxy(cached, current_user=User(\u0026#34;U001\u0026#34;, is_librarian=False)) # Client code: b1 = service.get_book_details(\u0026#34;R-9001\u0026#34;) # 2s — slow miss + redacted b2 = service.get_book_details(\u0026#34;R-9001\u0026#34;) # instant — cache hit, still redacted print(b1.full_details) # \u0026#34;[restricted — ...]\u0026#34; Proxy vs. Decorator — the same shape, different intent Look at the code structurally and CachingBookProxy looks exactly like a Decorator. Same shape: wraps the real object, implements the same interface, delegates internally.\nThe intent distinguishes them:\nDecorator adds behaviour — e.g., adding caramel to coffee. Proxy controls access — e.g., caching to make access faster, protection to make access conditional. Both are valid here; the convention is to call them Proxies because the intent is control of access (caching for performance control, protection for security control), not enhancement of capability.\nExtensibility check — a third proxy Want to log every query for analytics? Add a LoggingBookProxy(BookService) that wraps the chain. Zero changes to existing proxies or the real DB.\nProblem 2.5 — File System with Virtual Files (Level 3) Scenario. A file system manager displays folder hierarchies.\nA Folder can contain Files and other Folders. size() on a Folder returns the recursive total size of contents. display() on a Folder shows an indented tree view. Files have fixed sizes; Folders compute their size from contents. Additionally: some files are virtual — they\u0026rsquo;re generated on demand (e.g., a thumbnail computed from a source image only when first accessed). Virtual files must behave exactly like real files from the caller\u0026rsquo;s perspective. Their content must be generated lazily on first access.\nWhat you must produce:\nThe two patterns needed and where each one fits. Full implementation. Demonstration that the caller cannot distinguish a virtual file from a real one. Solution Recognition Two patterns are at work:\n\u0026ldquo;Folders contain Files and other Folders, treated identically by size() and display().\u0026rdquo; This is a tree where leaves and composites share an interface — the Composite pattern. \u0026ldquo;Virtual files generate content lazily on first access, behaving like real files.\u0026rdquo; This is a Proxy that delays expensive work and pretends to be the real thing. The proxy lives inside the Composite tree as a leaf. The Composite doesn\u0026rsquo;t know which of its leaves are real and which are proxies — and that\u0026rsquo;s the design\u0026rsquo;s beauty.\nThe implementation from abc import ABC, abstractmethod from typing import List, Callable # --------------------------------------------------------------- # The common interface — every node implements it # --------------------------------------------------------------- class FileSystemNode(ABC): @abstractmethod def size(self) -\u0026gt; int: ... @abstractmethod def display(self, indent: int = 0) -\u0026gt; None: ... @property @abstractmethod def name(self) -\u0026gt; str: ... # --------------------------------------------------------------- # A real file — leaf # --------------------------------------------------------------- class RealFile(FileSystemNode): def __init__(self, name: str, content: bytes): self._name = name self._content = content @property def name(self): return self._name def size(self): return len(self._content) def display(self, indent=0): print(\u0026#34; \u0026#34; * indent + f\u0026#34;📄 {self._name} ({self.size()} bytes)\u0026#34;) # --------------------------------------------------------------- # A Folder — the COMPOSITE # --------------------------------------------------------------- class Folder(FileSystemNode): def __init__(self, name: str): self._name = name self._children: List[FileSystemNode] = [] @property def name(self): return self._name def add(self, node: FileSystemNode) -\u0026gt; None: self._children.append(node) def size(self): # Recursive: ask each child for its size — and each child # decides for itself how that\u0026#39;s computed. Real file? Stored # content length. Sub-folder? Recurse. Virtual file? Trigger # generation. The folder doesn\u0026#39;t know or care. return sum(child.size() for child in self._children) def display(self, indent=0): print(\u0026#34; \u0026#34; * indent + f\u0026#34;📁 {self._name}/\u0026#34;) for child in self._children: child.display(indent + 2) # --------------------------------------------------------------- # A LazyFile — PROXY for an expensive-to-generate file # Implements the SAME FileSystemNode interface, so it can slot into # a Folder anywhere a RealFile could go. # --------------------------------------------------------------- class LazyFile(FileSystemNode): def __init__(self, name: str, generator: Callable[[], bytes]): self._name = name self._generator = generator # called only on demand self._real: RealFile | None = None # the real file, when created @property def name(self): return self._name def _materialize(self) -\u0026gt; RealFile: if self._real is None: print(f\u0026#34; (generating {self._name}...)\u0026#34;) content = self._generator() self._real = RealFile(self._name, content) return self._real def size(self): return self._materialize().size() def display(self, indent=0): # The first display() call triggers generation. self._materialize().display(indent) Demonstration — the caller cannot tell def make_thumbnail() -\u0026gt; bytes: # Imagine this opens the source image, resizes, and encodes. # Expensive. return b\u0026#34;FAKE_THUMBNAIL_BYTES_\u0026#34; * 100 root = Folder(\u0026#34;photos\u0026#34;) root.add(RealFile(\u0026#34;vacation.jpg\u0026#34;, b\u0026#34;x\u0026#34; * 5000)) root.add(LazyFile(\u0026#34;vacation_thumb.jpg\u0026#34;, make_thumbnail)) subfolder = Folder(\u0026#34;2024\u0026#34;) subfolder.add(RealFile(\u0026#34;birthday.jpg\u0026#34;, b\u0026#34;x\u0026#34; * 3000)) subfolder.add(LazyFile(\u0026#34;birthday_thumb.jpg\u0026#34;, make_thumbnail)) root.add(subfolder) # Display the tree — virtual files generate on demand. root.display() print(f\u0026#34;Total bytes: {root.size()}\u0026#34;) Output (the (generating…) lines reveal the lazy materialisation, but the caller doesn\u0026rsquo;t need to know):\n📁 photos/ 📄 vacation.jpg (5000 bytes) (generating vacation_thumb.jpg...) 📄 vacation_thumb.jpg (2000 bytes) 📁 2024/ 📄 birthday.jpg (3000 bytes) (generating birthday_thumb.jpg...) 📄 birthday_thumb.jpg (2000 bytes) Total bytes: 12000 Why this combination works The Composite pattern says: \u0026ldquo;give every node — real file, virtual file, folder — the same interface, and the parent can treat them uniformly.\u0026rdquo; The Proxy pattern says: \u0026ldquo;make the virtual file look exactly like a real file.\u0026rdquo; Stack them: the proxy is a leaf in the composite tree, indistinguishable from a real leaf at the interface level. The folder\u0026rsquo;s recursive size() works correctly without ever knowing about virtual files.\nExtensibility check A new node type — Shortcut (a symlink-like reference)? Implement FileSystemNode. Resolve the target on demand. Zero changes elsewhere. A \u0026ldquo;compressed file\u0026rdquo; leaf that decompresses on display? Same shape — a proxy that materialises on demand. Problem 2.6 — UI Rendering Across Platforms (Level 3) Scenario. A UI rendering system has two independent axes of variation:\nShape type — Circle, Square, Triangle. Rendering platform — SVG, Canvas, WebGL. The current team is creating one class per combination: SVGCircle, SVGSquare, CanvasCircle, CanvasSquare, WebGLCircle, etc. — 9 classes for 3 shapes × 3 platforms. Adding a 4th platform (Vulkan) means 3 more classes. Adding a 4th shape means 3 more classes. Refactor this.\nWhat you must produce:\nIdentify the pattern. Show the class-count before and after. Full implementation with at least 2 shapes × 2 platforms. Explain the \u0026ldquo;axes of variation\u0026rdquo; intuition that the pattern formalises. Solution Recognition The signal is class explosion from two independent axes. If you have N shapes × M platforms classes, and adding either an N+1th shape or an M+1th platform adds M or N new classes, you have the textbook Bridge pattern problem.\nThe Bridge separates the two axes:\nAn abstraction (Shape) for the what — circles and squares. An implementor (Renderer) for the how — SVG, Canvas, WebGL. Each shape holds a Renderer. To draw, the shape decomposes itself into primitives (draw a circle = draw an arc; draw a triangle = draw three lines) and asks the renderer to handle them.\nBefore vs. after — class count Naive (subclass-per-combination) Bridge 3 shapes × 3 platforms 9 classes 3 + 3 = 6 classes 3 shapes × 4 platforms 12 classes 3 + 4 = 7 classes 4 shapes × 4 platforms 16 classes 4 + 4 = 8 classes 10 shapes × 5 platforms 50 classes 10 + 5 = 15 classes Each axis grows linearly instead of multiplicatively. Add a new platform = 1 new renderer class, every existing shape works on it. Add a new shape = 1 new shape class, every existing renderer can draw it.\nThe implementation from abc import ABC, abstractmethod # --------------------------------------------------------------- # The IMPLEMENTOR axis — rendering primitives # A renderer knows how to draw a few low-level shapes; high-level # shapes compose themselves out of these primitives. # --------------------------------------------------------------- class Renderer(ABC): @abstractmethod def render_circle(self, cx: float, cy: float, radius: float) -\u0026gt; None: ... @abstractmethod def render_line(self, x1: float, y1: float, x2: float, y2: float) -\u0026gt; None: ... class SVGRenderer(Renderer): def render_circle(self, cx, cy, radius): print(f\u0026#39;\u0026lt;svg:circle cx=\u0026#34;{cx}\u0026#34; cy=\u0026#34;{cy}\u0026#34; r=\u0026#34;{radius}\u0026#34;/\u0026gt;\u0026#39;) def render_line(self, x1, y1, x2, y2): print(f\u0026#39;\u0026lt;svg:line x1=\u0026#34;{x1}\u0026#34; y1=\u0026#34;{y1}\u0026#34; x2=\u0026#34;{x2}\u0026#34; y2=\u0026#34;{y2}\u0026#34;/\u0026gt;\u0026#39;) class CanvasRenderer(Renderer): def render_circle(self, cx, cy, radius): print(f\u0026#34;ctx.arc({cx}, {cy}, {radius}, 0, 2 * Math.PI);\u0026#34;) def render_line(self, x1, y1, x2, y2): print(f\u0026#34;ctx.moveTo({x1}, {y1}); ctx.lineTo({x2}, {y2});\u0026#34;) # --------------------------------------------------------------- # The ABSTRACTION axis — shapes # A shape holds a Renderer and uses its primitives. # --------------------------------------------------------------- class Shape(ABC): def __init__(self, renderer: Renderer): self.renderer = renderer @abstractmethod def draw(self) -\u0026gt; None: ... class Circle(Shape): def __init__(self, renderer, cx, cy, radius): super().__init__(renderer) self.cx, self.cy, self.radius = cx, cy, radius def draw(self): # A circle decomposes to one renderer primitive. self.renderer.render_circle(self.cx, self.cy, self.radius) class Triangle(Shape): def __init__(self, renderer, p1, p2, p3): super().__init__(renderer) self.p1, self.p2, self.p3 = p1, p2, p3 def draw(self): # A triangle decomposes to three line primitives. self.renderer.render_line(*self.p1, *self.p2) self.renderer.render_line(*self.p2, *self.p3) self.renderer.render_line(*self.p3, *self.p1) Use site — any shape, any renderer, any combination svg = SVGRenderer() canvas = CanvasRenderer() shapes = [ Circle(svg, 100, 100, 50), Circle(canvas, 100, 100, 50), Triangle(svg, (0, 0), (10, 0), (5, 8)), Triangle(canvas, (0, 0), (10, 0), (5, 8)), ] for s in shapes: s.draw() Six \u0026ldquo;classes\u0026rdquo; of behaviour exist (2 shapes × 3 renderers) — but only 5 actual classes (Shape ABC + Circle + Triangle + Renderer ABC + 2 concrete renderers excluded already). The combinations are runtime combinations of orthogonal axes, not compile-time combinations of subclasses.\nExtensibility check Add Square (a new shape): one new class, Square(Shape). It decomposes into 4 line primitives. Works on every existing renderer. Add WebGL (a new platform): one new class, WebGLRenderer(Renderer). Every existing shape can now render to WebGL. Add Vulkan: same — one new renderer class. Compare with the naive version, where adding WebGL meant three new classes (WebGLCircle, WebGLSquare, WebGLTriangle). The savings compound.\nPattern reveal — Bridge, and the intuition Bridge. Use it when:\nYou have two (or more) independent axes of variation. A naive subclass-per-combination produces N×M classes. Both axes are likely to grow over time. The intuition. When you find yourself drawing a table where the rows are one kind of thing (shapes) and the columns are another (platforms), and each cell is a class — that table is screaming for Bridge. The rows become one class hierarchy, the columns become another, and the cells disappear because the \u0026ldquo;combination\u0026rdquo; lives in composition (one object holding the other), not in inheritance.\n","permalink":"https://blogs.codeverra.com/blog/low_level_design/06_design_patterns_practice/","summary":"\u003ch1 id=\"design-patterns--practice-workbook\"\u003eDesign Patterns — Practice Workbook\u003c/h1\u003e\n\u003cblockquote\u003e\n\u003cp\u003e\u003cstrong\u003eWho this is for:\u003c/strong\u003e Anyone who has worked through the three teaching files — \u003ccode\u003e03_design_patterns_creational.md\u003c/code\u003e, \u003ccode\u003e04_design_patterns_structural.md\u003c/code\u003e, and \u003ccode\u003e05_design_patterns_behavioral.md\u003c/code\u003e — and now wants to \u003cem\u003eprove they actually understand\u003c/em\u003e by solving problems independently.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003chr\u003e\n\u003ch2 id=\"table-of-contents\"\u003eTable of Contents\u003c/h2\u003e\n\u003col\u003e\n\u003cli\u003e\u003ca href=\"#preface\"\u003ePreface — How to Use This Workbook\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#section-1--creational-patterns-practice\"\u003eSection 1 — Creational Patterns Practice\u003c/a\u003e\n\u003cul\u003e\n\u003cli\u003e\u003ca href=\"#problem-11--application-configuration-level-1\"\u003eProblem 1.1 — Application Configuration (Level 1)\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#problem-12--document-export-flow-level-1\"\u003eProblem 1.2 — Document Export Flow (Level 1)\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#problem-13--food-delivery-order-level-2\"\u003eProblem 1.3 — Food Delivery Order (Level 2)\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#problem-14--game-enemy-spawning-level-2\"\u003eProblem 1.4 — Game Enemy Spawning (Level 2)\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#problem-15--cloud-environment-configurations-level-3\"\u003eProblem 1.5 — Cloud Environment Configurations (Level 3)\u003c/a\u003e\u003c/li\u003e\n\u003c/ul\u003e\n\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#section-2--structural-patterns-practice\"\u003eSection 2 — Structural Patterns Practice\u003c/a\u003e\n\u003cul\u003e\n\u003cli\u003e\u003ca href=\"#problem-21--third-party-payment-sdk-level-1\"\u003eProblem 2.1 — Third-Party Payment SDK (Level 1)\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#problem-22--coffee-shop-pricing-level-2\"\u003eProblem 2.2 — Coffee Shop Pricing (Level 2)\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#problem-23--smart-home-scenes-level-2\"\u003eProblem 2.3 — Smart Home Scenes (Level 2)\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#problem-24--library-book-database-level-2\"\u003eProblem 2.4 — Library Book Database (Level 2)\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#problem-25--file-system-with-virtual-files-level-3\"\u003eProblem 2.5 — File System with Virtual Files (Level 3)\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#problem-26--ui-rendering-across-platforms-level-3\"\u003eProblem 2.6 — UI Rendering Across Platforms (Level 3)\u003c/a\u003e\u003c/li\u003e\n\u003c/ul\u003e\n\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#section-3--behavioural-patterns-practice\"\u003eSection 3 — Behavioural Patterns Practice\u003c/a\u003e\n\u003cul\u003e\n\u003cli\u003e\u003ca href=\"#problem-31--stock-price-monitor-level-1\"\u003eProblem 3.1 — Stock Price Monitor (Level 1)\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#problem-32--e-commerce-shipping-level-1\"\u003eProblem 3.2 — E-commerce Shipping (Level 1)\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#problem-33--text-editor-undoredo-level-2\"\u003eProblem 3.3 — Text Editor Undo/Redo (Level 2)\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#problem-34--vending-machine-level-2\"\u003eProblem 3.4 — Vending Machine (Level 2)\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#problem-35--order-validation-pipeline-level-2\"\u003eProblem 3.5 — Order Validation Pipeline (Level 2)\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#problem-36--news-aggregator-level-3\"\u003eProblem 3.6 — News Aggregator (Level 3)\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#problem-37--chess-engine-level-3\"\u003eProblem 3.7 — Chess Engine (Level 3)\u003c/a\u003e\u003c/li\u003e\n\u003c/ul\u003e\n\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#section-4--combination-problems-multiple-patterns-together\"\u003eSection 4 — Combination Problems: Multiple Patterns Together\u003c/a\u003e\n\u003cul\u003e\n\u003cli\u003e\u003ca href=\"#problem-41--ride-sharing-app-level-3\"\u003eProblem 4.1 — Ride-Sharing App (Level 3)\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#problem-42--hotel-booking-platform-level-3\"\u003eProblem 4.2 — Hotel Booking Platform (Level 3)\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#problem-43--notification-service-level-3\"\u003eProblem 4.3 — Notification Service (Level 3)\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#problem-44--document-management-system-level-3--hardest\"\u003eProblem 4.4 — Document Management System (Level 3 — Hardest)\u003c/a\u003e\u003c/li\u003e\n\u003c/ul\u003e\n\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#section-5--anti-pattern-traps-when-not-to-use-a-pattern\"\u003eSection 5 — Anti-Pattern Traps: When NOT to Use a Pattern\u003c/a\u003e\n\u003cul\u003e\n\u003cli\u003e\u003ca href=\"#problem-51--a-simple-csv-script\"\u003eProblem 5.1 — A Simple CSV Script\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#problem-52--temperatureconverter\"\u003eProblem 5.2 — TemperatureConverter\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#problem-53--internal-bulk-rename-tool\"\u003eProblem 5.3 — Internal Bulk-Rename Tool\u003c/a\u003e\u003c/li\u003e\n\u003c/ul\u003e\n\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#summary--pattern-selection-reference-card\"\u003eSummary — Pattern Selection Reference Card\u003c/a\u003e\u003c/li\u003e\n\u003c/ol\u003e\n\u003chr\u003e\n\u003ch2 id=\"preface\"\u003ePreface\u003c/h2\u003e\n\u003cp\u003eYou\u0026rsquo;ve read the three teaching files. Each one \u003cem\u003egave you\u003c/em\u003e the pattern: \u0026ldquo;Here is Observer, here is what it solves, here is the example.\u0026rdquo; Reading those files makes you familiar with patterns. It does not make you good at using them.\u003c/p\u003e","tags":["lld","design-patterns","python"],"title":"Design Patterns: Practice Workbook"},{"content":"SOLID Principles Prerequisite: This material assumes familiarity with Python object-oriented programming, including classes, inheritance, polymorphism, and abstraction. The discussion builds directly on that foundation and extends it toward writing object-oriented code that remains maintainable as a system grows.\nTable of Contents Why SOLID Exists What SOLID Stands For Single Responsibility Principle (SRP) Open/Closed Principle (OCP) Liskov Substitution Principle (LSP) Interface Segregation Principle (ISP) Dependency Inversion Principle (DIP) How the Five Principles Reinforce Each Other Capstone Example: Applying All Five Together Common Pitfalls and Anti-Patterns When Not to Apply SOLID Summary and Key Takeaways Self-Assessment Questions Practice Exercises with Solutions 1. Why SOLID Exists A learner who has completed a course on object-oriented programming knows how to define classes, establish inheritance hierarchies, and use polymorphism. A natural question follows: if these tools are already available, why does a separate set of principles need to be learned before writing \u0026ldquo;good\u0026rdquo; object-oriented code?\nThe answer becomes clear once a codebase grows beyond the size of a classroom exercise. Consider a common scenario in professional software development. An engineer joins a team maintaining a system of fifty classes. A request arrives to add a new payment method. The change takes two days, touches six files, and breaks two unrelated tests. The engineer fixes them. The following week, a colleague adds an unrelated feature, and it breaks the first engineer\u0026rsquo;s code. Over several sprints, the word \u0026ldquo;refactor\u0026rdquo; becomes something the team avoids because no one is confident about the blast radius of a change. Development velocity declines steadily even though the team is not writing objectively bad code line by line.\nThis outcome is not caused by a flaw in Python or in object-oriented programming as a paradigm. It is a design problem. The classes exist and function correctly in isolation, but the relationships between them are tangled in a way that causes a change in one place to propagate unpredictably to others.\nObject-oriented programming supplies the raw constructs: classes, objects, inheritance, and polymorphism. These constructs do not, by themselves, guarantee a well-structured system, in the same way that possessing bricks and cement does not guarantee a structurally sound building. The arrangement of the materials determines whether the resulting structure is sound or fragile.\nSOLID is a set of five design principles that describe how object-oriented constructs should be arranged so that the resulting system exhibits four properties:\nProperty Definition Flexibility New features can be added with proportionally small effort. Robustness A change in one part of the system does not produce unintended effects elsewhere. Testability Individual components can be verified in isolation, without requiring the entire system to be running. Readability Each component has a clearly identifiable purpose, so a reader does not need to trace unrelated logic to understand it. Relating SOLID to OOP It is useful to think of object-oriented programming as supplying the vocabulary of a language, where classes act as nouns and methods act as verbs. SOLID supplies the grammar: the rules that determine how this vocabulary should be composed so that the resulting statements remain coherent as more are added. A system with correct vocabulary but poor grammar can still communicate its intent, but every addition makes the whole harder to parse and more prone to contradiction.\nHistorical Note The individual principles predate the acronym. Several were described by Bertrand Meyer and Barbara Liskov in the late 1980s. Robert C. Martin (commonly referred to as \u0026ldquo;Uncle Bob\u0026rdquo;) consolidated and popularized them in the early 2000s, and the acronym SOLID itself was coined later by Michael Feathers. These principles are not laws in the mathematical sense. They are heuristics distilled from decades of observed failure patterns in large codebases, and they should be applied with judgment rather than followed dogmatically. Section 11 addresses this distinction in detail.\n2. What SOLID Stands For Letter Principle Core Statement S Single Responsibility Principle A class should have only one reason to change. O Open/Closed Principle Software entities should be open for extension but closed for modification. L Liskov Substitution Principle Objects of a subtype must be substitutable for objects of their base type without altering the correctness of the program. I Interface Segregation Principle Clients should not be forced to depend on methods they do not use. D Dependency Inversion Principle High-level modules and low-level modules should both depend on abstractions, not on each other directly. Each principle is examined in detail in the sections that follow, including the specific problem it addresses, the mechanism by which it solves that problem, and the ways in which it can be misapplied.\n3. Single Responsibility Principle (SRP) \u0026ldquo;A class should have one, and only one, reason to change.\u0026rdquo; Robert C. Martin\nPrecise Definition A common but imprecise reading of SRP states that \u0026ldquo;a class should do only one thing.\u0026rdquo; This phrasing is too vague to apply consistently, since almost any class can be described as doing \u0026ldquo;one thing\u0026rdquo; at a sufficiently abstract level, and almost any class can also be decomposed into multiple smaller actions at a sufficiently granular level.\nThe precise formulation, as intended by Martin, concerns stakeholders, not method count:\nA class should have exactly one stakeholder, or one category of stakeholder, whose changing requirements would require modifying that class.\nIn this formulation, \u0026ldquo;reason to change\u0026rdquo; means \u0026ldquo;a distinct source of requirements that could independently demand a change.\u0026rdquo; If two responsibilities are typically modified by different people, for different business reasons, at different times, they represent two separate reasons to change and therefore belong in two separate classes.\nThe Problem Without SRP Consider the following class:\nclass Employee: def __init__(self, name, hours_worked, hourly_rate): self.name = name self.hours_worked = hours_worked self.hourly_rate = hourly_rate def calculate_pay(self): return self.hours_worked * self.hourly_rate def save_to_database(self): # Opens DB connection, writes employee, closes connection print(f\u0026#34;Saving {self.name} to DB...\u0026#34;) def generate_report(self, format): if format == \u0026#34;pdf\u0026#34;: print(f\u0026#34;Generating PDF for {self.name}...\u0026#34;) elif format == \u0026#34;html\u0026#34;: print(f\u0026#34;Generating HTML for {self.name}...\u0026#34;) This single class serves three distinct stakeholders:\nThe finance department, which owns the rules governing pay calculation. The database or infrastructure team, which owns how employee data is persisted. The reporting team, which owns how employee data is presented. Whenever finance revises pay rules, this class must be modified. Whenever the infrastructure team migrates to a new database technology, this class must be modified. Whenever reporting adds a new output format, this class must be modified. The class becomes what practitioners refer to as a hot file, a file that is edited frequently for unrelated reasons, which increases the probability that an unrelated change introduces a regression.\nApplying SRP Each responsibility is extracted into its own class:\nclass Employee: def __init__(self, name, hours_worked, hourly_rate): self.name = name self.hours_worked = hours_worked self.hourly_rate = hourly_rate class PayrollCalculator: def calculate_pay(self, employee): return employee.hours_worked * employee.hourly_rate class EmployeeRepository: def save(self, employee): print(f\u0026#34;Saving {employee.name} to DB...\u0026#34;) class EmployeeReportGenerator: def generate(self, employee, format): if format == \u0026#34;pdf\u0026#34;: print(f\u0026#34;PDF for {employee.name}\u0026#34;) elif format == \u0026#34;html\u0026#34;: print(f\u0026#34;HTML for {employee.name}\u0026#34;) With this structure, each stakeholder\u0026rsquo;s changes are isolated:\nA revision to pay rules affects only PayrollCalculator. A database migration affects only EmployeeRepository. A new report format affects only EmployeeReportGenerator. Each class now has exactly one reason to change, satisfying the principle.\nA Common Misreading SRP does not state that a class must have only one method, nor that classes must be kept extremely small. A class can legitimately contain many methods if all of them serve one cohesive responsibility. For example, a ShoppingCart class with methods add_item, remove_item, update_quantity, total_price, and clear has five methods but a single responsibility: managing the state of a cart. This design satisfies SRP despite having multiple methods, because all five methods change for the same reason, namely a change in how the cart\u0026rsquo;s contents are managed.\nIndicators of an SRP Violation Indicator Explanation The class name contains \u0026ldquo;and\u0026rdquo; Names such as UserAndEmailSender signal that two responsibilities were merged into one class. Groups of methods never share state If half the methods never touch the same instance attributes as the other half, the class is likely serving two purposes. Multiple teams edit the same class If different teams routinely modify the same file for unrelated reasons, responsibilities have not been separated. Business logic is mixed with I/O A class that both computes a result and performs database access, file access, or network access typically has more than one reason to change. Unrelated test failures A change to one method causing failures in tests for a seemingly unrelated feature indicates hidden coupling between responsibilities. Scope of SRP Beyond Classes The principle applies at multiple levels of granularity within a system:\nMethods: a single method should perform one logical operation. A three-hundred-line method that both validates input and performs a calculation and formats output is a violation at the method level. Modules and packages: a module should group functionality that serves one cohesive purpose. Services: in a distributed system, a microservice should own one business capability rather than several unrelated ones. The scale changes, but the underlying reasoning, isolating distinct sources of change, remains identical.\n4. Open/Closed Principle (OCP) \u0026ldquo;Software entities (classes, modules, functions) should be open for extension, but closed for modification.\u0026rdquo; Bertrand Meyer, 1988, popularized by Robert C. Martin\nPrecise Definition The principle states that new behavior should be addable to a system without altering the source code of existing, already-tested components. \u0026ldquo;Open for extension\u0026rdquo; means the system\u0026rsquo;s behavior can be extended as requirements grow. \u0026ldquo;Closed for modification\u0026rdquo; means that extending the behavior does not require editing the source code of the module that already works.\nThis does not mean a class can never be edited. Bug fixes and legitimate design corrections necessarily involve modification. The principle addresses the common, expected axis of change: when a system is known to require new variants of a certain behavior over time (new discount types, new payment methods, new shape types), the code should be structured so that adding a variant means adding new code rather than editing existing code.\nModifying working code carries risk. Every edit to an already-functioning class introduces the possibility of a regression, requires re-running and possibly rewriting existing tests, and can trigger changes in code that depends on the modified class. OCP reduces this risk by containing extension to new code paths.\nThe Problem Without OCP Consider a discount calculator implemented with conditional branching:\nclass DiscountCalculator: def calculate(self, customer_type, amount): if customer_type == \u0026#34;regular\u0026#34;: return amount * 0.05 elif customer_type == \u0026#34;premium\u0026#34;: return amount * 0.10 elif customer_type == \u0026#34;vip\u0026#34;: return amount * 0.20 else: return 0 When a new customer category, \u0026ldquo;corporate,\u0026rdquo; is introduced with a fifteen percent discount, the existing method must be opened and another elif branch added. When a \u0026ldquo;student\u0026rdquo; category follows with an eight percent discount, the method is edited again. Over time, as seasonal discounts, regional discounts, and first-time-buyer discounts accumulate, the method grows into a large conditional chain in which every new addition risks altering the behavior of an existing branch, and every addition requires re-verifying the entire method.\nApplying OCP Polymorphism is used to allow new behavior to be introduced through new classes rather than through modification of existing ones:\nfrom abc import ABC, abstractmethod class DiscountStrategy(ABC): @abstractmethod def calculate(self, amount): pass class RegularDiscount(DiscountStrategy): def calculate(self, amount): return amount * 0.05 class PremiumDiscount(DiscountStrategy): def calculate(self, amount): return amount * 0.10 class VIPDiscount(DiscountStrategy): def calculate(self, amount): return amount * 0.20 class DiscountCalculator: def calculate(self, strategy, amount): return strategy.calculate(amount) Adding a corporate discount now requires only a new class:\nclass CorporateDiscount(DiscountStrategy): def calculate(self, amount): return amount * 0.15 No line inside DiscountCalculator or any existing strategy class is changed. The system has been extended, not modified.\nRelationship to Polymorphism OCP depends on polymorphism as its enabling mechanism. Without polymorphism, extending behavior would require explicit type checks, such as if isinstance(x, RegularDiscount), and every new type would require editing this check, which is precisely the modification OCP seeks to avoid. Polymorphism allows the calling code, DiscountCalculator in this example, to remain unaware of how many strategy classes exist.\nA Necessary Qualification OCP does not imply that every part of a system should be made abstract and extensible. Attempting to make everything open for extension produces excessive layers of abstraction that make the system harder to understand without providing corresponding benefit. The engineering judgment required here is identifying which axes of change are likely to occur in practice and protecting only those axes with the appropriate abstraction. Stable, unlikely-to-change logic can remain concrete.\nIndicators of an OCP Violation Indicator Explanation Growing conditional chains Every new feature requires adding a branch to an existing if/elif or switch structure. Type checks scattered across files Adding a new type requires updating multiple files that each check for that type explicitly. Repeated re-testing of stable code Old, previously verified functionality must be re-tested every time a new feature is added. Maintenance comments Comments such as \u0026ldquo;remember to update this when a new type is added\u0026rdquo; indicate a hardcoded extension point. 5. Liskov Substitution Principle (LSP) \u0026ldquo;If S is a subtype of T, then objects of type T in a program may be replaced with objects of type S without altering any desirable properties of that program.\u0026rdquo; Barbara Liskov, 1987\nPrecise Definition LSP states that any code written to operate on a base class should continue to operate correctly when given an instance of any subclass of that base class, without the calling code needing to know which specific subclass it received.\nIf a class Sparrow inherits from Bird, then any function written to accept a Bird argument must behave correctly when a Sparrow instance is passed to it, with no special handling required. LSP formalizes what it means for an inheritance relationship to be behaviorally correct, as opposed to merely syntactically valid. It distinguishes a subclass that genuinely satisfies the contract of its parent from a subclass that shares a superficial resemblance to the parent but violates the parent\u0026rsquo;s guarantees when actually used.\nClassic Violation: The Rectangle and Square Problem Geometrically, a square is a special case of a rectangle where width equals height. Modeling this relationship through inheritance produces a behavioral contradiction:\nclass Rectangle: def __init__(self, width, height): self.width = width self.height = height def set_width(self, w): self.width = w def set_height(self, h): self.height = h def area(self): return self.width * self.height class Square(Rectangle): def __init__(self, side): super().__init__(side, side) def set_width(self, w): self.width = w self.height = w # Keep it square def set_height(self, h): self.width = h self.height = h Consider code written against the Rectangle interface:\ndef stretch_rectangle(rect: Rectangle): rect.set_width(10) rect.set_height(5) assert rect.area() == 50 # Expected for a rectangle Passing a genuine Rectangle(1, 1) produces an area of 50, as expected. Passing a Square(1) produces an area of 25, and the assertion fails. The Square subclass silently breaks an implicit contract of Rectangle, namely that width and height can be set independently of one another. Any code written against Rectangle, without knowledge of Square, contains a latent defect that surfaces only when a Square instance is supplied.\nThis demonstrates that LSP is a behavioral criterion, not a mathematical or taxonomic one. Despite the \u0026ldquo;is-a\u0026rdquo; relationship holding mathematically, Square is not substitutable for Rectangle in this design, and the principle is therefore violated. The correct resolution is to avoid modeling Square as a subclass of Rectangle. The two should be represented as separate types, each with its own contract.\nAnother Classic Violation: Birds and Flight class Bird: def fly(self): print(\u0026#34;Flying!\u0026#34;) class Sparrow(Bird): pass class Penguin(Bird): def fly(self): raise NotImplementedError(\u0026#34;Penguins can\u0026#39;t fly!\u0026#34;) Code written against the base class:\ndef make_birds_fly(birds): for bird in birds: bird.fly() If the list passed to make_birds_fly includes a Penguin instance, the program raises an exception at runtime, since Penguin cannot honor the contract implied by Bird.fly. This is again an LSP violation. The correction involves restructuring the hierarchy so that the base class contract reflects only what every subclass can genuinely guarantee:\nclass Bird(ABC): @abstractmethod def move(self): pass class FlyingBird(Bird): def move(self): print(\u0026#34;Flying!\u0026#34;) class FlightlessBird(Bird): def move(self): print(\u0026#34;Walking!\u0026#34;) class Sparrow(FlyingBird): pass class Penguin(FlightlessBird): pass With this hierarchy, any Bird instance can be asked to move(), and the guarantee holds for every subclass without exception.\nFormal Rules for Preserving LSP A subclass must satisfy the following conditions to remain substitutable for its parent:\nPreconditions cannot be strengthened. If the parent method accepts any integer, the child method cannot narrow this to accept only positive integers. Postconditions cannot be weakened. If the parent method guarantees a sorted list is returned, the child method cannot return an unsorted list. Invariants must be preserved. If the parent guarantees that an account balance never falls below zero, the child cannot introduce a code path that violates this invariant. The history constraint must hold. A subclass should not permit state transitions that the parent\u0026rsquo;s contract forbids. An Illustrative Comparison Consider a hiring scenario in which a role is defined as: report at 9 AM, drive from location A to location B, without additional conditions. A candidate who fulfills exactly this description can be substituted freely into any schedule built around this role. A second candidate states that they will only drive on Tuesdays and will not use highways. This second candidate may describe themselves using the same job title, but they cannot be substituted into a schedule that assumes the unconditional role, since doing so will produce failures whenever the schedule requires a non-Tuesday or a highway route.\nThe principle, translated to software, states that if a subclass\u0026rsquo;s actual behavior differs materially from what the base class\u0026rsquo;s contract promises, it should not be modeled as that subclass. It should instead be represented as a distinct type with its own, narrower contract.\nWhy LSP Matters A violation of LSP undermines the value of polymorphism itself. If subtypes are not genuinely substitutable, calling code cannot safely treat all subtypes uniformly and must instead perform type checks to handle exceptions, which defeats the purpose of using polymorphism in the first place. Since OCP depends on polymorphism functioning reliably, an LSP violation typically undermines OCP as well.\nIndicators of an LSP Violation Indicator Explanation Overridden methods raise NotImplementedError A subclass refuses to support an operation the parent guarantees. Overridden methods return a weaker guarantee The subclass returns a different type or provides a less specific result than the parent\u0026rsquo;s contract promises. Callers require isinstance() checks If code using a base class needs to branch based on the concrete subtype, substitutability has already broken down. Hidden side effects on state An overridden method silently mutates object state in ways the base class contract does not anticipate. 6. Interface Segregation Principle (ISP) \u0026ldquo;Clients should not be forced to depend on interfaces they do not use.\u0026rdquo; Robert C. Martin\nPrecise Definition ISP states that large interfaces containing many unrelated methods should be decomposed into smaller, focused interfaces, so that a class implementing an interface is only required to implement methods relevant to it, and client code depending on an interface only needs knowledge of the methods it actually calls.\nThe Problem Without ISP Consider an interface intended to represent office equipment:\nfrom abc import ABC, abstractmethod class OfficeMachine(ABC): @abstractmethod def print(self, doc): pass @abstractmethod def scan(self, doc): pass @abstractmethod def fax(self, doc): pass A multifunction printer implements this interface without difficulty:\nclass AllInOnePrinter(OfficeMachine): def print(self, doc): print(f\u0026#34;Printing {doc}\u0026#34;) def scan(self, doc): print(f\u0026#34;Scanning {doc}\u0026#34;) def fax(self, doc): print(f\u0026#34;Faxing {doc}\u0026#34;) A basic printer, however, cannot scan or fax, yet the interface forces it to declare these methods:\nclass BasicPrinter(OfficeMachine): def print(self, doc): print(f\u0026#34;Printing {doc}\u0026#34;) def scan(self, doc): raise NotImplementedError(\u0026#34;Can\u0026#39;t scan!\u0026#34;) def fax(self, doc): raise NotImplementedError(\u0026#34;Can\u0026#39;t fax!\u0026#34;) This design produces two distinct problems. First, BasicPrinter carries stub methods whose only function is to raise an exception, which is wasted code that provides no value. Second, BasicPrinter claims, through its type, to be a fully functional OfficeMachine, but breaks this claim the moment it is used polymorphically as one. This is a direct violation of LSP, which illustrates that a bloated interface (an ISP violation) frequently forces the classes that implement it into an LSP violation as well.\nApplying ISP The single, large interface is decomposed into narrower, single-purpose interfaces:\nclass Printer(ABC): @abstractmethod def print(self, doc): pass class Scanner(ABC): @abstractmethod def scan(self, doc): pass class Fax(ABC): @abstractmethod def fax(self, doc): pass class BasicPrinter(Printer): def print(self, doc): print(f\u0026#34;Printing {doc}\u0026#34;) class AllInOnePrinter(Printer, Scanner, Fax): def print(self, doc): print(f\u0026#34;Printing {doc}\u0026#34;) def scan(self, doc): print(f\u0026#34;Scanning {doc}\u0026#34;) def fax(self, doc): print(f\u0026#34;Faxing {doc}\u0026#34;) With this structure, BasicPrinter implements only the interface it can genuinely fulfill. AllInOnePrinter composes multiple narrow interfaces to represent its full functionality. Client code that requires only printing depends on the Printer interface alone and remains entirely unaware of scanning or faxing capability.\nISP in Python Python\u0026rsquo;s support for duck typing reduces the practical impact of this issue in some contexts, since code can be written to accept any object exposing a required method without formally declaring an interface. However, when interfaces are explicitly defined using abstract base classes or typing.Protocol, the same principle applies: interfaces should remain narrow.\nfrom typing import Protocol class Printable(Protocol): def print(self, doc): ... def print_many(docs, printer: Printable): for doc in docs: printer.print(doc) Any object that defines a print method, regardless of what other methods it defines, satisfies the Printable protocol. The interface remains narrow and focused on exactly the capability the client requires.\nIndicators of an ISP Violation Indicator Explanation NotImplementedError in subclasses A concrete class raises an error for a method it was forced to declare but cannot support. Interfaces with many rarely co-used methods Methods on the same interface are used together infrequently, suggesting they belong to different clients. Clients importing large interfaces for small usage Code depends on a large interface but only calls one or two of its methods. Bloated test doubles Mocks and stubs used in tests must implement many methods that the test does not actually exercise. 7. Dependency Inversion Principle (DIP) \u0026ldquo;High-level modules should not depend on low-level modules. Both should depend on abstractions. Abstractions should not depend on details. Details should depend on abstractions.\u0026rdquo; Robert C. Martin\nPrecise Definition DIP states that classes should depend on abstractions (interfaces or abstract base classes) rather than on concrete implementations, so that implementations can be substituted without modifying the code that uses them.\nTwo terms require precise definition here. A high-level module is one that contains business logic, that is, the policies and rules that define what the application actually does. A low-level module is one that contains mechanical detail, such as how data is read from or written to a database, a file system, or a network connection.\nIn a naive design, high-level modules depend directly on low-level modules: business logic calls the database access code directly. DIP requires inverting this relationship so that both the high-level and low-level modules depend on an abstraction positioned between them.\nThe Problem Without DIP class MySQLDatabase: def save(self, data): print(f\u0026#34;Saving {data} to MySQL\u0026#34;) class UserService: def __init__(self): self.db = MySQLDatabase() # Hardcoded dependency def register_user(self, name): self.db.save({\u0026#34;name\u0026#34;: name}) This design produces three distinct problems:\nTight coupling. UserService can operate only with MySQLDatabase. Switching to PostgreSQL requires editing UserService directly. Reduced testability. A unit test for UserService cannot run without a live MySQL server, which converts what should be a fast unit test into a slower integration test. Reduced flexibility. Every variation, such as logging to a file during development or publishing events to a message queue instead of a database, requires modifying the internals of UserService. UserService, which represents high-level business logic, depends directly on MySQLDatabase, a low-level mechanism. Any change to the storage mechanism therefore forces a change in the business logic that has nothing conceptually to do with storage.\nApplying DIP An abstraction is introduced between the two layers:\nfrom abc import ABC, abstractmethod class Database(ABC): # The abstraction @abstractmethod def save(self, data): pass class MySQLDatabase(Database): def save(self, data): print(f\u0026#34;Saving {data} to MySQL\u0026#34;) class PostgresDatabase(Database): def save(self, data): print(f\u0026#34;Saving {data} to Postgres\u0026#34;) class InMemoryDatabase(Database): # Useful for tests def __init__(self): self.data = [] def save(self, data): self.data.append(data) class UserService: def __init__(self, db: Database): # Depends on the abstraction self.db = db def register_user(self, name): self.db.save({\u0026#34;name\u0026#34;: name}) This design allows the following usages without modifying UserService:\n# Production service = UserService(MySQLDatabase()) # Development service = UserService(PostgresDatabase()) # Tests fake_db = InMemoryDatabase() service = UserService(fake_db) service.register_user(\u0026#34;Amit\u0026#34;) assert fake_db.data == [{\u0026#34;name\u0026#34;: \u0026#34;Amit\u0026#34;}] UserService has no knowledge of which concrete class implements Database. Substituting one implementation for another is a change made once, at the point where the object is constructed, rather than a change to the class\u0026rsquo;s internal logic.\nDependency Injection Dependency injection (DI) refers to the technique of supplying a class\u0026rsquo;s dependencies from outside the class, typically through its constructor, rather than having the class construct its own dependencies internally. DI is the standard practical mechanism through which DIP is achieved. When a dependency is supplied externally, it can be substituted freely for tests or for different runtime environments, and the class receiving the dependency has no knowledge of which implementation it was given.\nThree common injection styles are used in practice:\nStyle Description Constructor injection The dependency is passed as an argument to __init__. This is the most common style. Setter or method injection The dependency is supplied through a setter method or as a parameter to a specific method, rather than at construction time. Framework-based injection A framework automatically resolves and supplies dependencies, for example FastAPI\u0026rsquo;s Depends mechanism. An Illustrative Comparison Consider the distinction between a laptop with a charging port hardwired to accept only one proprietary charger design and a laptop that charges through a standard USB-C port. In the first case, losing the specific charger, or traveling to a region where it is unavailable, leaves the device unusable. In the second case, the laptop depends on an abstraction, namely conformance to the USB-C specification, and can therefore draw power from any compliant source: a wall adapter, a portable battery, or another laptop.\nThe laptop, representing the high-level component in this comparison, has no knowledge of the specific power source it is connected to. It depends only on the interface. Manufacturers of power sources, representing the low-level components, also conform to that same interface. Neither side is required to know the internal details of the other, and either side can be replaced independently as long as the interface is honored.\nWhy the Term \u0026ldquo;Inversion\u0026rdquo; In the naive design, the dependency arrow points from the high-level module toward the low-level module:\nUserService -\u0026gt; MySQLDatabase (high) (low) After applying DIP, both modules point toward the abstraction instead:\nUserService MySQLDatabase | | v v Database (abstraction) This reversal of direction, in which the low-level detail now conforms to an abstraction rather than the high-level module depending on the low-level detail directly, is the reason the principle is described as an \u0026ldquo;inversion\u0026rdquo; of the conventional dependency direction.\nIndicators of a DIP Violation Indicator Explanation Internal construction of dependencies A class constructs its own collaborators, for example self.db = MySQLDatabase(), rather than receiving them from outside. Tests require real infrastructure Unit tests cannot run without a real database, external API, or file system available. Wide-reaching implementation swaps Replacing one implementation with another requires editing many files rather than a single configuration point. Direct imports of infrastructure libraries in business logic Business logic modules directly import libraries such as a specific database driver or HTTP client library. 8. How the Five Principles Reinforce Each Other The five principles should not be treated as an independent checklist to be verified one at a time. In practice, they form a reinforcing chain, where satisfying one principle makes the next easier to satisfy, and violating one principle tends to produce violations of the others.\nThe chain can be summarized as follows. Applying SRP forces responsibilities to be split across multiple small, focused classes. Small, focused classes naturally lead to small, focused interfaces, which satisfies ISP. Small interfaces are easier for subclasses to implement honestly, which supports LSP. When classes behave honestly with respect to their declared contracts, substituting one implementation for another becomes safe, which enables DIP. Once code depends on abstractions rather than concrete types, adding new behavior becomes a matter of adding new implementations of that abstraction rather than modifying existing code, which satisfies OCP.\nSRP -\u0026gt; small classes -\u0026gt; small interfaces (ISP) -\u0026gt; honest subclasses (LSP) | safe polymorphism | abstractions trusted (DIP) | extension over modification (OCP) | system stays maintainable -\u0026gt; easier SRP next time The reverse pattern also holds. A single class accumulating multiple unrelated responsibilities (an SRP violation) typically exposes a bloated interface (an ISP violation). Subclasses of that interface are then forced to implement methods that do not apply to them, often by raising exceptions (an LSP violation). Because these subclasses cannot be trusted to behave correctly when substituted, code cannot safely depend on the abstraction (a DIP violation), which in turn forces the calling code to fall back on explicit type checks and conditional branches (an OCP violation).\n9. Capstone Example This section constructs a notification system from an initial version that violates all five principles, then refactors it step by step to satisfy each one.\nInitial Version class NotificationService: def send(self, user, message, channel): if channel == \u0026#34;email\u0026#34;: # Hardcoded SMTP stuff print(f\u0026#34;Connecting to smtp.example.com...\u0026#34;) print(f\u0026#34;Email to {user.email}: {message}\u0026#34;) elif channel == \u0026#34;sms\u0026#34;: # Hardcoded Twilio stuff print(f\u0026#34;Calling Twilio API...\u0026#34;) print(f\u0026#34;SMS to {user.phone}: {message}\u0026#34;) elif channel == \u0026#34;push\u0026#34;: print(f\u0026#34;Push to {user.device_id}: {message}\u0026#34;) # Also log to database print(f\u0026#34;Logging notification to MySQL...\u0026#34;) # Also update user\u0026#39;s last_notified_at user.last_notified_at = \u0026#34;now\u0026#34; The violations present in this version are summarized below.\nPrinciple Violation SRP The class mixes message sending, logging, and user record updates in a single method. OCP Adding a new channel, such as Slack or Telegram, requires editing this method directly. LSP Not directly applicable, since no substitutable types exist yet, but the monolithic design prevents introducing them. ISP Not directly applicable yet, since no interface has been defined, which is itself part of the problem. DIP SMTP, Twilio, and MySQL are hardcoded, making the class impossible to unit test without real external services. Refactoring Step by Step Step 1: Extract narrow interfaces (addresses ISP).\nfrom abc import ABC, abstractmethod class NotificationChannel(ABC): @abstractmethod def send(self, recipient, message): pass class NotificationLogger(ABC): @abstractmethod def log(self, recipient, message, channel_name): pass Step 2: Implement each channel independently (addresses OCP).\nclass EmailChannel(NotificationChannel): def send(self, recipient, message): print(f\u0026#34;Email to {recipient.email}: {message}\u0026#34;) class SMSChannel(NotificationChannel): def send(self, recipient, message): print(f\u0026#34;SMS to {recipient.phone}: {message}\u0026#34;) class PushChannel(NotificationChannel): def send(self, recipient, message): print(f\u0026#34;Push to {recipient.device_id}: {message}\u0026#34;) Introducing SlackChannel or TelegramChannel at a later date requires creating a new class only. No existing class is touched.\nStep 3: Implement logging as a separate responsibility (addresses SRP).\nclass DatabaseLogger(NotificationLogger): def log(self, recipient, message, channel_name): print(f\u0026#34;DB log: {channel_name} to {recipient}: {message}\u0026#34;) class InMemoryLogger(NotificationLogger): # For tests def __init__(self): self.entries = [] def log(self, recipient, message, channel_name): self.entries.append((recipient, message, channel_name)) Step 4: Introduce an orchestrator that depends on abstractions (addresses DIP).\nclass NotificationService: def __init__(self, channel: NotificationChannel, logger: NotificationLogger): self.channel = channel self.logger = logger def notify(self, recipient, message): self.channel.send(recipient, message) self.logger.log(recipient, message, self.channel.__class__.__name__) Step 5: Verify LSP. Any implementation of NotificationChannel can be substituted for another, since all of them honor the same contract, send(recipient, message), without exception or special-casing.\nUsage # Production email_service = NotificationService(EmailChannel(), DatabaseLogger()) email_service.notify(user, \u0026#34;Your order has shipped\u0026#34;) # Tests test_logger = InMemoryLogger() test_service = NotificationService(EmailChannel(), test_logger) test_service.notify(fake_user, \u0026#34;hello\u0026#34;) assert len(test_logger.entries) == 1 Verification Against Each Principle Principle How it is satisfied SRP Each class, NotificationChannel implementations, NotificationLogger implementations, and NotificationService, has exactly one reason to change. OCP New channels or loggers are added as new classes, with zero modification to NotificationService. LSP Any NotificationChannel implementation is substitutable for another, since all honor the same method signature and behavioral contract. ISP NotificationChannel exposes only send, and NotificationLogger exposes only log, keeping each interface narrow. DIP NotificationService depends entirely on the two abstractions, never on a specific channel or logger implementation. The resulting design is extensible, independently testable, and each component can be reasoned about without reference to the others.\n10. Common Pitfalls Over-Engineering Applying SOLID can become a justification for introducing unnecessary layers of abstraction. If an application has exactly one type of discount and there is no reasonable expectation of additional types in the foreseeable future, introducing four classes and an abstract base class for DiscountCalculator adds complexity without corresponding benefit.\nThe applicable rule is that SOLID should be applied when an axis of change genuinely exists or is reasonably anticipated based on the domain, not applied speculatively to every part of a system.\nMistaking Class Splitting for SRP Dividing one class into two classes does not automatically satisfy SRP. The relevant test is whether the two resulting classes have genuinely different reasons to change. If both classes continue to change together, for the same underlying reason, the split has merely distributed one responsibility across two files, which is a worse outcome than the original single class, since the coupling now spans multiple files instead of being visible in one.\nInterface Explosion from Over-Applying ISP Excessive application of ISP produces a large number of narrow interfaces, each used in exactly one place, which introduces noise rather than clarity. Interface segregation is justified when there are genuinely distinct clients with distinct needs. It is not justified purely for the sake of minimizing method count on every interface.\nForcing Inheritance to Enable Polymorphism LSP violations frequently originate from inheritance relationships introduced because two classes appear conceptually similar, such as the square and rectangle example in Section 5, even though their behavior under mutation differs. When the \u0026ldquo;is-a\u0026rdquo; relationship does not hold behaviorally, composition or separate class hierarchies should be used instead of inheritance.\nApplying DIP to Every Dependency Not every dependency requires an abstraction layer. A call to datetime.now(), for instance, rarely needs to be abstracted behind an interface. A dependency on a remote API that may change providers, or a database that needs to be replaced with an in-memory implementation for testing, is a stronger candidate for abstraction. The general rule is to abstract dependencies at system boundaries, meaning input/output operations, third-party services, and volatile external dependencies, while keeping internal business logic concrete wherever it is stable.\nDogmatic Application SOLID is a heuristic derived from observed patterns in long-lived, team-maintained software. A short-lived script, a throwaway prototype, or a small configuration parser does not require five layers of abstraction. The decision to apply SOLID should be based on the expected lifetime and complexity of the code in question, not applied uniformly regardless of context.\n11. When Not to Apply SOLID Context Reasoning Prototypes and technical spikes The purpose of this code is exploration. Premature abstraction slows down the exploration it is meant to support. Small, single-purpose scripts A fifty-line automation script intended for one-time or infrequent use does not justify the overhead of formal abstraction layers. Performance-critical inner loops Abstraction mechanisms such as virtual method dispatch can introduce measurable overhead in code paths where every unit of latency matters. Genuinely stable domains If a piece of logic is tied to a fact that will not change, such as a mathematical constant, designing an extension point for it provides no value. Cases where abstraction reduces clarity If introducing an abstraction makes the code harder for a future maintainer to read and understand than the direct alternative, the abstraction is counterproductive. The underlying skill required is judgment: apply SOLID where change is expected and the resulting flexibility will be used, and avoid it where it introduces friction without a corresponding benefit.\n12. Summary The following table consolidates the five principles for quick reference.\nPrinciple Core Idea Primary Benefit SRP A class should have exactly one reason to change. Changes remain localized, reducing unintended side effects. OCP Extend behavior through new code rather than modification of existing code. New features can be added without risking regressions in existing functionality. LSP Subclasses must honor the behavioral contract of their parent class. Polymorphism can be relied upon without special-case handling. ISP Interfaces should remain narrow and focused on a single client\u0026rsquo;s needs. Classes are not forced to implement irrelevant methods, and contracts remain clear. DIP Depend on abstractions rather than concrete implementations. Implementations become swappable and components become independently testable. Key Takeaways SOLID describes how object-oriented constructs should be arranged so that a system remains maintainable as it grows. Object-oriented programming supplies the constructs; SOLID supplies the arrangement rules. Every principle addresses, from a different angle, the goal of minimizing the extent to which a single change propagates through a system. The five principles are interdependent. Satisfying one tends to make the others easier to satisfy, and violating one tends to produce violations of the others. Nearly all five principles reduce, at some level, to the same underlying technique: depending on the correct abstraction, at the correct level of the system. SOLID is a set of heuristics, not a set of laws. It should be applied where the change patterns of a system justify the overhead, and it should be withheld where it does not. SOLID provides the conceptual foundation for design patterns. Most classical design patterns are concrete, named applications of one or more SOLID principles, for example the Strategy pattern applies OCP and DIP together, and the Adapter pattern applies LSP and DIP together. 13. Self-Assessment Questions The following questions test conceptual understanding rather than memorization. Attempt an answer before revealing the solution.\nQuestion 1 A class has fifteen methods, all operating on the same underlying state and all serving one cohesive purpose. Does this class violate SRP?\nAnswer: No. SRP concerns the number of distinct reasons a class might need to change, not the number of methods it defines. If all fifteen methods serve one responsibility and would only change for one reason, the class satisfies SRP. A ShoppingCart class with fifteen methods, all related to managing cart state, is a valid design under this principle.\nQuestion 2 An Animal class defines a method fly(). A new class Penguin needs to be added. What is the correct response under SOLID?\nAnswer: The class hierarchy should be restructured. Penguin should not override fly() to raise an exception, since this would violate LSP. Instead, the hierarchy should be split, for example into FlyingAnimal and FlightlessAnimal, or the fly() method should be removed from the base Animal class entirely. The base class\u0026rsquo;s contract should only promise behavior that every subclass can genuinely deliver.\nQuestion 3 Adding a new payment type requires modifying a central if/elif chain inside a PaymentProcessor class. Which principle is being violated?\nAnswer: This is primarily a violation of OCP, since existing code must be modified to extend behavior. It is often also a violation of DIP, since the class depends on concrete payment types rather than an abstraction. The fix is to introduce a PaymentStrategy interface with one implementing class per payment type, and to have PaymentProcessor depend on the interface rather than on concrete types.\nQuestion 4 A Printable interface defines print(), scan(), and fax(). A BasicPrinter class implements only print() and raises NotImplementedError for the other two methods. Which principle is being violated directly, and which principle is being violated indirectly?\nAnswer: This is a direct violation of ISP, since BasicPrinter is forced to implement methods it has no use for. It is an indirect violation of LSP, since any code that treats a Printable polymorphically will encounter a runtime exception when given a BasicPrinter. The fix is to split the interface into Printer, Scanner, and Fax, so that each concrete class implements only the interfaces it genuinely supports.\nQuestion 5 A class OrderService constructs a PostgresDatabase instance directly inside its constructor. As a result, tests for OrderService must run against a real PostgreSQL server. What principle is violated, and what is the fix?\nAnswer: This violates DIP, since the high-level OrderService depends directly on the low-level PostgresDatabase implementation. The fix is to extract a Database abstraction, have OrderService receive an instance of this abstraction through constructor injection, and supply an in-memory or mock implementation during testing.\nQuestion 6 True or false: if a class has only one method, it automatically satisfies SRP.\nAnswer: False. SRP concerns reasons to change, not the number of methods. A single method containing five hundred lines of tangled logic that performs several unrelated tasks still violates SRP. Conversely, a class with twenty methods can satisfy SRP if every method serves one cohesive responsibility.\nQuestion 7 A Duck class defines a quack() method. A subclass RubberDuck overrides quack() to play a squeaking sound rather than an actual duck call. Does this violate LSP?\nAnswer: Generally, no, provided the contract of the base class is understood as \u0026ldquo;produce some form of duck-associated sound,\u0026rdquo; which both a real duck and a rubber duck satisfy. LSP would only be violated if the base class\u0026rsquo;s contract made a stronger guarantee that the subclass fails to honor, for example if the contract specified \u0026ldquo;returns a recording of an actual duck,\u0026rdquo; and RubberDuck returned nothing at all. LSP concerns adherence to a stated contract, not identical implementation.\nQuestion 8 Can a design satisfy SRP while still violating OCP? Provide a brief example.\nAnswer: Yes. A DiscountCalculator class with a single, well-defined responsibility, calculating a discount, can still implement that responsibility using a large if/elif chain, which requires modification whenever a new discount type is introduced. SRP concerns the boundaries around a class\u0026rsquo;s responsibilities; OCP concerns the mechanism by which that responsibility is extended. The two principles are related but address distinct aspects of a design.\nQuestion 9 What is the relationship between Dependency Injection and the Dependency Inversion Principle?\nAnswer: DIP is the principle: high-level and low-level modules should both depend on abstractions. Dependency Injection is the most common practical technique used to satisfy DIP, whereby dependencies are supplied to a class from outside, through a constructor, a setter, or a framework, rather than being constructed internally by the class itself. DIP describes the goal; Dependency Injection describes one common method of achieving it.\nQuestion 10 A hundred-line script reads a CSV file and produces a summary report. Should all five SOLID principles be carefully applied to this script?\nAnswer: Generally, no. The overhead associated with SOLID is justified when code is expected to persist and evolve over time under changing requirements. For a short, throwaway script, applying all five principles typically over-engineers the solution relative to its actual lifespan and complexity. The appropriate judgment is to abstract where change is genuinely likely and to remain concrete where it is not.\n14. Practice Exercises Exercise 1: SRP Refactor The following class violates SRP. Identify the distinct responsibilities present, split the class accordingly, and justify the resulting structure.\nclass Invoice: def __init__(self, items): self.items = items # list of (name, price, qty) def calculate_total(self): return sum(price * qty for _, price, qty in self.items) def calculate_tax(self): return self.calculate_total() * 0.18 def save_to_pdf(self, filepath): print(f\u0026#34;Writing PDF to {filepath}\u0026#34;) def send_by_email(self, to): print(f\u0026#34;Sending invoice to {to}\u0026#34;) Solution\nThe class contains four distinct responsibilities:\nHolding invoice data, specifically the list of items. Performing financial calculations, specifically totals and tax, which belongs to the finance domain. Rendering the invoice as a PDF, which belongs to the presentation domain. Delivering the invoice by email, which belongs to the infrastructure domain. class Invoice: def __init__(self, items): self.items = items class InvoiceCalculator: TAX_RATE = 0.18 def total(self, invoice): return sum(price * qty for _, price, qty in invoice.items) def tax(self, invoice): return self.total(invoice) * self.TAX_RATE class InvoicePDFRenderer: def render(self, invoice, filepath): print(f\u0026#34;Writing PDF of {len(invoice.items)} items to {filepath}\u0026#34;) class InvoiceEmailSender: def send(self, invoice, to): print(f\u0026#34;Sending invoice with {len(invoice.items)} items to {to}\u0026#34;) Each resulting class can now change independently. A revision to the tax rate affects only InvoiceCalculator. A change to the PDF layout affects only InvoicePDFRenderer. A migration to a new email delivery system affects only InvoiceEmailSender.\nExercise 2: OCP Application The following class violates OCP.\nclass AreaCalculator: def compute(self, shapes): total = 0 for shape in shapes: if shape[\u0026#34;type\u0026#34;] == \u0026#34;circle\u0026#34;: total += 3.14 * shape[\u0026#34;radius\u0026#34;] ** 2 elif shape[\u0026#34;type\u0026#34;] == \u0026#34;square\u0026#34;: total += shape[\u0026#34;side\u0026#34;] ** 2 elif shape[\u0026#34;type\u0026#34;] == \u0026#34;rectangle\u0026#34;: total += shape[\u0026#34;width\u0026#34;] * shape[\u0026#34;height\u0026#34;] return total Adding a new shape type currently requires modifying AreaCalculator directly. Refactor this design to comply with OCP.\nSolution\nfrom abc import ABC, abstractmethod import math class Shape(ABC): @abstractmethod def area(self): pass class Circle(Shape): def __init__(self, radius): self.radius = radius def area(self): return math.pi * self.radius ** 2 class Square(Shape): def __init__(self, side): self.side = side def area(self): return self.side ** 2 class Rectangle(Shape): def __init__(self, width, height): self.width = width self.height = height def area(self): return self.width * self.height class AreaCalculator: def compute(self, shapes): return sum(shape.area() for shape in shapes) Introducing a Triangle type now requires only the addition of a new Triangle(Shape) class. AreaCalculator is not modified.\nExercise 3: LSP Check Examine the following two subclasses. Determine whether each violates LSP and justify the conclusion.\nclass FileReader: def read(self, path): with open(path) as f: return f.read() class LoggingFileReader(FileReader): def read(self, path): print(f\u0026#34;Reading {path}\u0026#34;) return super().read(path) class StrictFileReader(FileReader): def read(self, path): if not path.endswith(\u0026#34;.txt\u0026#34;): raise ValueError(\u0026#34;Only .txt supported!\u0026#34;) return super().read(path) Solution\nLoggingFileReader does not violate LSP. It adds a side effect, printing a log line, but continues to return the full file contents for any valid path, exactly as the parent\u0026rsquo;s contract guarantees.\nStrictFileReader does violate LSP. It strengthens the precondition of the parent method by restricting valid input to files with a .txt extension. Code written against FileReader that supplies a .json file would succeed with the parent class but raise an exception with StrictFileReader. Substitutability is broken.\nThe general lesson is that a subclass may extend the behavior of its parent, for example by adding a side effect, but must not restrict what the parent was willing to accept.\nExercise 4: ISP Design Design a set of interfaces for a workshop with the following constraints. A Carpenter requires only a cut_wood capability. A Plumber requires only a fix_pipe capability. A Handyman requires both capabilities. Neither specialist should be forced to implement the other\u0026rsquo;s capability.\nSolution\nfrom abc import ABC, abstractmethod class WoodWorker(ABC): @abstractmethod def cut_wood(self): pass class PipeWorker(ABC): @abstractmethod def fix_pipe(self): pass class Carpenter(WoodWorker): def cut_wood(self): print(\u0026#34;Cutting wood\u0026#34;) class Plumber(PipeWorker): def fix_pipe(self): print(\u0026#34;Fixing pipe\u0026#34;) class Handyman(WoodWorker, PipeWorker): def cut_wood(self): print(\u0026#34;Cutting wood\u0026#34;) def fix_pipe(self): print(\u0026#34;Fixing pipe\u0026#34;) Multiple inheritance from narrow interfaces allows Handyman to represent the combination of both skills without either Carpenter or Plumber carrying methods irrelevant to their role.\nExercise 5: DIP via Injection The following code is difficult to test because EmailService depends on a hardcoded SMTP client. Refactor it to comply with DIP.\nclass SmtpClient: def send(self, to, subject, body): print(f\u0026#34;SMTP -\u0026gt; {to}: {subject}\u0026#34;) class EmailService: def __init__(self): self.client = SmtpClient() def send_welcome(self, user): self.client.send(user.email, \u0026#34;Welcome!\u0026#34;, f\u0026#34;Hi {user.name}\u0026#34;) Solution\nfrom abc import ABC, abstractmethod class EmailClient(ABC): @abstractmethod def send(self, to, subject, body): pass class SmtpClient(EmailClient): def send(self, to, subject, body): print(f\u0026#34;SMTP -\u0026gt; {to}: {subject}\u0026#34;) class FakeEmailClient(EmailClient): # For tests def __init__(self): self.sent = [] def send(self, to, subject, body): self.sent.append((to, subject, body)) class EmailService: def __init__(self, client: EmailClient): self.client = client def send_welcome(self, user): self.client.send(user.email, \u0026#34;Welcome!\u0026#34;, f\u0026#34;Hi {user.name}\u0026#34;) # Usage prod_service = EmailService(SmtpClient()) # Testing fake = FakeEmailClient() test_service = EmailService(fake) test_service.send_welcome(SomeUser) assert len(fake.sent) == 1 EmailService now depends on the EmailClient abstraction rather than a concrete client. Implementations are supplied externally, and testing no longer requires a real SMTP connection.\nExercise 6: Full SOLID Refactor (Capstone) The following class, commonly referred to as a \u0026ldquo;god class,\u0026rdquo; violates every SOLID principle. Refactor it completely.\nclass OrderManager: def __init__(self): self.orders = [] def place_order(self, customer_type, items, payment_method): # Calculate total total = sum(price * qty for _, price, qty in items) # Apply discount if customer_type == \u0026#34;regular\u0026#34;: total *= 0.95 elif customer_type == \u0026#34;premium\u0026#34;: total *= 0.90 elif customer_type == \u0026#34;vip\u0026#34;: total *= 0.80 # Process payment if payment_method == \u0026#34;card\u0026#34;: print(f\u0026#34;Charging Rs.{total} on card\u0026#34;) elif payment_method == \u0026#34;upi\u0026#34;: print(f\u0026#34;Processing Rs.{total} via UPI\u0026#34;) elif payment_method == \u0026#34;cash\u0026#34;: print(f\u0026#34;Collecting Rs.{total} cash on delivery\u0026#34;) # Save to MySQL print(f\u0026#34;Saving order to MySQL...\u0026#34;) # Send email print(f\u0026#34;Sending confirmation email...\u0026#34;) self.orders.append({\u0026#34;items\u0026#34;: items, \u0026#34;total\u0026#34;: total}) return total Solution\nfrom abc import ABC, abstractmethod # --- Discount strategies (OCP, DIP) --- class DiscountStrategy(ABC): @abstractmethod def apply(self, amount): pass class RegularDiscount(DiscountStrategy): def apply(self, amount): return amount * 0.95 class PremiumDiscount(DiscountStrategy): def apply(self, amount): return amount * 0.90 class VIPDiscount(DiscountStrategy): def apply(self, amount): return amount * 0.80 # --- Payment processors (OCP, DIP) --- class PaymentProcessor(ABC): @abstractmethod def charge(self, amount): pass class CardProcessor(PaymentProcessor): def charge(self, amount): print(f\u0026#34;Card: Rs.{amount}\u0026#34;) class UPIProcessor(PaymentProcessor): def charge(self, amount): print(f\u0026#34;UPI: Rs.{amount}\u0026#34;) class CashProcessor(PaymentProcessor): def charge(self, amount): print(f\u0026#34;Cash on delivery: Rs.{amount}\u0026#34;) # --- Repository (SRP, DIP) --- class OrderRepository(ABC): @abstractmethod def save(self, order): pass class MySQLOrderRepository(OrderRepository): def save(self, order): print(f\u0026#34;MySQL: saved order {order}\u0026#34;) class InMemoryOrderRepository(OrderRepository): def __init__(self): self.orders = [] def save(self, order): self.orders.append(order) # --- Notification (SRP, DIP) --- class OrderNotifier(ABC): @abstractmethod def notify(self, order): pass class EmailOrderNotifier(OrderNotifier): def notify(self, order): print(f\u0026#34;Email: order {order} confirmed\u0026#34;) # --- Pricing (SRP) --- class PriceCalculator: def total(self, items): return sum(price * qty for _, price, qty in items) # --- Orchestrator (SRP: coordinates, does not perform every task itself) --- class OrderService: def __init__(self, calculator: PriceCalculator, discount: DiscountStrategy, payment: PaymentProcessor, repository: OrderRepository, notifier: OrderNotifier): self.calculator = calculator self.discount = discount self.payment = payment self.repository = repository self.notifier = notifier def place_order(self, items): total = self.calculator.total(items) total = self.discount.apply(total) self.payment.charge(total) order = {\u0026#34;items\u0026#34;: items, \u0026#34;total\u0026#34;: total} self.repository.save(order) self.notifier.notify(order) return total Usage:\nservice = OrderService( calculator=PriceCalculator(), discount=PremiumDiscount(), payment=UPIProcessor(), repository=MySQLOrderRepository(), notifier=EmailOrderNotifier(), ) service.place_order([(\u0026#34;book\u0026#34;, 300, 2), (\u0026#34;pen\u0026#34;, 50, 5)]) Verification against each principle\nPrinciple Verification SRP Each class has one reason to change. PriceCalculator, DiscountStrategy, PaymentProcessor, OrderRepository, OrderNotifier, and OrderService each own a distinct, non-overlapping responsibility. OCP New discount types, payment methods, repositories, or notifiers are added as new classes. OrderService remains unmodified. LSP Any implementation of DiscountStrategy, PaymentProcessor, OrderRepository, or OrderNotifier is substitutable within OrderService, since each honors its respective contract without exception. ISP Each interface is narrow: apply, charge, save, and notify are each defined on a separate interface, so no class is forced to implement a method irrelevant to its role. DIP OrderService depends entirely on abstractions. Production and test configurations differ only in which concrete implementations are supplied at construction time. With OOP supplying the raw constructs and SOLID supplying the rules for arranging them, the natural next step is the study of design patterns: well-documented, reusable solutions to recurring design problems, most of which are, on close inspection, a concrete application of one or more SOLID principles.\nThis content is part of Codeverra, a platform for learning coding, data science, DSA, and AI from scratch. Explore more: https://codeverra.com\n","permalink":"https://blogs.codeverra.com/blog/low_level_design/02_solid_principles---extra-draft/","summary":"\u003ch1 id=\"solid-principles\"\u003eSOLID Principles\u003c/h1\u003e\n\u003cp\u003e\u003cstrong\u003ePrerequisite:\u003c/strong\u003e This material assumes familiarity with Python object-oriented programming, including classes, inheritance, polymorphism, and abstraction. The discussion builds directly on that foundation and extends it toward writing object-oriented code that remains maintainable as a system grows.\u003c/p\u003e\n\u003chr\u003e\n\u003ch2 id=\"table-of-contents\"\u003eTable of Contents\u003c/h2\u003e\n\u003col\u003e\n\u003cli\u003e\u003ca href=\"#1-why-solid-exists\"\u003eWhy SOLID Exists\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#2-what-solid-stands-for\"\u003eWhat SOLID Stands For\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#3-single-responsibility-principle-srp\"\u003eSingle Responsibility Principle (SRP)\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#4-openclosed-principle-ocp\"\u003eOpen/Closed Principle (OCP)\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#5-liskov-substitution-principle-lsp\"\u003eLiskov Substitution Principle (LSP)\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#6-interface-segregation-principle-isp\"\u003eInterface Segregation Principle (ISP)\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#7-dependency-inversion-principle-dip\"\u003eDependency Inversion Principle (DIP)\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#8-how-the-five-principles-reinforce-each-other\"\u003eHow the Five Principles Reinforce Each Other\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#9-capstone-example\"\u003eCapstone Example: Applying All Five Together\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#10-common-pitfalls\"\u003eCommon Pitfalls and Anti-Patterns\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#11-when-not-to-apply-solid\"\u003eWhen Not to Apply SOLID\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#12-summary\"\u003eSummary and Key Takeaways\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#13-self-assessment-questions\"\u003eSelf-Assessment Questions\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#14-practice-exercises\"\u003ePractice Exercises with Solutions\u003c/a\u003e\u003c/li\u003e\n\u003c/ol\u003e\n\u003chr\u003e\n\u003ch2 id=\"1-why-solid-exists\"\u003e1. Why SOLID Exists\u003c/h2\u003e\n\u003cp\u003eA learner who has completed a course on object-oriented programming knows how to define classes, establish inheritance hierarchies, and use polymorphism. A natural question follows: if these tools are already available, why does a separate set of principles need to be learned before writing \u0026ldquo;good\u0026rdquo; object-oriented code?\u003c/p\u003e","tags":["lld","oop","python"],"title":"LLD Foundations - SOLID Principles"},{"content":"Object-Oriented Programming in Python — A Deep Dive Who this is for: Anyone who knows basic Python (variables, functions, lists, dicts) and wants to truly understand OOP — not just the syntax, but why it exists, what problems it solves, and how to wield it well in real systems.\nTable of Contents The World Before OOP — Why Do We Even Need It? Classes and Objects — The Blueprint and the Building The self Parameter — Python\u0026rsquo;s Most Misunderstood Keyword Attributes — Instance vs Class Attributes Methods — Instance, Class, and Static The Four Pillars of OOP Encapsulation — The Art of Hiding Inheritance — Building on What Exists Polymorphism — Many Forms, One Interface Abstraction — Focusing on What Matters Dunder Methods — Making Your Objects Feel Native Properties — Controlled Attribute Access Multiple Inheritance and the MRO 13A. Mixins as a Pattern — Composable Capabilities Composition vs Inheritance Dataclasses — OOP with Less Boilerplate 15A. ABC vs Protocol — Two Ways to Define an Interface 15B. @classmethod as a Factory — Named Constructors Summary \u0026amp; Key Takeaways Practice Problems 1. The World Before OOP Before we learn what OOP is, let\u0026rsquo;s understand why humanity invented it. Imagine you\u0026rsquo;ve been hired to build software for a ride-hailing company (think Uber). You start writing code procedurally — just functions and dictionaries.\n# Procedural approach — how most of us start driver1 = {\u0026#34;name\u0026#34;: \u0026#34;Amit\u0026#34;, \u0026#34;rating\u0026#34;: 4.8, \u0026#34;car\u0026#34;: \u0026#34;Swift\u0026#34;, \u0026#34;is_available\u0026#34;: True} driver2 = {\u0026#34;name\u0026#34;: \u0026#34;Priya\u0026#34;, \u0026#34;rating\u0026#34;: 4.5, \u0026#34;car\u0026#34;: \u0026#34;WagonR\u0026#34;, \u0026#34;is_available\u0026#34;: False} def start_ride(driver, rider_name, destination): if not driver[\u0026#34;is_available\u0026#34;]: print(f\u0026#34;{driver[\u0026#39;name\u0026#39;]} is busy!\u0026#34;) return driver[\u0026#34;is_available\u0026#34;] = False print(f\u0026#34;{driver[\u0026#39;name\u0026#39;]} is picking up {rider_name} to go to {destination}\u0026#34;) def end_ride(driver): driver[\u0026#34;is_available\u0026#34;] = True This works! For a while. Then reality starts biting:\nProblem 1: Data and behavior live apart. The driver1 dictionary holds data. The start_ride function holds behavior. But they\u0026rsquo;re conceptually one thing — a driver. Nothing stops someone from calling start_ride(\u0026quot;hello\u0026quot;, \u0026quot;Priya\u0026quot;, \u0026quot;Airport\u0026quot;). Python won\u0026rsquo;t complain until it crashes at runtime.\nProblem 2: No structure or invariants. What if someone does driver1[\u0026quot;ratng\u0026quot;] = 5.0 (typo)? Or sets rating = -9999? Nothing prevents it. There\u0026rsquo;s no enforced \u0026ldquo;shape\u0026rdquo; for a driver.\nProblem 3: Code duplication explodes. Now you need a Rider. And a DeliveryPartner. Each has similar fields (name, rating, availability) but also differences. You end up copy-pasting logic everywhere.\nProblem 4: Change is expensive. One day, product says: \u0026ldquo;Drivers now have a tier — Gold, Silver, Bronze — and their commission depends on tier.\u0026rdquo; You have to hunt down every function that touches drivers and update it. Miss one, and bugs ship.\nOOP was invented to solve exactly these pain points. The core insight is beautifully simple:\nData and the behaviors that operate on that data belong together, as one unit.\nThat unit is called an object. And the template used to build objects is called a class.\nAn Everyday Analogy Think of a cookie cutter and the cookies it produces.\nThe cookie cutter is the class. It defines the shape, but you can\u0026rsquo;t eat it. Each cookie is an object (also called an instance). It has its own existence, its own icing, its own bite marks. The cutter can make thousands of cookies, all with the same shape but independent lives. Or think of an architect\u0026rsquo;s blueprint for a house (class) and the actual houses built from it (objects). Each house has the same layout but its own residents, furniture, and wifi password.\nKeep these analogies in mind — we\u0026rsquo;ll refer back to them.\n2. Classes and Objects Let\u0026rsquo;s rewrite the driver example using a class.\nclass Driver: def __init__(self, name, rating, car): self.name = name self.rating = rating self.car = car self.is_available = True def start_ride(self, rider_name, destination): if not self.is_available: print(f\u0026#34;{self.name} is busy!\u0026#34;) return self.is_available = False print(f\u0026#34;{self.name} is picking up {rider_name} to go to {destination}\u0026#34;) def end_ride(self): self.is_available = True print(f\u0026#34;{self.name} is now available again\u0026#34;) # Creating objects (instances) from the class amit = Driver(\u0026#34;Amit\u0026#34;, 4.8, \u0026#34;Swift\u0026#34;) priya = Driver(\u0026#34;Priya\u0026#34;, 4.5, \u0026#34;WagonR\u0026#34;) amit.start_ride(\u0026#34;Rohan\u0026#34;, \u0026#34;Airport\u0026#34;) # Output: Amit is picking up Rohan to go to Airport priya.start_ride(\u0026#34;Sneha\u0026#34;, \u0026#34;Mall\u0026#34;) # Output: Priya is picking up Sneha to go to Mall Let\u0026rsquo;s dissect this carefully.\nclass Driver: — This is a declaration. We\u0026rsquo;re telling Python: \u0026ldquo;I\u0026rsquo;m defining a new type of thing called Driver.\u0026rdquo; Nothing is created yet — it\u0026rsquo;s just a blueprint.\n__init__ — This is the constructor (technically, it\u0026rsquo;s an initializer, but most people call it constructor). It runs automatically whenever a new Driver is created. Its job is to set up the initial state. When you write Driver(\u0026quot;Amit\u0026quot;, 4.8, \u0026quot;Swift\u0026quot;), Python:\nCreates a new empty Driver object in memory Calls __init__ on it, passing \u0026quot;Amit\u0026quot;, 4.8, \u0026quot;Swift\u0026quot; as arguments __init__ populates the object\u0026rsquo;s attributes Returns the fully-formed object self — The current object. We\u0026rsquo;ll cover this in depth next. For now, think of it as \u0026ldquo;me\u0026rdquo; — the specific object this method is being called on.\namit = Driver(...) — This creates a new instance. amit and priya are two completely independent cookies made from the same cutter. Modifying amit.is_available doesn\u0026rsquo;t touch priya at all.\nWhat Did We Gain? Compared to the procedural version:\nCohesion. Driver data and driver behavior live together. You can\u0026rsquo;t accidentally call start_ride on a random dict. Identity. amit is clearly a Driver. Type checkers, IDEs, and humans all know this. Encapsulation (even if weak so far). The driver\u0026rsquo;s state is managed through its own methods. Readability. amit.start_ride(...) reads like English. Contrast with start_ride(amit_dict, ...). 3. The self Parameter self confuses nearly every beginner. Let\u0026rsquo;s demystify it once and for all.\nWhen you write:\namit.start_ride(\u0026#34;Rohan\u0026#34;, \u0026#34;Airport\u0026#34;) Python translates this internally to roughly:\nDriver.start_ride(amit, \u0026#34;Rohan\u0026#34;, \u0026#34;Airport\u0026#34;) That is — Python takes the object before the dot (amit) and passes it as the first argument to the method. That first argument is conventionally named self. It\u0026rsquo;s just a reference to \u0026ldquo;the object this method was called on.\u0026rdquo;\nWhy is self explicit in Python? Languages like Java and C++ have an implicit this. Python made a deliberate design choice: explicit is better than implicit (a core Python philosophy). By requiring self, Python:\nMakes it crystal clear what\u0026rsquo;s an instance attribute (self.name) vs a local variable (name). Makes method definitions and function definitions look identical — no magic. Lets you rename it if you want (though please don\u0026rsquo;t — use self). A Mental Model When you call amit.start_ride(\u0026quot;Rohan\u0026quot;, \u0026quot;Airport\u0026quot;), imagine Python whispering to the method: \u0026ldquo;Hey start_ride, here\u0026rsquo;s the driver you\u0026rsquo;re working with (amit), and here are the other arguments.\u0026rdquo;\nThe method then uses self.name, self.is_available, etc., to access that specific driver\u0026rsquo;s data.\n# These two calls operate on DIFFERENT self objects amit.start_ride(\u0026#34;Rohan\u0026#34;, \u0026#34;Airport\u0026#34;) # self IS amit priya.start_ride(\u0026#34;Sneha\u0026#34;, \u0026#34;Mall\u0026#34;) # self IS priya This is how one method definition can serve thousands of different objects — each invocation automatically knows which object it\u0026rsquo;s dealing with.\n4. Attributes Attributes are variables attached to objects (or classes). Python distinguishes two kinds: instance attributes and class attributes.\nInstance Attributes These belong to individual objects. Each object has its own copy. self.name, self.rating, self.is_available — all instance attributes.\namit.name = \u0026#34;Amit Kumar\u0026#34; # Only affects amit priya.name # Still \u0026#34;Priya\u0026#34; Class Attributes These belong to the class itself — one copy shared across all instances. Useful for constants or shared state.\nclass Driver: company_name = \u0026#34;QuickRide\u0026#34; # Class attribute — shared total_drivers_created = 0 # Shared counter def __init__(self, name, rating, car): self.name = name # Instance attribute self.rating = rating self.car = car self.is_available = True Driver.total_drivers_created += 1 # Modify the class attribute amit = Driver(\u0026#34;Amit\u0026#34;, 4.8, \u0026#34;Swift\u0026#34;) priya = Driver(\u0026#34;Priya\u0026#34;, 4.5, \u0026#34;WagonR\u0026#34;) print(Driver.total_drivers_created) # 2 print(amit.company_name) # \u0026#34;QuickRide\u0026#34; — accessed via instance print(priya.company_name) # \u0026#34;QuickRide\u0026#34; The Subtle Trap Here\u0026rsquo;s something that burns many beginners. Consider:\nclass Driver: rides_taken = [] # Class attribute — a MUTABLE list, shared! def __init__(self, name): self.name = name def add_ride(self, dest): self.rides_taken.append(dest) amit = Driver(\u0026#34;Amit\u0026#34;) priya = Driver(\u0026#34;Priya\u0026#34;) amit.add_ride(\u0026#34;Airport\u0026#34;) print(priya.rides_taken) # [\u0026#39;Airport\u0026#39;] ← WHAT?! Because rides_taken is a class attribute and lists are mutable, every instance shares the same list. amit.rides_taken and priya.rides_taken both refer to the exact same list object.\nThe fix: use instance attributes for per-object mutable state:\nclass Driver: def __init__(self, name): self.name = name self.rides_taken = [] # Fresh list for each driver Rule of thumb: use class attributes for constants and immutable values. Use instance attributes (set in __init__) for per-object mutable state.\n5. Methods Python has three kinds of methods. Most beginners only know the first.\nInstance Methods The usual kind. Take self as first parameter. Operate on a specific object.\nclass Driver: def __init__(self, name): self.name = name def greet(self): # Instance method return f\u0026#34;Hi, I\u0026#39;m {self.name}\u0026#34; Class Methods Marked with @classmethod. Take cls (the class itself) as first parameter instead of self. Useful for alternative constructors or operations involving the class as a whole.\nclass Driver: def __init__(self, name, rating, car): self.name = name self.rating = rating self.car = car @classmethod def from_dict(cls, data): \u0026#34;\u0026#34;\u0026#34;Alternative constructor: build Driver from a dict.\u0026#34;\u0026#34;\u0026#34; return cls(data[\u0026#34;name\u0026#34;], data[\u0026#34;rating\u0026#34;], data[\u0026#34;car\u0026#34;]) @classmethod def rookie(cls, name): \u0026#34;\u0026#34;\u0026#34;Alternative constructor: a brand new driver.\u0026#34;\u0026#34;\u0026#34; return cls(name, rating=5.0, car=\u0026#34;Unknown\u0026#34;) amit = Driver.from_dict({\u0026#34;name\u0026#34;: \u0026#34;Amit\u0026#34;, \u0026#34;rating\u0026#34;: 4.8, \u0026#34;car\u0026#34;: \u0026#34;Swift\u0026#34;}) newbie = Driver.rookie(\u0026#34;Rahul\u0026#34;) Why use cls instead of hardcoding Driver? Because if someone subclasses Driver (say, PremiumDriver), cls will correctly refer to the subclass. You get polymorphism for free.\nStatic Methods Marked with @staticmethod. Take neither self nor cls. They\u0026rsquo;re just regular functions that happen to live inside a class\u0026rsquo;s namespace for logical grouping.\nclass Driver: @staticmethod def is_valid_rating(rating): return 0 \u0026lt;= rating \u0026lt;= 5 Driver.is_valid_rating(4.8) # True Why put it in the class at all? For organization. Callers see that this utility is logically related to Driver, even though it doesn\u0026rsquo;t touch any instance or class state.\nWhen to Use Which? Method Type First Param Use When Instance method self You need to read/modify a specific object\u0026rsquo;s state Class method cls You need the class itself (alt constructors, factories, operations on the class) Static method none Utility logically grouped with the class but needing no state 6. The Four Pillars of OOP Every OOP course teaches \u0026ldquo;the four pillars.\u0026rdquo; They\u0026rsquo;re not arbitrary — each pillar solves a specific category of software engineering pain.\nPillar One-Line Essence Pain It Solves Encapsulation Bundle data + behavior; hide internals Prevents random code from corrupting an object\u0026rsquo;s state Inheritance Create new classes by extending existing ones Eliminates duplication across similar types Polymorphism Different objects, same interface Lets code work with many types without knowing which Abstraction Expose what, hide how Reduces cognitive load; lets you change implementations We\u0026rsquo;ll cover each in depth. But first, remember: these aren\u0026rsquo;t separate features — they reinforce each other. Good OOP code tends to use all four naturally.\n7. Encapsulation Encapsulation = bundling data with the methods that operate on that data, AND controlling access to the data.\nThe second part is crucial. Without access control, you\u0026rsquo;re just bundling — which is nice but insufficient.\nThe Problem Without Encapsulation class BankAccount: def __init__(self, owner, balance): self.owner = owner self.balance = balance acc = BankAccount(\u0026#34;Amit\u0026#34;, 1000) acc.balance = -999999 # Nothing stops this absurdity! Anyone can set balance to any value. Invariants (balance \u0026gt;= 0) can\u0026rsquo;t be enforced.\nPython\u0026rsquo;s Approach: Naming Conventions Unlike Java (which has private/public keywords), Python uses naming conventions. Python\u0026rsquo;s philosophy is \u0026ldquo;we\u0026rsquo;re all consenting adults\u0026rdquo; — the language trusts you, but signals intent clearly.\nname (no underscore) → public. Part of the API. Use freely. _name (single underscore) → \u0026ldquo;protected\u0026rdquo; / internal. A convention meaning \u0026ldquo;don\u0026rsquo;t touch this unless you know what you\u0026rsquo;re doing.\u0026rdquo; __name (double underscore) → name-mangled. Python actually renames it to _ClassName__name behind the scenes to prevent accidental subclass overrides. class BankAccount: def __init__(self, owner, initial_balance): self.owner = owner # public self._balance = initial_balance # \u0026#34;private\u0026#34; by convention self.__pin = \u0026#34;1234\u0026#34; # name-mangled def deposit(self, amount): if amount \u0026lt;= 0: raise ValueError(\u0026#34;Deposit must be positive\u0026#34;) self._balance += amount def withdraw(self, amount): if amount \u0026gt; self._balance: raise ValueError(\u0026#34;Insufficient funds\u0026#34;) self._balance -= amount def get_balance(self): return self._balance acc = BankAccount(\u0026#34;Amit\u0026#34;, 1000) acc.deposit(500) acc.withdraw(200) print(acc.get_balance()) # 1300 # acc._balance = -999 ← Possible, but you\u0026#39;re breaking the contract # acc.__pin ← AttributeError! (actually stored as _BankAccount__pin) Now the invariant \u0026ldquo;balance is only modified via legal operations\u0026rdquo; is enforced (as long as users respect the underscore convention).\nEncapsulation\u0026rsquo;s Real Value It\u0026rsquo;s not just about hiding. It\u0026rsquo;s about creating a contract. The class says: \u0026ldquo;Here\u0026rsquo;s my public API. Use only this. I promise this API is stable. Internals may change without notice.\u0026rdquo;\nWithout this contract, every change ripples everywhere. With it, you can refactor internals freely.\n8. Inheritance Inheritance lets you define a new class based on an existing one, reusing its attributes and methods while adding or changing behavior.\nThe Problem It Solves Back to our ride-hailing app. You now have Drivers, DeliveryPartners, and TruckDrivers. All three have names, ratings, and availability. All three can start and end jobs. But each has specializations.\nWithout inheritance, you\u0026rsquo;d copy-paste a lot. With inheritance, you extract common parts into a base class.\nclass Worker: \u0026#34;\u0026#34;\u0026#34;Base class for anyone doing work on the platform.\u0026#34;\u0026#34;\u0026#34; def __init__(self, name, rating): self.name = name self.rating = rating self.is_available = True def start_job(self): if not self.is_available: print(f\u0026#34;{self.name} is busy!\u0026#34;) return False self.is_available = False return True def end_job(self): self.is_available = True class Driver(Worker): def __init__(self, name, rating, car): super().__init__(name, rating) # Delegate to parent\u0026#39;s __init__ self.car = car def start_job(self, rider, destination): if super().start_job(): print(f\u0026#34;{self.name} ({self.car}) picking up {rider} → {destination}\u0026#34;) class DeliveryPartner(Worker): def __init__(self, name, rating, vehicle_type): super().__init__(name, rating) self.vehicle_type = vehicle_type def start_job(self, order_id, address): if super().start_job(): print(f\u0026#34;{self.name} delivering order #{order_id} to {address}\u0026#34;) Anatomy of What Happened class Driver(Worker): → Driver inherits from Worker. Driver is now a \u0026ldquo;subclass\u0026rdquo; (or \u0026ldquo;child\u0026rdquo;); Worker is the \u0026ldquo;superclass\u0026rdquo; (or \u0026ldquo;parent\u0026rdquo;). super().__init__(name, rating) → Call Worker\u0026rsquo;s __init__ to do the common setup. Then we add our own (self.car = car). We overrode start_job in each subclass to add specialized logic, while still delegating the common \u0026ldquo;set availability\u0026rdquo; logic to the parent. The \u0026ldquo;Is-A\u0026rdquo; Relationship Inheritance models an is-a relationship. A Driver is a Worker. A DeliveryPartner is a Worker. If you can\u0026rsquo;t say \u0026ldquo;X is a Y\u0026rdquo; comfortably in English, don\u0026rsquo;t use inheritance.\n❌ A Car is not a Engine. Don\u0026rsquo;t make Car inherit Engine. Use composition (Car has an Engine). ✅ A Dog is an Animal. Inheritance fits.\nWhat Gets Inherited? All public and \u0026ldquo;protected\u0026rdquo; (_underscore) methods and attributes from the parent. Private (__double-underscore) attributes are name-mangled and won\u0026rsquo;t be directly accessible.\nThe super() Function super() returns a proxy object that lets you call methods from the parent class. It\u0026rsquo;s the mechanism that makes inheritance composable — you can extend parent behavior rather than replace it.\nclass Driver(Worker): def describe(self): base = super().describe() # \u0026#34;hypothetically, if Worker had describe\u0026#34; return base + f\u0026#34; Drives a {self.car}.\u0026#34; This way, if Worker\u0026rsquo;s describe changes, Driver automatically benefits.\n9. Polymorphism Polymorphism (Greek: \u0026ldquo;many shapes\u0026rdquo;) means that the same interface can refer to different underlying implementations.\nIn practice: you write code that works with a general type (like Worker), and it automatically works with every specific subtype (Driver, DeliveryPartner, etc.), each behaving appropriately.\nThe Classic Example class Animal: def speak(self): raise NotImplementedError class Dog(Animal): def speak(self): return \u0026#34;Woof!\u0026#34; class Cat(Animal): def speak(self): return \u0026#34;Meow!\u0026#34; class Cow(Animal): def speak(self): return \u0026#34;Moo!\u0026#34; animals = [Dog(), Cat(), Cow()] for animal in animals: print(animal.speak()) Notice how the loop doesn\u0026rsquo;t care what kind of animal it is. It just calls speak() and trusts that each animal knows how to do its thing.\nWhy This Is Powerful Imagine the alternative without polymorphism:\nfor animal in animals: if isinstance(animal, Dog): print(\u0026#34;Woof!\u0026#34;) elif isinstance(animal, Cat): print(\u0026#34;Meow!\u0026#34;) elif isinstance(animal, Cow): print(\u0026#34;Moo!\u0026#34;) # ...add a case every time a new animal is introduced Every time you add a new animal, you have to modify this loop. Polymorphism lets you add new behaviors without modifying existing code. (This is literally the Open-Closed Principle, which you\u0026rsquo;ll see later in SOLID.)\nPython\u0026rsquo;s Duck Typing Python takes polymorphism a step further with duck typing:\n\u0026ldquo;If it walks like a duck and quacks like a duck, it\u0026rsquo;s a duck.\u0026rdquo;\nIn Python, you don\u0026rsquo;t need a common parent class. As long as two objects have a method with the same name and signature, they can be used interchangeably.\nclass Dog: def speak(self): return \u0026#34;Woof!\u0026#34; class Car: def speak(self): return \u0026#34;Vroom!\u0026#34; # Cars don\u0026#39;t normally speak, but we\u0026#39;re being silly for thing in [Dog(), Car()]: print(thing.speak()) # Works fine — no shared parent needed This is looser than Java-style polymorphism but very Pythonic. The tradeoff: no compile-time guarantee that an object has the method. If it doesn\u0026rsquo;t, you get an AttributeError at runtime.\nMethod Overriding When a subclass defines a method with the same name as its parent, it overrides the parent\u0026rsquo;s version:\nclass Worker: def describe(self): return f\u0026#34;I\u0026#39;m a worker named {self.name}\u0026#34; class Driver(Worker): def describe(self): # Overrides the parent\u0026#39;s describe return f\u0026#34;I\u0026#39;m a driver named {self.name}, I drive a {self.car}\u0026#34; Method Overloading? Python does not support traditional method overloading (multiple methods with same name but different parameter lists) like Java/C++. Instead, Python uses default arguments and *args/**kwargs:\nclass Greeter: def greet(self, name=None, formal=False): if name is None: return \u0026#34;Hello!\u0026#34; return f\u0026#34;Good day, {name}\u0026#34; if formal else f\u0026#34;Hey {name}\u0026#34; One method, multiple behaviors based on arguments. Pythonic.\n10. Abstraction Abstraction = hiding complex implementation details behind a simple, understandable interface.\nWhen you press a car\u0026rsquo;s accelerator, you don\u0026rsquo;t think about fuel injection, combustion, or torque conversion. You just press the pedal; the car accelerates. That\u0026rsquo;s abstraction.\nEncapsulation and abstraction are closely related:\nEncapsulation is about protecting internals. Abstraction is about simplifying what users see. Same coin, different sides.\nAbstract Base Classes (ABCs) Sometimes you want to declare: \u0026ldquo;Any class claiming to be a Worker MUST implement these methods.\u0026rdquo; Python gives you the abc module for this.\nfrom abc import ABC, abstractmethod class PaymentProcessor(ABC): @abstractmethod def process(self, amount): pass @abstractmethod def refund(self, amount): pass class UPIProcessor(PaymentProcessor): def process(self, amount): print(f\u0026#34;Processing ₹{amount} via UPI\u0026#34;) def refund(self, amount): print(f\u0026#34;Refunding ₹{amount} via UPI\u0026#34;) class CardProcessor(PaymentProcessor): def process(self, amount): print(f\u0026#34;Charging ₹{amount} on card\u0026#34;) # Forgot to implement refund! # processor = PaymentProcessor() ← TypeError! Can\u0026#39;t instantiate abstract class. upi = UPIProcessor() # ✓ Works — all abstract methods implemented # card = CardProcessor() ← TypeError! `refund` is not implemented. Why this matters: in a large codebase, ABCs catch \u0026ldquo;I forgot to implement this\u0026rdquo; errors at instantiation time rather than at a random runtime moment. They document the contract explicitly.\nAbstraction Without ABCs You don\u0026rsquo;t always need ABCs. Duck typing often suffices. Use ABCs when:\nYou\u0026rsquo;re building a framework or library where others will implement your interfaces. You want explicit, enforced contracts. You have a family of related classes where \u0026ldquo;forgetting to implement\u0026rdquo; would be catastrophic. For simple, internal code, a common base class with NotImplementedError is often enough.\n11. Dunder Methods \u0026ldquo;Dunder\u0026rdquo; = \u0026ldquo;double underscore\u0026rdquo;. Methods like __init__, __str__, __len__. These are Python\u0026rsquo;s hooks for making your objects feel native — integrated with the language\u0026rsquo;s built-in operations.\nThe Problem They Solve Without dunders:\nclass Money: def __init__(self, amount): self.amount = amount a = Money(100) b = Money(50) print(a + b) # TypeError: unsupported operand type(s) for + print(a) # \u0026lt;__main__.Money object at 0x7f8a...\u0026gt; — useless len(a) # TypeError Your objects feel foreign. Dunders fix this.\nCommon Dunders class Money: def __init__(self, amount, currency=\u0026#34;INR\u0026#34;): self.amount = amount self.currency = currency def __repr__(self): \u0026#34;\u0026#34;\u0026#34;Unambiguous representation — for developers, debugging.\u0026#34;\u0026#34;\u0026#34; return f\u0026#34;Money({self.amount!r}, {self.currency!r})\u0026#34; def __str__(self): \u0026#34;\u0026#34;\u0026#34;Readable representation — for users.\u0026#34;\u0026#34;\u0026#34; return f\u0026#34;{self.currency} {self.amount:.2f}\u0026#34; def __eq__(self, other): return isinstance(other, Money) and self.amount == other.amount and self.currency == other.currency def __lt__(self, other): self._check_currency(other) return self.amount \u0026lt; other.amount def __add__(self, other): self._check_currency(other) return Money(self.amount + other.amount, self.currency) def __hash__(self): return hash((self.amount, self.currency)) def _check_currency(self, other): if self.currency != other.currency: raise ValueError(\u0026#34;Currency mismatch\u0026#34;) a = Money(100) b = Money(50) print(a + b) # INR 150.00 print(a == Money(100)) # True print(a \u0026gt; b) # True print(repr(a)) # Money(100, \u0026#39;INR\u0026#39;) Now Money feels like a first-class Python type.\nA Broader List Dunder When It\u0026rsquo;s Called __init__(self, ...) Object creation __repr__(self) repr(obj), debugger, REPL __str__(self) str(obj), print(obj) __eq__, __lt__, __gt__, etc. ==, \u0026lt;, \u0026gt;, etc. __add__, __sub__, __mul__ +, -, * __len__(self) len(obj) __getitem__(self, key) obj[key] __setitem__(self, key, value) obj[key] = value __iter__, __next__ for x in obj: __call__(self, ...) obj(...) — makes object callable like a function __enter__, __exit__ with obj: — context manager __hash__ Using obj as dict key or in a set Dunders are Python\u0026rsquo;s way of letting you plug into its entire ecosystem.\n11.1 __hash__ — The Companion of __eq__ Look back at the Money example. You may have noticed something we glossed over: when we defined __eq__, we also defined __hash__. That wasn\u0026rsquo;t optional. It\u0026rsquo;s a rule.\nThe rule: if you define __eq__, you must also define __hash__. Always. Together. Forever.\nWhy? Because the moment Python sees a custom __eq__, it silently sets __hash__ to None — meaning your object becomes unhashable. It can no longer go into a set or be used as a dict key.\nHere\u0026rsquo;s the bug almost every Python developer hits once:\nclass Point: def __init__(self, x, y): self.x = x self.y = y def __eq__(self, other): return self.x == other.x and self.y == other.y # Looks innocent enough... points = {Point(1, 2), Point(3, 4)} # TypeError: unhashable type: \u0026#39;Point\u0026#39; Why does Python do this? Because hashing has a contract: two objects that are equal must have the same hash. If you redefine equality but leave the default identity-based hash in place, you violate the contract — and dicts and sets break in mysterious ways. Python protects you by refusing to hash the object at all.\nThe fix: define both, and derive the hash from the same fields you use for equality.\nclass Point: def __init__(self, x, y): self.x = x self.y = y def __eq__(self, other): return self.x == other.x and self.y == other.y def __hash__(self): return hash((self.x, self.y)) # tuple of the fields used in __eq__ points = {Point(1, 2), Point(3, 4), Point(1, 2)} print(len(points)) # 2 — the duplicate (1, 2) was deduped correctly The formula is mechanical: hash(tuple_of_equality_fields). As long as you remember \u0026ldquo;same fields in __eq__ go into __hash__\u0026rdquo;, you\u0026rsquo;ll never write a bug here.\nThe shortcut: @dataclass(frozen=True) does both for you automatically. We\u0026rsquo;ll see this in Section 15.\nLLD connection. In a parking lot, suppose you want to know which Vehicle is parked in which ParkingSpot — a dict[ParkingSpot, Vehicle]. For that to work, ParkingSpot instances need to be hashable. The same applies to using Book as a key in a borrow-history dict, or Coordinates in a geo-cache. Whenever a value object needs to live in a set or a dict key, __hash__ is non-negotiable.\n11.2 __contains__ — Making in Work The in operator is one of the most readable bits of Python. \u0026quot;hello\u0026quot; in some_string, 5 in some_list. You can make your own classes work with it by defining __contains__.\nWithout it, callers have to peek into your internals:\nclass ParkingFloor: def __init__(self, spots): self.spots = spots # list of ParkingSpot # Caller has to know about .spots if some_spot in floor.spots: ... That\u0026rsquo;s a leak. The caller now knows that spots is a list. Tomorrow you switch to a set or a dict, and every caller breaks.\nWith __contains__, you expose the intent — \u0026ldquo;does this floor contain this spot?\u0026rdquo; — and hide the data structure:\nclass ParkingFloor: def __init__(self, spots): self._spots = set(spots) def __contains__(self, spot): return spot in self._spots # Reads like English. No internal knowledge needed. if some_spot in floor: print(\u0026#34;Spot belongs to this floor\u0026#34;) This is encapsulation paying interest. The internal data structure can change freely; the public interface (in) stays stable.\n11.3 __enter__ and __exit__ — Context Managers Some resources need cleanup: file handles, database connections, network sockets, locks. The painful version looks like this:\nconn = db.connect() try: conn.begin() conn.execute(\u0026#34;INSERT INTO orders ...\u0026#34;) conn.commit() except Exception: conn.rollback() raise finally: conn.close() Every single time you touch a connection, you must remember the try/except/finally choreography. Forget one rollback, leak one connection, and production pages you at 3am.\nAnalogy. Think of a library borrowing system. When you take a book, you must return it. Now imagine the library hands you a magic backpack: the moment you walk out the door (whether you finished the book, dropped it in a puddle, or it caught fire), the backpack auto-returns it. You can\u0026rsquo;t forget — the system handles it.\nThat magic backpack is a context manager. The with statement is \u0026ldquo;walking through the door.\u0026rdquo;\nclass DatabaseTransaction: def __init__(self, connection): self.connection = connection def __enter__(self): \u0026#34;\u0026#34;\u0026#34;Called when we enter the `with` block. Return the resource.\u0026#34;\u0026#34;\u0026#34; self.connection.begin() return self.connection def __exit__(self, exc_type, exc_val, exc_tb): \u0026#34;\u0026#34;\u0026#34; Called when we leave the `with` block — even on exception. - exc_type is None if the block exited normally. - Return True to suppress the exception, False/None to let it propagate. \u0026#34;\u0026#34;\u0026#34; if exc_type is None: self.connection.commit() else: self.connection.rollback() self.connection.close() return False # never silently swallow exceptions with DatabaseTransaction(conn) as txn: txn.execute(\u0026#34;INSERT INTO orders ...\u0026#34;) txn.execute(\u0026#34;UPDATE inventory ...\u0026#34;) # commit happens here automatically. # If anything above raised — rollback happens automatically. # Either way, the connection is closed. Read __exit__ carefully. The three parameters describe what (if anything) went wrong inside the with block. The return value decides whether to swallow that exception — and the answer is almost always no, let it propagate. Suppressing exceptions silently is how you write bugs that are impossible to debug.\nThe lighter alternative — contextlib.contextmanager. For simple cases, the full dunder approach is overkill. The contextlib decorator lets you write a context manager as a generator function:\nfrom contextlib import contextmanager @contextmanager def database_transaction(connection): connection.begin() try: yield connection # everything before yield = __enter__ except Exception: connection.rollback() raise else: connection.commit() finally: connection.close() # everything after yield = __exit__ with database_transaction(conn) as txn: txn.execute(\u0026#34;INSERT INTO orders ...\u0026#34;) Rule of thumb: use the decorator for one-off, simple cases. Use the class-based form when the context manager has its own state, multiple methods, or needs to be reused.\nLLD connections. Context managers are everywhere in real LLD:\nParking spot lock: acquire the spot\u0026rsquo;s lock on __enter__, release it on __exit__, even if the parking attempt raises. Hotel reservation hold: put the room into a 5-minute \u0026ldquo;tentative\u0026rdquo; state on entry; either commit it (on payment) or release it (on timeout/error). Chess board move attempt: snapshot the board state on entry; restore it if the move turns out to be illegal. Any time the rule is \u0026ldquo;set something up, do work, tear something down — even if work fails,\u0026rdquo; you want a context manager.\n11.4 __slots__ — Memory Optimisation for Many-Instance Classes By default, every Python object carries a per-instance __dict__ — a small dictionary that stores its attributes. That dict gives Python tremendous flexibility (you can add new attributes at runtime), but it also costs memory. For a class with a few instances, it doesn\u0026rsquo;t matter. For a class with millions of instances, it adds up fast.\nimport sys class PointDict: def __init__(self, x, y): self.x = x self.y = y class PointSlots: __slots__ = (\u0026#34;x\u0026#34;, \u0026#34;y\u0026#34;) def __init__(self, x, y): self.x = x self.y = y p1 = PointDict(1, 2) p2 = PointSlots(1, 2) # Crude approximation — actual savings include the __dict__ too print(sys.getsizeof(p1) + sys.getsizeof(p1.__dict__)) # ~152 bytes print(sys.getsizeof(p2)) # ~48 bytes __slots__ tells Python: \u0026ldquo;this class only ever has these attributes — skip the per-instance dict.\u0026rdquo; The savings are substantial. On modern Python, you typically save 40–60% memory per instance.\nThere\u0026rsquo;s a useful side effect too: __slots__ prevents typo bugs.\np = PointSlots(1, 2) p.z = 5 # AttributeError: \u0026#39;PointSlots\u0026#39; object has no attribute \u0026#39;z\u0026#39; A regular class would have happily added a z attribute, hiding the typo. The slotted class catches it at the point of the bug.\nCosts to know:\nNo __dict__ means no dynamic attribute addition. (Often a feature.) Inheritance with __slots__ requires every class in the chain to use it; otherwise the saving is lost. Can\u0026rsquo;t be mixed with class-level mutable defaults easily. LLD connections.\nA parking lot with 10,000 spots: ParkingSpot.__slots__ = (\u0026quot;id\u0026quot;, \u0026quot;level\u0026quot;, \u0026quot;type\u0026quot;, \u0026quot;status\u0026quot;). A ride-sharing app with 100,000 active trips in memory: Trip with slots. A chess engine evaluating 1M board states per second: BoardState with slots is the difference between fitting in cache and not. You don\u0026rsquo;t reach for __slots__ by default. You reach for it when profiling shows that object overhead is your bottleneck.\n12. Properties Sometimes you want attribute-like access but need to intercept reads/writes. Getters/setters, essentially. Python gives you @property for this — elegant and Pythonic.\nThe Setup Suppose we want to ensure temperature in Celsius never goes below absolute zero.\nThe Java-ish Way class Thermometer: def __init__(self, celsius): self._celsius = celsius def get_celsius(self): return self._celsius def set_celsius(self, value): if value \u0026lt; -273.15: raise ValueError(\u0026#34;Below absolute zero!\u0026#34;) self._celsius = value t = Thermometer(25) t.set_celsius(30) print(t.get_celsius()) Works, but verbose. Every access is obj.get_xxx()/obj.set_xxx().\nThe Pythonic Way class Thermometer: def __init__(self, celsius): self.celsius = celsius # Will trigger the setter below! @property def celsius(self): return self._celsius @celsius.setter def celsius(self, value): if value \u0026lt; -273.15: raise ValueError(\u0026#34;Below absolute zero!\u0026#34;) self._celsius = value @property def fahrenheit(self): \u0026#34;\u0026#34;\u0026#34;Computed attribute — no storage needed.\u0026#34;\u0026#34;\u0026#34; return self._celsius * 9 / 5 + 32 t = Thermometer(25) print(t.celsius) # 25 — looks like attribute access t.celsius = 30 # Looks like attribute assignment print(t.fahrenheit) # 86.0 — computed on the fly # t.celsius = -500 # ValueError @property lets you evolve a simple attribute into a computed or validated one without changing the class\u0026rsquo;s public API. Clients doing t.celsius don\u0026rsquo;t have to care whether it\u0026rsquo;s stored or computed.\nThis is a beautiful illustration of abstraction — the interface stays stable while the implementation can evolve.\n12.1 Lazy Initialisation — Compute Only When Needed Some attributes are expensive to produce. A user\u0026rsquo;s lifetime statistics may require a database query that scans months of order data. A PDF\u0026rsquo;s text layer may require a parse over thousands of pages. Computing these eagerly — in __init__ — punishes every caller, even those who never touch the attribute.\nAnalogy. A government office may commission a 100-page report on demand. It doesn\u0026rsquo;t print all reports for every citizen the moment they register. It prints each one only when someone actually asks for it — and then files a copy so the next request is fast.\nThat\u0026rsquo;s lazy initialisation. Compute on first access. Cache for subsequent ones.\nclass UserReport: def __init__(self, user_id): self.user_id = user_id self._statistics = None # not computed yet — sentinel @property def statistics(self): if self._statistics is None: self._statistics = self._fetch_from_db() # expensive return self._statistics def _fetch_from_db(self): # Imagine a heavy join across orders, refunds, ratings, etc. print(f\u0026#34;[db] Computing statistics for user {self.user_id}...\u0026#34;) return {\u0026#34;orders\u0026#34;: 42, \u0026#34;ltv\u0026#34;: 12350.0} u = UserReport(user_id=99) # No DB call yet. print(u.statistics) # [db] Computing... then {\u0026#39;orders\u0026#39;: 42, ...} print(u.statistics) # No DB call — cached. The first access pays the cost. Every subsequent access is free. Callers don\u0026rsquo;t have to know.\nWhy this is better than computing in __init__. Creating a UserReport is now cheap. You can build a thousand UserReports for a leaderboard view and only the ones the user clicks on will pay the DB cost. That\u0026rsquo;s a real performance win on a real API.\nA standard-library shortcut. Python\u0026rsquo;s functools.cached_property does exactly this in one decorator:\nfrom functools import cached_property class UserReport: def __init__(self, user_id): self.user_id = user_id @cached_property def statistics(self): return self._fetch_from_db() The manual version is worth knowing — it shows what cached_property is doing under the hood and works in environments where you need custom invalidation behaviour.\n12.2 The deleter — Cache Invalidation Done Properly @property has a third form: @property_name.deleter. It runs when someone does del obj.attribute. Most of the time you don\u0026rsquo;t need it. When you do need it, it\u0026rsquo;s the cleanest way to express cache invalidation.\nclass UserReport: def __init__(self, user_id): self.user_id = user_id self._statistics = None @property def statistics(self): if self._statistics is None: self._statistics = self._fetch_from_db() return self._statistics @statistics.deleter def statistics(self): \u0026#34;\u0026#34;\u0026#34;Invalidate the cache. Next access will refetch.\u0026#34;\u0026#34;\u0026#34; self._statistics = None def _fetch_from_db(self): print(f\u0026#34;[db] Fetching for {self.user_id}\u0026#34;) return {\u0026#34;orders\u0026#34;: 42} u = UserReport(99) print(u.statistics) # [db] Fetching... then {\u0026#39;orders\u0026#39;: 42} print(u.statistics) # cached # Later, after the user places a new order: del u.statistics # invalidate print(u.statistics) # [db] Fetching... fresh value The deleter is also useful for cleaning up external resources — closing a file handle, releasing a connection — when an attribute is no longer needed.\n12.3 Validated Setters Across Multiple Fields The earlier Thermometer example showed a setter that validates one field in isolation. Real systems often have invariants that span multiple fields. A DateRange must have end \u0026gt;= start. A ParkingFloor\u0026rsquo;s available_spots must satisfy 0 \u0026lt;= available_spots \u0026lt;= capacity. A Hotel\u0026rsquo;s check_out must be after check_in.\nThe naive trap is to validate only inside __init__. But fields can be mutated later — and the moment they are, the invariant is at risk.\nfrom datetime import date class DateRange: def __init__(self, start: date, end: date): # Set the private fields first so the setter can compare. self._start = start self._end = end self._validate() @property def start(self): return self._start @start.setter def start(self, value): if value \u0026gt; self._end: raise ValueError(f\u0026#34;start ({value}) cannot be after end ({self._end})\u0026#34;) self._start = value @property def end(self): return self._end @end.setter def end(self, value): if value \u0026lt; self._start: raise ValueError(f\u0026#34;end ({value}) cannot be before start ({self._start})\u0026#34;) self._end = value def _validate(self): if self._start \u0026gt; self._end: raise ValueError(f\u0026#34;start ({self._start}) cannot be after end ({self._end})\u0026#34;) dr = DateRange(date(2026, 1, 1), date(2026, 1, 31)) dr.end = date(2026, 2, 15) # OK # dr.end = date(2025, 12, 1) # ValueError — would violate invariant The pattern is clear: every setter that participates in an invariant must check the invariant before committing the change. The class can never enter an invalid state — not in __init__, not after any later assignment.\nLLD connection. Validated setters are how you enforce domain invariants — facts that must always be true. The available_spots count on a parking floor should never go below 0 or above capacity. The balance on a BankAccount (with overdraft disabled) should never go below 0. The occupants count in an elevator should never exceed capacity. These are precisely the kinds of bugs that bring systems down in production, and validated setters are the seam at which you catch them.\n13. Multiple Inheritance and the MRO Python, unlike Java, allows a class to inherit from multiple parents. Powerful, but comes with its own puzzle: if two parents have a method with the same name, which one wins?\nclass A: def greet(self): print(\u0026#34;Hi from A\u0026#34;) class B: def greet(self): print(\u0026#34;Hi from B\u0026#34;) class C(A, B): pass c = C() c.greet() # Which one runs? Answer: A. Because Python looks up methods following the MRO (Method Resolution Order), which goes left-to-right through parents.\nprint(C.__mro__) # (\u0026lt;class \u0026#39;C\u0026#39;\u0026gt;, \u0026lt;class \u0026#39;A\u0026#39;\u0026gt;, \u0026lt;class \u0026#39;B\u0026#39;\u0026gt;, \u0026lt;class \u0026#39;object\u0026#39;\u0026gt;) Python uses the C3 linearization algorithm to compute a consistent MRO — it ensures:\nA class appears before its parents. Order of parents in class X(A, B, C) is preserved. The Diamond Problem A / \\ B C \\ / D If B and C both inherit A and override some method, what does D see?\nclass A: def method(self): print(\u0026#34;A\u0026#34;) class B(A): def method(self): print(\u0026#34;B\u0026#34;); super().method() class C(A): def method(self): print(\u0026#34;C\u0026#34;); super().method() class D(B, C): def method(self): print(\u0026#34;D\u0026#34;); super().method() D().method() # D # B # C # A Thanks to C3 linearization, A.method is called exactly once, not twice. This is the key correctness guarantee.\nPractical Advice Multiple inheritance is powerful but often misused. Prefer:\nSingle inheritance + composition, OR Mixins — small, focused classes designed to be combined. class JSONSerializableMixin: def to_json(self): import json return json.dumps(self.__dict__) class LoggableMixin: def log(self, msg): print(f\u0026#34;[{self.__class__.__name__}] {msg}\u0026#34;) class User(JSONSerializableMixin, LoggableMixin): def __init__(self, name): self.name = name u = User(\u0026#34;Amit\u0026#34;) u.log(\u0026#34;created\u0026#34;) # [User] created print(u.to_json()) # {\u0026#34;name\u0026#34;: \u0026#34;Amit\u0026#34;} Mixins are a clean way to share orthogonal behaviors without deep hierarchies.\n13A. Mixins as a Pattern You just saw mixins in passing. They\u0026rsquo;re worth a section of their own — because once you start designing real systems, you\u0026rsquo;ll find that the same \u0026ldquo;horizontal\u0026rdquo; capabilities keep showing up across unrelated classes. Logging. Timestamps. Audit trails. Caching. Serialisation. These aren\u0026rsquo;t an inheritance hierarchy. They\u0026rsquo;re capabilities.\nWhy Mixins Exist Deep inheritance is fragile. A class hierarchy that\u0026rsquo;s six levels deep is a class hierarchy where changing the base class can break anything. But some behaviour genuinely needs to be shared across many unrelated types.\nThink about three classes in a system: User, Order, and Product. They have nothing in common as concepts. But all three probably need:\nA created_at and updated_at timestamp. An audit trail of changes. The ability to be serialised to JSON. You don\u0026rsquo;t want a TimestampedAuditableSerializableBase class that all three inherit from. That\u0026rsquo;s brittle, and it forces an unnatural \u0026ldquo;is-a\u0026rdquo; relationship: a User is not \u0026ldquo;a kind of TimestampedAuditableThing.\u0026rdquo; A User is a User that happens to have timestamping, auditing, and serialisation capabilities.\nThat\u0026rsquo;s what mixins encode.\nThe Analogy Think of skills on a LinkedIn profile. A person can list \u0026ldquo;Python,\u0026rdquo; \u0026ldquo;Public Speaking,\u0026rdquo; and \u0026ldquo;Data Analysis.\u0026rdquo; These aren\u0026rsquo;t an inheritance tree. You don\u0026rsquo;t say a Python developer \u0026ldquo;is a\u0026rdquo; public speaker. They\u0026rsquo;re independent, composable capabilities the person has.\nOr: a Swiss Army knife. The knife blade, screwdriver, scissors, and corkscrew are mixed into one handle. None of them \u0026ldquo;is-a\u0026rdquo; Swiss Army knife — they\u0026rsquo;re capabilities composed into one tool.\nMixins are the same idea in code: small, focused classes that contribute a single capability when mixed into a host class.\nWhat a Mixin Looks Like A well-formed mixin:\nAdds one specific, self-contained capability — not five. Is never instantiated on its own. It only exists to be combined. Has either no __init__, or one that uses cooperative super().__init__(*args, **kwargs) to play nicely with other classes in the MRO. Doesn\u0026rsquo;t carry state that has its own meaning — the state belongs to the host class. Here\u0026rsquo;s a TimestampMixin:\nfrom datetime import datetime class TimestampMixin: \u0026#34;\u0026#34;\u0026#34;Adds created_at and updated_at to any model.\u0026#34;\u0026#34;\u0026#34; def __init__(self, *args, **kwargs): super().__init__(*args, **kwargs) # cooperative — pass through self.created_at = datetime.now() self.updated_at = self.created_at def touch(self): \u0026#34;\u0026#34;\u0026#34;Call whenever the object is modified.\u0026#34;\u0026#34;\u0026#34; self.updated_at = datetime.now() And an AuditMixin:\nclass AuditMixin: \u0026#34;\u0026#34;\u0026#34;Adds a tamper-evident audit trail to any model.\u0026#34;\u0026#34;\u0026#34; def __init__(self, *args, **kwargs): super().__init__(*args, **kwargs) self._audit_log = [] def record_change(self, field: str, old_value, new_value, actor: str): self._audit_log.append({ \u0026#34;field\u0026#34;: field, \u0026#34;old\u0026#34;: old_value, \u0026#34;new\u0026#34;: new_value, \u0026#34;actor\u0026#34;: actor, \u0026#34;at\u0026#34;: datetime.now(), }) def audit_trail(self): return list(self._audit_log) # defensive copy Each mixin contributes one capability. Each calls super().__init__(*args, **kwargs) so that whatever class it sits next to in the MRO chain also gets its initialiser called.\nComposing Mixins on a Real Class Now combine them on a User that inherits from a BaseModel:\nclass BaseModel: \u0026#34;\u0026#34;\u0026#34;The \u0026#39;root\u0026#39; for our domain models — provides an ID.\u0026#34;\u0026#34;\u0026#34; def __init__(self, id: str): self.id = id class User(TimestampMixin, AuditMixin, BaseModel): def __init__(self, id: str, name: str, email: str): # Cooperative call — kicks off the chain along the MRO. super().__init__(id=id) self.name = name self.email = email def update_email(self, new_email: str, actor: str): old = self.email self.email = new_email self.touch() # from TimestampMixin self.record_change(\u0026#34;email\u0026#34;, old, new_email, actor) # from AuditMixin u = User(id=\u0026#34;u-1\u0026#34;, name=\u0026#34;Amit\u0026#34;, email=\u0026#34;amit@old.com\u0026#34;) u.update_email(\u0026#34;amit@new.com\u0026#34;, actor=\u0026#34;self\u0026#34;) print(u.created_at) # set by TimestampMixin print(u.updated_at) # updated by .touch() print(u.audit_trail()) # one entry from AuditMixin print(u.id, u.name) # id from BaseModel, name from User itself One User, four sources of behaviour, no deep hierarchy.\nTracing the MRO The magic of mixins lives in the MRO. When you call super().__init__(id=id) inside User.__init__, Python doesn\u0026rsquo;t just go to BaseModel. It walks the MRO in order.\nprint(User.__mro__) # (\u0026lt;class \u0026#39;__main__.User\u0026#39;\u0026gt;, # \u0026lt;class \u0026#39;__main__.TimestampMixin\u0026#39;\u0026gt;, # \u0026lt;class \u0026#39;__main__.AuditMixin\u0026#39;\u0026gt;, # \u0026lt;class \u0026#39;__main__.BaseModel\u0026#39;\u0026gt;, # \u0026lt;class \u0026#39;object\u0026#39;\u0026gt;) So the call sequence is:\nUser.__init__ calls super().__init__(id=id) → goes to TimestampMixin. TimestampMixin.__init__ calls super().__init__(*args, **kwargs) → goes to AuditMixin. AuditMixin.__init__ calls super().__init__(*args, **kwargs) → goes to BaseModel. BaseModel.__init__ sets self.id. Control unwinds back: AuditMixin sets _audit_log, TimestampMixin sets created_at/updated_at, User sets name/email. The order matters and is determined by C3 linearisation (the algorithm you met in Section 13). The rule of thumb most teams follow: mixins first, base class last. That way each mixin gets to enrich the object before the base class anchors it.\nWhy super().__init__(*args, **kwargs) in Mixins If you\u0026rsquo;ve written Python for a few years, you\u0026rsquo;ve probably written a class with super().__init__() and no arguments. That works for single inheritance. It breaks for mixins.\nclass TimestampMixin: def __init__(self): # BROKEN for multiple inheritance self.created_at = datetime.now() # no super() call class User(TimestampMixin, BaseModel): def __init__(self, id, name): super().__init__() # only TimestampMixin runs # BaseModel.__init__ never gets called → self.id is missing By using super().__init__(*args, **kwargs) in each mixin, every class in the chain gets its arguments and its turn. This is called cooperative multiple inheritance — every class in the chain cooperates by passing the call along.\nLLD Connections You\u0026rsquo;ll see this pattern in many real LLD designs:\nLibrary Management System. Book, Member, Loan all benefit from TimestampMixin and AuditMixin. The audit trail is critical for compliance — knowing who changed what about a member record. Hotel Management. Booking, Guest, Room all need timestamps. The booking system in particular needs a clean audit log of who modified a reservation. Ride-Hailing. Trip, Driver, Rider, Payment all need timestamping. Disputes are resolved by walking audit trails. When you spot the same capability appearing in unrelated classes, that\u0026rsquo;s a mixin asking to be born.\nWhen NOT to Use a Mixin Mixins are not a hammer for every nail. Reach for plain composition (the __init__ parameter — having an auditor field rather than inheriting AuditMixin) when:\nThe capability has its own meaningful lifetime (e.g., a logger that lives across many objects). You need to swap implementations at runtime. The mixin would need to know too much about the host class (e.g., it has to access many fields specifically). Composition is the default; mixins are for \u0026ldquo;I want this capability automatically present, without ceremony.\u0026rdquo;\n14. Composition vs Inheritance One of the most important judgment calls in OOP: should I inherit, or should I compose?\nInheritance = \u0026ldquo;is-a\u0026rdquo; relationship. Car is a Vehicle. Composition = \u0026ldquo;has-a\u0026rdquo; relationship. Car has an Engine. Why Composition Often Wins Inheritance creates tight coupling between parent and child. Changes in the parent ripple to every child. Deep hierarchies become rigid, fragile, and hard to reason about.\nComposition creates loose coupling. A Car holds an Engine object. If you want to change the engine type, swap in a different engine — no class hierarchy touched.\n# INHERITANCE — rigid class Vehicle: def start(self): ... def accelerate(self): ... class Car(Vehicle): ... class ElectricCar(Car): # What if electric cars don\u0026#39;t \u0026#34;start\u0026#34; the same way? def start(self): ... class HybridCar(Car, ElectricCar): # Combinatorial explosion of subclasses ... # COMPOSITION — flexible class PetrolEngine: def start(self): print(\u0026#34;Vroom!\u0026#34;) class ElectricEngine: def start(self): print(\u0026#34;Silent hum...\u0026#34;) class Car: def __init__(self, engine): self.engine = engine def start(self): self.engine.start() Car(PetrolEngine()).start() # Vroom! Car(ElectricEngine()).start() # Silent hum... Same Car class, different engines. Zero subclassing.\nA Famous Principle \u0026ldquo;Favor composition over inheritance.\u0026rdquo; — Design Patterns (Gang of Four)\nThis doesn\u0026rsquo;t mean never inherit. It means: before you inherit, ask \u0026ldquo;Is this really an \u0026lsquo;is-a\u0026rsquo; relationship, or is \u0026lsquo;has-a\u0026rsquo; more honest?\u0026rdquo;\n15. Dataclasses A common pain point: classes that are basically \u0026ldquo;bags of data\u0026rdquo; require a lot of boilerplate.\nclass Point: def __init__(self, x, y): self.x = x self.y = y def __repr__(self): return f\u0026#34;Point(x={self.x}, y={self.y})\u0026#34; def __eq__(self, other): return isinstance(other, Point) and self.x == other.x and self.y == other.y Ugh. Python 3.7+ gives us @dataclass:\nfrom dataclasses import dataclass @dataclass class Point: x: float y: float p1 = Point(1, 2) p2 = Point(1, 2) print(p1) # Point(x=1, y=2) print(p1 == p2) # True @dataclass auto-generates __init__, __repr__, __eq__, and more. You can customize:\n@dataclass(frozen=True, order=True) # Immutable, comparable class Point: x: float y: float p = Point(1, 2) # p.x = 5 ← FrozenInstanceError When to use dataclasses? When your class is primarily data with simple behavior. For rich behavior, use regular classes.\n15.1 field() — The Correct Way to Handle Mutable Defaults The single most common dataclass bug — every Python developer hits it exactly once:\nfrom dataclasses import dataclass @dataclass class ShoppingCart: user_id: str items: list = [] # ← BUG: looks innocent. Is not. cart_a = ShoppingCart(\u0026#34;user-1\u0026#34;) cart_b = ShoppingCart(\u0026#34;user-2\u0026#34;) cart_a.items.append(\u0026#34;book\u0026#34;) print(cart_b.items) # [\u0026#39;book\u0026#39;] ← what?! Both carts share the same list. Why? Because items: list = [] evaluates [] exactly once — at class definition time — and stores it as a class-level default. Every instance that doesn\u0026rsquo;t pass items ends up with a reference to that single shared list.\nThis is the classic \u0026ldquo;mutable default argument\u0026rdquo; bug, dressed up in dataclass clothing.\nThe fix: field(default_factory=list). The factory is called each time an instance is created, producing a fresh list.\nfrom dataclasses import dataclass, field @dataclass class ShoppingCart: user_id: str items: list = field(default_factory=list) cart_a = ShoppingCart(\u0026#34;user-1\u0026#34;) cart_b = ShoppingCart(\u0026#34;user-2\u0026#34;) cart_a.items.append(\u0026#34;book\u0026#34;) print(cart_b.items) # [] ← correct Use default_factory for any mutable default: lists, dicts, sets, custom mutable objects. For immutable defaults (numbers, strings, tuples, None), = value works fine — there\u0026rsquo;s nothing to share.\nIn fact, Python helps you here: if you try to write items: list = [] directly inside @dataclass, recent versions raise ValueError: mutable default \u0026lt;class 'list'\u0026gt; for field items is not allowed. The language itself is steering you toward field().\n15.2 __post_init__ — Validation and Derived Fields Dataclasses auto-generate __init__, which is great — but what if you need validation, or to compute a derived field? You don\u0026rsquo;t replace __init__ (that defeats the point). You add a __post_init__.\n@dataclass class Product: name: str price: float sku: str = field(default=\u0026#34;\u0026#34;) # derived if not provided def __post_init__(self): # Validation if self.price \u0026lt; 0: raise ValueError(f\u0026#34;price cannot be negative: {self.price}\u0026#34;) if not self.name.strip(): raise ValueError(\u0026#34;name cannot be empty\u0026#34;) # Normalisation self.name = self.name.strip().title() # Derived field if not self.sku: self.sku = f\u0026#34;{self.name.replace(\u0026#39; \u0026#39;, \u0026#39;-\u0026#39;).upper()}-{int(self.price)}\u0026#34; p = Product(name=\u0026#34; python book \u0026#34;, price=599.0) print(p.name) # \u0026#39;Python Book\u0026#39; print(p.sku) # \u0026#39;PYTHON-BOOK-599\u0026#39; # Product(name=\u0026#34;x\u0026#34;, price=-1) ← ValueError __post_init__ runs automatically after __init__. It\u0026rsquo;s the canonical place for validation logic, normalisation, and computing derived state in dataclasses.\n15.3 frozen=True — Immutable Value Objects There\u0026rsquo;s a category of objects in any system that don\u0026rsquo;t really have an identity — they represent a value. Money(100, \u0026quot;INR\u0026quot;), Coordinates(28.6, 77.2), DateRange(start, end), SpotType.COMPACT. Two Money(100, \u0026quot;INR\u0026quot;) objects aren\u0026rsquo;t \u0026ldquo;two different ₹100 amounts.\u0026rdquo; They\u0026rsquo;re the same value, twice.\nFor these, mutability is a liability, not a feature. If a Money were mutable, a method could quietly change its amount under your feet — like rewriting the number on a banknote. That\u0026rsquo;s nonsense for a value.\n@dataclass(frozen=True) enforces this:\n@dataclass(frozen=True) class Money: amount: float currency: str m = Money(100, \u0026#34;INR\u0026#34;) # m.amount = 200 ← dataclasses.FrozenInstanceError frozen=True does two important things:\nBlocks attribute assignment after construction. Any attempt raises FrozenInstanceError. Auto-generates __hash__ based on the dataclass fields. This means Money(100, \u0026quot;INR\u0026quot;) is now usable as a dict key or in a set. That second point is huge. It means frozen dataclasses are the cleanest way to write hashable value objects in Python — no need to write __eq__ and __hash__ by hand.\nprices = { Money(100, \u0026#34;INR\u0026#34;): \u0026#34;tea\u0026#34;, Money(250, \u0026#34;INR\u0026#34;): \u0026#34;coffee\u0026#34;, } print(prices[Money(100, \u0026#34;INR\u0026#34;)]) # \u0026#39;tea\u0026#39; — works because frozen → hashable LLD connection. Money, Coordinates, DateRange, SpotType, Address — all of these are value objects. All of them should be @dataclass(frozen=True). The pattern is so consistent that \u0026ldquo;is this a value object?\u0026rdquo; is the first question you should ask before deciding between a regular class and a dataclass.\n15.4 asdict and astuple — Serialisation Helpers Two utility functions from the dataclasses module that come up constantly when serialising objects (e.g., to JSON for an API response):\nfrom dataclasses import asdict, astuple @dataclass class Product: name: str price: float tags: list = field(default_factory=list) p = Product(name=\u0026#34;Book\u0026#34;, price=599, tags=[\u0026#34;education\u0026#34;, \u0026#34;python\u0026#34;]) print(asdict(p)) # {\u0026#39;name\u0026#39;: \u0026#39;Book\u0026#39;, \u0026#39;price\u0026#39;: 599, \u0026#39;tags\u0026#39;: [\u0026#39;education\u0026#39;, \u0026#39;python\u0026#39;]} print(astuple(p)) # (\u0026#39;Book\u0026#39;, 599, [\u0026#39;education\u0026#39;, \u0026#39;python\u0026#39;]) Both are recursive — nested dataclasses inside dataclasses get unpacked properly:\n@dataclass class Address: city: str pincode: str @dataclass class User: name: str address: Address u = User(name=\u0026#34;Amit\u0026#34;, address=Address(city=\u0026#34;Bengaluru\u0026#34;, pincode=\u0026#34;560001\u0026#34;)) print(asdict(u)) # {\u0026#39;name\u0026#39;: \u0026#39;Amit\u0026#39;, \u0026#39;address\u0026#39;: {\u0026#39;city\u0026#39;: \u0026#39;Bengaluru\u0026#39;, \u0026#39;pincode\u0026#39;: \u0026#39;560001\u0026#39;}} That single call replaces the hand-written to_dict() method you\u0026rsquo;d otherwise be writing on every model. Pair it with json.dumps(asdict(u)) and you have JSON serialisation in one line.\nNote: these are functions in the dataclasses module, not methods on the instance. You call asdict(u), not u.asdict(). Easy to forget once.\n15A. ABC vs Protocol Every design pattern that defines an \u0026ldquo;interface\u0026rdquo; — Strategy, Observer, Command, Factory — needs a way in Python to express that interface. Python gives you two ways: Abstract Base Classes (ABC) and Protocol. They look similar at first. They are not interchangeable. Picking the wrong one is the source of an entire family of integration headaches.\nThis section is one of the most practically important in the whole module. Every chapter on patterns that follows uses one or both of these.\nWhy Interfaces Matter at All Recall the Strategy pattern. You have a PaymentProcessor and multiple ways to pay — credit card, UPI, wallet, COD. You don\u0026rsquo;t want PaymentProcessor to know about each one specifically. You want a contract — \u0026ldquo;anything that can charge a customer\u0026rdquo; — and you want to plug in implementations.\nThat contract is the interface. The question is: how do you define the contract in Python?\nApproach 1 — ABC (Abstract Base Class) from abc import ABC, abstractmethod class PaymentProcessor(ABC): \u0026#34;\u0026#34;\u0026#34;A contract for anything that can process payments.\u0026#34;\u0026#34;\u0026#34; def log_attempt(self, amount, currency): \u0026#34;\u0026#34;\u0026#34;Concrete shared behaviour — every processor logs attempts.\u0026#34;\u0026#34;\u0026#34; print(f\u0026#34;[audit] Attempting {amount} {currency}\u0026#34;) @abstractmethod def charge(self, amount: float, currency: str) -\u0026gt; str: \u0026#34;\u0026#34;\u0026#34;Must be implemented by every subclass.\u0026#34;\u0026#34;\u0026#34; ... @abstractmethod def refund(self, transaction_id: str) -\u0026gt; bool: ... class StripeProcessor(PaymentProcessor): def charge(self, amount, currency): self.log_attempt(amount, currency) return f\u0026#34;stripe_tx_{amount}_{currency}\u0026#34; def refund(self, transaction_id): return True Two important properties:\nForced implementation. If StripeProcessor forgot refund(), Python would raise TypeError: Can't instantiate abstract class StripeProcessor with abstract method refund the moment you tried to create one. The contract is enforced at construction. Shared concrete behaviour. log_attempt is defined once and inherited by every subclass. You don\u0026rsquo;t repeat yourself. The Problem with ABC ABC requires explicit inheritance. The subclass must declare (PaymentProcessor) in its class definition. This is fine when you control all the implementations. It breaks the moment you don\u0026rsquo;t.\n# A third-party library you can\u0026#39;t modify class SlackClient: def send(self, message: str) -\u0026gt; bool: return True def get_delivery_status(self, message_id: str) -\u0026gt; str: return \u0026#34;delivered\u0026#34; # Your code expects a Notifier ABC class Notifier(ABC): @abstractmethod def send(self, message: str) -\u0026gt; bool: ... @abstractmethod def get_delivery_status(self, message_id: str) -\u0026gt; str: ... def alert_all(notifier: Notifier, message: str): notifier.send(message) # SlackClient has the right methods, but it doesn\u0026#39;t inherit from Notifier. alert_all(SlackClient(), \u0026#34;hello\u0026#34;) # Works at runtime in Python (duck typing), but: # - A type checker (mypy, pyright) will flag this # - isinstance(slack_client, Notifier) returns False # - You can\u0026#39;t enforce the contract on this third-party class You don\u0026rsquo;t own SlackClient. You can\u0026rsquo;t make it inherit from Notifier. But you\u0026rsquo;d like to use it where a Notifier is expected. This is exactly the case Protocol was designed for.\nApproach 2 — Protocol (Structural Subtyping) from typing import Protocol, runtime_checkable @runtime_checkable class Notifier(Protocol): \u0026#34;\u0026#34;\u0026#34;Anything with these methods qualifies as a Notifier.\u0026#34;\u0026#34;\u0026#34; def send(self, message: str) -\u0026gt; bool: ... def get_delivery_status(self, message_id: str) -\u0026gt; str: ... That\u0026rsquo;s it. There\u0026rsquo;s no (Notifier) on SlackClient. There doesn\u0026rsquo;t need to be.\ndef alert_all(notifier: Notifier, message: str): notifier.send(message) slack = SlackClient() print(isinstance(slack, Notifier)) # True — thanks to @runtime_checkable alert_all(slack, \u0026#34;hello\u0026#34;) # type checker and runtime both happy This is called structural subtyping — sometimes nicknamed \u0026ldquo;static duck typing.\u0026rdquo; If it walks like a duck and quacks like a duck, the type system agrees it\u0026rsquo;s a duck, even without an inheritance declaration.\nThe Decision Rule Use this flow, in this order:\nDo I control the hierarchy AND do subclasses share concrete methods? → ABC. You want forced implementation, and you want to share code (like log_attempt). Do I just need to express \u0026ldquo;anything with these methods qualifies\u0026rdquo;? → Protocol. No shared code. Possibly multiple unrelated classes (including third-party ones) satisfy the contract. Do I need isinstance() checks at runtime? → Protocol decorated with @runtime_checkable. Both? Sometimes you genuinely want both — an ABC for your team\u0026rsquo;s internal implementations (forced contract + shared code) plus a Protocol exposing the same shape for external integration. That\u0026rsquo;s a legitimate design, not a code smell. Side-by-Side: Same Problem, Both Approaches A Serializable contract — anything with to_dict() and to_json().\n# ABC version — would force every existing class to inherit from abc import ABC, abstractmethod class SerializableABC(ABC): @abstractmethod def to_dict(self) -\u0026gt; dict: ... @abstractmethod def to_json(self) -\u0026gt; str: ... # Problem: a dataclass already auto-generates fields; we\u0026#39;d have to wrap it # in a subclass of SerializableABC just to satisfy the type — annoying. # Protocol version — works for any class that happens to have these methods from typing import Protocol class Serializable(Protocol): def to_dict(self) -\u0026gt; dict: ... def to_json(self) -\u0026gt; str: ... # Now any dataclass with these methods is automatically a Serializable. # No inheritance changes. No code modification. For pure shape contracts — \u0026ldquo;anything that quacks like X\u0026rdquo; — Protocol is almost always the better choice. ABC earns its place when you want to share implementation, not just specify shape.\nLLD Connection Strategy pattern for a payment system where all gateways are written by your team → ABC with shared logging behaviour. Strategy pattern for a payment system that must integrate third-party gateways → Protocol. Observer pattern — Listener is naturally a Protocol. Any class with on_event(event) qualifies. Command pattern — Command is a Protocol. Any object with execute() and undo() qualifies. Every time a design pattern says \u0026ldquo;we need an interface,\u0026rdquo; start by asking: do the implementations share code, or just a shape? That answer decides ABC vs Protocol.\n15B. @classmethod as a Factory Constructors (__init__) can only ever create one kind of object. But objects often have multiple natural starting points: from a dict, from a row, from defaults, from another object. Stuffing all these into one __init__ with optional parameters is how __init__ signatures end up with 15 parameters and three contradictory branches.\nThe pattern: define multiple @classmethod \u0026ldquo;named constructors,\u0026rdquo; each capturing one creation intent. This is sometimes called the factory class method pattern, and it\u0026rsquo;s the simplest form of the Factory pattern you\u0026rsquo;ll meet.\nWhy Named Constructors Beat Boolean Flags Compare these two styles. First, the boolean-flag style:\n@dataclass class Member: name: str email: str tier: str borrow_limit: int # Caller has to know magic strings and magic numbers m = Member(name=\u0026#34;Amit\u0026#34;, email=\u0026#34;a@b.com\u0026#34;, tier=\u0026#34;premium\u0026#34;, borrow_limit=10) The caller has to remember: what tier strings are valid? what\u0026rsquo;s the right borrow_limit for premium? Every call site repeats this logic and risks mistakes — tier=\u0026quot;Premium\u0026quot; (wrong case), borrow_limit=15 (wrong policy).\nNow the factory-method style:\n@dataclass class Member: name: str email: str tier: str borrow_limit: int @classmethod def regular(cls, name: str, email: str) -\u0026gt; \u0026#34;Member\u0026#34;: return cls(name=name, email=email, tier=\u0026#34;regular\u0026#34;, borrow_limit=3) @classmethod def premium(cls, name: str, email: str) -\u0026gt; \u0026#34;Member\u0026#34;: return cls(name=name, email=email, tier=\u0026#34;premium\u0026#34;, borrow_limit=10) @classmethod def from_dict(cls, data: dict) -\u0026gt; \u0026#34;Member\u0026#34;: return cls( name=data[\u0026#34;name\u0026#34;], email=data[\u0026#34;email\u0026#34;], tier=data.get(\u0026#34;tier\u0026#34;, \u0026#34;regular\u0026#34;), borrow_limit=data.get(\u0026#34;borrow_limit\u0026#34;, 3), ) # Call sites are clear about intent. No magic strings. m1 = Member.regular(name=\u0026#34;Amit\u0026#34;, email=\u0026#34;a@b.com\u0026#34;) m2 = Member.premium(name=\u0026#34;Priya\u0026#34;, email=\u0026#34;p@b.com\u0026#34;) m3 = Member.from_dict({\u0026#34;name\u0026#34;: \u0026#34;Raj\u0026#34;, \u0026#34;email\u0026#34;: \u0026#34;r@b.com\u0026#34;}) Member.premium(\u0026quot;Priya\u0026quot;, \u0026quot;p@b.com\u0026quot;) reads exactly like English. The policy (what premium means in terms of borrow_limit) is captured once, inside the class. Every call site stays clean.\nWhy cls and Not Member Notice the methods use cls(...) rather than Member(...). That\u0026rsquo;s deliberate. cls refers to whatever subclass invoked the method. If someone later defines:\nclass GoldMember(Member): pass g = GoldMember.premium(name=\u0026#34;Geet\u0026#34;, email=\u0026#34;g@b.com\u0026#34;) print(type(g)) # GoldMember — not Member The cls ensures the right type comes back. Hard-coding Member(...) would silently return a Member even when called on GoldMember. Always use cls in factory methods.\nCommon Factory-Method Patterns A few patterns you\u0026rsquo;ll see repeatedly:\n@dataclass class User: id: str name: str email: str created_at: datetime @classmethod def from_dict(cls, data: dict) -\u0026gt; \u0026#34;User\u0026#34;: \u0026#34;\u0026#34;\u0026#34;For deserialisation from JSON / API payloads.\u0026#34;\u0026#34;\u0026#34; return cls( id=data[\u0026#34;id\u0026#34;], name=data[\u0026#34;name\u0026#34;], email=data[\u0026#34;email\u0026#34;], created_at=datetime.fromisoformat(data[\u0026#34;created_at\u0026#34;]), ) @classmethod def from_db_row(cls, row: tuple) -\u0026gt; \u0026#34;User\u0026#34;: \u0026#34;\u0026#34;\u0026#34;For loading from a database row.\u0026#34;\u0026#34;\u0026#34; id, name, email, created_at = row return cls(id=id, name=name, email=email, created_at=created_at) @classmethod def anonymous(cls) -\u0026gt; \u0026#34;User\u0026#34;: \u0026#34;\u0026#34;\u0026#34;A pre-configured default — useful for tests and guest users.\u0026#34;\u0026#34;\u0026#34; return cls( id=\u0026#34;guest\u0026#34;, name=\u0026#34;Anonymous\u0026#34;, email=\u0026#34;guest@example.com\u0026#34;, created_at=datetime.now(), ) Each method is small, named for its intent, and makes call-site code dramatically more readable.\nLLD Connection This is the simplest form of the Factory pattern — before you ever need a full FactoryMethod class. In an LLD interview, if you\u0026rsquo;re asked \u0026ldquo;how would you create different kinds of Vehicle based on type?\u0026rdquo; — and there are only a handful of vehicle types — factory class methods are often the right answer:\n@dataclass class Vehicle: type: str plate: str spot_size: str @classmethod def bike(cls, plate: str) -\u0026gt; \u0026#34;Vehicle\u0026#34;: return cls(type=\u0026#34;BIKE\u0026#34;, plate=plate, spot_size=\u0026#34;SMALL\u0026#34;) @classmethod def car(cls, plate: str) -\u0026gt; \u0026#34;Vehicle\u0026#34;: return cls(type=\u0026#34;CAR\u0026#34;, plate=plate, spot_size=\u0026#34;MEDIUM\u0026#34;) @classmethod def truck(cls, plate: str) -\u0026gt; \u0026#34;Vehicle\u0026#34;: return cls(type=\u0026#34;TRUCK\u0026#34;, plate=plate, spot_size=\u0026#34;LARGE\u0026#34;) You only reach for a full Factory class (a separate VehicleFactory with subclasses) when creation logic is itself complex — say, vehicles must be loaded from external configuration or built up from multiple sources. For the common case, factory class methods are enough.\n16. Summary The mental map you should carry away:\nOOP is a response to real pain: scattered data+behavior, duplication, tight coupling, and change-resistance in procedural code. A class is a blueprint; an object is an instance built from it. Data lives as attributes; behavior lives as methods. self is simply the object the method was called on — made explicit for clarity. Instance attributes are per-object; class attributes are shared. Don\u0026rsquo;t use mutable class attributes unintentionally. Three kinds of methods: instance (self), class (cls), static (neither). Pick based on what state the method needs. The four pillars: Encapsulation — bundle + protect state, define a contract. Inheritance — share code along \u0026ldquo;is-a\u0026rdquo; relationships. Polymorphism — one interface, many implementations. Enables open/closed design. Abstraction — expose simple interfaces; hide complex internals. Dunders plug your objects into Python\u0026rsquo;s ecosystem so they feel native. __eq__ always travels with __hash__. Context managers (__enter__/__exit__) guarantee cleanup. __slots__ removes per-instance overhead when objects are many. Properties give attribute-like access with validation or computation. They also unlock lazy initialisation for expensive attributes and multi-field invariants via validated setters. Multiple inheritance uses C3 MRO; mixins turn that machinery into a clean pattern for composing orthogonal capabilities (timestamping, auditing, serialisation) onto otherwise unrelated classes. Composition (\u0026ldquo;has-a\u0026rdquo;) is often preferable to inheritance (\u0026ldquo;is-a\u0026rdquo;) for flexibility. Dataclasses remove boilerplate. field(default_factory=...) prevents the mutable-default trap. __post_init__ is the place for validation. frozen=True makes value objects immutable and hashable in one move. ABC vs Protocol: ABC when you control the hierarchy and share concrete code; Protocol when you just need to express \u0026ldquo;anything with these methods qualifies.\u0026rdquo; @classmethod factories are named constructors. They capture creation intent at the call site without needing a separate Factory class. Key Takeaways OOP is a tool, not a religion. Use it where it clarifies; don\u0026rsquo;t force it. Model with real-world relationships. \u0026ldquo;Is-a\u0026rdquo; → inheritance; \u0026ldquo;has-a\u0026rdquo; → composition. The public API is sacred. Design it carefully; hide everything else. Favor composition over inheritance unless \u0026ldquo;is-a\u0026rdquo; is genuinely true. Polymorphism is how OOP enables extension without modification — the seed of SOLID. Reach for the right interface mechanism. Define ABCs for hierarchies you own; define Protocols for shapes that anything can satisfy. Treat value objects as immutable. @dataclass(frozen=True) is the default starting point for anything that represents a value rather than an entity. 17. Practice Problems Problem 1 — Foundational Design a Rectangle class with:\nwidth and height as attributes (must be positive; raise ValueError otherwise). Methods area() and perimeter(). Proper __repr__ and __eq__. Solution:\nclass Rectangle: def __init__(self, width, height): if width \u0026lt;= 0 or height \u0026lt;= 0: raise ValueError(\u0026#34;Dimensions must be positive\u0026#34;) self.width = width self.height = height def area(self): return self.width * self.height def perimeter(self): return 2 * (self.width + self.height) def __repr__(self): return f\u0026#34;Rectangle({self.width}, {self.height})\u0026#34; def __eq__(self, other): return (isinstance(other, Rectangle) and self.width == other.width and self.height == other.height) Problem 2 — Inheritance + Polymorphism Build a small shape hierarchy:\nAbstract Shape with abstract area() method. Concrete Circle and Square subclasses. A function total_area(shapes) that works for any list of shapes. Solution:\nfrom abc import ABC, abstractmethod import math class Shape(ABC): @abstractmethod def area(self): pass class Circle(Shape): def __init__(self, radius): self.radius = radius def area(self): return math.pi * self.radius ** 2 class Square(Shape): def __init__(self, side): self.side = side def area(self): return self.side ** 2 def total_area(shapes): return sum(shape.area() for shape in shapes) print(total_area([Circle(1), Square(2), Circle(3)])) Notice total_area doesn\u0026rsquo;t know or care about specific shape types. Adding a Triangle later requires zero changes here.\nProblem 3 — Spot the Bug Find the bug. Explain why. Fix it.\nclass Team: members = [] def __init__(self, name): self.name = name def add_member(self, member): self.members.append(member) t1 = Team(\u0026#34;Alpha\u0026#34;) t1.add_member(\u0026#34;Amit\u0026#34;) t2 = Team(\u0026#34;Bravo\u0026#34;) print(t2.members) # Why is this non-empty?! Solution:\nmembers = [] is a class attribute, shared across all instances. add_member mutates it, so t2.members reflects changes made via t1.\nFix:\nclass Team: def __init__(self, name): self.name = name self.members = [] # Instance attribute — independent per team Problem 4 — Design Exercise You\u0026rsquo;re modeling a library system. It has:\nBook (title, author, ISBN, is_available) Member (name, member_id, list of borrowed books) Library (holds books, members; supports borrow(member, book), return_book(member, book)). Design the classes. Ensure:\nA book can\u0026rsquo;t be borrowed if unavailable. Members can borrow at most 3 books. Returning a book updates both the book\u0026rsquo;s availability and member\u0026rsquo;s list. Solution (one valid design):\nclass Book: def __init__(self, title, author, isbn): self.title = title self.author = author self.isbn = isbn self.is_available = True def __repr__(self): return f\u0026#34;Book({self.title!r})\u0026#34; class Member: MAX_BOOKS = 3 def __init__(self, name, member_id): self.name = name self.member_id = member_id self.borrowed_books = [] def can_borrow(self): return len(self.borrowed_books) \u0026lt; Member.MAX_BOOKS class Library: def __init__(self): self.books = [] self.members = [] def add_book(self, book): self.books.append(book) def register_member(self, member): self.members.append(member) def borrow(self, member, book): if not book.is_available: raise ValueError(f\u0026#34;{book} is not available\u0026#34;) if not member.can_borrow(): raise ValueError(f\u0026#34;{member.name} has reached the borrow limit\u0026#34;) book.is_available = False member.borrowed_books.append(book) def return_book(self, member, book): if book not in member.borrowed_books: raise ValueError(\u0026#34;Member did not borrow this book\u0026#34;) member.borrowed_books.remove(book) book.is_available = True Discussion questions to ponder:\nWhere did we use encapsulation? (State is modified only through Library methods.) Could we use inheritance here? (Perhaps Book → Ebook, PrintBook — only if they actually differ in behavior.) What if borrow limits differ per member type (student vs faculty)? (Hint: subclass Member or compose a BorrowingPolicy object. We\u0026rsquo;ll revisit this in SOLID.) Problem 5 — Dunder Method Practice Create a Vector2D class supporting:\nv1 + v2 (vector addition) v * 3 (scalar multiplication) v1 == v2 len(v) returning magnitude (approximate — rounded int) Printable via print(v) Solution:\nimport math class Vector2D: def __init__(self, x, y): self.x = x self.y = y def __add__(self, other): return Vector2D(self.x + other.x, self.y + other.y) def __mul__(self, scalar): return Vector2D(self.x * scalar, self.y * scalar) def __eq__(self, other): return isinstance(other, Vector2D) and self.x == other.x and self.y == other.y def __len__(self): return int(math.sqrt(self.x ** 2 + self.y ** 2)) def __repr__(self): return f\u0026#34;Vector2D({self.x}, {self.y})\u0026#34; v1 = Vector2D(3, 4) v2 = Vector2D(1, 2) print(v1 + v2) # Vector2D(4, 6) print(v1 * 2) # Vector2D(6, 8) print(v1 == Vector2D(3, 4)) # True print(len(v1)) # 5 Advanced Python OOP Practice These problems target the deeper toolkit — __hash__, context managers, frozen dataclasses, ABC vs Protocol, mixins, and factory class methods. Work them in order; later problems build on earlier insights.\nType 1 — Spot the Bug P1.1 — The missing __hash__\nclass BookISBN: def __init__(self, isbn: str): self.isbn = isbn def __eq__(self, other): return isinstance(other, BookISBN) and self.isbn == other.isbn # What happens here? Why? isbns = {BookISBN(\u0026#34;978-0-13-468599-1\u0026#34;), BookISBN(\u0026#34;978-0-13-468599-1\u0026#34;)} What does the snippet do, and what\u0026rsquo;s the fix?\nSolution. The code raises TypeError: unhashable type: 'BookISBN'. Defining __eq__ without __hash__ causes Python to set __hash__ to None. The rule of thumb is always pair __eq__ and __hash__. The hash must be derived from the same fields as the equality check, otherwise the dict/set invariant (\u0026ldquo;equal objects have equal hashes\u0026rdquo;) is violated.\nclass BookISBN: def __init__(self, isbn: str): self.isbn = isbn def __eq__(self, other): return isinstance(other, BookISBN) and self.isbn == other.isbn def __hash__(self): return hash(self.isbn) Or — the one-liner version with @dataclass(frozen=True):\nfrom dataclasses import dataclass @dataclass(frozen=True) class BookISBN: isbn: str frozen=True generates __eq__ and __hash__ based on the fields automatically.\nP1.2 — Mutable dataclass default\nfrom dataclasses import dataclass @dataclass class ShoppingCart: user_id: str items: list = [] cart_a = ShoppingCart(\u0026#34;user-1\u0026#34;) cart_b = ShoppingCart(\u0026#34;user-2\u0026#34;) cart_a.items.append(\u0026#34;book\u0026#34;) print(cart_b.items) What is wrong with this code, and what would print(cart_b.items) output? Fix it.\nSolution. Modern Python actually refuses to define this class at all: it raises ValueError: mutable default \u0026lt;class 'list'\u0026gt; for field items is not allowed: use default_factory. Older Python (and the manual equivalent of this dataclass) would let it through, and both carts would share the same list — so print(cart_b.items) would output ['book'].\nThe fix is field(default_factory=list). The factory is called per instance, producing a fresh list each time.\nfrom dataclasses import dataclass, field @dataclass class ShoppingCart: user_id: str items: list = field(default_factory=list) Now cart_b.items is [], as a clean new cart should be.\nP1.3 — ABC used where Protocol was needed\nfrom abc import ABC, abstractmethod class Drawable(ABC): @abstractmethod def draw(self) -\u0026gt; None: ... # A third-party class — you cannot modify this class SVGCircle: def draw(self): print(\u0026#34;\u0026lt;circle cx=\u0026#39;10\u0026#39; cy=\u0026#39;10\u0026#39; r=\u0026#39;5\u0026#39;/\u0026gt;\u0026#34;) def render_all(drawables: list[Drawable]): for d in drawables: d.draw() render_all([SVGCircle()]) What\u0026rsquo;s the issue here? What\u0026rsquo;s the right fix?\nSolution. SVGCircle has the draw method but doesn\u0026rsquo;t inherit from Drawable. At runtime, Python\u0026rsquo;s duck typing will let render_all work — but the type system flags it, isinstance(svg_circle, Drawable) is False, and any code that does an isinstance check (e.g., for filtering) will exclude SVGCircle. You also can\u0026rsquo;t enforce that callers actually pass valid drawables.\nABC requires explicit inheritance. We don\u0026rsquo;t own SVGCircle, so inheritance isn\u0026rsquo;t an option. The right tool is Protocol: a shape contract that any class with the right methods satisfies automatically.\nfrom typing import Protocol, runtime_checkable @runtime_checkable class Drawable(Protocol): def draw(self) -\u0026gt; None: ... def render_all(drawables: list[Drawable]): for d in drawables: d.draw() print(isinstance(SVGCircle(), Drawable)) # True render_all([SVGCircle()]) # works, type-checks correctly Use ABC when you control the hierarchy and want to share concrete code. Use Protocol when you just want to say \u0026ldquo;anything with this shape qualifies.\u0026rdquo;\nType 2 — Implement It P2.1 — Context manager for a parking spot\nImplement a ParkingSpotLock context manager. On entry, it marks the given spot as occupied. On exit, it releases the spot — even if an exception was raised inside the with block. Demonstrate this behaviour by including a usage example where the parking session raises an exception, and show that the spot is still released.\nSolution.\nclass ParkingSpot: def __init__(self, spot_id: str): self.spot_id = spot_id self.is_occupied = False class ParkingSpotLock: \u0026#34;\u0026#34;\u0026#34;Acquires a parking spot on entry; releases it on exit unconditionally.\u0026#34;\u0026#34;\u0026#34; def __init__(self, spot: ParkingSpot): self.spot = spot def __enter__(self): if self.spot.is_occupied: raise RuntimeError(f\u0026#34;Spot {self.spot.spot_id} is already occupied\u0026#34;) self.spot.is_occupied = True return self.spot def __exit__(self, exc_type, exc_val, exc_tb): # Release the spot regardless of whether the block raised. # exc_type is None on a clean exit, or the exception class otherwise. self.spot.is_occupied = False # Return False so any exception inside the `with` block still propagates. return False # --- Usage demonstrating exception safety --- spot = ParkingSpot(\u0026#34;L1-A-07\u0026#34;) try: with ParkingSpotLock(spot) as locked_spot: print(f\u0026#34;In block: occupied = {locked_spot.is_occupied}\u0026#34;) # True raise ValueError(\u0026#34;Payment failed mid-parking\u0026#34;) except ValueError as e: print(f\u0026#34;Caught: {e}\u0026#34;) print(f\u0026#34;After block: occupied = {spot.is_occupied}\u0026#34;) # False — released Two important details:\n__exit__ releases the spot before it returns. The release happens whether the block ran cleanly or raised. Returning False (or None) from __exit__ means the exception is allowed to propagate. Returning True would silently swallow it — almost always a bug. P2.2 — Frozen value object: Money\nImplement a Money value object that:\nHas amount: float and currency: str. Is immutable. Supports __add__ for adding two Money of the same currency. Raises ValueError on currency mismatch. Has a __repr__ of the form Money(₹500.00) for INR, Money($500.00) for USD, etc. Demonstrate that Money is hashable (usable as a dict key) thanks to frozen=True.\nSolution.\nfrom dataclasses import dataclass CURRENCY_SYMBOLS = {\u0026#34;INR\u0026#34;: \u0026#34;₹\u0026#34;, \u0026#34;USD\u0026#34;: \u0026#34;$\u0026#34;, \u0026#34;EUR\u0026#34;: \u0026#34;€\u0026#34;, \u0026#34;GBP\u0026#34;: \u0026#34;£\u0026#34;} @dataclass(frozen=True) class Money: amount: float currency: str def __add__(self, other: \u0026#34;Money\u0026#34;) -\u0026gt; \u0026#34;Money\u0026#34;: if not isinstance(other, Money): return NotImplemented if self.currency != other.currency: raise ValueError( f\u0026#34;Currency mismatch: {self.currency} vs {other.currency}\u0026#34; ) # Return a new Money — never mutate (we\u0026#39;re frozen anyway). return Money(amount=self.amount + other.amount, currency=self.currency) def __repr__(self) -\u0026gt; str: symbol = CURRENCY_SYMBOLS.get(self.currency, self.currency + \u0026#34; \u0026#34;) return f\u0026#34;Money({symbol}{self.amount:.2f})\u0026#34; m1 = Money(500.0, \u0026#34;INR\u0026#34;) m2 = Money(250.0, \u0026#34;INR\u0026#34;) print(m1 + m2) # Money(₹750.00) # m1.amount = 0 # FrozenInstanceError — immutability enforced # Hashable because of frozen=True — usable as a dict key prices = {Money(500.0, \u0026#34;INR\u0026#34;): \u0026#34;headphones\u0026#34;, Money(2500.0, \u0026#34;INR\u0026#34;): \u0026#34;keyboard\u0026#34;} print(prices[Money(500.0, \u0026#34;INR\u0026#34;)]) # \u0026#39;headphones\u0026#39; # Currency mismatch raises try: print(Money(100, \u0026#34;INR\u0026#34;) + Money(100, \u0026#34;USD\u0026#34;)) except ValueError as e: print(f\u0026#34;Caught: {e}\u0026#34;) Why frozen=True is the right call here: Money is a value object. There\u0026rsquo;s no notion of \u0026ldquo;the same Money changing its amount over time.\u0026rdquo; Two Money(500, \u0026quot;INR\u0026quot;) instances are the same value and should be interchangeable — exactly the case for immutability and equality-by-value. frozen=True gives you all of that, plus __hash__, in one decorator.\nP2.3 — Mixin composition\nImplement three classes:\nSerializableMixin — adds a to_dict() method (using vars(self)) and to_json() (using json.dumps). ValidatableMixin — adds a validate() method that returns a list of error strings. Subclasses provide rules by overriding _validation_rules(). A Product class inheriting from both mixins (plus a BaseModel), with concrete validation rules. Show that Product.__mro__ resolves methods in the expected order, and demonstrate both mixins working on the same instance.\nSolution.\nimport json from datetime import datetime class BaseModel: \u0026#34;\u0026#34;\u0026#34;Root model. Holds an id and a created_at.\u0026#34;\u0026#34;\u0026#34; def __init__(self, *, id: str): self.id = id self.created_at = datetime.now() class SerializableMixin: \u0026#34;\u0026#34;\u0026#34;Adds .to_dict() and .to_json() to any model.\u0026#34;\u0026#34;\u0026#34; def to_dict(self) -\u0026gt; dict: # vars(self) returns the instance\u0026#39;s __dict__ — all attributes. # Strip private attributes and non-serializable values for safety. return { key: value.isoformat() if isinstance(value, datetime) else value for key, value in vars(self).items() if not key.startswith(\u0026#34;_\u0026#34;) } def to_json(self) -\u0026gt; str: return json.dumps(self.to_dict(), default=str) class ValidatableMixin: \u0026#34;\u0026#34;\u0026#34;Adds .validate() — returns a list of error strings (empty if valid).\u0026#34;\u0026#34;\u0026#34; def validate(self) -\u0026gt; list[str]: errors: list[str] = [] for rule_name, rule in self._validation_rules().items(): if not rule(): errors.append(rule_name) return errors def _validation_rules(self) -\u0026gt; dict: \u0026#34;\u0026#34;\u0026#34;Subclasses override this with their own rule mapping.\u0026#34;\u0026#34;\u0026#34; return {} class Product(SerializableMixin, ValidatableMixin, BaseModel): def __init__(self, *, id: str, name: str, price: float): super().__init__(id=id) self.name = name self.price = price def _validation_rules(self) -\u0026gt; dict: return { \u0026#34;name must be non-empty\u0026#34;: lambda: bool(self.name.strip()), \u0026#34;price must be positive\u0026#34;: lambda: self.price \u0026gt; 0, } # --- Demonstration --- p = Product(id=\u0026#34;p-1\u0026#34;, name=\u0026#34;Headphones\u0026#34;, price=2499.0) print(p.to_dict()) # {\u0026#39;id\u0026#39;: \u0026#39;p-1\u0026#39;, \u0026#39;created_at\u0026#39;: \u0026#39;2026-05-21T...\u0026#39;, \u0026#39;name\u0026#39;: \u0026#39;Headphones\u0026#39;, \u0026#39;price\u0026#39;: 2499.0} print(p.validate()) # [] — no errors bad = Product(id=\u0026#34;p-2\u0026#34;, name=\u0026#34; \u0026#34;, price=-1) print(bad.validate()) # [\u0026#39;name must be non-empty\u0026#39;, \u0026#39;price must be positive\u0026#39;] print(Product.__mro__) # (Product, SerializableMixin, ValidatableMixin, BaseModel, object) The MRO matches the rule \u0026ldquo;mixins first, base class last.\u0026rdquo; When Product.__init__ calls super().__init__(id=id), Python walks the MRO: SerializableMixin has no __init__ of its own, neither does ValidatableMixin, so the call ultimately lands on BaseModel.__init__, which sets self.id and self.created_at.\nP2.4 — ABC vs Protocol: side-by-side comparison\nDefine a Notifier contract twice — once as an ABC, once as a Protocol — with methods send(message: str) -\u0026gt; bool and get_delivery_status(message_id: str) -\u0026gt; str. Then:\n(a) Write an SMSNotifier class that satisfies both versions. (b) Show that an existing third-party SlackClient (which has the right methods but doesn\u0026rsquo;t inherit from anything) satisfies the Protocol but not the ABC. (c) Explain in code comments why the Protocol version is better for integration with code you don\u0026rsquo;t control.\nSolution.\nfrom abc import ABC, abstractmethod from typing import Protocol, runtime_checkable # --- ABC version --- class NotifierABC(ABC): @abstractmethod def send(self, message: str) -\u0026gt; bool: ... @abstractmethod def get_delivery_status(self, message_id: str) -\u0026gt; str: ... # --- Protocol version --- @runtime_checkable class NotifierProto(Protocol): def send(self, message: str) -\u0026gt; bool: ... def get_delivery_status(self, message_id: str) -\u0026gt; str: ... # (a) A class that satisfies both — explicit inheritance for ABC, # structural conformance for Protocol. class SMSNotifier(NotifierABC): def send(self, message: str) -\u0026gt; bool: print(f\u0026#34;[SMS] {message}\u0026#34;) return True def get_delivery_status(self, message_id: str) -\u0026gt; str: return \u0026#34;delivered\u0026#34; # (b) A third-party class we can\u0026#39;t modify. class SlackClient: def send(self, message: str) -\u0026gt; bool: return True def get_delivery_status(self, message_id: str) -\u0026gt; str: return \u0026#34;delivered\u0026#34; slack = SlackClient() sms = SMSNotifier() print(isinstance(sms, NotifierABC)) # True print(isinstance(slack, NotifierABC)) # False — no inheritance print(isinstance(sms, NotifierProto)) # True print(isinstance(slack, NotifierProto)) # True — structural match # (c) Why Protocol wins for integration: # To make SlackClient satisfy NotifierABC, we\u0026#39;d have to monkey-patch its bases # or wrap it in an adapter class. Both add ceremony and risk for no real benefit # — the class already does exactly what we need. # NotifierProto accepts SlackClient as-is, which is the whole point of structural # subtyping. Use ABC when you want a contract AND shared concrete behaviour. # Use Protocol when you just need a shape that anything (including third-party # code) can satisfy. Type 3 — Design Judgment P3.1 — Choose the right interface mechanism\nYou\u0026rsquo;re building a payment system. Today, the gateways StripeGateway, RazorpayGateway, and PayPalGateway are all written by your team in the same codebase. In the future, the platform will support third-party payment providers — written by external developers, dropped in as plugins.\nShould PaymentGateway be an ABC or a Protocol? Justify your answer. Is there a way to support both?\nSolution.\nThere isn\u0026rsquo;t a single \u0026ldquo;right\u0026rdquo; answer — it depends on what matters most. Here\u0026rsquo;s the reasoning:\nArguments for ABC for the internal gateways.\nYou control the hierarchy. You probably want to share concrete logic — logging payment attempts, recording metrics, retry policies. ABC lets you put these in one place. You want the type system (and runtime) to force your team to implement required methods. Forgetting refund() should be a TypeError at class instantiation, not a NoneType has no attribute 'refund' bug at 3am. Arguments for Protocol for the external/plugin gateways.\nThird-party developers can\u0026rsquo;t be forced to inherit from your ABC — and you wouldn\u0026rsquo;t want to force a foreign dependency on them anyway. You only need to express the shape of a payment gateway, not share code with third parties. The pragmatic design: do both.\nfrom abc import ABC, abstractmethod from typing import Protocol, runtime_checkable @runtime_checkable class PaymentGateway(Protocol): \u0026#34;\u0026#34;\u0026#34;Public contract — anything with these methods works.\u0026#34;\u0026#34;\u0026#34; def charge(self, amount: float, currency: str) -\u0026gt; str: ... def refund(self, transaction_id: str) -\u0026gt; bool: ... class InternalGateway(ABC): \u0026#34;\u0026#34;\u0026#34;For your team\u0026#39;s gateways. Forces implementation + shared behaviour.\u0026#34;\u0026#34;\u0026#34; def log_attempt(self, amount, currency): # shared concrete code ... @abstractmethod def charge(self, amount: float, currency: str) -\u0026gt; str: ... @abstractmethod def refund(self, transaction_id: str) -\u0026gt; bool: ... class StripeGateway(InternalGateway): def charge(self, amount, currency): ... def refund(self, tx_id): ... # A third-party plugin gateway needs no inheritance to be acceptable here. def process(gateway: PaymentGateway, amount: float, currency: str): return gateway.charge(amount, currency) Every InternalGateway automatically satisfies the PaymentGateway Protocol because of structural subtyping. External plugins satisfy PaymentGateway without knowing your ABC exists. You get the best of both: forced contracts and shared code for your team, structural compatibility for the world.\nAlternative. If you don\u0026rsquo;t have meaningful shared code, drop the ABC entirely and rely on Protocol everywhere. Simpler. Pick this if internal code-sharing isn\u0026rsquo;t actually paying off.\nP3.2 — When is a dataclass NOT the right tool?\nFor each of the following, decide whether @dataclass (or @dataclass(frozen=True)) is appropriate, or whether a regular class is better. Justify.\n(a) A BankAccount with a balance that must never go negative, and methods deposit(), withdraw(), transfer(). (b) A Coordinates(lat, lon) value object used as a dict key in a geographic cache. (c) An EventBus that manages subscriptions and dispatches events to listeners.\nSolution.\n(a) Regular class. A BankAccount is an entity with identity (the account number) and rich behaviour (deposit/withdraw/transfer with rules). Dataclasses excel at data + simple behaviour; they don\u0026rsquo;t help much when the class is mostly methods that enforce invariants. You can still use a dataclass — for the fields — and add methods, but the dataclass-generated __eq__ may not make sense for an account (two accounts with the same balance aren\u0026rsquo;t \u0026ldquo;equal\u0026rdquo;). A regular class avoids the ambiguity.\n(b) @dataclass(frozen=True). This is the textbook case for a frozen dataclass. Coordinates is a pure value object with no identity, used as a dict key. frozen=True gives you immutability, structural equality, and __hash__ in a single decorator. Writing this by hand is just busywork.\n@dataclass(frozen=True) class Coordinates: lat: float lon: float (c) Regular class. An EventBus is behaviour-first: it has subscribers, methods to subscribe/unsubscribe, methods to dispatch. There\u0026rsquo;s barely any \u0026ldquo;data\u0026rdquo; to auto-generate boilerplate for. A dataclass would auto-generate __eq__ and __repr__ based on internal subscriber lists — both meaningless for an EventBus. A regular class is clearer and avoids unwanted dunders.\nThe principle: dataclasses are for data; regular classes are for behaviour. Value objects and DTOs love @dataclass. Services, controllers, and stateful entities are better as regular classes.\nType 4 — Capstone P4.1 — Design a model layer for a Library Management System\nDesign a small model layer using everything from this module. Requirements:\nBook — value object identified by ISBN; immutable; usable as a dict key. Member — entity with timestamps, an audit trail of changes, and validation (name must be non-empty, borrow limit must be positive). Borrowable — a Protocol that any lendable item (Book, Magazine, DVD) must satisfy, with is_available() and get_due_date(). LibraryItem — an ABC for items the library owns, with abstract catalogue_entry() and shared age_in_collection(). Then walk through one scenario: a Member borrows a Book, the Book\u0026rsquo;s availability changes, and the Member\u0026rsquo;s audit trail records the borrow event.\nSolution.\nfrom abc import ABC, abstractmethod from dataclasses import dataclass, field from datetime import datetime, date, timedelta from typing import Protocol, runtime_checkable # ---------- Value object: Book ---------- @dataclass(frozen=True) class Book: \u0026#34;\u0026#34;\u0026#34; A book is identified by its ISBN. The same ISBN means the same book (logically) — equality and hashing are by all fields, which suit a pure value object. Frozen so it can sit in dicts and sets. \u0026#34;\u0026#34;\u0026#34; isbn: str title: str author: str # ---------- Mixins for entities ---------- class TimestampMixin: def __init__(self, *args, **kwargs): super().__init__(*args, **kwargs) self.created_at = datetime.now() self.updated_at = self.created_at def touch(self): self.updated_at = datetime.now() class AuditMixin: def __init__(self, *args, **kwargs): super().__init__(*args, **kwargs) self._audit_log: list[dict] = [] def record_change(self, action: str, details: dict): self._audit_log.append({ \u0026#34;action\u0026#34;: action, \u0026#34;details\u0026#34;: details, \u0026#34;at\u0026#34;: datetime.now(), }) def audit_trail(self) -\u0026gt; list[dict]: return list(self._audit_log) # ---------- Base for entities ---------- class BaseEntity: def __init__(self, *, id: str): self.id = id # ---------- Member (entity, not value object) ---------- class Member(TimestampMixin, AuditMixin, BaseEntity): def __init__(self, *, id: str, name: str, borrow_limit: int): super().__init__(id=id) self._validate(name, borrow_limit) self.name = name self.borrow_limit = borrow_limit self.borrowed: dict[str, date] = {} # isbn -\u0026gt; due_date def _validate(self, name, borrow_limit): if not name.strip(): raise ValueError(\u0026#34;Member name cannot be empty\u0026#34;) if borrow_limit \u0026lt;= 0: raise ValueError(\u0026#34;borrow_limit must be positive\u0026#34;) def can_borrow_more(self) -\u0026gt; bool: return len(self.borrowed) \u0026lt; self.borrow_limit def borrow(self, book: Book, due_date: date): if not self.can_borrow_more(): raise RuntimeError(\u0026#34;Borrow limit reached\u0026#34;) self.borrowed[book.isbn] = due_date self.touch() self.record_change( \u0026#34;borrowed\u0026#34;, {\u0026#34;isbn\u0026#34;: book.isbn, \u0026#34;title\u0026#34;: book.title, \u0026#34;due\u0026#34;: due_date.isoformat()}, ) # ---------- Protocol: structural contract for lendable items ---------- @runtime_checkable class Borrowable(Protocol): \u0026#34;\u0026#34;\u0026#34;Any class with these methods is borrowable — no inheritance required.\u0026#34;\u0026#34;\u0026#34; def is_available(self) -\u0026gt; bool: ... def get_due_date(self) -\u0026gt; date | None: ... # ---------- ABC: items the library physically owns ---------- class LibraryItem(ABC): \u0026#34;\u0026#34;\u0026#34; For things the library owns and tracks. ABC because we want forced implementation AND shared concrete behaviour (age_in_collection). \u0026#34;\u0026#34;\u0026#34; def __init__(self, acquired_on: date): self.acquired_on = acquired_on def age_in_collection(self) -\u0026gt; timedelta: return date.today() - self.acquired_on @abstractmethod def catalogue_entry(self) -\u0026gt; str: \u0026#34;\u0026#34;\u0026#34;One-line catalogue summary.\u0026#34;\u0026#34;\u0026#34; # ---------- A concrete library item satisfying both LibraryItem and Borrowable ---------- class BookCopy(LibraryItem): \u0026#34;\u0026#34;\u0026#34; A *physical copy* of a Book. The Book is a value object; the BookCopy is the physical thing on the shelf — it can be borrowed, has a state. \u0026#34;\u0026#34;\u0026#34; def __init__(self, copy_id: str, book: Book, acquired_on: date): super().__init__(acquired_on=acquired_on) self.copy_id = copy_id self.book = book self._due_date: date | None = None def is_available(self) -\u0026gt; bool: return self._due_date is None def get_due_date(self) -\u0026gt; date | None: return self._due_date def borrow_until(self, due: date): if not self.is_available(): raise RuntimeError(f\u0026#34;Copy {self.copy_id} is already borrowed\u0026#34;) self._due_date = due def catalogue_entry(self) -\u0026gt; str: return f\u0026#34;[{self.copy_id}] {self.book.title} by {self.book.author}\u0026#34; # ---------- Scenario walkthrough ---------- book = Book(isbn=\u0026#34;978-0-13-468599-1\u0026#34;, title=\u0026#34;Effective Python\u0026#34;, author=\u0026#34;Brett Slatkin\u0026#34;) copy = BookCopy(copy_id=\u0026#34;C-001\u0026#34;, book=book, acquired_on=date(2024, 1, 15)) member = Member(id=\u0026#34;M-001\u0026#34;, name=\u0026#34;Amit\u0026#34;, borrow_limit=3) # Confirm the Borrowable Protocol is satisfied — structurally, no inheritance. assert isinstance(copy, Borrowable) # The member borrows the copy. due = date.today() + timedelta(days=14) copy.borrow_until(due) member.borrow(book, due) print(copy.catalogue_entry()) # [C-001] Effective Python by Brett Slatkin print(copy.is_available()) # False print(copy.get_due_date()) # in 14 days print(member.borrowed) # {\u0026#39;978-0-13-468599-1\u0026#39;: date(...)} print(member.audit_trail()) # one \u0026#39;borrowed\u0026#39; entry Why each design decision was made:\nBook is a frozen dataclass. Books are values, not entities — two Book objects with the same ISBN are interchangeable. frozen=True gives us immutability and a free __hash__ so books can live in dicts/sets. Member is a regular class with mixins. Members are entities (they have an id, lifecycle, mutable state). Timestamping and auditing are reused across entities, so they become mixins. The MRO is Member → TimestampMixin → AuditMixin → BaseEntity → object — mixins first, base last. Borrowable is a Protocol. A magazine or DVD class (potentially defined elsewhere) should be borrowable without having to inherit from anything. Protocol enables this with no ceremony. LibraryItem is an ABC. Things the library owns share concrete behaviour (age_in_collection) and need forced implementation of catalogue_entry. ABC is the right fit. BookCopy is the bridge. It inherits from the ABC LibraryItem (because it\u0026rsquo;s a library-owned thing) and structurally satisfies the Borrowable Protocol (because it has is_available and get_due_date). One class can satisfy both an ABC and a Protocol — they\u0026rsquo;re complementary, not exclusive. This single example uses: dataclasses (frozen), mixins with cooperative super(), ABC, Protocol, validated state, and audit trails — i.e., the entire toolkit working together.\nYou\u0026rsquo;re now fluent in Python OOP. Next up: SOLID Principles — the five rules that separate \u0026ldquo;code that uses classes\u0026rdquo; from \u0026ldquo;code that uses classes well.\u0026rdquo;\nThis content is part of Codeverra — a platform for learning coding, data science, DSA, and AI from scratch. Explore more: https://codeverra.com\n","permalink":"https://blogs.codeverra.com/blog/low_level_design/01_oop_in_python/","summary":"\u003ch1 id=\"object-oriented-programming-in-python--a-deep-dive\"\u003eObject-Oriented Programming in Python — A Deep Dive\u003c/h1\u003e\n\u003cblockquote\u003e\n\u003cp\u003e\u003cstrong\u003eWho this is for:\u003c/strong\u003e Anyone who knows basic Python (variables, functions, lists, dicts) and wants to truly understand OOP — not just the syntax, but \u003cem\u003ewhy\u003c/em\u003e it exists, \u003cem\u003ewhat problems\u003c/em\u003e it solves, and \u003cem\u003ehow\u003c/em\u003e to wield it well in real systems.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003chr\u003e\n\u003ch2 id=\"table-of-contents\"\u003eTable of Contents\u003c/h2\u003e\n\u003col\u003e\n\u003cli\u003e\u003ca href=\"#1-the-world-before-oop\"\u003eThe World Before OOP — Why Do We Even Need It?\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#2-classes-and-objects\"\u003eClasses and Objects — The Blueprint and the Building\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#3-the-self-parameter\"\u003eThe \u003ccode\u003eself\u003c/code\u003e Parameter — Python\u0026rsquo;s Most Misunderstood Keyword\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#4-attributes\"\u003eAttributes — Instance vs Class Attributes\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#5-methods\"\u003eMethods — Instance, Class, and Static\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#6-the-four-pillars-of-oop\"\u003eThe Four Pillars of OOP\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#7-encapsulation\"\u003eEncapsulation — The Art of Hiding\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#8-inheritance\"\u003eInheritance — Building on What Exists\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#9-polymorphism\"\u003ePolymorphism — Many Forms, One Interface\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#10-abstraction\"\u003eAbstraction — Focusing on What Matters\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#11-dunder-methods\"\u003eDunder Methods — Making Your Objects Feel Native\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#12-properties\"\u003eProperties — Controlled Attribute Access\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#13-multiple-inheritance-and-the-mro\"\u003eMultiple Inheritance and the MRO\u003c/a\u003e\n13A. \u003ca href=\"#13a-mixins-as-a-pattern\"\u003eMixins as a Pattern — Composable Capabilities\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#14-composition-vs-inheritance\"\u003eComposition vs Inheritance\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#15-dataclasses\"\u003eDataclasses — OOP with Less Boilerplate\u003c/a\u003e\n15A. \u003ca href=\"#15a-abc-vs-protocol\"\u003e\u003ccode\u003eABC\u003c/code\u003e vs \u003ccode\u003eProtocol\u003c/code\u003e — Two Ways to Define an Interface\u003c/a\u003e\n15B. \u003ca href=\"#15b-classmethod-as-a-factory\"\u003e\u003ccode\u003e@classmethod\u003c/code\u003e as a Factory — Named Constructors\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#16-summary\"\u003eSummary \u0026amp; Key Takeaways\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#17-practice-problems\"\u003ePractice Problems\u003c/a\u003e\u003c/li\u003e\n\u003c/ol\u003e\n\u003chr\u003e\n\u003ch2 id=\"1-the-world-before-oop\"\u003e1. The World Before OOP\u003c/h2\u003e\n\u003cp\u003eBefore we learn \u003cem\u003ewhat\u003c/em\u003e OOP is, let\u0026rsquo;s understand \u003cem\u003ewhy\u003c/em\u003e humanity invented it. Imagine you\u0026rsquo;ve been hired to build software for a ride-hailing company (think Uber). You start writing code procedurally — just functions and dictionaries.\u003c/p\u003e","tags":["lld","oop","python","low level design","system design"],"title":"Object-Oriented Programming in Python: A Deep Dive"},{"content":"SOLID Principles: A Complete Deep Dive Prerequisite: Working familiarity with Python object-oriented programming, specifically classes, objects, inheritance, method overriding, polymorphism, and abstract base classes. If any of those terms are unfamiliar, study the OOP document first, because every argument in this document is expressed in that vocabulary.\nWhat this document delivers: A rigorous treatment of the five SOLID principles. For each principle you will find its formal statement, a precise interpretation of every term in that statement, the concrete engineering problem it solves, a worked example of the problem and its resolution, the alternative techniques available for the same goal, the costs of applying the principle, and the conditions under which it should not be applied.\nHow to read it: Sequentially. Section 4 defines the vocabulary used throughout. Sections 5 to 9 develop the principles one at a time, each building on terms established earlier. Section 10 explains how the five interact as a system, and Section 11 applies all five to a single realistic codebase.\nTable of Contents How This Document Is Organised The Problem SOLID Solves: The Economics of Change The Five Principles at a Glance Foundational Vocabulary Single Responsibility Principle (SRP) Open/Closed Principle (OCP) Liskov Substitution Principle (LSP) Interface Segregation Principle (ISP) Dependency Inversion Principle (DIP) How the Five Principles Interact Capstone: Refactoring an Order Processing System SOLID Beyond Classes: Functions, Modules, and Services Anti-Patterns and Misapplications When SOLID Should Not Be Applied Summary and Key Takeaways Self-Assessment Questions with Answers Practice Exercises with Solutions Where SOLID Leads Next 1. How This Document Is Organised Each of the five principles is presented using an identical ten-part structure. The uniformity is deliberate: once you internalise the structure, you can navigate any principle quickly and you can compare principles against each other along the same dimensions.\nPart Content Question It Answers 1 Formal statement What exactly was claimed, and by whom 2 Term-by-term interpretation What each word in the statement means precisely 3 The problem it solves Why the principle exists at all 4 Worked violation What the failure looks like in real code 5 Diagnosis How to recognise the failure systematically 6 Resolution How to restructure the code 7 Alternative techniques What other ways exist to reach the same goal 8 Costs and limits What the principle charges you, and when the price is too high 9 Detection heuristics Signals that indicate a violation 10 Checkpoint questions Immediate verification of understanding Every code example is written in Python 3. Abstract base classes are used to make contracts explicit, because explicit contracts are pedagogically clearer than implicit ones, even though Python\u0026rsquo;s duck typing would often permit the abstraction to be omitted. Section 8.7 discusses when the explicit form is worth its cost and when structural typing via typing.Protocol is preferable.\n2. The Problem SOLID Solves: The Economics of Change 2.1 The observation behind the principles A program is written once and modified many times. Industry studies of software cost consistently attribute the majority of total lifetime expenditure to maintenance rather than to initial construction. The practical consequence is that the dominant quality attribute of a codebase is not how quickly it was written but how cheaply it can be changed.\nDefinition (cost of change). The total effort required to implement a modification to a system, including the effort to understand the existing code, to write the modification, to update the affected tests, to verify that unrelated behaviour is unaffected, and to repair defects introduced by the modification.\nSOLID is a set of five structural heuristics whose shared purpose is to reduce the cost of change. Each principle attacks the problem from a different direction, but the target is the same.\n2.2 The four symptoms of degrading design Robert C. Martin catalogued four observable symptoms that indicate a design is decaying. These are the diagnostic vocabulary of the rest of this document.\nSymptom Definition Typical observation Rigidity The difficulty of changing the software, because every change forces a cascade of further changes in dependent modules. A one-line business rule change turns into a three-day task touching fourteen files. Fragility The tendency of the software to break in places that have no conceptual relationship to the part that was changed. Modifying the invoice printer breaks the login flow. Immobility The inability to reuse a component in another system, because extracting it drags along unwanted dependencies. The pricing logic cannot be reused because it is entangled with the database connection and the email client. Viscosity The condition in which preserving the design is harder than violating it, so developers repeatedly choose the design-damaging shortcut. Adding the feature \u0026ldquo;properly\u0026rdquo; needs three new classes, but adding one elif takes thirty seconds. A fifth symptom, opacity, refers to code that is difficult to read and understand. Opacity is not addressed directly by SOLID, but it is usually reduced as a side effect, because each principle pushes toward smaller units with narrower purposes.\n2.3 Coupling and cohesion: the underlying variables Every SOLID principle can be restated as an instruction to lower coupling or raise cohesion. These two properties are therefore worth defining precisely before proceeding.\nDefinition (coupling). The degree of interdependence between two software modules. Module A is coupled to module B if a change to B can require a change to A. Coupling is measured by the number of assumptions A makes about B and by how specific those assumptions are.\nDefinition (cohesion). The degree to which the elements inside a single module belong together. A module is cohesive when its methods and fields all contribute to a single, well-defined purpose, and when removing any one of them would leave that purpose incomplete.\nThe design objective is low coupling and high cohesion. The two are related: when a module contains unrelated responsibilities (low cohesion), it necessarily accumulates dependencies belonging to all of them, which raises its coupling to the rest of the system.\nCoupling comes in degrees, not merely in the binary of present or absent. The following ordering, from most damaging to least, is useful when judging a design.\nForm of coupling Description Example Content coupling One module modifies the internal state of another directly. order._items.append(x) reaching into a private attribute. Common coupling Multiple modules share global mutable state. Several classes read and write a module-level CONFIG dictionary. Control coupling One module passes a flag that dictates the internal control flow of another. report.generate(fmt=\u0026quot;pdf\u0026quot;) where the flag selects an internal branch. Stamp coupling A module receives a composite object but uses only a fragment of it. A function takes an entire User object only to read user.email. Data coupling Modules communicate only through simple parameters that are all used. calculate_tax(amount, rate). Abstraction coupling A module depends only on an interface, not on any concrete implementation. UserService depends on the abstract Repository. The lower two rows are the targets. The Dependency Inversion Principle in Section 9 is precisely a technique for converting the upper rows into the bottom row.\n2.4 Why object-oriented programming alone is not sufficient Object-oriented programming supplies four mechanisms: encapsulation, abstraction, inheritance, and polymorphism. These are mechanisms, not policies. They determine what is expressible, not what is advisable.\nConsider the analogy of a language and its grammar, which is used here because the correspondence is exact rather than decorative. Vocabulary determines which words exist; grammar determines which arrangements of those words produce meaning. A speaker who knows ten thousand words but no grammar produces sequences that are lexically valid and semantically useless. Object-oriented programming is the vocabulary of classes, methods, and inheritance relationships. SOLID is a grammar that constrains how those elements may be arranged so that the resulting structure remains comprehensible and modifiable.\nConcretely, inheritance permits a subclass to override a method in a way that contradicts the parent\u0026rsquo;s guarantees; nothing in the language prevents it. Encapsulation permits a class to hide fifteen unrelated responsibilities behind a clean public interface; the compiler is satisfied. The mechanisms are neutral. SOLID supplies the missing judgement.\n2.5 Historical origin The principles were formulated separately over roughly two decades and later assembled under one acronym.\nPrinciple Originator Year of first clear formulation Open/Closed Bertrand Meyer, in Object-Oriented Software Construction 1988 Liskov Substitution Barbara Liskov, in a keynote address; formalised with Jeannette Wing 1987, formalised 1994 Single Responsibility Robert C. Martin, drawing on earlier work on cohesion by Tom DeMarco and Meilir Page-Jones 1990s Interface Segregation Robert C. Martin, arising from consulting work at Xerox 1990s Dependency Inversion Robert C. Martin 1996 Martin collected the five in his writings during the late 1990s. Michael Feathers proposed the SOLID mnemonic in the early 2000s. The acronym imposes a memorable ordering but not a logical one: the principles were not designed as a sequence and need not be applied in the order S, O, L, I, D.\n2.6 What the principles are, and what they are not SOLID principles are heuristics, meaning rules of thumb that produce good outcomes in the majority of cases within their domain of applicability, but which admit exceptions and require judgement to apply. They are not theorems, not language rules, and not universal laws. A design that violates a SOLID principle is not automatically defective; it is a design that has accepted a specific risk, and the engineer should be able to state why that risk is acceptable.\nSection 14 treats this point at length, because the most common failure mode among developers newly exposed to SOLID is not under-application but over-application.\n3. The Five Principles at a Glance Letter Name Formal essence Primary variable improved Chief mechanism S Single Responsibility Principle A module should have one, and only one, reason to change. Cohesion Separation of responsibilities into distinct classes O Open/Closed Principle Software entities should be open for extension but closed for modification. Rigidity Polymorphic dispatch behind a stable abstraction L Liskov Substitution Principle Subtypes must be substitutable for their base types without altering program correctness. Reliability of polymorphism Contract preservation in subclasses I Interface Segregation Principle Clients should not be forced to depend on methods they do not use. Coupling Decomposition of wide interfaces into narrow, client-specific ones D Dependency Inversion Principle High-level modules and low-level modules should both depend on abstractions. Coupling and testability Inversion of the dependency direction through an interface Read the table once now and again after finishing Section 9. The second reading will carry substantially more information, which is a reasonable test of whether the intervening material has been absorbed.\n4. Foundational Vocabulary The five principles are stated in a compact technical vocabulary. Ambiguity in these terms is the primary cause of misapplication, so each is defined here and used consistently thereafter.\nTerm Definition Module Any cohesive unit of code with a boundary: a function, a class, a package, or a deployable service. SOLID statements that say \u0026ldquo;class\u0026rdquo; apply to all of these; the class is simply the most common granularity. Client The code that calls a module. If OrderService calls PaymentGateway, then OrderService is the client of PaymentGateway. The direction matters: principles such as ISP are stated from the client\u0026rsquo;s point of view. Abstraction A description of what a module does, stated without reference to how it does it. In Python this is realised as an abstract base class, a typing.Protocol, or an informal duck-typed convention. Concretion A specific implementation of an abstraction. MySQLRepository is a concretion of the abstraction Repository. Interface The set of operations a module exposes, together with the guarantees attached to them. Python has no interface keyword; the role is played by abstract base classes and protocols. Contract The complete agreement between a module and its callers: what the caller must guarantee before invocation (preconditions), what the module guarantees on return (postconditions), and what remains true throughout (invariants). Contracts include behaviour, not just method signatures. Dependency A relationship in which module A requires module B in order to compile, import, or execute correctly. The arrow points from the dependent to the dependency. High-level module A module that expresses policy: the business rules and workflows that give the application its purpose. Example: \u0026ldquo;an order must be paid before it is shipped.\u0026rdquo; Low-level module A module that expresses mechanism: the technical details of how an operation is physically performed. Example: opening a TCP connection to a payment provider. Axis of change A dimension along which requirements are expected to vary over time, such as \u0026ldquo;new payment methods will be added\u0026rdquo; or \u0026ldquo;the tax rate will change.\u0026rdquo; Identifying the correct axes is the central skill in applying OCP. Blast radius The set of modules that must be inspected, modified, or retested as a consequence of a single change. Minimising blast radius is the practical objective of SRP and DIP. Polymorphism The ability of a single call site to invoke different implementations depending on the runtime type of the receiving object. This is the language mechanism on which OCP, LSP, and DIP all depend. Composition Building behaviour by holding references to other objects and delegating to them, as opposed to inheriting behaviour from a parent class. Dependency injection The technique of supplying a module\u0026rsquo;s dependencies from outside rather than having the module construct them internally. Discussed in Section 9.7. Two of these deserve immediate elaboration, because they are the most frequently confused.\nInterface versus contract. An interface is a syntactic construct: a list of method names, parameters, and return types. A contract is a semantic one: it additionally specifies what the methods must actually do. A subclass can satisfy an interface perfectly while violating the contract, for instance by implementing withdraw(amount) with the correct signature but permitting the balance to go negative when the parent guaranteed it would not. The Liskov Substitution Principle in Section 7 is concerned with contracts, not interfaces, and this distinction is the entire subject of that section.\nPolicy versus mechanism. The distinction between high-level and low-level modules is not about position in a call stack or about how many lines of code a module contains. It is about whether the module states a rule of the business domain or a technique for accomplishing something. \u0026ldquo;Apply a 10 percent discount to premium customers\u0026rdquo; is policy and would remain true if the company migrated from PostgreSQL to DynamoDB. \u0026ldquo;Execute an INSERT statement against a PostgreSQL connection pool\u0026rdquo; is mechanism and has no meaning in the business domain. Policy is stable and valuable; mechanism is volatile and replaceable. The Dependency Inversion Principle in Section 9 exists to ensure that stable policy never depends on volatile mechanism.\n5. Single Responsibility Principle (SRP) 5.1 Formal statement \u0026ldquo;A class should have one, and only one, reason to change.\u0026rdquo;\nRobert C. Martin\nMartin later restated the principle in a form he considered less prone to misreading:\n\u0026ldquo;A module should be responsible to one, and only one, actor.\u0026rdquo;\nBoth formulations are used below, because each clarifies a different aspect of the same idea.\n5.2 Term-by-term interpretation The principle contains three terms whose ordinary-language meanings are misleading in this context.\n\u0026ldquo;Responsibility.\u0026rdquo; In SRP a responsibility is not a task, a method, or a feature. A responsibility is a reason to change, and a reason to change originates with a person or group who has the authority to request that change. Finance owns the rules for computing pay. Operations owns the rules for scheduling shifts. The database administration team owns the storage schema. Each of these is a distinct source of change requests.\n\u0026ldquo;Reason to change.\u0026rdquo; This phrase is deliberately about causes, not effects. Two pieces of code that happen to change at the same time do not share a reason to change if the underlying causes are unrelated. Conversely, two pieces of code that change at different times can share a reason if both changes trace back to the same decision by the same group.\n\u0026ldquo;One.\u0026rdquo; The principle does not restrict the number of methods, lines, or fields. A class with twenty methods that all serve a single purpose and all change together in response to requests from a single actor satisfies SRP. A class with two methods that answer to two different departments violates it.\nDefinition (Single Responsibility Principle). A module should have exactly one actor, meaning one person or group of stakeholders, whose changing requirements can cause that module to be modified.\n5.3 The problem it solves Consider what happens structurally when a single class answers to three actors. Every request from any of the three actors causes the same file to be edited. Three consequences follow mechanically.\nMerge contention. Three teams edit the same file concurrently. Version control conflicts are frequent and their resolution requires someone who understands all three domains.\nUnintended coupling of independent concerns. A change requested by Finance is deployed together with whatever half-finished work the Reporting team left in the same file. The two concerns share a release, a review, and a rollback unit even though they are logically unrelated. This is the mechanism by which fragility (Section 2.2) arises: the person changing the pay calculation had no reason to suspect the report generator, so they did not test it.\nLoss of independent reusability. Another system needs only the pay calculation. Importing the class drags in the database driver and the PDF renderer, because they live in the same module. This is immobility.\nThe principle addresses all three by ensuring that the unit of code corresponds to the unit of organisational responsibility.\n5.4 Worked violation The following class is representative of code that accumulates naturally when features are added without periodic restructuring.\nclass Employee: \u0026#34;\u0026#34;\u0026#34; Represents an employee AND computes their pay AND persists them AND renders them into reports. The conjunction in that sentence is the defect. \u0026#34;\u0026#34;\u0026#34; def __init__(self, name, hours_worked, hourly_rate): self.name = name self.hours_worked = hours_worked self.hourly_rate = hourly_rate def calculate_pay(self): # Owned by Finance. Overtime rules, tax slabs, and bonus policy # will all eventually land inside this method. return self.hours_worked * self.hourly_rate def save(self): # Owned by the database team. The choice of engine, the schema, # the connection handling, and the retry policy all live here. print(f\u0026#34;INSERT INTO employees VALUES (\u0026#39;{self.name}\u0026#39;, ...)\u0026#34;) def generate_report(self, output_format): # Owned by the Reporting team. Layout, branding, and supported # export formats change on their schedule, not Finance\u0026#39;s. if output_format == \u0026#34;pdf\u0026#34;: print(f\u0026#34;Rendering PDF for {self.name}\u0026#34;) elif output_format == \u0026#34;html\u0026#34;: print(f\u0026#34;Rendering HTML for {self.name}\u0026#34;) 5.5 Diagnosis The diagnostic procedure is mechanical and should be applied literally rather than intuitively.\nStep 1: List every method and ask who requests changes to it.\nMethod Actor who owns the rules Example change request calculate_pay Finance \u0026ldquo;Overtime above 40 hours is paid at 1.5 times the base rate.\u0026rdquo; save Database administration \u0026ldquo;We are migrating from MySQL to PostgreSQL.\u0026rdquo; generate_report Reporting and Compliance \u0026ldquo;Payslips must now be exportable as CSV for the auditor.\u0026rdquo; Step 2: Count the distinct actors. Three distinct actors implies three reasons to change, which implies three responsibilities in one class. The principle is violated.\nStep 3: Verify by constructing a concrete failure scenario. Suppose Finance changes the overtime rule and, while editing the file, a developer alters a shared helper used by generate_report. The payroll test suite passes because it does not exercise reporting. The defect reaches production in a module that no one associated with the change. This scenario is not hypothetical; it is the standard mechanism by which fragility manifests.\nThere is a subtler diagnostic worth internalising. Examine which methods use which fields. In the class above, calculate_pay uses hours_worked and hourly_rate; generate_report uses only name. When the methods of a class partition cleanly into groups that touch disjoint sets of fields, the class is almost certainly implementing multiple responsibilities. This measure is formalised in the software metrics literature as Lack of Cohesion of Methods (LCOM), and while the numeric metric is rarely worth computing by hand, the underlying observation is a reliable signal.\n5.6 Resolution Each responsibility is extracted into a class whose name states its single purpose.\nclass Employee: \u0026#34;\u0026#34;\u0026#34; A pure data holder for employee attributes. Design decision: this class deliberately contains no behaviour beyond holding state. It changes only when the *definition* of an employee changes, for instance when HR introduces an employment-type field. It does not change when pay rules, storage, or report layouts change. \u0026#34;\u0026#34;\u0026#34; def __init__(self, name, hours_worked, hourly_rate): self.name = name self.hours_worked = hours_worked self.hourly_rate = hourly_rate class PayrollCalculator: \u0026#34;\u0026#34;\u0026#34; Owns all monetary computation for employees. Answers to Finance. Overtime rules, tax handling, and bonus policy will be added here as the domain grows, and none of those additions can affect persistence or reporting, because this class shares no code with them. \u0026#34;\u0026#34;\u0026#34; OVERTIME_THRESHOLD_HOURS = 40 OVERTIME_MULTIPLIER = 1.5 def calculate_pay(self, employee): # The overtime split is expressed as explicit arithmetic rather # than a branch on hours, so that both components remain visible # and independently testable when tax rules are layered on later. regular_hours = min(employee.hours_worked, self.OVERTIME_THRESHOLD_HOURS) overtime_hours = max(0, employee.hours_worked - self.OVERTIME_THRESHOLD_HOURS) regular_pay = regular_hours * employee.hourly_rate overtime_pay = overtime_hours * employee.hourly_rate * self.OVERTIME_MULTIPLIER return regular_pay + overtime_pay class EmployeeRepository: \u0026#34;\u0026#34;\u0026#34; Owns persistence of employees. Answers to the database team. The name \u0026#39;Repository\u0026#39; is conventional and signals a specific contract: this class translates between domain objects and storage, and it contains no business rules. A tax calculation appearing in this class would itself be an SRP violation. \u0026#34;\u0026#34;\u0026#34; def save(self, employee): print(f\u0026#34;Persisting {employee.name}\u0026#34;) def find_by_name(self, name): print(f\u0026#34;Loading {name}\u0026#34;) class PayslipRenderer: \u0026#34;\u0026#34;\u0026#34; Owns presentation of employee pay. Answers to Reporting. Note that this class receives the already-computed pay rather than an Employee, so that it has no opportunity to recompute or reinterpret monetary values. Keeping the calculation out of the renderer is what prevents the two responsibilities from silently merging again. \u0026#34;\u0026#34;\u0026#34; def render(self, employee, net_pay, output_format): if output_format == \u0026#34;pdf\u0026#34;: print(f\u0026#34;PDF payslip for {employee.name}: {net_pay}\u0026#34;) elif output_format == \u0026#34;html\u0026#34;: print(f\u0026#34;HTML payslip for {employee.name}: {net_pay}\u0026#34;) The coordination that previously lived inside Employee now lives at the call site or in an explicit orchestrator.\nemployee = Employee(\u0026#34;Ananya\u0026#34;, hours_worked=45, hourly_rate=500) calculator = PayrollCalculator() repository = EmployeeRepository() renderer = PayslipRenderer() net_pay = calculator.calculate_pay(employee) repository.save(employee) renderer.render(employee, net_pay, output_format=\u0026#34;pdf\u0026#34;) The effect on blast radius is the point of the exercise and can be tabulated.\nChange request Before refactoring After refactoring New overtime rule Employee edited; payroll, persistence, and reporting tests all re-run PayrollCalculator edited only Migrate to PostgreSQL Employee edited; all tests re-run EmployeeRepository edited only Add CSV payslip export Employee edited; all tests re-run PayslipRenderer edited only Add an employee_id field Employee edited Employee edited, plus repository mapping The final row is instructive. Adding a field genuinely does touch two classes, because storage must know about the new field. SRP does not promise that every change touches exactly one file; it promises that each file changes for exactly one reason. The repository changes because storage rules changed, which is its own reason.\n5.7 Alternative techniques for the same goal Extracting classes is the standard resolution, but it is not the only one, and choosing it reflexively is itself a mistake.\nModule-level separation without new classes. In a small program, moving calculate_pay, save, and render into three separate modules as plain functions achieves the same separation of change reasons at lower ceremony. Python\u0026rsquo;s module system is a first-class unit of encapsulation, and a function in payroll.py is separated from a function in storage.py just as effectively as two classes would be. Classes earn their place when the behaviour needs configuration, state, or polymorphic substitution.\nThe Facade pattern to preserve caller convenience. Splitting a class multiplies the number of objects a caller must assemble. When the original single-call convenience matters, a thin facade restores it without recombining the responsibilities.\nclass PayrollService: \u0026#34;\u0026#34;\u0026#34; A facade that orchestrates the specialised collaborators. This class has exactly one reason to change: the *sequence* of steps in the payroll workflow. It contains no pay arithmetic, no SQL, and no formatting, so a change to any of those does not reach it. \u0026#34;\u0026#34;\u0026#34; def __init__(self, calculator, repository, renderer): self.calculator = calculator self.repository = repository self.renderer = renderer def process(self, employee, output_format=\u0026#34;pdf\u0026#34;): net_pay = self.calculator.calculate_pay(employee) self.repository.save(employee) self.renderer.render(employee, net_pay, output_format) return net_pay The facade is worth studying because it demonstrates that \u0026ldquo;orchestrating a workflow\u0026rdquo; is itself a legitimate single responsibility. The workflow changes when the business changes the order or composition of steps, which is a genuine and distinct reason to change.\nDeliberate non-separation for value objects. A class such as Money or Date may legitimately contain arithmetic, comparison, parsing, and formatting. These appear to be several responsibilities but are all owned by the same actor, namely whoever defines what money or a date means in the domain, and they all change together. Splitting them would produce anaemic fragments and increase, not decrease, the cost of change.\nThe following table summarises the decision.\nSituation Recommended approach Reason Methods answer to different departments Extract separate classes Distinct actors, distinct release cadences Small script, no state, no substitution needed Separate modules with plain functions Same separation, lower ceremony Callers need a single convenient entry point Extract classes and add a facade Preserves separation and convenience simultaneously Cohesive value object with many methods Leave as one class One actor, methods change together 5.8 Costs and limits SRP is not free, and the costs are the reason it must be applied with judgement.\nIncreased indirection. After splitting, understanding the payroll flow requires reading four files instead of one. For a reader unfamiliar with the codebase this is a genuine comprehension cost. The cost is worth paying when the classes change independently; it is pure overhead when they do not.\nAnaemic domain models. Aggressively stripping behaviour from entities produces classes that are little more than dictionaries with attribute access, while all logic accumulates in service classes. This is a recognised anti-pattern, because it discards the principal benefit of object orientation, namely the co-location of data with the operations that maintain its invariants. Behaviour that enforces an entity\u0026rsquo;s own invariants, such as Order.add_item rejecting a negative quantity, belongs on the entity. Behaviour that coordinates across entities or reaches outside the process belongs elsewhere.\nPremature decomposition. Splitting a class before the axes of change are known frequently produces the wrong split, and an incorrect boundary is more expensive than no boundary, because it must first be dismantled before the correct one can be created.\n5.9 Detection heuristics Signal Why it indicates an SRP violation The class name contains \u0026ldquo;and\u0026rdquo;, or is vague (\u0026ldquo;Manager\u0026rdquo;, \u0026ldquo;Processor\u0026rdquo;, \u0026ldquo;Helper\u0026rdquo;, \u0026ldquo;Util\u0026rdquo;) A name that cannot be stated precisely usually reflects a purpose that is not singular. Methods partition into groups that use disjoint sets of fields Low cohesion; the groups are separate concepts sharing a file by accident. The class contains both business rules and input/output operations Business rules change with the domain; input/output changes with infrastructure. These are always distinct actors. Version control history shows commits from several teams for unrelated reasons The most reliable empirical signal, because it measures actual change reasons rather than predicted ones. Unit testing the class requires mocking a database, a network client, or a clock Direct evidence that non-domain responsibilities have leaked in. The file appears near the top of a \u0026ldquo;most frequently modified files\u0026rdquo; report High change frequency concentrated in one file usually means many reasons to change converge there. The fourth and sixth signals deserve emphasis because they are empirical rather than speculative. Running git log --format=format: --name-only | sort | uniq -c | sort -rg | head -20 over a repository lists the files that change most often. Those files are the practical candidates for SRP analysis, since they are where the cost of poor separation is actually being paid.\n5.10 Checkpoint questions Q1. A ShoppingCart class exposes add_item, remove_item, update_quantity, apply_coupon, subtotal, and clear. Six public methods. Does this violate SRP?\nAnswer Not on the evidence given. All six methods operate on the same state, namely the collection of cart lines, and all six change for the same reason, namely a change to what a cart is or how cart contents are manipulated. Method count is irrelevant to SRP. The one method worth examining further is apply_coupon: if coupon eligibility involves promotional rules owned by a marketing team, that logic constitutes a second actor and should be extracted into a separate pricing or promotion component, leaving the cart to merely record the resulting adjustment.\nQ2. A developer splits a 300-line ReportGenerator into ReportGeneratorPartOne and ReportGeneratorPartTwo, each 150 lines. Has SRP been applied?\nAnswer No. SRP concerns reasons to change, not file size. Both halves still change for the same reasons and must be modified together, so the single responsibility has merely been distributed across two files. The result is worse than the original, because a reader must now hold two files in mind to understand one concept, with no compensating reduction in blast radius. A correct split would divide the class along actor boundaries, for example separating data aggregation, which changes when the business defines new metrics, from layout, which changes when the presentation requirements change.\nQ3. Two classes, InvoiceCalculator and TaxCalculator, are always modified in the same commit. Does this prove an SRP violation?\nAnswer It is a signal warranting investigation, not proof. If both change because the same tax authority issued a new rule, they share a reason to change and should probably be one module. If they change together merely because the team happens to schedule finance work in batches, the underlying reasons remain distinct and the separation is correct. The test is causal, not temporal: identify the originating decision, not the calendar date.\n6. Open/Closed Principle (OCP) 6.1 Formal statement \u0026ldquo;Software entities (classes, modules, functions) should be open for extension, but closed for modification.\u0026rdquo;\nBertrand Meyer, Object-Oriented Software Construction, 1988\n6.2 Term-by-term interpretation \u0026ldquo;Open for extension.\u0026rdquo; The behaviour of the module can be augmented. New cases, new variants, and new capabilities can be introduced.\n\u0026ldquo;Closed for modification.\u0026rdquo; The augmentation is achieved without editing the module\u0026rsquo;s existing source code. Its file remains untouched, its compiled artefact remains valid, and its existing tests remain relevant without revision.\nThe two clauses appear contradictory, since adding behaviour to something usually means changing it. The contradiction dissolves once a level of indirection is introduced. The module is written against an abstraction; new behaviour arrives as a new implementation of that abstraction; the module never learns that the new implementation exists. Polymorphic dispatch is the mechanism that makes the apparent contradiction tractable, which is why OCP cannot be discussed independently of polymorphism.\nDefinition (Open/Closed Principle). A module satisfies OCP with respect to a particular axis of change if new variations along that axis can be added by writing new code rather than by editing existing code.\nThe qualifier \u0026ldquo;with respect to a particular axis of change\u0026rdquo; is essential and is omitted in most informal presentations. No module can be closed against every conceivable change. A discount system may be closed against the addition of new discount types while remaining wide open to a change in the signature of the discount calculation itself. Meyer\u0026rsquo;s principle is therefore a directive to identify the axes along which change is likely and to arrange the design so that those specific axes require extension rather than modification.\n6.3 The problem it solves Modifying working code carries a category of risk that adding new code does not.\nRisk of modifying existing code Explanation Regression Existing behaviour may be altered inadvertently. Code that was previously verified must be re-verified. Test invalidation Tests covering the modified unit must be reviewed, and often rewritten, even though they concern behaviour that has not changed. Ripple through dependents Every module that depends on the modified one may be affected, expanding the review and retest surface. Deployment coupling The new feature and all pre-existing behaviour in the same unit are released together and must be rolled back together. Adding a new class that no existing code references carries none of these risks in principle. The existing code cannot behave differently, because it has not changed. Verification of the new class is confined to the new class. This asymmetry between modification and addition is the entire economic argument for OCP.\n6.4 Worked violation class DiscountCalculator: \u0026#34;\u0026#34;\u0026#34;Computes a discount based on the customer\u0026#39;s category.\u0026#34;\u0026#34;\u0026#34; def calculate(self, customer_type, amount): if customer_type == \u0026#34;regular\u0026#34;: return amount * 0.05 elif customer_type == \u0026#34;premium\u0026#34;: return amount * 0.10 elif customer_type == \u0026#34;vip\u0026#34;: return amount * 0.20 else: return 0 The requirement history of such a class is predictable. A corporate tier is introduced at 15 percent, so an elif is added. A student tier follows at 8 percent. A first-purchase promotion is introduced, which is not a customer type at all but a condition, so a boolean parameter is threaded through the signature. A regional festival discount is added, requiring a date parameter. Within a year the method is several hundred lines long, its parameter list encodes four orthogonal concerns, and no developer can modify one branch with confidence that the others are unaffected.\nObserve precisely which property has been lost. Each new customer type requires editing a file that already contains working, tested logic for every previous customer type. The blast radius of adding a customer type is the entire discount subsystem.\n6.5 Diagnosis The conditional-on-type test. Locate every conditional that branches on a type, a category name, an enumeration value, or a string tag. Each such conditional is a point at which the addition of a new value forces modification of existing code. This is the single most reliable indicator of an OCP violation.\nThe shotgun surgery test. Trace what a realistic new requirement would touch. If adding one conceptual thing, such as a customer tier, requires edits at several locations, and especially if it requires edits in several files that must be kept mutually consistent, the design is not closed along that axis. The failure mode where a developer updates three of the four required locations and ships a partially implemented tier is the direct consequence.\nDistinguishing genuine violations from acceptable conditionals. Not every conditional is an OCP violation. A conditional is a violation when it branches on a set of values that is expected to grow. A conditional that branches on a genuinely fixed set, such as the three possible outcomes of a comparison, is stable and should not be replaced with polymorphism.\n6.6 Resolution The resolution introduces an abstraction that represents the varying concept, then supplies one implementation per variant.\nfrom abc import ABC, abstractmethod class DiscountPolicy(ABC): \u0026#34;\u0026#34;\u0026#34; The stable abstraction. Everything that varies between customer tiers is pushed behind this single method, and everything that does not vary stays in the calling code. Naming note: \u0026#39;Policy\u0026#39; rather than \u0026#39;Strategy\u0026#39; is used here because the concept is a business rule. The structural pattern is identical to the Strategy pattern, which is examined in the behavioural patterns document; OCP is the principle, Strategy is one pattern that realises it. \u0026#34;\u0026#34;\u0026#34; @abstractmethod def discount_for(self, amount: float) -\u0026gt; float: \u0026#34;\u0026#34;\u0026#34; Return the absolute discount amount, not the discounted total. Contract note: returning the discount rather than the final price means the caller retains control over how the discount is applied, combined with other discounts, or reported. Returning the final price would have embedded that decision inside every policy and made discount stacking impossible to implement later. \u0026#34;\u0026#34;\u0026#34; ... class RegularCustomerDiscount(DiscountPolicy): def discount_for(self, amount): return amount * 0.05 class PremiumCustomerDiscount(DiscountPolicy): def discount_for(self, amount): return amount * 0.10 class VIPCustomerDiscount(DiscountPolicy): def discount_for(self, amount): return amount * 0.20 class NoDiscount(DiscountPolicy): \u0026#34;\u0026#34;\u0026#34; The Null Object variant. Its existence removes the need for callers to check whether a policy is present, which eliminates a null check at every call site and prevents the reappearance of conditionals in the client code. \u0026#34;\u0026#34;\u0026#34; def discount_for(self, amount): return 0.0 The consuming code is written once against the abstraction and is then never modified again on this axis.\nclass Checkout: def final_price(self, amount: float, policy: DiscountPolicy) -\u0026gt; float: # This method contains no knowledge of which policies exist. Adding # a hundred new policies leaves this line untouched, which is the # operational meaning of \u0026#34;closed for modification\u0026#34;. return amount - policy.discount_for(amount) Introducing a corporate tier is now purely additive.\nclass CorporateDiscount(DiscountPolicy): \u0026#34;\u0026#34;\u0026#34; Added later. No existing file was edited to introduce this class, so no existing test could have been invalidated by its addition. \u0026#34;\u0026#34;\u0026#34; def discount_for(self, amount): return amount * 0.15 6.7 Alternative techniques for the same goal Polymorphic classes are the canonical realisation of OCP, but several alternatives achieve extensibility at different cost and expressiveness points. Selecting among them is a genuine engineering decision.\nAlternative 1: dictionary dispatch. The mapping from key to behaviour is stored in a data structure rather than encoded in control flow.\nDISCOUNT_RATES = { \u0026#34;regular\u0026#34;: 0.05, \u0026#34;premium\u0026#34;: 0.10, \u0026#34;vip\u0026#34;: 0.20, } def discount_for(customer_type: str, amount: float) -\u0026gt; float: # Adding a tier means adding a dictionary entry. The function is # untouched, so it is closed for modification along this axis. return amount * DISCOUNT_RATES.get(customer_type, 0.0) This is appropriate when every variant differs only by a value. It becomes unsuitable the moment a variant needs behaviour rather than a constant, for example a tier whose discount is capped at a maximum amount or depends on order history.\nAlternative 2: first-class functions. Python treats functions as objects, so a policy can be a function rather than a class instance.\ndef regular_discount(amount): return amount * 0.05 def volume_discount(amount): # Behaviour, not merely a constant, so the dictionary-of-rates approach # would not have accommodated this variant. return amount * 0.15 if amount \u0026gt; 100_000 else amount * 0.05 def final_price(amount, discount_fn): return amount - discount_fn(amount) The trade-off is explicit. Functions are lighter, require no class declaration, and are trivially testable. They lose the formal contract that an abstract base class enforces, they cannot hold configured state without closures, and they cannot be extended with additional operations later without changing every call site. Functions are preferable when the abstraction has exactly one operation and no state; classes are preferable otherwise.\nAlternative 3: registry with self-registration. The set of implementations is discovered rather than enumerated, which closes the composition root as well as the consuming code.\nclass DiscountRegistry: \u0026#34;\u0026#34;\u0026#34; Maps a tier name to a policy instance. Without a registry, some module must contain a mapping from string to class, and that mapping is itself a place that must be edited whenever a tier is added. The registry moves that edit into the new class\u0026#39;s own file, so the addition remains confined to a single new file. \u0026#34;\u0026#34;\u0026#34; _policies: dict = {} @classmethod def register(cls, tier_name): def decorator(policy_class): cls._policies[tier_name] = policy_class() return policy_class return decorator @classmethod def get(cls, tier_name) -\u0026gt; DiscountPolicy: return cls._policies.get(tier_name, NoDiscount()) @DiscountRegistry.register(\u0026#34;corporate\u0026#34;) class CorporateDiscountV2(DiscountPolicy): def discount_for(self, amount): return amount * 0.15 The cost of a registry is reduced traceability: static analysis and simple text search no longer reveal where a policy is used, because the connection is made at runtime. Registries are justified in plugin architectures and frameworks, where the set of implementations is genuinely open-ended and may be supplied by third parties. They are over-engineering for a fixed internal set of five tiers.\nAlternative 4: accepting the conditional. When the set of variants is small, stable, and unlikely to grow, an if statement is the correct design. It is shorter, immediately readable, and imposes no indirection. The engineering error is not the use of a conditional; it is the use of a conditional along an axis that is known to be growing.\nThe following table summarises the selection criteria.\nTechnique Use when Principal cost Conditional (if/elif) Variants are few and the set is stable Every new variant modifies existing, tested code Dictionary of values Variants differ only by data Cannot express variant-specific behaviour First-class functions Abstraction has one operation and no state No enforced contract; hard to extend the abstraction later Polymorphic classes Variants have behaviour, possibly state, and the set will grow More files, more indirection, more ceremony Registry The implementation set is open-ended or plugin-supplied Runtime wiring reduces traceability and complicates debugging 6.8 Costs and limits Every abstraction is a bet on a predicted axis of change. If the prediction is correct, the abstraction pays for itself the first time a variant is added. If the prediction is wrong, the codebase carries permanent indirection that serves nothing and obstructs the change that actually arrives. Predicting the wrong axis is a common and expensive error, which is why the widely quoted guideline is to write the conditional first and introduce the abstraction when the second or third variant appears. By then the axis is established by evidence rather than by speculation.\nClosing one axis usually opens another. In the discount example, the design is now closed against new tiers but open to modification if the discount_for signature must change, since every implementation would then require editing. Abstractions redistribute the cost of change; they do not eliminate it. Choosing which axis to protect is therefore a decision about which changes are most probable.\nIndirection has a comprehension cost. Tracing a call through an abstract base class to an unknown implementation is harder than reading a conditional. In a codebase with dozens of unnecessary abstractions, this cost accumulates into the opacity described in Section 2.2.\n6.9 Detection heuristics Signal Why it indicates an OCP violation A conditional branching on a type name, category, or enumeration Each new value forces modification of existing code. Use of isinstance to select behaviour An explicit substitute for polymorphism, and it must be edited when a new subtype appears. A comment such as \u0026ldquo;add new types here as well\u0026rdquo; Direct evidence that the design requires coordinated edits at multiple sites. The same elif chain replicated in several files The strongest form of the violation: adding a variant requires several consistent edits, and inconsistency between them is a defect. A parameter list that has accumulated flags over time Each flag was almost certainly added to accommodate a variant that should have been a separate implementation. 6.10 Checkpoint questions Q1. A function branches on three cases: less than, equal to, and greater than. Does this violate OCP?\nAnswer No. The set of possible orderings between two comparable values is mathematically closed at three; no fourth case can ever be added. OCP protects against growth along an axis, and this axis cannot grow. Replacing this conditional with polymorphism would add indirection with no possibility of future benefit.\nQ2. After refactoring to polymorphic discount policies, some code must still decide which policy object to construct for a given customer. Has the conditional simply moved rather than disappeared?\nAnswer The conditional has moved, and that is precisely the intended outcome rather than a failure of the refactoring. The decision about which implementation to use is now isolated at a single point, conventionally called the composition root or a factory, and it is the only place that must change when a tier is added. Previously the tier knowledge was mixed into the calculation logic, so a tier addition and a calculation change occupied the same file and the same tests. Concentrating unavoidable variability at one deliberate location, rather than diffusing it through business logic, is the achievable form of OCP. Section 6.7 shows how a registry can shrink even that remaining edit to zero.\nQ3. A codebase has one discount type and the product manager states no others are planned. Should the polymorphic design be applied?\nAnswer No. With a single variant, the abstraction has no variation to absorb and constitutes speculative generality. The appropriate action is to write the direct implementation and introduce the abstraction when a second variant appears, at which point the refactoring is mechanical and the axis of change has been demonstrated by evidence rather than assumed.\n7. Liskov Substitution Principle (LSP) 7.1 Formal statement \u0026ldquo;Let φ(x) be a property provable about objects x of type T. Then φ(y) should be true for objects y of type S where S is a subtype of T.\u0026rdquo;\nBarbara Liskov and Jeannette Wing, A Behavioral Notion of Subtyping, 1994\nThe engineering restatement, which is the form used in practice:\nCode written against a base type must continue to behave correctly when given any subtype, without knowing that it received a subtype.\n7.2 Term-by-term interpretation \u0026ldquo;Property provable about objects of type T.\u0026rdquo; This includes every guarantee a caller may legitimately rely upon: the range of accepted inputs, the guarantees on returned values, the exceptions that may be raised, the invariants maintained across the object\u0026rsquo;s lifetime, and the observable effects of calling methods in a particular order. It does not include implementation details such as which algorithm is used internally.\n\u0026ldquo;Subtype.\u0026rdquo; Liskov\u0026rsquo;s notion of subtype is behavioural, not syntactic. Declaring class Square(Rectangle) establishes a syntactic subtype relationship in Python. Whether Square is a behavioural subtype of Rectangle is a separate question decided entirely by whether the contract is preserved. LSP asserts that the syntactic relationship should be created only when the behavioural relationship holds.\nDefinition (behavioural subtyping). Type S is a behavioural subtype of type T if every program that is correct when using instances of T remains correct when those instances are replaced by instances of S.\nDefinition (Liskov Substitution Principle). Inheritance should be used only when the derived type is a behavioural subtype of the base type. Where the behavioural relationship does not hold, the inheritance relationship must not be declared.\n7.3 The rules that constitute the contract Liskov and Wing decomposed behavioural subtyping into a small set of verifiable conditions. These are the operational form of the principle and the checklist that should be applied to any proposed override.\nRule Statement Practical meaning Preconditions cannot be strengthened The subtype must accept at least every input the base type accepts. If the base accepts any integer, the subtype cannot reject negative ones. Postconditions cannot be weakened The subtype must guarantee at least everything the base type guarantees. If the base returns a sorted list, the subtype cannot return an unsorted one. Invariants must be preserved Every condition the base type maintains across its lifetime must continue to hold. If the base guarantees a non-negative balance, the subtype cannot permit a negative one. History constraint The subtype must not permit state transitions the base type forbids. If the base type is immutable after construction, the subtype cannot introduce a mutator. Exception constraint The subtype must not raise exception types the base type\u0026rsquo;s contract does not permit. A subtype cannot raise NotImplementedError for an operation the base declares as supported. Return type covariance The subtype may return a more specific type than the base, but never a more general one. Returning Sparrow where Bird is promised is acceptable; returning object is not. Parameter type contravariance The subtype may accept a more general parameter type than the base, but never a more specific one. Accepting Iterable where list was promised is acceptable; accepting only SortedList is not. The last two rules follow directly from the first two. A narrower parameter type is a strengthened precondition; a broader return type is a weakened postcondition. They are listed separately because type checkers can verify them mechanically while the behavioural rules generally cannot be verified automatically.\nA memory device that captures the asymmetry: a subtype may demand less and deliver more, never the reverse.\n7.4 Worked violation 1: the Square and Rectangle problem This example is canonical because it demonstrates that a relationship which is true in the problem domain can be false in the behavioural domain.\nclass Rectangle: \u0026#34;\u0026#34;\u0026#34; Contract: width and height are independently mutable. Setting one has no effect on the other. This guarantee is not written in any method signature, yet every caller depends on it. \u0026#34;\u0026#34;\u0026#34; def __init__(self, width, height): self._width = width self._height = height def set_width(self, width): self._width = width def set_height(self, height): self._height = height def area(self): return self._width * self._height class Square(Rectangle): \u0026#34;\u0026#34;\u0026#34; Mathematically a square is a rectangle, so inheritance appears justified. The invariant width == height must be maintained, which forces each setter to modify both dimensions. \u0026#34;\u0026#34;\u0026#34; def __init__(self, side): super().__init__(side, side) def set_width(self, width): self._width = width self._height = width # Silently violates the independence guarantee def set_height(self, height): self._width = height self._height = height The violation becomes observable in any client written against the base contract.\ndef resize_and_verify(rectangle: Rectangle): \u0026#34;\u0026#34;\u0026#34; A perfectly reasonable function written against Rectangle\u0026#39;s contract. It relies only on the independence of width and height, which the base class guarantees. \u0026#34;\u0026#34;\u0026#34; rectangle.set_width(10) rectangle.set_height(5) assert rectangle.area() == 50, f\u0026#34;Expected 50, computed {rectangle.area()}\u0026#34; resize_and_verify(Rectangle(1, 1)) # Passes: area is 50 resize_and_verify(Square(1)) # Fails: area is 25 The failure is not in Square considered in isolation, which is internally consistent, nor in resize_and_verify, which uses only documented behaviour. The failure is in the declaration that Square is a subtype of Rectangle. Square strengthens an invariant, namely that the two dimensions are always equal, and any strengthened invariant on a mutable object is visible to clients as weakened postconditions on the setters.\nA widespread misdiagnosis holds that the problem is mathematical. It is not. Geometry is untouched. The problem is that Rectangle was defined as a mutable type with independent dimensions, and that specific contract admits no square. The following demonstrates the point by changing the contract rather than the geometry.\nclass ImmutableRectangle: \u0026#34;\u0026#34;\u0026#34; With no setters, the independence guarantee has no observable consequence, because dimensions can never be changed after construction. The contract that Square violated no longer exists. \u0026#34;\u0026#34;\u0026#34; def __init__(self, width, height): self._width = width self._height = height @property def width(self): return self._width @property def height(self): return self._height def area(self): return self._width * self._height def with_width(self, width): # Returns a new object rather than mutating this one. A subtype # can legitimately return a different concrete type here as long # as it remains an ImmutableRectangle. return ImmutableRectangle(width, self._height) class ImmutableSquare(ImmutableRectangle): \u0026#34;\u0026#34;\u0026#34; A legitimate behavioural subtype. Every guarantee ImmutableRectangle makes is upheld, because there is no mutation through which the equality invariant could surprise a caller. \u0026#34;\u0026#34;\u0026#34; def __init__(self, side): super().__init__(side, side) The lesson generalises well beyond this example: mutability is what converts a benign specialisation into an LSP violation, because mutation is the mechanism by which a subtype\u0026rsquo;s stricter invariant becomes observable to a client that did not expect it.\n7.5 Resolution options for the Square and Rectangle problem Three resolutions are available, and the choice among them is determined by how the types are actually used.\nOption 1: eliminate the inheritance relationship. Model both as independent implementations of a shared abstraction that promises only what both can deliver.\nfrom abc import ABC, abstractmethod class Shape(ABC): \u0026#34;\u0026#34;\u0026#34; The abstraction promises only area computation, which both shapes can honour unconditionally. Nothing about mutable dimensions is promised, so nothing about mutable dimensions can be violated. \u0026#34;\u0026#34;\u0026#34; @abstractmethod def area(self) -\u0026gt; float: ... class Rectangle(Shape): def __init__(self, width, height): self.width = width self.height = height def area(self): return self.width * self.height class Square(Shape): def __init__(self, side): self.side = side def area(self): return self.side ** 2 Option 2: make the base type immutable, as shown in Section 7.4. Appropriate when the domain naturally treats shapes as values rather than as entities with identity and mutable state.\nOption 3: weaken the base contract explicitly. If Rectangle documents that setting width may adjust height in some subtypes, then Square no longer violates anything. This resolution is technically valid and almost always undesirable, because the base contract has become so weak that clients can rely on nothing, and the value of the abstraction collapses.\nOption When appropriate Consequence Remove inheritance, share an abstraction The types share operations but not mutation semantics Cleanest and most common resolution Make the base immutable The domain treats these as value objects Removes the class of violation entirely Weaken the base contract Rarely appropriate Preserves the hierarchy at the cost of a useless abstraction 7.6 Worked violation 2: the unsupported operation The second canonical violation occurs when a subtype cannot perform an operation the base type declares.\nclass Bird: def fly(self): print(\u0026#34;Flying\u0026#34;) class Penguin(Bird): def fly(self): # Violates the exception constraint. The base contract states that # fly() succeeds; this subtype converts a normal call into a crash. raise NotImplementedError(\u0026#34;Penguins cannot fly\u0026#34;) Any generic loop over birds now fails on a subset of its inputs, which forces callers to reintroduce type checks and thereby destroys the benefit of polymorphism.\nThe correct resolution is to restructure the hierarchy so that each abstraction promises only what all of its subtypes can deliver.\nfrom abc import ABC, abstractmethod class Bird(ABC): \u0026#34;\u0026#34;\u0026#34; The abstraction is narrowed to locomotion in general, which every bird supports. Flight is not part of the base contract because it is not universal among birds. \u0026#34;\u0026#34;\u0026#34; @abstractmethod def move(self) -\u0026gt; None: ... class FlyingBird(Bird): def move(self): self.fly() def fly(self): print(\u0026#34;Flying\u0026#34;) class FlightlessBird(Bird): def move(self): print(\u0026#34;Walking\u0026#34;) class Sparrow(FlyingBird): pass class Penguin(FlightlessBird): def move(self): print(\u0026#34;Swimming\u0026#34;) # Legitimate override: still locomotion, contract intact Code that requires flight now declares FlyingBird in its parameter type, and the type system prevents a penguin from reaching it. Code that requires only locomotion declares Bird and accepts every subtype safely.\nThe general principle extracted from this example: an abstraction must promise only the intersection of what all its implementations can deliver, never the union. Promising the union guarantees that some implementation will have to break the contract.\n7.7 Violations that are not violations Several patterns superficially resemble LSP violations but do not violate the principle. Recognising them prevents unnecessary refactoring.\nAdding new methods. A subtype may introduce operations the base type does not declare. Existing clients cannot call methods they do not know about, so no existing guarantee is affected.\nChanging the implementation while preserving guarantees. A subtype may sort using a different algorithm, cache results, log calls, or open a network connection. As long as the observable contract holds, the substitution is safe. This is exactly why the Decorator and Proxy patterns are legitimate.\nReturning a more specific type. Covariant returns strengthen the postcondition and are permitted.\nAccepting a broader parameter type. Contravariant parameters weaken the precondition and are permitted.\nThe following pair illustrates the boundary precisely.\nclass FileReader: def read(self, path: str) -\u0026gt; str: with open(path) as handle: return handle.read() class LoggingFileReader(FileReader): \u0026#34;\u0026#34;\u0026#34; Does NOT violate LSP. A side effect is added, but every path the parent accepts is still accepted, and the returned content is unchanged. \u0026#34;\u0026#34;\u0026#34; def read(self, path): print(f\u0026#34;Reading {path}\u0026#34;) return super().read(path) class TextOnlyFileReader(FileReader): \u0026#34;\u0026#34;\u0026#34; VIOLATES LSP. The precondition has been strengthened from \u0026#34;any readable path\u0026#34; to \u0026#34;any readable path ending in .txt\u0026#34;. A client that worked with the parent will crash when handed this subtype and a JSON file. \u0026#34;\u0026#34;\u0026#34; def read(self, path): if not path.endswith(\u0026#34;.txt\u0026#34;): raise ValueError(\u0026#34;Only .txt files are supported\u0026#34;) return super().read(path) 7.8 LSP in Python specifically Python\u0026rsquo;s dynamic typing changes how violations are detected, not whether they matter.\nNo compile-time enforcement. Python performs no static verification of overrides by default. A subclass may change a method\u0026rsquo;s signature entirely and the interpreter will accept it. Violations therefore surface at runtime, typically in production, and typically in code far from the subclass that caused them.\nStatic type checkers verify the mechanical rules. Tools such as mypy verify parameter contravariance and return covariance when type annotations are present. They cannot verify behavioural rules such as invariant preservation, because those are semantic rather than syntactic. Type annotations therefore catch a useful subset of LSP violations at low cost, and behavioural violations remain the responsibility of design review and tests.\nDuck typing narrows the scope of the problem. When a function accepts any object exposing a read method, no inheritance relationship is declared and the base contract is defined implicitly by what the function actually requires. This does not eliminate LSP concerns; it relocates them. The implicit contract still exists, and an object that exposes read but returns an integer instead of a string still breaks callers. typing.Protocol makes such implicit contracts explicit without requiring inheritance, and is discussed in Section 8.7.\nContract tests are the practical enforcement mechanism. Because Python cannot verify behavioural contracts, the standard technique is to write one test suite against the abstraction and run it against every implementation.\nimport pytest class DiscountPolicyContract: \u0026#34;\u0026#34;\u0026#34; A reusable contract test. Every concrete policy must pass every test defined here, which turns the informal contract into an executable one. Design note: this class is deliberately not named with a Test prefix, so the test runner does not attempt to execute it directly. Subclasses supply the implementation under test and inherit the assertions. \u0026#34;\u0026#34;\u0026#34; def create_policy(self): raise NotImplementedError(\u0026#34;Subclasses supply the implementation\u0026#34;) def test_discount_is_never_negative(self): policy = self.create_policy() assert policy.discount_for(1000) \u0026gt;= 0 def test_discount_never_exceeds_the_amount(self): policy = self.create_policy() assert policy.discount_for(1000) \u0026lt;= 1000 def test_zero_amount_yields_zero_discount(self): policy = self.create_policy() assert policy.discount_for(0) == 0 class TestVIPDiscount(DiscountPolicyContract): def create_policy(self): return VIPCustomerDiscount() class TestCorporateDiscount(DiscountPolicyContract): def create_policy(self): return CorporateDiscount() Any future implementation that violates the shared contract fails the inherited tests immediately, at the point where the violation was introduced rather than at the point where a client happens to encounter it.\n7.9 Costs and limits Contracts must be documented to be enforceable. LSP presupposes that the base type\u0026rsquo;s guarantees are known. In practice, many base classes carry undocumented guarantees that clients discovered empirically. A subtype author cannot preserve a contract that was never stated, so writing contracts down is a precondition for applying the principle at all.\nStrict application discourages inheritance broadly. Applied rigorously, LSP eliminates most inheritance hierarchies, because genuine behavioural subtyping is rarer than it appears. This is generally a benefit rather than a cost. Composition and delegation avoid the entire class of problems, which is the basis of the long-standing guidance to prefer composition over inheritance.\nSome violations are economically acceptable. A subtype that violates a rarely exercised guarantee in an internal tool may be cheaper to tolerate than to restructure. The requirement is that the violation be identified, documented, and consciously accepted, rather than discovered later as a production defect.\n7.10 Detection heuristics Signal Why it indicates an LSP violation An override raises NotImplementedError or UnsupportedOperationException Direct violation of the exception constraint; the subtype cannot honour a declared operation. An override begins with a validation that the parent does not perform Strengthened precondition. Client code uses isinstance to handle particular subtypes differently Empirical proof that substitution is not actually safe, since clients have been forced to compensate. An override returns None where the parent returns a value Weakened postcondition. An override silently ignores an argument the parent honours Weakened postcondition, and particularly dangerous because it fails silently. A subclass documents \u0026ldquo;do not use method X on this class\u0026rdquo; The contract has been broken and the breakage delegated to documentation. A subclass overrides a method to do nothing Usually indicates the operation does not belong in the base abstraction. 7.11 Checkpoint questions Q1. A CachingRepository subclasses Repository and returns cached results rather than querying storage on every call. Does this violate LSP?\nAnswer It depends entirely on whether the base contract promises freshness. If Repository.find guarantees only that it returns the entity with the given identifier, caching is an implementation detail and substitution is safe. If it guarantees that the returned data reflects the current state of storage, caching weakens that postcondition and the substitution is unsafe in any client that depends on immediate consistency. This case demonstrates why LSP cannot be evaluated from code structure alone; the contract must be known.\nQ2. A subclass overrides a method to accept Iterable where the parent accepted list. Is this permitted?\nAnswer Yes. Every list is an Iterable, so every input the parent accepted is still accepted, and the precondition has been weakened rather than strengthened. This is parameter contravariance and it is explicitly permitted. The reverse, accepting only list where the parent accepted any Iterable, would be a violation.\nQ3. Stack inherits from list in order to reuse append and pop. What is the LSP concern?\nAnswer Inheriting from list exposes the full list interface, including insert, __setitem__, and slice assignment, all of which permit modification at arbitrary positions. A stack\u0026rsquo;s defining invariant is that access is restricted to one end. Any client holding the object as a list may legally violate that invariant, so the subtype cannot maintain it. The correct design is composition: Stack holds a list privately and exposes only push, pop, and peek. This is also an Interface Segregation concern, because the stack was forced to inherit an interface far wider than its clients require.\n8. Interface Segregation Principle (ISP) 8.1 Formal statement \u0026ldquo;Clients should not be forced to depend upon interfaces that they do not use.\u0026rdquo;\nRobert C. Martin\n8.2 Term-by-term interpretation \u0026ldquo;Client.\u0026rdquo; The calling code, not the implementing class. ISP is stated from the perspective of the consumer. This orientation is what distinguishes ISP from SRP and is the most frequently missed aspect of the principle.\n\u0026ldquo;Forced to depend upon.\u0026rdquo; A client depends on every method declared in the interface it references, whether or not it calls them. The dependency is real even when unused, because a change to any method in that interface, including a signature change or the addition of a new abstract method, propagates to everyone who references the interface.\n\u0026ldquo;Interfaces they do not use.\u0026rdquo; Methods present in the declared type but irrelevant to the particular client.\nDefinition (fat interface). An interface that aggregates operations serving several distinct client groups, such that no single client uses more than a fraction of it.\nDefinition (Interface Segregation Principle). Wide interfaces should be decomposed into several narrow, client-specific interfaces, so that each client depends only on the operations it actually invokes.\n8.3 The problem it solves Two distinct costs arise from fat interfaces, and they affect implementers and clients respectively.\nCost imposed on implementers. A class that can perform only part of the interface must nonetheless supply every declared method. The residual methods become stubs that raise exceptions or return meaningless values, which is simultaneously an LSP violation, since the object cannot honour the contract it advertises. This is the mechanism by which ISP violations manufacture LSP violations, and it explains why the two principles are almost always discussed together.\nCost imposed on clients. A client that requires one operation but declares a dependency on a twelve-method interface is coupled to eleven methods it never invokes. Three concrete consequences follow. Recompilation and redeployment are triggered by changes the client does not care about. Test doubles must implement all twelve methods, making every test more expensive to write and more fragile. The reader of the client code cannot determine from the type declaration what the client actually needs, which obscures the true dependency structure of the system.\n8.4 Worked violation from abc import ABC, abstractmethod class OfficeMachine(ABC): \u0026#34;\u0026#34;\u0026#34; A single abstraction covering every capability any office device might have. The defect is that no real device has all of them. \u0026#34;\u0026#34;\u0026#34; @abstractmethod def print_document(self, document): ... @abstractmethod def scan_document(self, document): ... @abstractmethod def fax_document(self, document): ... @abstractmethod def staple(self, document): ... class AllInOnePrinter(OfficeMachine): \u0026#34;\u0026#34;\u0026#34;Satisfies the interface honestly, since it genuinely has every capability.\u0026#34;\u0026#34;\u0026#34; def print_document(self, document): print(f\u0026#34;Printing {document}\u0026#34;) def scan_document(self, document): print(f\u0026#34;Scanning {document}\u0026#34;) def fax_document(self, document): print(f\u0026#34;Faxing {document}\u0026#34;) def staple(self, document): print(f\u0026#34;Stapling {document}\u0026#34;) class BudgetPrinter(OfficeMachine): \u0026#34;\u0026#34;\u0026#34; Cannot honour three quarters of the contract it advertises. The stubs are not a cosmetic problem: any client holding this object as an OfficeMachine may legally call scan_document and will crash. \u0026#34;\u0026#34;\u0026#34; def print_document(self, document): print(f\u0026#34;Printing {document}\u0026#34;) def scan_document(self, document): raise NotImplementedError def fax_document(self, document): raise NotImplementedError def staple(self, document): raise NotImplementedError The client-side cost is equally concrete.\ndef print_all(documents, machine: OfficeMachine): # This function uses exactly one of the four declared operations, yet it # is coupled to all four. Adding a fifth abstract method to OfficeMachine # would break every test double written for this function, even though # the function\u0026#39;s own behaviour is unaffected. for document in documents: machine.print_document(document) 8.5 Diagnosis Step 1: build a client-to-method usage matrix. For each client of the interface, record which methods it invokes.\nClient print_document scan_document fax_document staple print_all used unused unused unused ArchiveService unused used unused unused LegalNoticeSender unused unused used unused BindingStation unused unused unused used Step 2: examine the pattern. A diagonal pattern, in which each client uses a different single method, is definitive evidence that the interface is an arbitrary aggregation rather than a coherent abstraction. A dense matrix, in which most clients use most methods, indicates a genuinely cohesive interface that should not be split.\nStep 3: count the stubs. Every method in every implementation that raises, returns a placeholder, or does nothing is a location where the interface promised something the implementer cannot deliver.\n8.6 Resolution The interface is decomposed along the boundaries revealed by the usage matrix, and implementations compose the capabilities they actually possess.\nfrom abc import ABC, abstractmethod class Printer(ABC): @abstractmethod def print_document(self, document): ... class Scanner(ABC): @abstractmethod def scan_document(self, document): ... class FaxMachine(ABC): @abstractmethod def fax_document(self, document): ... class Stapler(ABC): @abstractmethod def staple(self, document): ... class BudgetPrinter(Printer): \u0026#34;\u0026#34;\u0026#34; Declares exactly one capability and implements it. There is no stub, no exception, and no possibility of a client calling an operation this device cannot perform, because the type system prevents it. \u0026#34;\u0026#34;\u0026#34; def print_document(self, document): print(f\u0026#34;Printing {document}\u0026#34;) class AllInOnePrinter(Printer, Scanner, FaxMachine, Stapler): \u0026#34;\u0026#34;\u0026#34; Multiple inheritance is used here as interface composition rather than implementation reuse. Each base contributes one abstract method and no state, so the diamond and initialisation problems associated with multiple inheritance of concrete classes do not arise. \u0026#34;\u0026#34;\u0026#34; def print_document(self, document): print(f\u0026#34;Printing {document}\u0026#34;) def scan_document(self, document): print(f\u0026#34;Scanning {document}\u0026#34;) def fax_document(self, document): print(f\u0026#34;Faxing {document}\u0026#34;) def staple(self, document): print(f\u0026#34;Stapling {document}\u0026#34;) Clients now declare precisely what they need.\ndef print_all(documents, machine: Printer): # The signature now documents the actual requirement. A reader learns # from the type alone that this function neither scans nor faxes, and a # test double needs exactly one method. for document in documents: machine.print_document(document) def archive(documents, scanner: Scanner): for document in documents: scanner.scan_document(document) def print_and_staple(documents, machine): \u0026#34;\u0026#34;\u0026#34; A client requiring two capabilities. Python\u0026#39;s structural flexibility permits passing any object that has both methods; a statically checked version would declare an intersection type or a dedicated Protocol. \u0026#34;\u0026#34;\u0026#34; for document in documents: machine.print_document(document) machine.staple(document) 8.7 Alternative techniques and the Python-specific dimension Role interfaces versus header interfaces. Martin Fowler distinguishes two styles of interface definition, and the distinction is the practical core of ISP.\nStyle Definition Consequence Header interface The interface mirrors the full public surface of an existing implementation class, typically extracted mechanically. Reproduces the implementation\u0026rsquo;s shape, so it is fat by construction and every client depends on everything. Role interface The interface describes one role that a client requires, defined from the client\u0026rsquo;s needs rather than from any implementation. Narrow by construction, and one class may implement several roles. ISP is effectively an instruction to define role interfaces. The practical technique is to name the interface after the capability the client requires, not after the class that happens to provide it. Printer is a role; HPLaserJetInterface is a header.\nClient-owned interfaces. A stronger form of the same idea places the interface definition in the client\u0026rsquo;s module rather than the implementer\u0026rsquo;s. The client declares what it needs; the implementer conforms. This inverts the ownership of the abstraction and is the structural basis of the ports and adapters architecture discussed in Section 9.8.\ntyping.Protocol as the Pythonic realisation. Python 3.8 introduced structural subtyping through typing.Protocol, which allows an interface to be declared and statically checked without any inheritance relationship.\nfrom typing import Protocol class SupportsPrint(Protocol): \u0026#34;\u0026#34;\u0026#34; A structural interface. Any object with a matching print_document method satisfies it, with no inheritance and no import dependency from the implementer to this declaration. This is the strongest available form of ISP in Python: the client declares its requirement, implementations remain entirely unaware of the declaration, and mypy still verifies conformance statically. \u0026#34;\u0026#34;\u0026#34; def print_document(self, document) -\u0026gt; None: ... def print_all(documents, machine: SupportsPrint) -\u0026gt; None: for document in documents: machine.print_document(document) The trade-off between the two mechanisms is worth stating precisely.\nMechanism Advantages Disadvantages Abstract base class (ABC) Explicit, discoverable through the class hierarchy, prevents instantiation of incomplete subclasses, permits shared default implementations Requires implementers to import and inherit, creating a compile-time dependency on the abstraction typing.Protocol No dependency from implementer to interface, works with third-party classes that cannot be modified, statically checked Conformance is not visible in the implementing class, no shared default implementations, requires a type checker to obtain any enforcement Duck typing without any declaration. Passing an object and calling a method, with no interface declared at all, is the lightest option and remains common in Python. The interface still exists; it is simply implicit and undocumented. This is acceptable for small internal code and unsuitable for module boundaries, where the absence of a written contract makes both LSP verification and safe evolution impossible.\n8.8 Costs and limits Interface proliferation. Segregating without evidence produces dozens of single-method interfaces, each with one implementation and one client. This is not modularity; it is indirection without benefit. The correct trigger for segregation is the observed existence of distinct client groups with distinct needs, demonstrated by the usage matrix of Section 8.5.\nComposition complexity at the point of use. When a client legitimately needs three capabilities, it must express a compound requirement. Python has no first-class intersection type, so the options are a combined Protocol, multiple parameters, or an untyped parameter with a documented expectation. Each of these carries some awkwardness.\nSegregation does not reduce total surface area. Splitting one four-method interface into four one-method interfaces leaves the same number of methods in the system. The benefit is in the dependency graph, not in the amount of code. If no client actually benefits from the narrower dependency, the split has produced cost with no return.\n8.9 Detection heuristics Signal Why it indicates an ISP violation Implementations contain methods that raise NotImplementedError The interface promises capabilities that implementers do not have. A test double must stub methods the test never exercises Direct measurement of unnecessary client dependency. The client-to-method usage matrix is sparse or diagonal Distinct client groups are sharing an interface for no reason. An interface name is a noun of convenience such as Service, Manager, or Handler Such names rarely describe a single role and usually indicate aggregation. Adding a method to an interface breaks several unrelated implementations The interface is serving groups that should have been separate. A client imports a large interface and calls one method The dependency declared far exceeds the dependency required. 8.10 Checkpoint questions Q1. An interface has eight methods and every one of its four clients uses all eight. Does this violate ISP?\nAnswer No. ISP prohibits forced dependency on unused methods. When all clients use all methods, the interface is cohesive and the dependency is entirely warranted. Splitting it would impose composition overhead on every client with no reduction in coupling. Method count alone never establishes a violation; the usage matrix does.\nQ2. How are ISP and SRP different, given that both push toward smaller units?\nAnswer They are distinguished by perspective and by subject. SRP concerns the implementation and asks how many actors can cause it to change; it is evaluated from the producer\u0026rsquo;s side. ISP concerns the interface and asks whether clients are coupled to operations they do not invoke; it is evaluated from the consumer\u0026rsquo;s side. The two are independent: a class with a single responsibility can still expose a fat interface if different clients use disjoint parts of it, and a narrow interface can be backed by an implementation that violates SRP internally.\nQ3. In Python, where duck typing means a function can simply call whatever method it needs, is ISP still relevant?\nAnswer Yes, though the mechanism of violation changes. Duck typing removes the compile-time coupling but not the semantic coupling: a function that calls four methods on its parameter requires all four from every caller, and every test double must supply all four. The interface exists whether or not it is declared. Duck typing makes it easier to satisfy ISP accidentally, because a function tends to require only what it calls, and harder to verify deliberately, because the requirement is nowhere written down. typing.Protocol exists precisely to make these implicit requirements explicit and checkable.\n9. Dependency Inversion Principle (DIP) 9.1 Formal statement \u0026ldquo;A. High-level modules should not depend on low-level modules. Both should depend on abstractions. B. Abstractions should not depend on details. Details should depend on abstractions.\u0026rdquo;\nRobert C. Martin\n9.2 Term-by-term interpretation The definitions of high-level module, low-level module, abstraction, and concretion were established in Section 4 and are used here without restatement.\nClause A prohibits a direct dependency from policy to mechanism. The business rule must not name the database driver.\nClause B is the clause that gives the principle its name and is the one most often overlooked. It states that the abstraction must be defined in terms of the domain rather than in terms of any particular implementation. An interface named Repository with a method save(entity) satisfies clause B. An interface named Repository with a method execute_sql(query) does not, because although it is technically abstract, its vocabulary is that of a relational database and no document store could implement it sensibly. Such an interface is a leaky abstraction: it has inverted the syntax of the dependency while preserving its semantics.\n\u0026ldquo;Inversion.\u0026rdquo; The term is comparative and refers to the reversal of the conventional layered dependency direction. In traditional layered design, the dependency arrows point downward from user interface to business logic to data access. DIP reverses the lower arrow so that data access depends on business logic through an abstraction that business logic owns.\nDefinition (Dependency Inversion Principle). Source-level dependencies should point toward abstractions that are owned by the higher-level, more stable module, so that volatile implementation details depend on stable policy rather than the reverse.\n9.3 The problem it solves class MySQLDatabase: def save(self, record): print(f\u0026#34;INSERT INTO users ... {record}\u0026#34;) class UserService: \u0026#34;\u0026#34;\u0026#34; High-level policy: what it means to register a user. The constructor hard-codes a specific storage technology, which produces three separate defects examined below. \u0026#34;\u0026#34;\u0026#34; def __init__(self): self.database = MySQLDatabase() # The defect is on this line def register(self, name, email): if \u0026#34;@\u0026#34; not in email: raise ValueError(\u0026#34;Invalid email address\u0026#34;) self.database.save({\u0026#34;name\u0026#34;: name, \u0026#34;email\u0026#34;: email}) Defect Explanation Rigidity Changing the storage technology requires editing UserService, which contains no storage logic and should have no opinion about storage. Untestability The registration rule cannot be unit tested without a running MySQL server. A test of a three-line validation rule acquires a database dependency, becomes slow, and becomes capable of failing for reasons unrelated to the code under test. Immobility UserService cannot be reused in any context lacking MySQL, including a command-line tool, a batch job, or a different product line. The structural statement of the problem is that the arrow points in the wrong direction. Business policy is the stable, valuable part of a system and typically outlives several generations of storage technology. Storage is volatile. A dependency from stable to volatile means the stable component is destabilised by every change to the volatile one.\n9.4 Resolution An abstraction is introduced, defined in the vocabulary of the domain, and every concrete mechanism conforms to it.\nfrom abc import ABC, abstractmethod class UserRepository(ABC): \u0026#34;\u0026#34;\u0026#34; The abstraction, expressed entirely in domain vocabulary. There is no mention of tables, connections, queries, or files, which is what allows a relational store, a document store, an in-memory structure, or a remote service to implement it without distortion. Ownership note: conceptually this interface belongs to the business layer, not to the persistence layer, because it exists to state what the business layer requires. Placing it in the persistence package would satisfy the letter of the principle while defeating its purpose. \u0026#34;\u0026#34;\u0026#34; @abstractmethod def save(self, user: dict) -\u0026gt; None: ... @abstractmethod def find_by_email(self, email: str) -\u0026gt; dict | None: ... class MySQLUserRepository(UserRepository): def save(self, user): print(f\u0026#34;MySQL insert: {user}\u0026#34;) def find_by_email(self, email): print(f\u0026#34;MySQL select where email = {email}\u0026#34;) return None class InMemoryUserRepository(UserRepository): \u0026#34;\u0026#34;\u0026#34; A production-quality implementation for tests, not a mock. It has real behaviour and can therefore verify that the service actually stored what it claimed to store, which a bare mock cannot do without replicating the assertion logic in every test. \u0026#34;\u0026#34;\u0026#34; def __init__(self): self._users = {} def save(self, user): self._users[user[\u0026#34;email\u0026#34;]] = user def find_by_email(self, email): return self._users.get(email) class UserService: def __init__(self, repository: UserRepository): # The dependency arrives from outside. This class now names no # storage technology anywhere in its source, so no change of # storage technology can require editing it. self.repository = repository def register(self, name, email): if \u0026#34;@\u0026#34; not in email: raise ValueError(\u0026#34;Invalid email address\u0026#34;) if self.repository.find_by_email(email) is not None: raise ValueError(\u0026#34;Email is already registered\u0026#34;) self.repository.save({\u0026#34;name\u0026#34;: name, \u0026#34;email\u0026#34;: email}) Testing the business rule now requires no infrastructure whatsoever.\ndef test_duplicate_registration_is_rejected(): repository = InMemoryUserRepository() service = UserService(repository) service.register(\u0026#34;Ananya\u0026#34;, \u0026#34;ananya@example.com\u0026#34;) try: service.register(\u0026#34;Ananya Again\u0026#34;, \u0026#34;ananya@example.com\u0026#34;) assert False, \u0026#34;Expected a duplicate registration to be rejected\u0026#34; except ValueError: pass The test executes in microseconds, has no external dependency, and fails only when the business rule is wrong. This improvement in testability is the most immediately measurable benefit of DIP and is usually the argument that persuades teams to adopt it.\n9.5 Why the relationship is called an inversion Before applying the principle, the source-level dependency runs downward:\nUserService ----\u0026gt; MySQLDatabase (policy) (mechanism) After applying it, the dependency from the mechanism has been reversed:\nUserService ----\u0026gt; UserRepository \u0026lt;---- MySQLUserRepository (policy) (abstraction) (mechanism) Two properties of the second diagram deserve attention. First, the arrow from MySQLUserRepository points upward, which is the inversion the principle is named for. Second, UserService and UserRepository together form a self-contained unit that compiles, imports, and tests without any knowledge that MySQLUserRepository exists. That unit is the reusable, valuable core of the system.\nThe runtime call still flows from UserService into MySQLUserRepository. Only the source-level dependency has been inverted. Distinguishing the direction of control flow from the direction of source dependency is the conceptual key to the principle, and confusing the two is the most common source of misunderstanding.\n9.6 Where the abstraction should live Clause B of the formal statement has a structural consequence that is frequently ignored. An interface placed in the same package as its implementation, and shaped around that implementation\u0026rsquo;s capabilities, provides syntactic indirection without reducing coupling.\nPlacement Effect Interface defined in the persistence package, shaped by the database Business layer imports from the persistence package, so the dependency remains. The abstraction is decorative. Interface defined in the business package, shaped by business needs Persistence package imports from the business package. The dependency is genuinely inverted and the business package can be extracted and reused. The practical test is to ask whether the business package can be packaged and shipped with its persistence package deleted. If it can, the inversion is real.\n9.7 Dependency injection: the technique that realises DIP Definition (dependency injection). The practice of supplying a module\u0026rsquo;s collaborators from outside the module rather than having the module construct or locate them itself.\nDependency inversion is the principle, which describes what the dependency structure should look like. Dependency injection is a technique, which describes how the correct structure is achieved at runtime. A third term, inversion of control, is broader still.\nTerm Category Meaning Dependency Inversion Principle Design principle Depend on abstractions; abstractions should be owned by the higher-level module. Dependency Injection Implementation technique Supply collaborators from outside rather than constructing them internally. Inversion of Control Architectural style The framework or the caller, rather than the module itself, determines flow and wiring. Dependency injection is one instance of it. Three injection styles are in common use.\nConstructor injection, which is the default choice.\nclass ReportService: def __init__(self, repository: UserRepository, formatter): # Required collaborators are supplied at construction, so an # instance cannot exist in a partially configured state. This is # the reason constructor injection is preferred by default. self.repository = repository self.formatter = formatter Setter injection, for optional or replaceable collaborators.\nclass ReportService: def __init__(self, repository: UserRepository): self.repository = repository self._logger = NullLogger() # Safe default, so the object is always usable def set_logger(self, logger): # Optional collaborator. Using setter injection here signals that # logging is not required for correct operation, which constructor # injection would have obscured by making it mandatory. self._logger = logger Method injection, when the collaborator varies per call rather than per object.\nclass Checkout: def final_price(self, amount: float, policy: DiscountPolicy) -\u0026gt; float: # The policy differs for every customer, so binding it to the # object would force a new Checkout instance per customer. return amount - policy.discount_for(amount) Framework-based injection delegates the wiring to a container, as with FastAPI\u0026rsquo;s Depends or the various dependency injection libraries. The benefit is reduced boilerplate in large applications; the cost is that the object graph is assembled by machinery that is harder to trace than explicit construction.\nRegardless of style, the concrete implementations must be selected somewhere. That location is called the composition root, and it is conventionally the application\u0026rsquo;s entry point.\ndef main(): \u0026#34;\u0026#34;\u0026#34; The composition root. This is the only function in the application that names concrete infrastructure classes. Every other module refers to abstractions, which is what makes the rest of the system portable. \u0026#34;\u0026#34;\u0026#34; repository = MySQLUserRepository() service = UserService(repository) service.register(\u0026#34;Ananya\u0026#34;, \u0026#34;ananya@example.com\u0026#34;) 9.8 Architectural consequences DIP applied consistently across a whole system produces a recognisable architecture. Business rules occupy the centre and depend on nothing external. Interfaces for everything the business rules require, such as persistence, messaging, and time, are defined in the centre. All infrastructure sits at the periphery and depends inward by implementing those interfaces.\nThis arrangement appears in the literature under several names: hexagonal architecture, ports and adapters, onion architecture, and clean architecture. They differ in terminology and in how many layers they name, but each is fundamentally an application of the Dependency Inversion Principle at the scale of an entire system. The interfaces defined by the centre are the ports; the infrastructure implementations are the adapters.\n9.9 Costs and limits Not every dependency should be inverted. The criterion is volatility combined with the need to substitute. Stable dependencies with no realistic alternative implementation should be used directly.\nDependency Invert? Reason Database, message broker, payment gateway, third-party HTTP API Yes Volatile, externally owned, and must be substituted in tests. The system clock, when time-dependent logic is tested Yes Non-determinism makes tests unreliable unless time is injectable. Standard library data structures such as list and dict No Stable, ubiquitous, and never substituted. Pure functions with no input or output, such as arithmetic helpers No Deterministic and independently testable already. A domain value object such as Money No Part of the domain itself, not a mechanism behind it. Indirection increases the cost of comprehension. Following a call from policy to the executing code requires knowing which implementation was wired at the composition root. In a large object graph this is genuine friction, and it is the reason explicit composition roots are preferable to magical container configuration in codebases of moderate size.\nAbstractions defined too early are usually wrong. An interface extracted from a single implementation before a second one exists tends to reproduce the first implementation\u0026rsquo;s shape and then requires reworking when the second arrives. Extracting the interface at the moment the second implementation is genuinely needed produces a better abstraction with less rework.\n9.10 Detection heuristics Signal Why it indicates a DIP violation A class constructs its own collaborators, as in self.db = MySQLDatabase() The class has chosen a concrete implementation, so no substitution is possible. Business logic modules import infrastructure libraries such as psycopg2, requests, or boto3 Policy has acquired a direct dependency on mechanism. A unit test requires a database, a network connection, or a running service The unit under test cannot be isolated, which is the observable symptom of an uninverted dependency. Calls to datetime.now() or random() inside logic that must be tested deterministically Non-substitutable dependencies on ambient state. An interface whose method names use infrastructure vocabulary, such as execute_query Violation of clause B: the abstraction has been shaped by the detail. Changing an implementation requires edits in several unrelated modules Concrete types have leaked into multiple call sites. 9.11 Checkpoint questions Q1. A team introduces an interface IUserRepository whose only method is execute_sql(query: str). Has DIP been satisfied?\nAnswer No. Clause A has been satisfied syntactically, since the service now refers to an interface, but clause B has been violated. The abstraction is expressed in the vocabulary of relational databases, so a document store or an in-memory implementation cannot honour it without absurdity, and the caller must construct SQL, which means SQL knowledge has been pushed into the business layer. The dependency has been renamed rather than inverted. A correct abstraction would expose domain operations such as find_by_email and save.\nQ2. Is datetime.now() a DIP violation?\nAnswer Only when the surrounding logic must be tested deterministically or must support alternative time sources. Logging a timestamp does not warrant abstraction. A subscription expiry rule does, because the test must control the current time in order to verify boundary behaviour, and a hard-coded call to the system clock makes that impossible. The general criterion applies here: invert a dependency when it is volatile or non-deterministic and substitution is required, not merely because it is external.\nQ3. Does dependency injection by itself guarantee dependency inversion?\nAnswer No. Injecting a concrete MySQLDatabase through a constructor improves testability slightly and permits substitution by a subclass, but the parameter type still names a concrete class, so the high-level module remains coupled to a specific mechanism. Inversion requires that the declared dependency be an abstraction owned by the high-level module. Injection is the delivery mechanism; the abstraction is what makes the delivery meaningful.\n10. How the Five Principles Interact 10.1 The principles are not independent Presenting SOLID as five separate rules obscures the fact that they describe one structural property from five angles. Each principle, correctly applied, makes the others easier to satisfy, and each violation makes the others harder.\nThe causal chain runs as follows. Separating responsibilities produces classes with narrow purposes (SRP). Classes with narrow purposes naturally expose narrow interfaces, because there is little to expose (ISP). Narrow interfaces are easy for a subtype to honour completely, since the number of guarantees is small (LSP). When subtypes reliably honour their contracts, depending on an abstraction becomes safe rather than speculative (DIP). When clients depend on trustworthy abstractions, new behaviour can be introduced by adding implementations rather than editing existing code (OCP). Systems that are extended by addition accumulate less entanglement, which makes the next round of responsibility separation cheaper (SRP again).\nThe reverse chain is equally mechanical and describes how codebases decay. A class that accumulates responsibilities (SRP violation) exposes a wide interface (ISP violation). A wide interface cannot be fully honoured by every implementation, so implementations begin stubbing methods (LSP violation). Clients learn that the abstraction is unreliable and start checking concrete types to compensate (DIP violation). Type checks are conditionals on type, so each new variant requires editing them (OCP violation). At that point all five have failed, and the original cause was a single class that was allowed to grow.\n10.2 Dependency map Principle Depends on Enables SRP Nothing; it is the entry point Narrow interfaces, therefore ISP ISP Cohesive classes produced by SRP Contracts small enough to honour, therefore LSP LSP Narrow contracts produced by ISP Trustworthy substitution, therefore DIP and OCP DIP Reliable substitution guaranteed by LSP Extension points, therefore OCP OCP Abstractions established by DIP; substitutability from LSP Additive change, which preserves SRP over time 10.3 Grouping by purpose A different organisation of the same material is useful when deciding which principle to reach for.\nConcern Principles Question to ask How code is partitioned SRP, ISP Is this unit doing one thing, and is its interface no wider than its clients require? How code is extended OCP Can the expected new variant be added without editing existing code? How code is connected DIP Does policy depend on abstractions that policy itself defines? Whether abstractions can be trusted LSP Can any implementation be substituted without a client noticing? LSP occupies a distinct position among the five. The other four describe how to structure code; LSP describes a correctness condition that the structure must satisfy for the others to deliver their benefits. An OCP design built on subtypes that violate their contracts is not extensible, it is merely indirect.\n11. Capstone: Refactoring an Order Processing System This section applies all five principles to a single realistic module. The starting point is code of a kind that appears in most codebases that have grown without periodic restructuring. The refactoring proceeds in explicit stages, and each stage states which principle motivates it and what would go wrong if the stage were skipped.\n11.1 The starting point class OrderManager: \u0026#34;\u0026#34;\u0026#34; Handles the complete order lifecycle. The word \u0026#39;complete\u0026#39; in that sentence is the entire problem. \u0026#34;\u0026#34;\u0026#34; def __init__(self): self.orders = [] def place_order(self, customer_type, items, payment_method, email): # Responsibility 1: pricing arithmetic total = sum(price * quantity for _, price, quantity in items) # Responsibility 2: discount policy, owned by the commercial team if customer_type == \u0026#34;regular\u0026#34;: total *= 0.95 elif customer_type == \u0026#34;premium\u0026#34;: total *= 0.90 elif customer_type == \u0026#34;vip\u0026#34;: total *= 0.80 # Responsibility 3: payment execution, owned by the payments team if payment_method == \u0026#34;card\u0026#34;: print(f\u0026#34;Charging {total} to card via the card gateway\u0026#34;) elif payment_method == \u0026#34;upi\u0026#34;: print(f\u0026#34;Collecting {total} via UPI\u0026#34;) elif payment_method == \u0026#34;cash\u0026#34;: print(f\u0026#34;Recording {total} as cash on delivery\u0026#34;) # Responsibility 4: persistence, owned by the platform team print(\u0026#34;Executing INSERT INTO orders ...\u0026#34;) # Responsibility 5: customer communication, owned by the growth team print(f\u0026#34;Sending confirmation email to {email}\u0026#34;) order = {\u0026#34;items\u0026#34;: items, \u0026#34;total\u0026#34;: total} self.orders.append(order) return total 11.2 Diagnosis against all five principles Principle Status Specific evidence SRP Violated Five actors can require changes to this single method: commercial, payments, platform, growth, and whoever owns pricing arithmetic. OCP Violated Two conditionals branch on values that are certain to grow: customer tiers and payment methods. LSP Not yet applicable No abstractions exist, so there is nothing to substitute. The absence of abstraction is itself the finding. ISP Not yet applicable No interfaces exist. DIP Violated The payment gateway, the database, and the email transport are all named directly inside business logic, which makes the method impossible to unit test. The order in which these are addressed matters. Extracting responsibilities (SRP) is performed first, because it produces the units against which the remaining principles are then applied. Attempting OCP or DIP inside a class that still does five things produces abstractions along the wrong boundaries.\n11.3 Stage 1, separate responsibilities (SRP) Each responsibility identified in the diagnosis becomes its own concept. At this stage the classes remain concrete; abstraction is introduced deliberately in later stages, only where an axis of change justifies it.\nclass OrderLine: \u0026#34;\u0026#34;\u0026#34; A value object representing one line of an order. Introduced because tuples of (name, price, quantity) carry no meaning at the point of use, permit silent argument reordering, and cannot enforce their own invariants. \u0026#34;\u0026#34;\u0026#34; def __init__(self, product_name: str, unit_price: float, quantity: int): if unit_price \u0026lt; 0: raise ValueError(\u0026#34;Unit price cannot be negative\u0026#34;) if quantity \u0026lt;= 0: raise ValueError(\u0026#34;Quantity must be positive\u0026#34;) self.product_name = product_name self.unit_price = unit_price self.quantity = quantity @property def line_total(self) -\u0026gt; float: # Behaviour placed on the data it operates on, which avoids the # anaemic model problem discussed in Section 5.8. return self.unit_price * self.quantity class Order: \u0026#34;\u0026#34;\u0026#34; The order entity. It owns its own invariants and nothing else. It does not know how it is priced, paid for, stored, or announced. \u0026#34;\u0026#34;\u0026#34; def __init__(self, customer_email: str): self.customer_email = customer_email self.lines: list[OrderLine] = [] self.total: float | None = None def add_line(self, line: OrderLine) -\u0026gt; None: self.lines.append(line) @property def subtotal(self) -\u0026gt; float: return sum(line.line_total for line in self.lines) 11.4 Stage 2, make the varying policies extensible (OCP) The two conditionals identified in the diagnosis are replaced by abstractions, because both branch on sets that are known to grow.\nfrom abc import ABC, abstractmethod class DiscountPolicy(ABC): \u0026#34;\u0026#34;\u0026#34; Axis of change: the commercial team introduces new customer tiers and promotional schemes several times a year. This is the clearest possible justification for an abstraction, since the axis is demonstrated rather than predicted. \u0026#34;\u0026#34;\u0026#34; @abstractmethod def discount_for(self, subtotal: float) -\u0026gt; float: \u0026#34;\u0026#34;\u0026#34;Return the absolute discount amount, never the discounted total.\u0026#34;\u0026#34;\u0026#34; ... class NoDiscount(DiscountPolicy): def discount_for(self, subtotal): return 0.0 class RegularDiscount(DiscountPolicy): def discount_for(self, subtotal): return subtotal * 0.05 class PremiumDiscount(DiscountPolicy): def discount_for(self, subtotal): return subtotal * 0.10 class VIPDiscount(DiscountPolicy): def discount_for(self, subtotal): return subtotal * 0.20 class CappedDiscount(DiscountPolicy): \u0026#34;\u0026#34;\u0026#34; A composite policy that wraps another and limits its effect. Its existence is the strongest evidence that a class-based abstraction was the right choice over a dictionary of percentage rates: this variant expresses behaviour that no table of constants could represent, and it was added without modifying any existing policy. \u0026#34;\u0026#34;\u0026#34; def __init__(self, inner: DiscountPolicy, maximum: float): self.inner = inner self.maximum = maximum def discount_for(self, subtotal): return min(self.inner.discount_for(subtotal), self.maximum) 11.5 Stage 3, invert the infrastructure dependencies (DIP and ISP) Payment, persistence, and notification are all mechanisms behind stable policy. Each receives an abstraction expressed in domain vocabulary, and each abstraction is deliberately narrow so that no client depends on operations it does not use.\nclass PaymentGateway(ABC): \u0026#34;\u0026#34;\u0026#34; One method, because the order workflow requires exactly one operation. Refunds and settlement reports are genuine capabilities but belong to separate interfaces used by separate clients, in accordance with ISP. \u0026#34;\u0026#34;\u0026#34; @abstractmethod def charge(self, amount: float, reference: str) -\u0026gt; bool: \u0026#34;\u0026#34;\u0026#34;Return True when the charge succeeds, False when it is declined.\u0026#34;\u0026#34;\u0026#34; ... class OrderRepository(ABC): @abstractmethod def save(self, order: Order) -\u0026gt; None: ... class OrderNotifier(ABC): @abstractmethod def notify_confirmed(self, order: Order) -\u0026gt; None: ... class CardGateway(PaymentGateway): def charge(self, amount, reference): print(f\u0026#34;Card charge of {amount} for {reference}\u0026#34;) return True class UPIGateway(PaymentGateway): def charge(self, amount, reference): print(f\u0026#34;UPI collection of {amount} for {reference}\u0026#34;) return True class CashOnDeliveryGateway(PaymentGateway): \u0026#34;\u0026#34;\u0026#34; Honours the contract without moving money at authorisation time. LSP check: the contract states that True means the charge succeeded, where success is defined as the payment being validly arranged. Cash on delivery arranges payment at delivery, which satisfies that definition. Had the contract stated that True means funds have settled, this class would violate it and would require a different abstraction. \u0026#34;\u0026#34;\u0026#34; def charge(self, amount, reference): print(f\u0026#34;Cash on delivery recorded: {amount} for {reference}\u0026#34;) return True class SQLOrderRepository(OrderRepository): def save(self, order): print(f\u0026#34;Persisting order for {order.customer_email}\u0026#34;) class InMemoryOrderRepository(OrderRepository): def __init__(self): self.saved: list[Order] = [] def save(self, order): self.saved.append(order) class EmailNotifier(OrderNotifier): def notify_confirmed(self, order): print(f\u0026#34;Confirmation email to {order.customer_email}\u0026#34;) class NullNotifier(OrderNotifier): \u0026#34;\u0026#34;\u0026#34;Used in tests and batch imports where no customer contact is desired.\u0026#34;\u0026#34;\u0026#34; def notify_confirmed(self, order): pass 11.6 Stage 4, the orchestrator The remaining class coordinates the workflow and does nothing else. Its single reason to change is a change to the sequence or composition of steps in placing an order.\nclass PaymentDeclined(Exception): \u0026#34;\u0026#34;\u0026#34; A domain-level exception. The service raises this rather than a gateway-specific error, so callers never need to know which gateway was used. Leaking a gateway exception here would reintroduce the dependency that the PaymentGateway abstraction removed. \u0026#34;\u0026#34;\u0026#34; class OrderService: def __init__( self, discount_policy: DiscountPolicy, payment_gateway: PaymentGateway, repository: OrderRepository, notifier: OrderNotifier, ): self.discount_policy = discount_policy self.payment_gateway = payment_gateway self.repository = repository self.notifier = notifier def place_order(self, order: Order) -\u0026gt; float: subtotal = order.subtotal discount = self.discount_policy.discount_for(subtotal) order.total = subtotal - discount # Payment precedes persistence so that a declined payment leaves no # order record. The alternative ordering, persisting first and # marking the order failed afterwards, is also defensible and is # chosen when an audit trail of failed attempts is required. The # decision belongs here, in the workflow, which is exactly why the # workflow is a responsibility in its own right. if not self.payment_gateway.charge(order.total, order.customer_email): raise PaymentDeclined(f\u0026#34;Payment declined for {order.customer_email}\u0026#34;) self.repository.save(order) self.notifier.notify_confirmed(order) return order.total 11.7 Composition and testing def main(): \u0026#34;\u0026#34;\u0026#34;The composition root, and the only place concrete classes are named.\u0026#34;\u0026#34;\u0026#34; order = Order(\u0026#34;ananya@example.com\u0026#34;) order.add_line(OrderLine(\u0026#34;Textbook\u0026#34;, 1200.0, 2)) order.add_line(OrderLine(\u0026#34;Notebook\u0026#34;, 80.0, 5)) service = OrderService( discount_policy=CappedDiscount(VIPDiscount(), maximum=500.0), payment_gateway=UPIGateway(), repository=SQLOrderRepository(), notifier=EmailNotifier(), ) service.place_order(order) def test_discount_is_applied_and_order_is_persisted(): \u0026#34;\u0026#34;\u0026#34; A complete test of the ordering workflow with no infrastructure. This is the practical payoff of the refactoring and would have been impossible against the original OrderManager. \u0026#34;\u0026#34;\u0026#34; class DecliningGateway(PaymentGateway): def charge(self, amount, reference): return False repository = InMemoryOrderRepository() order = Order(\u0026#34;test@example.com\u0026#34;) order.add_line(OrderLine(\u0026#34;Book\u0026#34;, 1000.0, 1)) service = OrderService( discount_policy=RegularDiscount(), payment_gateway=DecliningGateway(), repository=repository, notifier=NullNotifier(), ) try: service.place_order(order) assert False, \u0026#34;A declined payment should prevent order placement\u0026#34; except PaymentDeclined: pass assert repository.saved == [], \u0026#34;No order should be persisted after a decline\u0026#34; 11.8 Verification against each principle Principle How the final design satisfies it SRP OrderLine owns line arithmetic, Order owns order invariants, each policy owns one pricing rule, each gateway owns one payment mechanism, the repository owns persistence, the notifier owns communication, and OrderService owns the workflow sequence. Each has exactly one actor. OCP A new customer tier is a new DiscountPolicy subclass. A new payment method is a new PaymentGateway subclass. Neither requires editing OrderService or any existing implementation. LSP Every implementation honours its contract without stubs or unsupported operations. The CashOnDeliveryGateway docstring records the contract reasoning explicitly, which is what makes the substitution verifiable rather than assumed. ISP Each interface declares exactly one method, because each client requires exactly one operation. No implementation raises NotImplementedError. DIP OrderService names no concrete infrastructure. All three infrastructure abstractions are expressed in domain vocabulary, so alternative implementations are possible without distortion. Concrete selection occurs only in main. 11.9 An honest accounting of the cost The original implementation was one class of roughly thirty lines. The refactored version is thirteen classes across roughly a hundred and fifty lines. That expansion is a real cost and must be justified rather than assumed.\nThe justification rests on specific, observed facts about this domain: customer tiers change several times a year, payment methods are added when the business enters new markets, and the ordering workflow must be tested without infrastructure because it encodes revenue-critical rules. Each abstraction in the final design corresponds to one of those documented axes of change.\nHad the business supported exactly one customer tier and one payment method with no expectation of growth, the original class would have been the better design, and the refactoring would have been speculative generality. The engineering judgement is not \u0026ldquo;apply SOLID\u0026rdquo; but \u0026ldquo;identify the axes along which this system will actually change, and pay for abstraction only along those axes.\u0026rdquo;\n12. SOLID Beyond Classes: Functions, Modules, and Services The principles are usually taught with classes because that is the granularity at which they are easiest to demonstrate. They apply unchanged at every level of decomposition, and recognising this generalisation is a mark of maturity in applying them.\nPrinciple At function level At module or package level At service level SRP A function does one thing; its name fully describes its effect, with no hidden side effects. A package serves one area of concern and does not mix domain logic with infrastructure. A service owns one business capability and one data store. OCP New behaviour is added by passing a different function argument rather than adding a branch. New capability arrives as a new module registered through an existing extension point. New behaviour arrives as a new subscriber to an existing event stream, with no change to the publisher. LSP A replacement function accepts the same inputs and returns the same guarantees. A replacement module honours the package\u0026rsquo;s public contract. A new version of a service remains backward compatible with existing consumers. ISP A function takes only the parameters it uses, rather than a large configuration object. A package exposes a narrow public surface and keeps the rest private. A service exposes purpose-specific endpoints instead of one generic endpoint driven by a mode flag. DIP A function receives its collaborators as parameters rather than importing them at module scope. Domain packages define the interfaces that infrastructure packages implement. Services communicate through published contracts and message schemas rather than through each other\u0026rsquo;s databases. Two entries in this table deserve elaboration.\nStamp coupling at function level. A function that accepts an entire Order object in order to read order.total is coupled to the whole of Order. Accepting total: float removes that coupling, makes the function reusable in contexts that have no Order, and states the actual requirement in the signature. This is ISP applied to a parameter list.\nThe shared database as a DIP violation between services. When two services read and write the same tables, each is coupled to the other\u0026rsquo;s internal storage schema, which is the most concrete detail either service has. Neither can change its schema independently, so the services are separately deployable but not separately modifiable, which forfeits the main benefit of separating them. Communicating through published contracts restores the inversion.\n13. Anti-Patterns and Misapplications The failures below are all committed by developers who have learned SOLID and are applying it earnestly. They are more common in practice than outright ignorance of the principles.\n13.1 Speculative generality An abstraction is created for a variation that has never occurred and has not been requested. The codebase carries an interface, a factory, and a configuration entry to support exactly one implementation.\nThe diagnostic is quantitative: count the implementations of each abstraction in the codebase. An abstraction with a single implementation and no documented plan for a second is a candidate for removal. The counter-consideration is testability, since an abstraction with one production implementation and one test implementation genuinely has two, and that is sufficient justification when the dependency is infrastructural.\n13.2 Mechanical splitting mistaken for SRP A large class is divided by line count or by alphabetical grouping of methods rather than by reason to change. The result is two classes that must always be modified together, so the blast radius is unchanged while the comprehension cost has doubled. The corrective question is not \u0026ldquo;is this class large\u0026rdquo; but \u0026ldquo;which distinct actors can require this class to change.\u0026rdquo;\n13.3 The anaemic domain model Every behaviour is extracted into a service class, leaving entities as bare attribute containers. Invariants that the entity should enforce are enforced instead by whichever service remembers to check them, which means they are eventually enforced inconsistently. Behaviour that maintains an entity\u0026rsquo;s own invariants belongs on the entity; behaviour that coordinates several entities or reaches outside the process does not.\n13.4 Interface proliferation Every class is given a corresponding interface, frequently generated mechanically and named by prefixing the class name. These are header interfaces in the sense of Section 8.7: they mirror an implementation rather than describing a client\u0026rsquo;s role, so they add a file and an indirection without reducing coupling.\n13.5 Abstraction in the wrong vocabulary An interface is introduced, satisfying clause A of DIP, but its methods are named after the implementation\u0026rsquo;s operations, violating clause B. IEmailSender.send_smtp_message is abstract in form and concrete in meaning. The test is whether a genuinely different implementation could satisfy the interface without distortion.\n13.6 Configuration-driven complexity Extensibility is pursued through configuration files and runtime registries in a system where the set of variants is small, internal, and stable. The result is behaviour that cannot be traced by reading code, that fails at runtime rather than at import time, and that requires reading two artefacts to understand one flow. Registries are appropriate for genuine plugin systems and are overhead for five internal variants.\n13.7 Applying principles without measuring the outcome The strongest available evidence for whether a design decision worked is the version control history. If a class was split six months ago and both halves have changed together in every subsequent commit, the split was wrong and should be reversed. Design principles produce predictions about future change; the repository records what actually happened, and consulting it converts design from an aesthetic exercise into an empirical one.\n14. When SOLID Should Not Be Applied Each principle imposes a cost in indirection, file count, and comprehension effort. That cost is justified only when the corresponding benefit is realised, and the benefit is realised only when the code changes in the anticipated way.\nContext Recommendation Reason Exploratory prototypes and spikes Do not apply The purpose is to discover requirements. Abstractions built on unknown requirements are built on the wrong boundaries and will be discarded with the prototype. Scripts under a few hundred lines with a short lifetime Do not apply Total lifetime cost is dominated by writing, not by maintenance, so there is no maintenance cost to reduce. Stable domains with no realistic variation Apply selectively Physical constants and closed mathematical enumerations do not grow. Abstracting them yields indirection with no possible return. Performance-critical inner loops Apply selectively and measure Dynamic dispatch and object allocation carry measurable cost in tight loops. The correct procedure is to profile first, since intuitions about hot paths are frequently wrong. Code with a single implementation and no test substitution need Defer Introduce the abstraction when the second implementation appears, at which point the correct shape is known from evidence. Long-lived business logic in an evolving domain Apply thoroughly This is the context the principles were formulated for, and the context in which they repay their cost most reliably. Any module at a boundary with external systems Apply DIP in particular External systems are the most volatile and least controllable dependencies, and they are the ones that make testing expensive. Two general guidelines summarise the judgement required.\nThe rule of three. Write the direct implementation for the first case. Duplicate or extend it for the second while observing what actually varies. Introduce the abstraction at the third, when the axis of change has been demonstrated by evidence rather than predicted. This sequence produces better abstractions than up-front design, because it is informed by real variation.\nAbstract the boundaries, keep the core concrete. Apply DIP aggressively where the system meets databases, networks, clocks, file systems, and third-party services, because these are volatile and make testing expensive. Keep pure domain computation concrete, direct, and free of indirection, because it is stable, deterministic, and already easy to test.\n15. Summary and Key Takeaways 15.1 Principle reference Principle Statement Problem addressed Primary mechanism Chief cost SRP One reason to change per module Merge contention, fragility, immobility Separation by actor More files, indirection, risk of anaemic entities OCP Open for extension, closed for modification Regression risk when adding variants Polymorphic dispatch behind a stable abstraction Wrong axis prediction produces useless indirection LSP Subtypes must honour base contracts Unreliable polymorphism and defensive type checks Contract preservation, verified by contract tests Requires documented contracts and discourages inheritance ISP No forced dependency on unused methods Unnecessary coupling and stubbed implementations Decomposition into role interfaces Interface proliferation, composition complexity DIP Depend on abstractions owned by policy Untestable, rigid, immobile business logic Inverted source dependency plus injection Object graph is harder to trace 15.2 Diagnostic quick reference Observed symptom Most likely violated principle First corrective action One file is edited by several teams for unrelated reasons SRP Identify actors and split along actor boundaries Adding a variant requires editing a conditional OCP Introduce an abstraction for the varying concept Implementations raise NotImplementedError ISP, and consequently LSP Split the interface along the client usage matrix Clients use isinstance to handle subtypes differently LSP Restructure the hierarchy so the base promises only what all subtypes deliver A unit test requires a database or a network DIP Extract a domain-vocabulary interface and inject the implementation Changing a technology requires edits across many modules DIP Locate the concrete type references and replace them with an abstraction The same conditional chain appears in several files OCP Consolidate into one polymorphic abstraction 15.3 Key takeaways The principles share a single objective, which is reducing the cost of change. Each addresses a different mechanism by which change becomes expensive, and every one of them can be restated as an instruction to lower coupling or raise cohesion.\nThe principles form a reinforcing system rather than a checklist. Correct application of one makes the others easier to satisfy, and a violation of one propagates predictably into violations of the rest, beginning almost always with an unchecked accumulation of responsibilities.\nContracts are behavioural, not syntactic. A subclass that matches every signature and violates a single guarantee has broken the abstraction as thoroughly as one that fails to compile. This is the reason LSP occupies a different position from the other four.\nAbstraction is a bet on a predicted axis of change. Correct predictions repay their cost at the first variation; incorrect predictions impose permanent indirection that serves nothing. Deferring abstraction until the axis is demonstrated by evidence produces better designs than predicting it in advance.\nTestability is the most reliable available proxy for good design. Code that can be unit tested without infrastructure has, almost by construction, separated policy from mechanism, depends on abstractions, and has narrow interfaces. When a design decision is genuinely unclear, asking how the result will be tested resolves it more often than any other question.\nThe most common failure among practitioners is over-application, not under-application. Speculative abstraction, mechanical splitting, and interface proliferation are the characteristic errors of developers who have recently learned SOLID, and they are more expensive than the entangled code they were intended to prevent.\nSOLID is the conceptual foundation of the design patterns catalogue. Nearly every pattern in the Gang of Four catalogue is a concrete, named realisation of one or more of these principles, which is why the patterns become substantially easier to understand once the principles are secure.\n16. Self-Assessment Questions with Answers Answer each question in writing before expanding the response. Questions test conceptual discrimination rather than recall, and several are deliberately constructed so that the obvious answer is incorrect.\nQ1. A Report class has eighteen methods, all operating on the same dataset and all changing whenever the reporting requirements change. Does it violate SRP?\nAnswer Not on the evidence given. SRP counts actors, not methods. If all eighteen methods change in response to requests from the reporting stakeholder and no other, the class has one reason to change and is cohesive. The class should be examined for a second actor, for instance if some methods perform data retrieval owned by a platform team, or if some perform file output owned by an infrastructure team. If such methods exist, the violation lies in those specific methods and not in the method count.\nQ2. Is it possible to satisfy SRP fully while violating OCP badly?\nAnswer Yes, and the combination is common. A DiscountCalculator whose sole purpose is calculating discounts has exactly one reason to change and satisfies SRP. If it implements that purpose with a conditional chain over customer tiers, every new tier requires editing it, which violates OCP. SRP governs where boundaries are drawn between modules; OCP governs how variation is handled inside a boundary. They are orthogonal.\nQ3. A subclass overrides a method to add caching. Under what circumstances is this an LSP violation?\nAnswer It is a violation when the base contract promises freshness and the cache can return stale data, since that weakens a postcondition. It is not a violation when the base contract promises only to return the entity with the given identifier, since caching is then an unobservable implementation detail. The determining factor is the documented contract, not the presence of caching. This is a case where the correct answer cannot be derived from the code alone.\nQ4. Explain the difference between the Dependency Inversion Principle and dependency injection, and state whether either implies the other.\nAnswer DIP is a principle governing the shape of the dependency graph: policy should depend on abstractions that policy itself defines, and mechanisms should depend on those abstractions. Dependency injection is a technique for supplying collaborators from outside a module. Neither implies the other. Injecting a concrete MySQLDatabase uses injection without achieving inversion, since the declared dependency is still a concrete class. Conversely, a module could depend on an abstraction and obtain its instance from a service locator, achieving inversion without injection, though at the cost of a hidden dependency on the locator. In practice DIP is normally realised through injection because injection makes the dependency explicit at the constructor.\nQ5. An interface declares save, load, delete, export_to_csv, and send_by_email. Which principle is most clearly violated, and what is the secondary consequence?\nAnswer ISP is most clearly violated, since the five operations serve at least three distinct client groups: persistence clients, reporting clients, and communication clients. No client requires all five. The secondary consequence is an LSP violation, because implementations that can persist but cannot send email will supply stubs that raise exceptions, which breaks the contract they advertise. There is also an SRP concern on the implementation side, since persistence, export, and communication answer to different actors. This question illustrates the cascade described in Section 10.1.\nQ6. Under what circumstances should a conditional chain be preferred over polymorphic dispatch?\nAnswer When the set of cases is closed and cannot grow, such as the three orderings between two comparable values. When the set is small, stable, and entirely local, so that the cost of indirection exceeds the cost of an occasional edit. When the branches are trivial value lookups, in which case a dictionary is simpler than either a conditional or a class hierarchy. When the code is a short-lived script whose total maintenance cost is negligible. The error is never the conditional itself; it is the use of a conditional along an axis that is known to be growing.\nQ7. A team introduces IUserService, IOrderService, and IPaymentService, each with exactly one implementation and each mirroring its implementation\u0026rsquo;s full public surface. Evaluate this design.\nAnswer These are header interfaces, described in Section 8.7. They provide syntactic indirection without reducing coupling, because their shape is determined by the implementation rather than by any client\u0026rsquo;s requirement, and any change to an implementation propagates to its interface and then to all clients. They also constitute speculative generality if no second implementation is anticipated. The defensible exception is when the interface enables test substitution at a genuine infrastructure boundary. For internal application services, the interfaces should be removed, and reintroduced later as narrow role interfaces defined by the clients that need them.\nQ8. Two classes have been separate for a year. Version control shows they have been modified in the same commit on every occasion. What does this indicate?\nAnswer It indicates that the original separation did not follow an actor boundary, so the two classes share a reason to change and should probably be merged. This is the empirical test described in Section 13.7: design principles produce predictions about future change, and the repository records what actually happened. Consistent co-modification over a long period is strong evidence that the predicted boundary was incorrect. The counter-consideration is that the classes may be co-changing because of a third, poorly placed dependency, in which case relocating that dependency is the better remedy.\nQ9. Why is the Liskov Substitution Principle described as occupying a different position from the other four?\nAnswer The other four are structural directives that describe how code should be arranged: how it is partitioned, how it is extended, how wide its interfaces are, and which direction its dependencies point. LSP is a correctness condition on the arrangement rather than a directive about it. If subtypes do not honour their contracts, then abstractions cannot be trusted, and every design built on those abstractions inherits the defect: OCP produces extension points that are unreliable, and DIP produces indirection without genuine substitutability. LSP is therefore a precondition for the other principles delivering their claimed benefits.\nQ10. A senior engineer rejects a pull request that introduces four new interfaces to support a single feature. What questions should the reviewer be asking?\nAnswer How many implementations does each interface have, counting test implementations separately from production ones. Which documented or observed axis of change does each interface protect. What would break if the interface were removed and the concrete class used directly. Is each interface shaped by a client\u0026rsquo;s requirement or by an implementation\u0026rsquo;s public surface. Has a similar abstraction in this codebase previously been introduced and never used. These questions convert the review from a matter of taste into an examination of evidence, which is the only basis on which abstraction decisions can be settled reliably.\n17. Practice Exercises with Solutions Attempt each exercise before reading the solution. Solutions include the reasoning behind the design choices, not only the resulting code, because the reasoning is the transferable part.\nExercise 1: Diagnose and repair an SRP violation Identify every responsibility in the class below, name the actor associated with each, and restructure accordingly.\nclass Invoice: def __init__(self, items): self.items = items # list of (name, price, quantity) def calculate_total(self): return sum(price * quantity for _, price, quantity in self.items) def calculate_tax(self): return self.calculate_total() * 0.18 def save_to_pdf(self, filepath): print(f\u0026#34;Writing PDF to {filepath}\u0026#34;) def send_by_email(self, recipient): print(f\u0026#34;Emailing invoice to {recipient}\u0026#34;) def record_in_ledger(self): print(\u0026#34;INSERT INTO ledger ...\u0026#34;) Solution Responsibilities and actors.\nResponsibility Actor Trigger for change Holding invoice data Domain modelling A new field such as a purchase order reference is required Monetary computation Finance The tax rate changes, or tax becomes jurisdiction-dependent PDF rendering Presentation and branding The layout or logo changes Email delivery Infrastructure The mail provider changes Ledger persistence Platform and accounting systems The schema or storage engine changes Five actors, therefore five reasons to change, therefore five responsibilities in one class.\nclass InvoiceLine: def __init__(self, description: str, unit_price: float, quantity: int): self.description = description self.unit_price = unit_price self.quantity = quantity @property def line_total(self) -\u0026gt; float: return self.unit_price * self.quantity class Invoice: \u0026#34;\u0026#34;\u0026#34; Holds invoice state and computes only what is intrinsic to it. The subtotal is intrinsic, because it is a direct consequence of the lines present. Tax is not intrinsic, because it depends on jurisdiction and on rules owned outside the invoice. \u0026#34;\u0026#34;\u0026#34; def __init__(self, lines: list[InvoiceLine]): self.lines = lines @property def subtotal(self) -\u0026gt; float: return sum(line.line_total for line in self.lines) class TaxCalculator: \u0026#34;\u0026#34;\u0026#34; Owned by Finance. Made a class rather than a constant so that jurisdiction-specific subclasses or configured rates can be introduced later without changing any caller. \u0026#34;\u0026#34;\u0026#34; def __init__(self, rate: float = 0.18): self.rate = rate def tax_for(self, invoice: Invoice) -\u0026gt; float: return invoice.subtotal * self.rate class InvoicePdfRenderer: def render(self, invoice: Invoice, tax: float, filepath: str) -\u0026gt; None: # Receives the computed tax rather than a TaxCalculator, so that it # cannot recompute or reinterpret monetary values. This keeps the # two responsibilities from silently merging again. print(f\u0026#34;PDF at {filepath}: subtotal {invoice.subtotal}, tax {tax}\u0026#34;) class InvoiceEmailSender: def send(self, recipient: str, filepath: str) -\u0026gt; None: print(f\u0026#34;Emailing {filepath} to {recipient}\u0026#34;) class LedgerRepository: def record(self, invoice: Invoice, tax: float) -\u0026gt; None: print(f\u0026#34;Recording {invoice.subtotal + tax} in the ledger\u0026#34;) Note on the subtotal decision. Leaving subtotal on Invoice while extracting tax is deliberate. The subtotal is a direct arithmetic consequence of the invoice\u0026rsquo;s own contents and changes only when the definition of an invoice line changes. Tax depends on external rules and changes when a tax authority acts. Extracting the subtotal as well would produce the anaemic model described in Section 5.8.\nExercise 2: Apply OCP, then evaluate the alternatives The class below requires modification whenever a new shape is supported. Refactor it, then state under what circumstances a simpler approach would have been preferable.\nclass AreaCalculator: def total_area(self, shapes): total = 0 for shape in shapes: if shape[\u0026#34;type\u0026#34;] == \u0026#34;circle\u0026#34;: total += 3.14159 * shape[\u0026#34;radius\u0026#34;] ** 2 elif shape[\u0026#34;type\u0026#34;] == \u0026#34;square\u0026#34;: total += shape[\u0026#34;side\u0026#34;] ** 2 elif shape[\u0026#34;type\u0026#34;] == \u0026#34;rectangle\u0026#34;: total += shape[\u0026#34;width\u0026#34;] * shape[\u0026#34;height\u0026#34;] return total Solution from abc import ABC, abstractmethod import math class Shape(ABC): \u0026#34;\u0026#34;\u0026#34; The abstraction promises only area computation, which every shape can honour unconditionally. Adding a perimeter method would force every future shape to implement it, so operations are added here only when all conceivable shapes can supply them. \u0026#34;\u0026#34;\u0026#34; @abstractmethod def area(self) -\u0026gt; float: ... class Circle(Shape): def __init__(self, radius: float): self.radius = radius def area(self): return math.pi * self.radius ** 2 class Square(Shape): def __init__(self, side: float): self.side = side def area(self): return self.side ** 2 class Rectangle(Shape): def __init__(self, width: float, height: float): self.width = width self.height = height def area(self): return self.width * self.height class AreaCalculator: def total_area(self, shapes: list[Shape]) -\u0026gt; float: # No knowledge of any concrete shape. Adding a triangle requires # writing one new class and editing nothing. return sum(shape.area() for shape in shapes) When a simpler approach would suffice. If the three shapes were fixed by the problem domain and no fourth could arise, the original conditional would be acceptable and shorter. The refactoring is justified here by two additional benefits beyond extensibility. First, the dictionary representation permitted invalid data, such as a circle without a radius, which the class-based version prevents at construction. Second, the original computed area with a truncated value of pi, an error that is easy to miss in a conditional chain and easy to isolate and test in a dedicated class.\nExercise 3: Determine whether LSP is violated For each subclass, state whether LSP is violated and identify the specific rule involved.\nclass Account: \u0026#34;\u0026#34;\u0026#34;Contract: balance never becomes negative; withdraw raises on insufficient funds.\u0026#34;\u0026#34;\u0026#34; def __init__(self, balance: float): self._balance = balance def withdraw(self, amount: float) -\u0026gt; float: if amount \u0026gt; self._balance: raise ValueError(\u0026#34;Insufficient funds\u0026#34;) self._balance -= amount return self._balance class AuditedAccount(Account): def withdraw(self, amount): print(f\u0026#34;Audit: withdrawal of {amount}\u0026#34;) return super().withdraw(amount) class OverdraftAccount(Account): def withdraw(self, amount): self._balance -= amount # Permits a negative balance return self._balance class MinimumBalanceAccount(Account): def withdraw(self, amount): if self._balance - amount \u0026lt; 1000: raise ValueError(\u0026#34;Balance would fall below the minimum\u0026#34;) self._balance -= amount return self._balance Solution Subclass Violates LSP Rule involved Explanation AuditedAccount No None A side effect is added while every guarantee is preserved. Inputs accepted and outputs produced are unchanged. This is the Decorator pattern\u0026rsquo;s structure and is always safe when the contract holds. OverdraftAccount Yes Invariant preservation The base guarantees a non-negative balance. This subclass permits a negative balance, so any client relying on the invariant, such as a report that assumes no negative totals, is broken by substitution. MinimumBalanceAccount Yes Precondition strengthening The base accepts any amount up to the current balance. This subclass rejects amounts that would breach a minimum, so inputs the base accepted are now refused, and a client that verified amount \u0026lt;= balance before calling will still encounter an exception. Resolution. The base abstraction promises too much. Restructure so that Account declares only that withdraw either succeeds or raises when its own policy forbids the withdrawal, with the specific policy left to each implementation. Clients then handle the exception in all cases rather than relying on a universal rule about balances. Alternatively, introduce an explicit WithdrawalPolicy collaborator, which converts the variation from an inheritance concern into a composition concern and eliminates the substitution question entirely.\nExercise 4: Design segregated interfaces Design the interfaces for a workshop in which a carpenter cuts wood only, a plumber fits pipes only, an electrician wires circuits only, and a general contractor performs all three. No specialist may be forced to implement another\u0026rsquo;s operation.\nSolution from abc import ABC, abstractmethod class WoodCutter(ABC): @abstractmethod def cut_wood(self, plank) -\u0026gt; None: ... class PipeFitter(ABC): @abstractmethod def fit_pipe(self, pipe) -\u0026gt; None: ... class Electrician(ABC): @abstractmethod def wire_circuit(self, circuit) -\u0026gt; None: ... class Carpenter(WoodCutter): def cut_wood(self, plank): print(f\u0026#34;Cutting {plank}\u0026#34;) class Plumber(PipeFitter): def fit_pipe(self, pipe): print(f\u0026#34;Fitting {pipe}\u0026#34;) class GeneralContractor(WoodCutter, PipeFitter, Electrician): \u0026#34;\u0026#34;\u0026#34; Multiple inheritance is used purely for interface composition. Each base contributes one abstract method and no state, so none of the usual hazards of multiple inheritance apply. \u0026#34;\u0026#34;\u0026#34; def cut_wood(self, plank): print(f\u0026#34;Contractor cutting {plank}\u0026#34;) def fit_pipe(self, pipe): print(f\u0026#34;Contractor fitting {pipe}\u0026#34;) def wire_circuit(self, circuit): print(f\u0026#34;Contractor wiring {circuit}\u0026#34;) def build_frame(planks, worker: WoodCutter) -\u0026gt; None: # The signature states the exact requirement. A plumber cannot be passed # here, and a test double needs exactly one method. for plank in planks: worker.cut_wood(plank) Alternative worth considering. Modelling capability as data rather than as type is sometimes preferable, for instance a Worker class holding a set of Skill values. That design supports runtime capability changes, such as a worker gaining a certification, which a type hierarchy cannot express without replacing the object. The trade-off is that capability errors then surface at runtime rather than being prevented by the type system. Choose the type-based design when capabilities are fixed at construction, and the data-based design when they vary during an object\u0026rsquo;s lifetime.\nExercise 5: Invert a dependency and make the code testable Refactor the class below so that its logic can be unit tested without a network connection and without waiting for real time to pass.\nimport requests from datetime import datetime class SubscriptionChecker: def is_active(self, user_id): response = requests.get(f\u0026#34;https://api.billing.example.com/users/{user_id}\u0026#34;) expiry = datetime.fromisoformat(response.json()[\u0026#34;expires_at\u0026#34;]) return expiry \u0026gt; datetime.now() Solution Two dependencies require inversion, and they are inverted for different reasons. The HTTP client is inverted because it is external, volatile, and slow. The clock is inverted because it is non-deterministic, and boundary conditions around expiry cannot otherwise be tested.\nfrom abc import ABC, abstractmethod from datetime import datetime class SubscriptionRepository(ABC): \u0026#34;\u0026#34;\u0026#34; Expressed in domain vocabulary. There is no mention of HTTP, URLs, or JSON, which is what permits a database-backed or in-memory implementation to satisfy it without distortion. An interface with a method named get_json would have violated clause B of DIP. \u0026#34;\u0026#34;\u0026#34; @abstractmethod def expiry_for(self, user_id: str) -\u0026gt; datetime | None: ... class Clock(ABC): @abstractmethod def now(self) -\u0026gt; datetime: ... class SystemClock(Clock): def now(self): return datetime.now() class FixedClock(Clock): \u0026#34;\u0026#34;\u0026#34;Test implementation permitting exact control of boundary conditions.\u0026#34;\u0026#34;\u0026#34; def __init__(self, moment: datetime): self.moment = moment def now(self): return self.moment class HttpSubscriptionRepository(SubscriptionRepository): \u0026#34;\u0026#34;\u0026#34; The only class that knows the billing service exists. All HTTP concerns, including retries, timeouts, and response parsing, are confined here. \u0026#34;\u0026#34;\u0026#34; def __init__(self, http_client, base_url: str): self.http_client = http_client self.base_url = base_url def expiry_for(self, user_id): response = self.http_client.get(f\u0026#34;{self.base_url}/users/{user_id}\u0026#34;) payload = response.json() if payload.get(\u0026#34;expires_at\u0026#34;) is None: return None return datetime.fromisoformat(payload[\u0026#34;expires_at\u0026#34;]) class InMemorySubscriptionRepository(SubscriptionRepository): def __init__(self, expiries: dict[str, datetime]): self.expiries = expiries def expiry_for(self, user_id): return self.expiries.get(user_id) class SubscriptionChecker: \u0026#34;\u0026#34;\u0026#34; Contains the business rule and nothing else. The rule is that a subscription is active when an expiry exists and lies in the future. That rule is now independently testable. \u0026#34;\u0026#34;\u0026#34; def __init__(self, repository: SubscriptionRepository, clock: Clock): self.repository = repository self.clock = clock def is_active(self, user_id: str) -\u0026gt; bool: expiry = self.repository.expiry_for(user_id) if expiry is None: return False return expiry \u0026gt; self.clock.now() Boundary behaviour can now be tested exactly.\ndef test_subscription_expiring_exactly_now_is_inactive(): moment = datetime(2026, 1, 1, 12, 0, 0) repository = InMemorySubscriptionRepository({\u0026#34;u1\u0026#34;: moment}) checker = SubscriptionChecker(repository, FixedClock(moment)) # The rule uses a strict comparison, so an expiry equal to the current # instant is inactive. This boundary is untestable without an # injectable clock, and it is exactly the kind of case that produces # production defects. assert checker.is_active(\u0026#34;u1\u0026#34;) is False Exercise 6: Decide whether abstraction is warranted A developer proposes introducing ILoggerFactory, ILogFormatter, ILogSink, and ILogLevelResolver for an internal tool that writes messages to standard output. The tool is used by four people and has no plan for structured logging. Evaluate the proposal.\nAnswer The proposal should be rejected in its current form, and the reasoning matters more than the conclusion.\nEvidence against. Each abstraction would have exactly one implementation with no documented second, which is the definition of speculative generality given in Section 13.1. The axis of change has not been demonstrated: no requirement for structured logging, alternative sinks, or configurable levels exists. Four abstractions to write a line to standard output inverts the ratio of ceremony to value, and the resulting indirection imposes a permanent comprehension cost on every reader.\nEvidence that might change the conclusion. If the tool\u0026rsquo;s output must be asserted in tests, a single narrow abstraction over the sink is warranted, since a test implementation is a genuine second implementation. If the tool is a library consumed by other teams, allowing callers to supply a logger is a reasonable interface decision rather than speculative abstraction.\nRecommended response. Use the standard library\u0026rsquo;s logging module directly, which already provides handler and formatter extensibility without any code being written. This illustrates a general point that is frequently overlooked: before building an abstraction, check whether the platform already supplies one.\nExercise 7: Full refactoring with justification The class below violates all five principles. Refactor it, and for each abstraction you introduce, state the axis of change that justifies it. Any abstraction you cannot justify should not be introduced.\nclass ReportManager: def generate(self, report_type, start_date, end_date, output_format, email_to): if report_type == \u0026#34;sales\u0026#34;: rows = self._query(\u0026#34;SELECT * FROM sales WHERE date BETWEEN ...\u0026#34;) elif report_type == \u0026#34;inventory\u0026#34;: rows = self._query(\u0026#34;SELECT * FROM inventory WHERE date BETWEEN ...\u0026#34;) if output_format == \u0026#34;csv\u0026#34;: content = \u0026#34;,\u0026#34;.join(str(r) for r in rows) elif output_format == \u0026#34;html\u0026#34;: content = \u0026#34;\u0026lt;table\u0026gt;\u0026#34; + \u0026#34;\u0026#34;.join(f\u0026#34;\u0026lt;tr\u0026gt;{r}\u0026lt;/tr\u0026gt;\u0026#34; for r in rows) + \u0026#34;\u0026lt;/table\u0026gt;\u0026#34; print(f\u0026#34;Sending report to {email_to}\u0026#34;) return content def _query(self, sql): print(f\u0026#34;Executing {sql}\u0026#34;) return [1, 2, 3] Solution Axes of change, established before any code is written.\nAxis Evidence that it will change Abstraction justified Report types New reports are requested routinely by the business Yes, a ReportSource abstraction Output formats Formats are added when new consumers appear, such as an auditor requiring PDF Yes, a ReportFormatter abstraction Delivery mechanism Delivery may move to file storage or a message queue, and tests must not send email Yes, a ReportDelivery abstraction Date range handling The concept of a date range is stable No abstraction; a value object suffices The report workflow itself The sequence of fetch, format, deliver is stable No abstraction; a concrete orchestrator suffices The final two rows are the important part of the answer. Recognising which axes do not warrant abstraction is as much a part of the exercise as recognising those that do.\nfrom abc import ABC, abstractmethod from datetime import date class DateRange: \u0026#34;\u0026#34;\u0026#34;A value object. Concrete because the concept does not vary.\u0026#34;\u0026#34;\u0026#34; def __init__(self, start: date, end: date): if end \u0026lt; start: raise ValueError(\u0026#34;End date cannot precede start date\u0026#34;) self.start = start self.end = end class ReportSource(ABC): \u0026#34;\u0026#34;\u0026#34;Axis: new report types. One implementation per report.\u0026#34;\u0026#34;\u0026#34; @abstractmethod def rows_for(self, period: DateRange) -\u0026gt; list[dict]: ... class ReportFormatter(ABC): \u0026#34;\u0026#34;\u0026#34;Axis: new output formats. Formatting is independent of data source.\u0026#34;\u0026#34;\u0026#34; @abstractmethod def format(self, rows: list[dict]) -\u0026gt; str: ... class ReportDelivery(ABC): \u0026#34;\u0026#34;\u0026#34;Axis: new delivery channels, and the need for a test implementation.\u0026#34;\u0026#34;\u0026#34; @abstractmethod def deliver(self, content: str, destination: str) -\u0026gt; None: ... class SalesReportSource(ReportSource): def __init__(self, database): # The database is injected rather than constructed, so this class is # itself testable and is not bound to a specific engine. self.database = database def rows_for(self, period): return self.database.query( \u0026#34;SELECT * FROM sales WHERE date BETWEEN ? AND ?\u0026#34;, period.start, period.end, ) class CsvFormatter(ReportFormatter): def format(self, rows): if not rows: return \u0026#34;\u0026#34; headers = list(rows[0].keys()) lines = [\u0026#34;,\u0026#34;.join(headers)] lines.extend(\u0026#34;,\u0026#34;.join(str(row[h]) for h in headers) for row in rows) return \u0026#34;\\n\u0026#34;.join(lines) class HtmlFormatter(ReportFormatter): def format(self, rows): body = \u0026#34;\u0026#34;.join( \u0026#34;\u0026lt;tr\u0026gt;\u0026#34; + \u0026#34;\u0026#34;.join(f\u0026#34;\u0026lt;td\u0026gt;{value}\u0026lt;/td\u0026gt;\u0026#34; for value in row.values()) + \u0026#34;\u0026lt;/tr\u0026gt;\u0026#34; for row in rows ) return f\u0026#34;\u0026lt;table\u0026gt;{body}\u0026lt;/table\u0026gt;\u0026#34; class EmailDelivery(ReportDelivery): def __init__(self, mail_client): self.mail_client = mail_client def deliver(self, content, destination): self.mail_client.send(destination, content) class CapturingDelivery(ReportDelivery): \u0026#34;\u0026#34;\u0026#34;Test implementation. Its existence is the second implementation that justifies the ReportDelivery abstraction independently of future channels.\u0026#34;\u0026#34;\u0026#34; def __init__(self): self.delivered: list[tuple[str, str]] = [] def deliver(self, content, destination): self.delivered.append((destination, content)) class ReportService: \u0026#34;\u0026#34;\u0026#34; Concrete, because the workflow does not vary. Its single reason to change is a change to the sequence of steps, for instance if reports must be archived before delivery. \u0026#34;\u0026#34;\u0026#34; def __init__(self, source: ReportSource, formatter: ReportFormatter, delivery: ReportDelivery): self.source = source self.formatter = formatter self.delivery = delivery def produce(self, period: DateRange, destination: str) -\u0026gt; str: rows = self.source.rows_for(period) content = self.formatter.format(rows) self.delivery.deliver(content, destination) return content Principle verification.\nPrinciple How it is satisfied SRP Data retrieval, formatting, delivery, and workflow coordination are four separate concerns owned by four different groups, and each occupies its own class. OCP New report types and new formats are new classes. ReportService is never edited. LSP Every formatter accepts any list of rows and returns a string; every source accepts any date range. No implementation restricts inputs its abstraction accepts. ISP Each interface declares one method, matching what each client actually invokes. DIP ReportService names no database, no template engine, and no mail transport. SalesReportSource receives its database rather than constructing it. On the two rejected abstractions. A DateRangeStrategy would have added indirection for a concept with no variation. A ReportWorkflow interface would have added a substitution point for a sequence that no requirement suggests will vary. Declining to build them is a design decision of the same weight as building the three that were justified.\n18. Where SOLID Leads Next The five principles describe properties that a good design exhibits. They do not supply a catalogue of concrete solutions to recurring problems. That catalogue is the subject of design patterns, and the relationship between the two bodies of knowledge is direct: most patterns are named, reusable structures that realise one or more SOLID principles for a specific class of problem.\nPattern Principles it primarily realises Problem it addresses Strategy OCP, DIP Selecting among interchangeable algorithms at runtime Factory Method and Abstract Factory OCP, DIP Deferring the choice of concrete class to a single, isolated location Adapter DIP, LSP Making an existing incompatible interface conform to a required abstraction Decorator OCP, LSP, SRP Adding behaviour to an object without modifying it or its class Observer OCP, SRP Notifying an open-ended set of dependents without the subject knowing them Template Method OCP, LSP Fixing an algorithm\u0026rsquo;s skeleton while allowing specific steps to vary Repository DIP, SRP Separating domain logic from persistence mechanism Composite LSP, OCP Treating individual objects and compositions of objects uniformly Studying patterns before the principles produces memorisation of structures without understanding of purpose, which is why patterns applied in that state tend to be applied where they do not belong. Studying the principles first, as this document has done, means each pattern arrives as a familiar idea in a specific form: Strategy is recognisable immediately as the OCP resolution from Section 6.6, and Repository is recognisable as the DIP resolution from Section 9.4.\nThe next documents in this series treat the creational, structural, and behavioural pattern families in that light.\nThis content is part of Codeverra, a platform for learning coding, data science, DSA, and AI from scratch. Explore more: https://codeverra.com\n","permalink":"https://blogs.codeverra.com/blog/low_level_design/02_solid_principles/","summary":"\u003ch1 id=\"solid-principles-a-complete-deep-dive\"\u003eSOLID Principles: A Complete Deep Dive\u003c/h1\u003e\n\u003cblockquote\u003e\n\u003cp\u003e\u003cstrong\u003ePrerequisite:\u003c/strong\u003e Working familiarity with Python object-oriented programming, specifically classes, objects, inheritance, method overriding, polymorphism, and abstract base classes. If any of those terms are unfamiliar, study the OOP document first, because every argument in this document is expressed in that vocabulary.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eWhat this document delivers:\u003c/strong\u003e A rigorous treatment of the five SOLID principles. For each principle you will find its formal statement, a precise interpretation of every term in that statement, the concrete engineering problem it solves, a worked example of the problem and its resolution, the alternative techniques available for the same goal, the costs of applying the principle, and the conditions under which it should not be applied.\u003c/p\u003e","tags":["lld","solid","python","oop"],"title":"SOLID Principles: A Complete Deep Dive"},{"content":"SOLID Principles: Practice Workbook Prerequisite: 02_solid_principles.md. This workbook assumes the five principles have already been defined and demonstrated. It contains no new theory. Its purpose is to convert conceptual knowledge into diagnostic skill.\nTable of Contents How to Use This Workbook Vocabulary and the Diagnostic Method Section A: Recognition Drills Section B: Single Responsibility Principle Section C: Open/Closed Principle Section D: Liskov Substitution Principle Section E: Interface Segregation Principle Section F: Dependency Inversion Principle Section G: Combined Violations Section H: Capstone Refactor Self-Review Checklist Summary and Key Takeaways Appendix: Violation Signature Reference 1. How to Use This Workbook 1.1 What this document is for Knowing the definition of the Single Responsibility Principle and being able to detect a Single Responsibility violation in unfamiliar code are two different competencies. The first is declarative knowledge and can be acquired by reading. The second is a diagnostic skill and can only be acquired by repeated exposure to examples, followed by correction.\nThis workbook supplies that exposure. It contains 24 problems arranged so that each principle is encountered three times at increasing levels of difficulty, then encountered again in combination with other principles, and finally encountered inside a single large class that violates all five at once. The last arrangement is the one that occurs in production code and in design interviews.\nThe objective is stated precisely as follows. On completing this workbook, a learner presented with an unfamiliar class of 40 to 60 lines should be able to state which principles are under strain, justify each claim by pointing to specific lines, and describe the direction of the correction without needing to consult reference material.\n1.2 The three difficulty levels Every section progresses through three levels. The levels are not merely longer problems; they exercise different cognitive operations.\nLevel Name Input given to the learner Operation exercised Expected time 1 Recognition A short code fragment, 10 to 20 lines Identify the violated principle and state the reason in one or two sentences. No code is written. 2 to 4 minutes 2 Correction A defective class, 20 to 45 lines Diagnose the defect, then produce corrected code that preserves existing behaviour 15 to 25 minutes 3 Construction A prose requirement only, no code Derive the class structure independently, define the abstractions, and justify the design 25 to 45 minutes Level 1 tests whether the violation signature is recognised. Level 2 tests whether the correction is known. Level 3 tests whether the principle has been internalised well enough to prevent the violation from occurring in the first place, which is the only level that transfers directly to real work.\nLearners who find Level 2 comfortable should not treat Level 3 as optional. Correcting supplied code is a substantially easier task than designing structure from a requirement, because the supplied code already contains the domain vocabulary, the method names, and the data shapes. At Level 3 none of that is given.\n1.3 Working method Each problem states its requirements completely before the solution appears. The solution is preceded by a horizontal rule and a ### Solution heading so that it can be scrolled past. The recommended procedure is:\nRead the problem statement and the code, if code is supplied. Write the diagnosis in your own words before writing any code. A diagnosis that cannot be written in two sentences is usually incomplete. Produce the correction or the design. Read the solution and compare, paying attention to the reasoning rather than to superficial differences in naming. Differences in class names, in the choice between abc.ABC and typing.Protocol, and in the placement of small helper methods are not errors. Differences in where the dependencies are created, in which class owns which decision, and in whether new behaviour requires editing existing code are errors, and they are the differences worth studying.\n1.4 Recommended schedule The workbook is designed for two passes separated by at least one week.\nPass Timing Method Purpose First Immediately after studying 02_solid_principles.md Reference material permitted for Level 3 problems Establish the vocabulary and the correction patterns Second Seven to ten days later No reference material, no notes Measure retention and expose the principles that were memorised rather than understood The difference between the two passes is the useful measurement. A principle that is diagnosed correctly in the first pass and incorrectly in the second was never learned; it was recalled.\n2. Vocabulary and the Diagnostic Method 2.1 Terms used throughout the solutions Precise vocabulary is what allows a diagnosis to be stated in two sentences instead of two paragraphs. The following terms appear repeatedly in the solutions and are defined here for reference.\nTerm Definition Responsibility A single reason for a module to change, expressed as the concern of one stakeholder role. Formatting rules are the concern of a presentation owner; storage schema is the concern of a database owner. These are two responsibilities regardless of how few lines each occupies. Axis of change An independent direction along which a requirement can vary. Payment provider, output format, and persistence technology are three axes. A class touched by movement along more than one axis holds more than one responsibility. Contract The complete set of promises a callable makes: what it requires before being invoked, what it guarantees afterwards, and what it may raise. A contract includes obligations that are documented informally or implied by the type signature, not only those enforced by the interpreter. Precondition A condition that must hold before a method is invoked for the method to behave as specified. Example: withdraw(amount) requires amount \u0026gt; 0. Postcondition A condition the method guarantees on return. Example: after a successful withdraw(amount), the balance has decreased by exactly amount. Invariant A condition true of an object throughout its observable lifetime. Example: a savings account balance never falls below zero. Abstraction A named contract with no implementation commitment, expressed in Python as an abstract base class (abc.ABC) or a structural type (typing.Protocol). Role interface An abstraction defined by what a specific client needs rather than by what an implementer can do. Readable is a role interface; DataManager with six unrelated methods is not. Seam A point at which behaviour can be substituted without editing the surrounding code. A constructor parameter accepting an abstraction is a seam; a constructor body that instantiates a concrete class is not. Composition root The single location, normally the application entry point, at which concrete implementations are selected and assembled into an object graph. Every other module receives its collaborators rather than choosing them. Change magnet A class that appears in the diff of most feature branches. Its presence is empirical evidence of a responsibility or extensibility defect, independent of any reading of the code. Shotgun surgery The condition in which a single logical change requires coordinated edits across many files. It is the symptom that results when one responsibility has been scattered rather than localised. 2.2 A repeatable scan Diagnosis is faster and more reliable when it follows a fixed order rather than an intuition. The following five questions, asked in this sequence, will locate the majority of violations in a class of moderate size. The ordering matters: responsibility defects are addressed first because they frequently cause the others, and a class that has been correctly split often turns out to have no remaining extensibility or dependency problem.\nEnumerate the reasons this class would be edited. List them as concrete requirement changes, not as abstract categories. If the list has more than one entry, the Single Responsibility Principle is under strain. Search for conditional chains keyed on a type, a mode, or a string constant. Each such chain is a location that must be edited when a new variant appears, which is an Open/Closed defect. Inspect every override for narrowed behaviour. An override that raises an exception the base does not declare, that rejects inputs the base accepts, or that guarantees less than the base promises is a Liskov violation. Inspect every implementer for unused inherited obligations. A method implemented as pass, as return None, or as raise NotImplementedError indicates that the interface obliges the implementer to declare capability it does not have, which is an Interface Segregation defect. Locate every point at which the class obtains a collaborator. A collaborator constructed inside the class, imported as a module-level singleton, or read from global configuration is a Dependency Inversion defect, because it removes the seam at which substitution would occur. 2.3 The relationship between the answers The five questions are not independent, and the solutions in this workbook repeatedly rely on the following relationships.\nObservation Principle primarily violated Principle usually violated as a consequence One class formats, stores, and transmits SRP DIP, because storage and transport are concrete infrastructure Conditional chain on a variant code OCP SRP, because each branch is a distinct policy living in a shared method Subclass raises NotImplementedError ISP, if the base declared too much LSP, because callers holding the base type now fail unpredictably Caller uses isinstance before calling a method LSP OCP, because each new subtype adds a branch to the guard Dependency constructed in __init__ DIP None directly, but it makes the class untestable in isolation The practical consequence is that a diagnosis naming one violation is usually incomplete. Section G and Section H are built specifically to train the habit of enumerating all of them.\n3. Section A: Recognition Drills Five short fragments follow. For each one, name the principle under strain and state the reason in a single sentence. No code is to be written. The intended time is two to four minutes per fragment, which is short enough to force pattern recognition rather than deliberation.\nThe fragments are drawn from five different domains deliberately. A learner who practises on a single domain begins to recognise the domain rather than the defect.\nDrill A.1 class PatientRecord: def __init__(self, patient_id: str, name: str, contact: str): self.patient_id = patient_id self.name = name self.contact = contact self.visits = [] def update_contact(self, new_contact: str) -\u0026gt; None: self.contact = new_contact def add_visit(self, visit) -\u0026gt; None: self.visits.append(visit) def send_discharge_email(self, smtp_host: str, smtp_user: str) -\u0026gt; None: import smtplib server = smtplib.SMTP(smtp_host) server.login(smtp_user, \u0026#34;hunter2\u0026#34;) server.sendmail(smtp_user, self.contact, f\u0026#34;Discharge summary for {self.name}\u0026#34;) server.quit() def generate_pdf_summary(self, output_path: str) -\u0026gt; None: with open(output_path, \u0026#34;wb\u0026#34;) as handle: handle.write(b\u0026#34;%PDF-1.4 ... rendered visit history ...\u0026#34;) def save_to_database(self, connection) -\u0026gt; None: connection.execute( \u0026#34;INSERT INTO patients (id, name, contact) VALUES (?, ?, ?)\u0026#34;, (self.patient_id, self.name, self.contact), ) Question: Which principle is violated, and why?\nSolution A.1 Single Responsibility Principle.\nThe class holds four independent reasons to change: the structure of patient data, the mail transport mechanism, the layout of the printed discharge summary, and the relational schema used for persistence. A migration from SMTP to a transactional email API, a change to the PDF layout, and a change to the database schema are three unrelated requirements originating from three different stakeholder roles, yet all three arrive at this one file.\nThe direction of correction is to reduce PatientRecord to the data and the rules that govern that data, and to relocate mail transport, document rendering, and persistence into separate collaborators. Note that the mail and persistence methods also constitute a Dependency Inversion defect, because smtplib is imported and used directly rather than being reached through an abstraction. The responsibility defect is the primary one, because correcting it is what creates the classes in which the dependency can then be injected.\nDrill A.2 class ShippingCostCalculator: def calculate(self, order, courier: str) -\u0026gt; float: weight = order.total_weight_kg if courier == \u0026#34;bluedart\u0026#34;: return 40.0 + (12.0 * weight) elif courier == \u0026#34;delhivery\u0026#34;: return 35.0 + (10.5 * weight) + (20.0 if order.is_cod else 0.0) elif courier == \u0026#34;indiapost\u0026#34;: base = 25.0 + (8.0 * weight) return base * 0.5 if order.is_book_media else base elif courier == \u0026#34;fedex\u0026#34;: return 150.0 + (30.0 * weight) if order.is_international else 90.0 + (18.0 * weight) else: raise ValueError(f\u0026#34;Unsupported courier: {courier}\u0026#34;) Question: Which principle is violated, and why?\nSolution A.2 Open/Closed Principle.\nEvery additional courier requires an edit inside calculate, which means the class is not closed against a change that is certain to recur. The specific cost of this arrangement is that adding a courier requires re-testing the rate logic of every existing courier, because all of it lives in one method that has just been modified.\nThe direction of correction is to define an abstraction representing a courier rate card with a single method that returns a cost for an order, to implement one class per courier, and to reduce the calculator to a lookup and a delegation. Adding a courier then consists of adding a class and registering it, with no edit to any existing rate logic.\nA secondary observation worth making at this stage is that the four branches contain genuinely different business rules: a cash-on-delivery surcharge, a media discount, and an international multiplier. These are separate policies that have been co-located purely because they share a method, which is also a responsibility defect.\nDrill A.3 class SavingsAccount: def __init__(self, account_no: str, balance: float): self.account_no = account_no self.balance = balance def deposit(self, amount: float) -\u0026gt; None: self.balance += amount def withdraw(self, amount: float) -\u0026gt; float: if amount \u0026gt; self.balance: raise ValueError(\u0026#34;Insufficient funds\u0026#34;) self.balance -= amount return amount class FixedDepositAccount(SavingsAccount): def withdraw(self, amount: float) -\u0026gt; float: raise PermissionError(\u0026#34;Fixed deposits cannot be withdrawn before maturity\u0026#34;) def sweep_idle_funds(account: SavingsAccount, target) -\u0026gt; None: if account.balance \u0026gt; 100_000: surplus = account.balance - 100_000 account.withdraw(surplus) target.deposit(surplus) Question: Which principle is violated, and why?\nSolution A.3 Liskov Substitution Principle.\nsweep_idle_funds is written against the contract of SavingsAccount, which promises that withdraw succeeds whenever the requested amount does not exceed the balance. FixedDepositAccount is a declared subtype and is therefore accepted by the parameter, but it violates that promise unconditionally by raising PermissionError, an exception type the base contract never declares. The caller has no way to anticipate the failure short of testing the concrete type, which is precisely what subtype polymorphism is supposed to eliminate.\nThe direction of correction is to reject the inheritance relationship rather than to defend against it at the call site. A fixed deposit is not a savings account with one method disabled; it is a distinct product whose funds are not withdrawable on demand. Section D develops the two acceptable corrections in detail: separating the withdrawable capability into its own abstraction, or widening the base contract so that a refusal to withdraw is a declared and expected outcome rather than a surprise.\nDrill A.4 from abc import ABC, abstractmethod class RideVehicle(ABC): @abstractmethod def start_ride(self, rider_id: str) -\u0026gt; None: ... @abstractmethod def end_ride(self, rider_id: str) -\u0026gt; float: ... @abstractmethod def load_luggage(self, pieces: int) -\u0026gt; None: ... @abstractmethod def install_child_seat(self) -\u0026gt; None: ... @abstractmethod def enable_air_conditioning(self) -\u0026gt; None: ... class Motorbike(RideVehicle): def start_ride(self, rider_id: str) -\u0026gt; None: print(f\u0026#34;Bike ride started for {rider_id}\u0026#34;) def end_ride(self, rider_id: str) -\u0026gt; float: return 48.0 def load_luggage(self, pieces: int) -\u0026gt; None: pass def install_child_seat(self) -\u0026gt; None: pass def enable_air_conditioning(self) -\u0026gt; None: pass Question: Which principle is violated, and why?\nSolution A.4 Interface Segregation Principle.\nRideVehicle obliges every implementer to declare five capabilities, but a motorbike possesses only two of them. The three empty implementations are not harmless boilerplate; each one is a false statement about the object\u0026rsquo;s capability, and it is a statement that client code is entitled to rely on. A booking flow holding a RideVehicle reference will call install_child_seat and receive silent success, which is worse than an error because the failure surfaces later, at the kerb, rather than at the point of the mistaken call.\nThe direction of correction is to decompose the interface along the lines of client need: a Rideable abstraction carrying the two lifecycle methods, and separate LuggageCapable, ChildSeatCapable, and ClimateControlled abstractions. Motorbike then implements only Rideable, and its declared type contains no untrue promises.\nNote the coupling to the Liskov Substitution Principle. Had the three unsupported methods raised NotImplementedError instead of silently passing, the same fragment would additionally be a Liskov violation. The underlying defect is identical in both variants: the interface promises more than the implementer can deliver, and the implementer is forced to choose between lying quietly and failing loudly.\nDrill A.5 class InventoryService: def __init__(self): self.repository = MySQLInventoryRepository( host=\u0026#34;prod-inventory.internal\u0026#34;, user=\u0026#34;svc_inv\u0026#34;, password=\u0026#34;s3cr3t\u0026#34; ) self.notifier = FirebasePushSender(project_id=\u0026#34;retail-prod-4417\u0026#34;) def reserve(self, sku: str, quantity: int, customer_id: str) -\u0026gt; bool: available = self.repository.available_quantity(sku) if available \u0026lt; quantity: self.notifier.send(customer_id, f\u0026#34;{sku} is out of stock\u0026#34;) return False self.repository.decrement(sku, quantity) self.notifier.send(customer_id, f\u0026#34;Reserved {quantity} units of {sku}\u0026#34;) return True Question: Which principle is violated, and why?\nSolution A.5 Dependency Inversion Principle.\nThe reservation policy, which is the high-level concern of this class, is expressed in terms of two concrete low-level implementations that it constructs for itself. Two consequences follow directly. The reservation rule cannot be exercised in a test without a reachable MySQL instance and a valid Firebase project, because both objects are created during __init__ and the failure occurs before any policy code runs. Substituting a different data store or a different notification channel requires editing this class, even though the reservation rule itself is unchanged.\nThe direction of correction is to define InventoryRepository and Notifier abstractions expressing only what the reservation policy requires, to accept both through the constructor, and to move the selection of MySQL and Firebase to the composition root. The credentials embedded in the constructor are a separate defect of a different kind and should also be relocated to configuration, but the structural problem is the absence of a seam.\nSection A review Drill Domain Principle Signature that identifies it A.1 Hospital records SRP Data, transport, rendering, and persistence in one class A.2 E-commerce shipping OCP Conditional chain keyed on a variant string A.3 Banking LSP Override raises an exception the base does not declare A.4 Ride-hailing ISP Implementer supplies empty bodies for unsupported obligations A.5 Retail inventory DIP Collaborators constructed inside __init__ The rightmost column is the operative part of this table. Each signature is a syntactic pattern that can be spotted without understanding the domain, which is what makes rapid diagnosis possible in unfamiliar code.\n4. Section B: Single Responsibility Principle The principle under examination states that a module should have exactly one reason to change. The formal statement given by Robert C. Martin is that a module should be responsible to one, and only one, actor. The practical test used throughout this section is the enumeration described in Section 2.2: list the concrete requirement changes that would force an edit to the class, and count the distinct stakeholder roles those changes originate from.\nProblem B.1 (Level 1) A content platform contains the following class.\nclass BlogPost: def __init__(self, author_id: str): self.author_id = author_id self.title = \u0026#34;\u0026#34; self.body = \u0026#34;\u0026#34; self.tags: list[str] = [] self.published = False def set_title(self, title: str) -\u0026gt; None: if len(title) \u0026gt; 120: raise ValueError(\u0026#34;Title exceeds 120 characters\u0026#34;) self.title = title def set_body(self, body: str) -\u0026gt; None: self.body = body def format_as_html(self) -\u0026gt; str: tag_markup = \u0026#34;\u0026#34;.join(f\u0026#34;\u0026lt;span class=\u0026#39;tag\u0026#39;\u0026gt;{t}\u0026lt;/span\u0026gt;\u0026#34; for t in self.tags) return f\u0026#34;\u0026lt;article\u0026gt;\u0026lt;h1\u0026gt;{self.title}\u0026lt;/h1\u0026gt;{tag_markup}\u0026lt;div\u0026gt;{self.body}\u0026lt;/div\u0026gt;\u0026lt;/article\u0026gt;\u0026#34; def format_as_markdown(self) -\u0026gt; str: tag_line = \u0026#34; \u0026#34;.join(f\u0026#34;#{t}\u0026#34; for t in self.tags) return f\u0026#34;# {self.title}\\n\\n{tag_line}\\n\\n{self.body}\u0026#34; def save_to_db(self, conn) -\u0026gt; None: conn.execute( \u0026#34;REPLACE INTO posts (author, title, body) VALUES (?, ?, ?)\u0026#34;, (self.author_id, self.title, self.body), ) def email_to_subscribers(self, subscriber_emails: list[str]) -\u0026gt; None: for address in subscriber_emails: print(f\u0026#34;POST /sendgrid/v3/mail/send to {address}: {self.title}\u0026#34;) Questions:\nHow many responsibilities does this class hold? Name each one and identify the stakeholder role it belongs to. Which requirement changes would force an edit to this class despite having nothing to do with the data or rules of a blog post? Solution B.1 Four responsibilities, belonging to four stakeholder roles.\nResponsibility Methods Stakeholder role requesting change Example change Post state and validation rules __init__, set_title, set_body Editorial team Raise the title limit to 200 characters Presentation format_as_html, format_as_markdown Front-end or design team Wrap tags in a different element for a redesign Persistence save_to_db Database or platform team Split the posts table and change the write Subscriber notification email_to_subscribers Growth or marketing team Migrate from SendGrid to Amazon SES Answer to question 2. A redesign of the article markup, a change to the storage schema, and a change of email vendor are three requirement changes that arrive from three different teams and none of which concerns what a blog post is or what makes one valid. Each of them nevertheless forces an edit to BlogPost. The direct operational consequence is that these four teams contend for the same file, so their changes must be sequenced, reviewed together, and released together, and a defect introduced by any one of them is attributed to a class the other three also depend on.\nA useful supplementary observation concerns the naming test. The class is called BlogPost, which correctly describes only the first responsibility. A name that described the class as it currently stands would be something on the order of BlogPostWithRenderingAndStorageAndNotification, and the awkwardness of that name is a reliable signal. When an honest name for a class requires a conjunction, the class holds more than one responsibility.\nProblem B.2 (Level 2) The following class belongs to a human resources reporting tool. It works correctly. Diagnose its structural defects and produce a corrected version that preserves the existing behaviour.\nclass EmployeeReport: def __init__(self, employees): self.employees = employees def get_top_performers(self): return [e for e in self.employees if e.performance_score \u0026gt; 8.0] def format_as_table(self): header = f\u0026#34;{\u0026#39;Name\u0026#39;:\u0026lt;20} {\u0026#39;Score\u0026#39;:\u0026lt;10} {\u0026#39;Department\u0026#39;:\u0026lt;15}\\n\u0026#34; rows = \u0026#34;\\n\u0026#34;.join( f\u0026#34;{e.name:\u0026lt;20} {e.performance_score:\u0026lt;10} {e.department:\u0026lt;15}\u0026#34; for e in self.employees ) return header + rows def save_to_csv(self, filepath): import csv with open(filepath, \u0026#34;w\u0026#34;) as f: writer = csv.DictWriter(f, fieldnames=[\u0026#34;name\u0026#34;, \u0026#34;score\u0026#34;, \u0026#34;department\u0026#34;]) writer.writeheader() for e in self.employees: writer.writerow({ \u0026#34;name\u0026#34;: e.name, \u0026#34;score\u0026#34;: e.performance_score, \u0026#34;department\u0026#34;: e.department, }) def send_to_hr(self, email_address): print(f\u0026#34;Emailing report to {email_address}\u0026#34;) Required:\nEnumerate the responsibilities. Refactor into separate classes, each with one reason to change. State what each split makes possible that was not possible before. Solution B.2 Diagnosis.\nThe class holds three responsibilities, and the fourth method reveals a dependency defect that becomes visible only after the split.\nThe first responsibility is analysis: deciding which employees qualify as top performers. The threshold of 8.0 is a business rule owned by the human resources policy team, and it is hard-coded into a method that cannot be reused with a different threshold.\nThe second responsibility is presentation: converting employee records into fixed-width text. Column widths, column order, and the choice of which fields appear are owned by whoever consumes the report.\nThe third responsibility is transport and serialisation: writing CSV to a filesystem path and sending mail. These are infrastructure concerns owned by the platform team, and they carry the additional problem that save_to_csv reaches directly for the filesystem and send_to_hr reaches directly for a mail mechanism, so neither can be exercised without side effects.\nThere is a fourth defect that is a consequence of the first three. format_as_table ignores the result of get_top_performers and formats self.employees in its entirety, while the class name promises a report about top performers. When several responsibilities share mutable state, inconsistencies of this kind become easy to introduce and hard to notice, which is one of the concrete costs of the violation rather than an accident of this particular example.\nCorrected version.\nfrom __future__ import annotations import csv from dataclasses import dataclass from typing import Iterable, Protocol, Sequence @dataclass(frozen=True) class Employee: name: str performance_score: float department: str # Responsibility 1: analysis. Owns the question of who qualifies. class PerformanceAnalyser: def __init__(self, threshold: float = 8.0): self._threshold = threshold def top_performers(self, employees: Iterable[Employee]) -\u0026gt; list[Employee]: return [e for e in employees if e.performance_score \u0026gt; self._threshold] # Responsibility 2: presentation. Owns layout only, and returns text # rather than writing it anywhere. class TableFormatter: COLUMNS = (\u0026#34;Name\u0026#34;, \u0026#34;Score\u0026#34;, \u0026#34;Department\u0026#34;) def render(self, employees: Sequence[Employee]) -\u0026gt; str: header = f\u0026#34;{self.COLUMNS[0]:\u0026lt;20} {self.COLUMNS[1]:\u0026lt;10} {self.COLUMNS[2]:\u0026lt;15}\u0026#34; rows = [ f\u0026#34;{e.name:\u0026lt;20} {e.performance_score:\u0026lt;10} {e.department:\u0026lt;15}\u0026#34; for e in employees ] return \u0026#34;\\n\u0026#34;.join([header, *rows]) # Responsibility 3: serialisation to a tabular file format. class CsvExporter: FIELDS = (\u0026#34;name\u0026#34;, \u0026#34;score\u0026#34;, \u0026#34;department\u0026#34;) def export(self, employees: Sequence[Employee], filepath: str) -\u0026gt; None: with open(filepath, \u0026#34;w\u0026#34;, newline=\u0026#34;\u0026#34;) as handle: writer = csv.DictWriter(handle, fieldnames=self.FIELDS) writer.writeheader() for e in employees: writer.writerow({ \u0026#34;name\u0026#34;: e.name, \u0026#34;score\u0026#34;: e.performance_score, \u0026#34;department\u0026#34;: e.department, }) # Responsibility 4: delivery, expressed as an abstraction so that the # coordinator does not depend on any particular mail vendor. class ReportDelivery(Protocol): def deliver(self, recipient: str, subject: str, body: str) -\u0026gt; None: ... class ConsoleDelivery: def __init__(self) -\u0026gt; None: self.delivered: list[tuple[str, str, str]] = [] def deliver(self, recipient: str, subject: str, body: str) -\u0026gt; None: self.delivered.append((recipient, subject, body)) print(f\u0026#34;To {recipient}: {subject}\u0026#34;) # Coordination. Owns the sequence of steps, none of the steps themselves. class TopPerformerReport: def __init__( self, analyser: PerformanceAnalyser, formatter: TableFormatter, delivery: ReportDelivery, ): self._analyser = analyser self._formatter = formatter self._delivery = delivery def send(self, employees: Sequence[Employee], recipient: str) -\u0026gt; str: qualifying = self._analyser.top_performers(employees) body = self._formatter.render(qualifying) self._delivery.deliver(recipient, \u0026#34;Top performers\u0026#34;, body) return body What each split makes possible.\nChange request Before the split After the split Human resources raises the threshold to 8.5 for one department Edit the class that also formats and exports Construct PerformanceAnalyser(8.5) with no code change at all Report is required in JSON Add a fourth method to the same class Add a JsonFormatter class, edit nothing existing Migrate from console printing to a transactional mail API Edit the reporting class Add a class implementing ReportDelivery, select it at the composition root Verify the qualification rule in a unit test Requires an object graph that can write files and send mail Instantiate PerformanceAnalyser alone and assert on a returned list The last row is the most important, because it is the one that changes daily working conditions. The qualification rule is the part of this code most likely to contain a business defect, and after the split it is testable in three lines with no filesystem and no mail transport.\nTwo points of technique deserve comment. First, TableFormatter.render returns a string and writes nothing, which is what allows the same output to be delivered by mail, written to a file, or asserted in a test. A formatter that performs its own output has silently absorbed a second responsibility. Second, ReportDelivery is declared as a Protocol rather than an abstract base class, which means ConsoleDelivery satisfies it structurally without inheriting from it. This matters when the implementer is a third-party object that cannot be made to inherit from an abstraction defined in this codebase.\nProblem B.3 (Level 3) Scenario. A restaurant management system must support the following order lifecycle.\nA waiter creates an order against a table and adds items to it. Items may be added or removed while the order is open. When the waiter places the order, the kitchen must be notified of the items to prepare. The kitchen marks items ready, and when all items are ready the order is served. When the order is closed, the billing system must compute the payable amount including a service charge and applicable tax, an invoice must be produced in a printable form, and the customer must receive a receipt by either SMS or email according to the contact details held for the table\u0026rsquo;s guest.\nOrders may be cancelled while open, in which case no notification, no bill, and no receipt are produced. Cancellation after placement requires a manager override and must be recorded for audit.\nRequired:\nIdentify the responsibilities present in this scenario and assign each to a class. Present the design as a table of classes, responsibilities, and key method signatures. Write the code for the coordinating class and the signatures of the collaborators. Full implementations of the collaborators are not required. Write a short justification of the design decisions, stating in each case which future change the decision protects against. Solution B.3 Step 1: separating the responsibilities in the requirement text.\nThe requirement contains six distinct concerns, and identifying them from prose is the skill this problem exercises. A reliable technique is to underline each noun phrase that names an artefact or a rule, then group the verbs that act on it.\nConcern Evidence in the requirement Why it is separate Order state and its legal transitions Created, items added or removed while open, placed, served, closed, cancelled Owned by operations. Changes when the workflow changes, for example if a \u0026ldquo;held\u0026rdquo; state is introduced. Kitchen notification Kitchen must be notified of items to prepare Owned by kitchen operations. Changes when the kitchen display system changes. Bill computation Service charge and applicable tax Owned by finance and by tax regulation. Changes when a tax rate changes, which is frequent and externally imposed. Invoice rendering Printable form Owned by whoever specifies the printed document. Changes independently of the arithmetic. Receipt dispatch SMS or email according to contact details Owned by the messaging platform. Changes when a vendor changes. Audit recording Cancellation after placement must be recorded Owned by compliance. Changes when retention or record format requirements change. Step 2: the design.\nClass Responsibility Key methods OrderItem Immutable record of one ordered line Data only: menu_item_id, name, unit_price, quantity Order Holds order state and enforces legal transitions add_item, remove_item, place, mark_item_ready, serve, close, cancel, items, status OrderStatus Enumeration of the lifecycle states OPEN, PLACED, SERVED, CLOSED, CANCELLED KitchenNotifier (abstraction) Delivers a prepared-items instruction to the kitchen notify(order) -\u0026gt; None BillCalculator Computes the payable amount from an order calculate(order) -\u0026gt; Bill Bill Immutable result of a calculation Data only: subtotal, service_charge, tax, total InvoiceRenderer (abstraction) Converts an order and a bill into a document render(order, bill) -\u0026gt; str ReceiptSender (abstraction) Delivers a receipt to a contact address send(contact, document) -\u0026gt; None AuditLog (abstraction) Records events that compliance requires record(event_type, payload) -\u0026gt; None OrderService Coordinates the collaborators at the correct lifecycle moments place_order, close_order, cancel_order Step 3: the code.\nfrom __future__ import annotations from dataclasses import dataclass, field from enum import Enum from typing import Protocol class OrderStatus(Enum): OPEN = \u0026#34;open\u0026#34; PLACED = \u0026#34;placed\u0026#34; SERVED = \u0026#34;served\u0026#34; CLOSED = \u0026#34;closed\u0026#34; CANCELLED = \u0026#34;cancelled\u0026#34; @dataclass(frozen=True) class OrderItem: menu_item_id: str name: str unit_price: float quantity: int @property def line_total(self) -\u0026gt; float: return self.unit_price * self.quantity @dataclass(frozen=True) class Bill: subtotal: float service_charge: float tax: float @property def total(self) -\u0026gt; float: return self.subtotal + self.service_charge + self.tax class IllegalTransition(Exception): \u0026#34;\u0026#34;\u0026#34;Raised when a lifecycle operation is invalid for the current status.\u0026#34;\u0026#34;\u0026#34; class Order: \u0026#34;\u0026#34;\u0026#34;Holds order state and guards its own invariants. Knows nothing about kitchens, money, documents, messaging, or audit.\u0026#34;\u0026#34;\u0026#34; def __init__(self, order_id: str, table_no: int): self.order_id = order_id self.table_no = table_no self._items: list[OrderItem] = [] self._ready: set[str] = set() self._status = OrderStatus.OPEN @property def status(self) -\u0026gt; OrderStatus: return self._status @property def items(self) -\u0026gt; tuple[OrderItem, ...]: return tuple(self._items) def add_item(self, item: OrderItem) -\u0026gt; None: self._require(OrderStatus.OPEN, \u0026#34;add items\u0026#34;) self._items.append(item) def remove_item(self, menu_item_id: str) -\u0026gt; None: self._require(OrderStatus.OPEN, \u0026#34;remove items\u0026#34;) self._items = [i for i in self._items if i.menu_item_id != menu_item_id] def place(self) -\u0026gt; None: self._require(OrderStatus.OPEN, \u0026#34;place\u0026#34;) if not self._items: raise IllegalTransition(\u0026#34;Cannot place an order with no items\u0026#34;) self._status = OrderStatus.PLACED def mark_item_ready(self, menu_item_id: str) -\u0026gt; None: self._require(OrderStatus.PLACED, \u0026#34;mark items ready\u0026#34;) self._ready.add(menu_item_id) def all_items_ready(self) -\u0026gt; bool: return {i.menu_item_id for i in self._items} \u0026lt;= self._ready def serve(self) -\u0026gt; None: self._require(OrderStatus.PLACED, \u0026#34;serve\u0026#34;) if not self.all_items_ready(): raise IllegalTransition(\u0026#34;Cannot serve before all items are ready\u0026#34;) self._status = OrderStatus.SERVED def close(self) -\u0026gt; None: self._require(OrderStatus.SERVED, \u0026#34;close\u0026#34;) self._status = OrderStatus.CLOSED def cancel(self) -\u0026gt; None: if self._status in (OrderStatus.CLOSED, OrderStatus.CANCELLED): raise IllegalTransition(f\u0026#34;Cannot cancel an order that is {self._status.value}\u0026#34;) self._status = OrderStatus.CANCELLED def _require(self, expected: OrderStatus, action: str) -\u0026gt; None: if self._status is not expected: raise IllegalTransition( f\u0026#34;Cannot {action} while order is {self._status.value}\u0026#34; ) # ---------- Collaborator contracts, signatures only ---------- class KitchenNotifier(Protocol): def notify(self, order: Order) -\u0026gt; None: ... class BillCalculator(Protocol): def calculate(self, order: Order) -\u0026gt; Bill: ... class InvoiceRenderer(Protocol): def render(self, order: Order, bill: Bill) -\u0026gt; str: ... class ReceiptSender(Protocol): def send(self, contact: str, document: str) -\u0026gt; None: ... class AuditLog(Protocol): def record(self, event_type: str, payload: dict) -\u0026gt; None: ... # ---------- Coordination ---------- class OrderService: \u0026#34;\u0026#34;\u0026#34;Owns the sequence in which collaborators are invoked. Contains no kitchen logic, no arithmetic, no formatting, and no transport.\u0026#34;\u0026#34;\u0026#34; def __init__( self, kitchen: KitchenNotifier, calculator: BillCalculator, renderer: InvoiceRenderer, receipts: ReceiptSender, audit: AuditLog, ): self._kitchen = kitchen self._calculator = calculator self._renderer = renderer self._receipts = receipts self._audit = audit def place_order(self, order: Order) -\u0026gt; None: order.place() self._kitchen.notify(order) def close_order(self, order: Order, guest_contact: str) -\u0026gt; str: order.close() bill = self._calculator.calculate(order) invoice = self._renderer.render(order, bill) self._receipts.send(guest_contact, invoice) return invoice def cancel_order(self, order: Order, manager_id: str | None = None) -\u0026gt; None: requires_override = order.status is not OrderStatus.OPEN if requires_override and manager_id is None: raise PermissionError(\u0026#34;Manager override required after placement\u0026#34;) previous = order.status order.cancel() if requires_override: self._audit.record( \u0026#34;order_cancelled_after_placement\u0026#34;, { \u0026#34;order_id\u0026#34;: order.order_id, \u0026#34;previous_status\u0026#34;: previous.value, \u0026#34;manager_id\u0026#34;: manager_id, }, ) Step 4: design decisions and the changes they protect against.\nThe lifecycle rules live in Order, not in OrderService. The condition that an order cannot be served before all its items are ready is an invariant of an order, and placing it in the object that owns the state means no code path can bypass it. Had the check lived in OrderService.serve_order, a second caller added later, for example a bulk-close routine at end of shift, would be able to reach an inconsistent state. This decision protects against invariant violations introduced by new callers.\nOrderService contains sequencing and nothing else. Every method in it reads as a short list of steps delegated elsewhere. This is the correct shape for a coordinator, and it provides a test that can be applied to any coordinator: if arithmetic, string formatting, or if statements about business values appear in it, a responsibility has leaked in.\nBill calculation is separated from invoice rendering. These are frequently combined because both concern the printed bill, and the separation is the single decision in this design most likely to be omitted. They change for unrelated reasons. A change in the tax rate is imposed externally, must be applied on a specific date, and must be verifiable by finance. A change in the printed layout comes from operations and has no financial consequence. Keeping them separate means the tax change is a two-line edit to a class whose entire content is arithmetic and is therefore straightforward to test exhaustively.\nReceipt dispatch is one abstraction, not two. The requirement mentions SMS and email, which suggests two classes. The correct reading is that both are instances of the same role, delivering a document to a contact address, so both implement ReceiptSender. The decision of which to use for a given guest is a selection among implementations and belongs either at the composition root or in a small dispatching implementation of the same interface. This protects against the arrival of a third channel, which then requires no change to OrderService.\nAudit recording is invoked by the coordinator rather than by Order. Auditing is an obligation of the enclosing process, not a property of an order. If Order.cancel wrote audit records, then constructing an order in a test would require an audit sink, and the domain object would depend on infrastructure.\nCancellation before placement produces no bill, no receipt, and no audit record. This is visible in the code as the absence of calls rather than as a conditional, because the requirement was read carefully: the manager override and the audit record are both consequences of the same condition, that the order has already been placed. Deriving requires_override once and using it twice keeps the two consequences in agreement, which is preferable to writing two independent status checks that a later edit could allow to diverge.\nSection B review Problem Domain Level Central lesson B.1 Content platform 1 Count stakeholder roles, not lines. Four short methods can be four responsibilities. B.2 Human resources 2 Formatters return values rather than performing output, which is what makes them reusable and testable. B.3 Restaurant 3 Invariants belong to the object holding the state. Coordinators sequence and delegate only. 5. Section C: Open/Closed Principle The principle under examination states that a module should be open for extension and closed for modification. New behaviour should be addable by writing new code rather than by editing code that already works and is already tested.\nThe practical significance is a matter of risk rather than of elegance. Code that has been in production for a year has been validated by that year of operation. Editing it discards part of that validation, because the tests that pass now were written against the previous behaviour. Adding a new class alongside it discards none of it. The Open/Closed Principle is therefore best understood as a technique for confining the blast radius of a change.\nProblem C.1 (Level 1) A payments service contains the following method.\nclass PaymentProcessor: def process(self, payment) -\u0026gt; dict: if payment.method == \u0026#34;card\u0026#34;: self._validate_card(payment.card_number, payment.cvv) fee = payment.amount * 0.02 return {\u0026#34;status\u0026#34;: \u0026#34;captured\u0026#34;, \u0026#34;fee\u0026#34;: fee, \u0026#34;gateway\u0026#34;: \u0026#34;razorpay\u0026#34;} elif payment.method == \u0026#34;upi\u0026#34;: if not payment.vpa.endswith((\u0026#34;@okhdfcbank\u0026#34;, \u0026#34;@ybl\u0026#34;, \u0026#34;@paytm\u0026#34;)): return {\u0026#34;status\u0026#34;: \u0026#34;rejected\u0026#34;, \u0026#34;reason\u0026#34;: \u0026#34;unsupported VPA\u0026#34;} return {\u0026#34;status\u0026#34;: \u0026#34;captured\u0026#34;, \u0026#34;fee\u0026#34;: 0.0, \u0026#34;gateway\u0026#34;: \u0026#34;npci\u0026#34;} elif payment.method == \u0026#34;netbanking\u0026#34;: fee = 15.0 return {\u0026#34;status\u0026#34;: \u0026#34;pending\u0026#34;, \u0026#34;fee\u0026#34;: fee, \u0026#34;gateway\u0026#34;: payment.bank_code} elif payment.method == \u0026#34;wallet\u0026#34;: if payment.wallet_balance \u0026lt; payment.amount: return {\u0026#34;status\u0026#34;: \u0026#34;rejected\u0026#34;, \u0026#34;reason\u0026#34;: \u0026#34;insufficient balance\u0026#34;} return {\u0026#34;status\u0026#34;: \u0026#34;captured\u0026#34;, \u0026#34;fee\u0026#34;: 0.0, \u0026#34;gateway\u0026#34;: \u0026#34;internal\u0026#34;} else: raise ValueError(f\u0026#34;Unknown payment method: {payment.method}\u0026#34;) def _validate_card(self, number: str, cvv: str) -\u0026gt; None: if len(number) != 16 or len(cvv) != 3: raise ValueError(\u0026#34;Invalid card\u0026#34;) Questions:\nWhich principle is violated? Describe precisely what must happen when a cryptocurrency method is added, and state the risk that arises. What is the direction of the correction? Solution C.1 Answer to question 1. The Open/Closed Principle. process must be modified for every new payment method, so the class is not closed against the change that the business is most certain to request.\nAnswer to question 2. Adding a cryptocurrency method requires inserting a branch into process, which means editing a method that currently handles four working payment flows. Three risks follow, and they should be stated in this order of severity.\nThe first is regression risk. Card, UPI, netbanking, and wallet payments all flow through the modified method, so a mistake made while adding the new branch can break a payment method that is currently taking live money. The change cannot be reviewed in isolation, because the reviewer must reason about the whole method.\nThe second is test invalidation. The existing tests for process were written against a method that no longer exists in the same form. They may still pass, but they no longer constitute complete coverage of the method as modified, and determining what additional coverage is required is manual work that grows with each branch.\nThe third is deployment coupling. The cryptocurrency feature cannot be released independently of the existing payment methods, because it ships inside the same class. If the new method must be disabled after release, the rollback affects code paths that were working.\nAnswer to question 3. Each payment method becomes a class implementing a common abstraction with a single method that accepts a payment and returns a result. PaymentProcessor holds a mapping from method identifier to handler and delegates to the selected handler. The conditional chain disappears entirely, because the mapping lookup performs the selection that the chain was performing. Adding cryptocurrency then consists of writing one new class and adding one entry to the registry, and the existing four classes are neither edited nor recompiled nor re-reviewed.\nTwo further observations are worth recording. First, the four branches contain markedly different logic: card validation, VPA suffix checking, a flat fee, and a balance comparison. These are four independent policies, which is why the method also violates the Single Responsibility Principle. The two violations have the same correction, which is common. Second, the final else clause that raises ValueError is the only part of the chain that survives the refactor, appearing as the failure branch of the registry lookup.\nProblem C.2 (Level 2) An observability library contains the following class. Refactor it so that adding an XML format or an HTML format requires no modification to any existing class.\nclass LogFormatter: def format(self, log_entry, output_format: str) -\u0026gt; str: if output_format == \u0026#34;plain\u0026#34;: return f\u0026#34;[{log_entry.level}] {log_entry.timestamp}: {log_entry.message}\u0026#34; elif output_format == \u0026#34;json\u0026#34;: import json return json.dumps({ \u0026#34;level\u0026#34;: log_entry.level, \u0026#34;timestamp\u0026#34;: str(log_entry.timestamp), \u0026#34;message\u0026#34;: log_entry.message, }) elif output_format == \u0026#34;csv\u0026#34;: return f\u0026#34;{log_entry.level},{log_entry.timestamp},{log_entry.message}\u0026#34; else: raise ValueError(f\u0026#34;Unknown format: {output_format}\u0026#34;) Required:\nState the defect and the specific cost it imposes. Produce the refactored design. Show how a caller selects a format, and show what the addition of an XML formatter looks like. Explain why the output_format string parameter disappears from the formatting call. Solution C.2 Diagnosis.\nThe class conflates two decisions that occur at different times. The decision of which format to use is made once, at configuration time, by whoever operates the application. The decision of how to render a single entry is made on every log line. By taking output_format as a parameter of format, the class re-evaluates a configuration-time decision on every invocation, and it must contain every possible answer to that decision in order to do so.\nThe cost is that the class accumulates one branch per format and becomes a change magnet for a concern that is not shared between formats. The CSV rendering and the JSON rendering have nothing in common beyond their input type, so a defect fixed in one has no relationship to the other, yet both are edited in the same file and shipped together. A secondary cost is that the class cannot be extended by a consumer of the library at all. A user of this logging package who requires a proprietary format has no option but to modify the package source, which is the clearest possible demonstration that the class is not closed for modification.\nRefactored design.\nfrom __future__ import annotations import json from abc import ABC, abstractmethod from dataclasses import dataclass from datetime import datetime from xml.sax.saxutils import escape @dataclass(frozen=True) class LogEntry: level: str timestamp: datetime message: str class LogFormatter(ABC): \u0026#34;\u0026#34;\u0026#34;The abstraction. One responsibility: render a single entry as text.\u0026#34;\u0026#34;\u0026#34; @abstractmethod def format(self, entry: LogEntry) -\u0026gt; str: ... class PlainFormatter(LogFormatter): def format(self, entry: LogEntry) -\u0026gt; str: return f\u0026#34;[{entry.level}] {entry.timestamp}: {entry.message}\u0026#34; class JsonFormatter(LogFormatter): def format(self, entry: LogEntry) -\u0026gt; str: return json.dumps({ \u0026#34;level\u0026#34;: entry.level, \u0026#34;timestamp\u0026#34;: entry.timestamp.isoformat(), \u0026#34;message\u0026#34;: entry.message, }) class CsvFormatter(LogFormatter): def format(self, entry: LogEntry) -\u0026gt; str: message = entry.message.replace(\u0026#39;\u0026#34;\u0026#39;, \u0026#39;\u0026#34;\u0026#34;\u0026#39;) return f\u0026#39;{entry.level},{entry.timestamp.isoformat()},\u0026#34;{message}\u0026#34;\u0026#39; The consumer of a formatter holds the abstraction and never learns which implementation it received.\nclass Logger: def __init__(self, formatter: LogFormatter, sink): self._formatter = formatter self._sink = sink def log(self, level: str, message: str) -\u0026gt; None: entry = LogEntry(level=level, timestamp=datetime.now(), message=message) self._sink.write(self._formatter.format(entry) + \u0026#34;\\n\u0026#34;) Adding a format is now an addition of a file, with no edit anywhere else.\nclass XmlFormatter(LogFormatter): def format(self, entry: LogEntry) -\u0026gt; str: return ( \u0026#34;\u0026lt;log\u0026gt;\u0026#34; f\u0026#34;\u0026lt;level\u0026gt;{escape(entry.level)}\u0026lt;/level\u0026gt;\u0026#34; f\u0026#34;\u0026lt;timestamp\u0026gt;{entry.timestamp.isoformat()}\u0026lt;/timestamp\u0026gt;\u0026#34; f\u0026#34;\u0026lt;message\u0026gt;{escape(entry.message)}\u0026lt;/message\u0026gt;\u0026#34; \u0026#34;\u0026lt;/log\u0026gt;\u0026#34; ) Where configuration supplies a format as a string, the mapping from string to implementation belongs in one place at the edge of the system rather than inside the formatting call path.\n_FORMATTERS: dict[str, type[LogFormatter]] = { \u0026#34;plain\u0026#34;: PlainFormatter, \u0026#34;json\u0026#34;: JsonFormatter, \u0026#34;csv\u0026#34;: CsvFormatter, \u0026#34;xml\u0026#34;: XmlFormatter, } def formatter_from_config(name: str) -\u0026gt; LogFormatter: try: return _FORMATTERS[name]() except KeyError: raise ValueError(f\u0026#34;Unknown log format: {name}\u0026#34;) from None Answer to question 4. The output_format parameter disappears because the selection has been moved from call time to construction time, where it belongs. A Logger is constructed once with a JsonFormatter and thereafter produces JSON; there is no per-call decision to make. This relocation is the substance of the refactor, and the elimination of the conditional chain is its consequence rather than its goal. A refactor that kept the parameter and merely replaced the chain with a dictionary lookup inside format would remove the branches without removing the coupling, since the class would still need to know every available format.\nTwo points of technique appear in the code above and are worth noting because they are frequently missed. The registry maps names to classes rather than to instances, so a formatter that holds per-logger state remains safe to use. The CsvFormatter quotes and escapes the message field, and the XmlFormatter escapes its text nodes; these correctness details were absent from the original and become straightforward to add once each format occupies its own class, which is a practical benefit of the separation beyond extensibility.\nProblem C.3 (Level 3) Scenario. An e-commerce platform applies promotions to orders at checkout. Three promotion types exist today.\nA flat discount reduces the order total by a fixed amount, for example 200 rupees off. A percentage discount reduces the total by a proportion, for example 10 percent, and is subject to a maximum discount cap. A buy-one-get-one promotion applies to a specified product and gives 50 percent off the second and subsequent qualifying units.\nThe marketing team introduces new promotion types every quarter and has already asked about free shipping above a threshold, a first-order discount, and a bundle price for a fixed set of products. Promotions may be combined on a single order, in which case they apply in a defined sequence, and the platform must be able to display a line-by-line breakdown of which promotion contributed which reduction.\nRequired:\nDesign a promotions system that is open to new promotion types without modification of the pricing engine. Implement the abstraction, the three existing promotion types, and the pricing engine. Support the line-by-line breakdown requirement. Demonstrate the addition of a free-shipping promotion, and state precisely which existing files are edited. Write the design decisions, including the reason for the chosen return type of the promotion method. Solution C.3 Step 1: locating the axis of variation.\nThe requirement contains one axis that varies without limit, which is the set of promotion types, and one axis that is stable, which is the process of applying a list of promotions to an order and reporting the result. The Open/Closed Principle is applied by expressing the varying axis as implementations of an abstraction and the stable axis as a single class that depends only on that abstraction.\nThe breakdown requirement determines the shape of the abstraction and is the part of this problem most often designed incorrectly. If a promotion returns a number, the engine can compute a total but cannot report which promotion produced which reduction, and the natural repair is for the engine to inspect the promotion type in order to label the line, which reintroduces exactly the conditional chain that the design exists to eliminate. The abstraction must therefore return a value that carries its own description.\nStep 2 and 3: implementation.\nfrom __future__ import annotations from abc import ABC, abstractmethod from dataclasses import dataclass from typing import Sequence @dataclass(frozen=True) class OrderLine: product_id: str unit_price: float quantity: int @property def line_total(self) -\u0026gt; float: return self.unit_price * self.quantity @dataclass(frozen=True) class Order: order_id: str customer_id: str lines: tuple[OrderLine, ...] shipping_cost: float is_first_order: bool = False @property def item_subtotal(self) -\u0026gt; float: return sum(line.line_total for line in self.lines) @dataclass(frozen=True) class Discount: \u0026#34;\u0026#34;\u0026#34;The result of one promotion. Carries its own label so that the engine never needs to inspect the promotion type to describe the reduction.\u0026#34;\u0026#34;\u0026#34; label: str amount: float applies_to: str = \u0026#34;items\u0026#34; # \u0026#34;items\u0026#34; or \u0026#34;shipping\u0026#34; @staticmethod def none(label: str) -\u0026gt; \u0026#34;Discount\u0026#34;: return Discount(label=label, amount=0.0) @property def is_zero(self) -\u0026gt; bool: return self.amount \u0026lt;= 0.0 class Promotion(ABC): \u0026#34;\u0026#34;\u0026#34;The abstraction. A promotion decides whether it applies to an order and how large a reduction it produces. It never mutates the order and never knows about other promotions.\u0026#34;\u0026#34;\u0026#34; @property @abstractmethod def code(self) -\u0026gt; str: ... @abstractmethod def apply(self, order: Order, running_total: float) -\u0026gt; Discount: ... class FlatDiscount(Promotion): def __init__(self, amount: float, minimum_order_value: float = 0.0): self._amount = amount self._minimum = minimum_order_value @property def code(self) -\u0026gt; str: return f\u0026#34;FLAT{int(self._amount)}\u0026#34; def apply(self, order: Order, running_total: float) -\u0026gt; Discount: if running_total \u0026lt; self._minimum: return Discount.none(self.code) return Discount(self.code, min(self._amount, running_total)) class PercentageDiscount(Promotion): def __init__(self, percent: float, maximum_discount: float | None = None): self._percent = percent self._cap = maximum_discount @property def code(self) -\u0026gt; str: return f\u0026#34;PCT{int(self._percent)}\u0026#34; def apply(self, order: Order, running_total: float) -\u0026gt; Discount: raw = running_total * (self._percent / 100.0) amount = min(raw, self._cap) if self._cap is not None else raw return Discount(self.code, amount) class BuyOneGetHalfOff(Promotion): def __init__(self, product_id: str): self._product_id = product_id @property def code(self) -\u0026gt; str: return f\u0026#34;BOGO50:{self._product_id}\u0026#34; def apply(self, order: Order, running_total: float) -\u0026gt; Discount: qualifying = [l for l in order.lines if l.product_id == self._product_id] reduction = sum( line.unit_price * 0.5 * (line.quantity - 1) for line in qualifying if line.quantity \u0026gt; 1 ) return Discount(self.code, reduction) The engine holds a sequence of promotions and knows nothing about any of them beyond the abstraction.\n@dataclass(frozen=True) class PricedOrder: item_subtotal: float shipping_cost: float breakdown: tuple[Discount, ...] @property def total_discount(self) -\u0026gt; float: return sum(d.amount for d in self.breakdown) @property def payable(self) -\u0026gt; float: return round( self.item_subtotal + self.shipping_cost - self.total_discount, 2 ) class PricingEngine: \u0026#34;\u0026#34;\u0026#34;Stable. Applies promotions in the configured sequence and reports the result. Contains no promotion-specific logic and no conditional chain.\u0026#34;\u0026#34;\u0026#34; def __init__(self, promotions: Sequence[Promotion]): self._promotions = tuple(promotions) def price(self, order: Order) -\u0026gt; PricedOrder: running_total = order.item_subtotal shipping_remaining = order.shipping_cost applied: list[Discount] = [] for promotion in self._promotions: discount = promotion.apply(order, running_total) if discount.is_zero: continue if discount.applies_to == \u0026#34;shipping\u0026#34;: capped = Discount( discount.label, min(discount.amount, shipping_remaining), \u0026#34;shipping\u0026#34; ) shipping_remaining -= capped.amount applied.append(capped) else: capped = Discount( discount.label, min(discount.amount, running_total), \u0026#34;items\u0026#34; ) running_total -= capped.amount applied.append(capped) return PricedOrder( item_subtotal=order.item_subtotal, shipping_cost=order.shipping_cost, breakdown=tuple(applied), ) Step 4: adding a promotion type.\nclass FreeShippingAbove(Promotion): def __init__(self, threshold: float): self._threshold = threshold @property def code(self) -\u0026gt; str: return f\u0026#34;FREESHIP{int(self._threshold)}\u0026#34; def apply(self, order: Order, running_total: float) -\u0026gt; Discount: if running_total \u0026lt; self._threshold: return Discount.none(self.code) return Discount(self.code, order.shipping_cost, applies_to=\u0026#34;shipping\u0026#34;) The files edited in order to introduce free shipping are: one new file containing FreeShippingAbove, and the composition root or campaign configuration where the active promotions for a campaign are listed. PricingEngine, FlatDiscount, PercentageDiscount, BuyOneGetHalfOff, Discount, and PricedOrder are all unmodified. No existing test requires revision, and the existing promotions require no re-review.\nA worked example of the composition and the resulting breakdown:\norder = Order( order_id=\u0026#34;ORD-9001\u0026#34;, customer_id=\u0026#34;CUST-22\u0026#34;, lines=( OrderLine(\u0026#34;BOOK-1\u0026#34;, 499.0, 3), OrderLine(\u0026#34;MUG-7\u0026#34;, 250.0, 1), ), shipping_cost=60.0, is_first_order=True, ) engine = PricingEngine([ BuyOneGetHalfOff(\u0026#34;BOOK-1\u0026#34;), PercentageDiscount(percent=10, maximum_discount=150.0), FlatDiscount(amount=200.0, minimum_order_value=1000.0), FreeShippingAbove(threshold=750.0), ]) priced = engine.price(order) for line in priced.breakdown: print(f\u0026#34;{line.label:\u0026lt;20} -{line.amount:\u0026gt;8.2f} ({line.applies_to})\u0026#34;) print(f\u0026#34;{\u0026#39;Payable\u0026#39;:\u0026lt;20} {priced.payable:\u0026gt;8.2f}\u0026#34;) Step 5: design decisions.\nThe promotion method returns a Discount rather than a float or a modified order. Three candidate return types were available and the choice determines whether the design holds. Returning a float satisfies the arithmetic but destroys the breakdown, and it forces the engine to label lines by inspecting types. Returning a modified Order gives each promotion the power to alter any part of the order, which makes the effect of a promotion sequence unpredictable and makes each promotion far harder to test. Returning a descriptive value object gives the engine everything it needs to both compute and report while confining each promotion to producing a single number and a label. This decision protects against the breakdown requirement being satisfied by a type check, which is the failure mode this design exists to prevent.\nPromotions receive the running total as an explicit parameter. The requirement states that promotions apply in a defined sequence, which means a percentage promotion applied after a flat discount must compute its percentage on the reduced amount. Passing running_total makes the ordering semantics explicit at the call site and keeps promotions free of mutable state. The alternative, in which each promotion reads a mutable amount from the order, produces promotions whose results depend on invocation history in ways that are not visible in their signatures.\nCapping is enforced by the engine, not trusted to each promotion. A promotion that returns a reduction larger than the amount remaining would drive the payable total negative. Because promotions are written by many people over many quarters, the engine treats their output as untrusted and clamps it. This is a deliberate placement of a system-wide invariant in the one class that cannot be bypassed, and it is the same reasoning applied to Order in Problem B.3.\nA promotion decides for itself whether it applies. The zero-valued Discount returned when a condition is unmet allows eligibility rules to live inside the promotion that owns them. The alternative of a separate is_applicable predicate on the abstraction requires the engine to call two methods in the correct order and permits an inconsistent implementation in which the predicate and the calculation disagree.\nShipping is modelled as a separate target rather than folded into the item total. Free shipping is already on the roadmap, and a promotion reducing shipping must not be permitted to reduce the item subtotal. Introducing the applies_to field costs one attribute and one branch in the engine, and it prevents an entire class of pricing defect. This is a case where a small anticipatory generalisation is justified, because the requirement explicitly names free shipping as forthcoming rather than merely conceivable.\nSection C review Problem Domain Level Central lesson C.1 Payments 1 The cost of an Open/Closed defect is regression risk in working code, not inelegance C.2 Logging 2 Move the selection from call time to construction time. Removing the branches is a consequence. C.3 E-commerce promotions 3 The return type of the abstraction determines whether the design survives its reporting requirements 6. Section D: Liskov Substitution Principle The principle under examination states that if S is a subtype of T, then objects of type T may be replaced with objects of type S without altering the correctness of the program. Barbara Liskov\u0026rsquo;s original formulation concerns the properties provable about supertype objects remaining provable about subtype objects.\nThe operational content of the principle is a set of rules governing what an override is permitted to do. An override may accept the same inputs or a wider range of inputs, but it may not narrow the accepted range, which is stated as the rule that preconditions may not be strengthened in a subtype. An override may guarantee the same outcome or a stronger outcome, but it may not guarantee less, which is stated as the rule that postconditions may not be weakened. An override may raise the exception types declared by the supertype or their subtypes, but it may not introduce new categories of failure. An override may not invalidate any invariant the supertype maintains.\nViolations are diagnosed by reading the override against the base contract rather than by reading the override alone. This is what makes Liskov violations harder to spot than the other four: the defect is a mismatch between two locations, and neither location is wrong in isolation.\nProblem D.1 (Level 1) class Rectangle: def __init__(self, width: float, height: float): self._width = width self._height = height def set_width(self, width: float) -\u0026gt; None: self._width = width def set_height(self, height: float) -\u0026gt; None: self._height = height def area(self) -\u0026gt; float: return self._width * self._height class Square(Rectangle): def __init__(self, side: float): super().__init__(side, side) def set_width(self, width: float) -\u0026gt; None: self._width = width self._height = width def set_height(self, height: float) -\u0026gt; None: self._width = height self._height = height def resize_and_verify(shape: Rectangle) -\u0026gt; None: shape.set_width(5) shape.set_height(10) assert shape.area() == 50, f\u0026#34;Expected 50, got {shape.area()}\u0026#34; Questions:\nWhat happens when resize_and_verify receives a Square? Which rule of the principle is broken, stated precisely? Why is adding an isinstance check inside resize_and_verify not a correction? Solution D.1 Answer to question 1. Setting the width to 5 also sets the height to 5. Setting the height to 10 also sets the width to 10. The area is 100 rather than 50, and the assertion fails. The function is correct with respect to the contract it was written against, and it fails only because it was handed an object whose declared type promises behaviour the object does not exhibit.\nAnswer to question 2. The rule broken is that invariants of the supertype must be preserved. Rectangle maintains the invariant that width and height are independent, which is observable through its public interface: after set_width(w) followed by set_height(h), the area is w * h for any w and h. Square cannot preserve this invariant, because its own invariant requires the two dimensions to be equal. Stated more sharply, Square weakens the postcondition of set_width from \u0026ldquo;the width is w and the height is unchanged\u0026rdquo; to \u0026ldquo;the width is w\u0026rdquo;, and every caller relying on the discarded clause is broken.\nAnswer to question 3. A type check inside the consumer is not a correction for three reasons, and the third is the decisive one.\nIt does not remove the defect; it documents it at every call site. The next function written against Rectangle will need the same guard, and the guard will be omitted somewhere.\nIt violates the Open/Closed Principle in the consumer. Each new shape subtype requires a new branch in every guard, so the number of edits grows as the product of subtypes and consumers.\nIt contradicts the purpose of subtype polymorphism. The reason to declare a parameter as Rectangle is to avoid knowing which concrete shape arrived. If the function must know, the type declaration is not providing the abstraction it appears to provide, and the inheritance relationship is providing no benefit while imposing a cost.\nThe correction is to recognise that the inheritance relationship is unsound as modelled. A square is a rectangle in geometry, where shapes are immutable values, and is not a rectangle in this design, where shapes have independently mutable dimensions. Two sound alternatives exist. Immutable shapes with a read-only area can be related by inheritance without difficulty, because there are no setters whose postconditions can be weakened. Alternatively, Square and Rectangle can be siblings implementing a common Shape abstraction that declares area and no dimension setters. The general lesson is that whether inheritance is valid depends on the mutability and the contract of the base, not on the taxonomy of the concepts.\nProblem D.2 (Level 2) A storage library contains the following hierarchy.\nclass FileStorage: def read(self, path: str) -\u0026gt; bytes: raise NotImplementedError def write(self, path: str, data: bytes) -\u0026gt; None: raise NotImplementedError def delete(self, path: str) -\u0026gt; None: raise NotImplementedError class LocalFileStorage(FileStorage): def __init__(self, root: str): self._root = root def read(self, path: str) -\u0026gt; bytes: with open(f\u0026#34;{self._root}/{path}\u0026#34;, \u0026#34;rb\u0026#34;) as handle: return handle.read() def write(self, path: str, data: bytes) -\u0026gt; None: with open(f\u0026#34;{self._root}/{path}\u0026#34;, \u0026#34;wb\u0026#34;) as handle: handle.write(data) def delete(self, path: str) -\u0026gt; None: import os os.remove(f\u0026#34;{self._root}/{path}\u0026#34;) class ReadOnlyArchiveStorage(FileStorage): def __init__(self, bucket: str): self._bucket = bucket def read(self, path: str) -\u0026gt; bytes: return b\u0026#34;...object bytes from immutable archive...\u0026#34; def write(self, path: str, data: bytes) -\u0026gt; None: raise PermissionError(\u0026#34;Archive storage is immutable\u0026#34;) def delete(self, path: str) -\u0026gt; None: raise PermissionError(\u0026#34;Archive storage is immutable\u0026#34;) def archive_and_purge(storage: FileStorage, path: str, destination: str) -\u0026gt; None: payload = storage.read(path) storage.write(destination, payload) storage.delete(path) Required:\nState the violation precisely and identify which caller is broken. Explain why catching PermissionError in archive_and_purge is not the correction. Redesign the hierarchy so that the defect is unrepresentable, and show the corrected signature of archive_and_purge. Identify which additional principle the original code violates. Solution D.2 Diagnosis.\nFileStorage declares three capabilities, and its contract therefore promises that any instance can read, write, and delete. ReadOnlyArchiveStorage is a declared subtype and satisfies the interface syntactically while contradicting it semantically: two of its three methods fail unconditionally with an exception type the base contract never mentions. archive_and_purge is written against the base contract, is correct with respect to that contract, and fails at the second statement when handed an archive instance. The failure occurs after the read has succeeded, which means the function fails partway through a multi-step operation, and this is the practical reason the defect is expensive rather than merely inelegant.\nAnswer to question 2. Catching the exception addresses the symptom at one call site and leaves the defect in place. Three specific objections apply.\nThe handler has no correct action available. If the write fails because the target is immutable, the archiving operation cannot be completed, so the handler can only re-raise a different exception or silently abandon the work. Neither outcome is better than the original failure, and the second is worse.\nThe knowledge is in the wrong place. The call site learns, by experiment at runtime, a fact that was known statically at construction time, namely that this particular storage cannot be written to. Discovering statically available information through exception handling is a design defect independent of the principle being discussed.\nIt does not generalise. Every function in the library that writes or deletes requires the same handler, and each is an opportunity to omit it. The compiler and the type checker provide no assistance, because the declared type continues to promise the capability.\nCorrected hierarchy.\nThe correction separates the capabilities into distinct abstractions, so that a function requiring write access cannot be handed an object that lacks it.\nfrom abc import ABC, abstractmethod class ReadableStorage(ABC): \u0026#34;\u0026#34;\u0026#34;Contract: read returns the bytes at path, or raises FileNotFoundError if no object exists at that path.\u0026#34;\u0026#34;\u0026#34; @abstractmethod def read(self, path: str) -\u0026gt; bytes: ... class WritableStorage(ReadableStorage): \u0026#34;\u0026#34;\u0026#34;Contract: everything ReadableStorage promises, and in addition write stores data at path and delete removes it. Neither refuses on grounds of the store being immutable.\u0026#34;\u0026#34;\u0026#34; @abstractmethod def write(self, path: str, data: bytes) -\u0026gt; None: ... @abstractmethod def delete(self, path: str) -\u0026gt; None: ... class LocalFileStorage(WritableStorage): def __init__(self, root: str): self._root = root def read(self, path: str) -\u0026gt; bytes: with open(f\u0026#34;{self._root}/{path}\u0026#34;, \u0026#34;rb\u0026#34;) as handle: return handle.read() def write(self, path: str, data: bytes) -\u0026gt; None: with open(f\u0026#34;{self._root}/{path}\u0026#34;, \u0026#34;wb\u0026#34;) as handle: handle.write(data) def delete(self, path: str) -\u0026gt; None: import os os.remove(f\u0026#34;{self._root}/{path}\u0026#34;) class ReadOnlyArchiveStorage(ReadableStorage): \u0026#34;\u0026#34;\u0026#34;Implements only what it can honour. There is no write method to break.\u0026#34;\u0026#34;\u0026#34; def __init__(self, bucket: str): self._bucket = bucket def read(self, path: str) -\u0026gt; bytes: return b\u0026#34;...object bytes from immutable archive...\u0026#34; The consumers now declare the narrowest capability they require, which is what converts the runtime failure into a static error.\ndef archive_and_purge( source: ReadableStorage, sink: WritableStorage, path: str, destination: str ) -\u0026gt; None: \u0026#34;\u0026#34;\u0026#34;Reads from any readable store and writes to any writable store. The parameter types make it impossible to pass an immutable store as the sink.\u0026#34;\u0026#34;\u0026#34; payload = source.read(path) sink.write(destination, payload) def load_manifest(storage: ReadableStorage) -\u0026gt; bytes: \u0026#34;\u0026#34;\u0026#34;Accepts both implementations, because both can genuinely read.\u0026#34;\u0026#34;\u0026#34; return storage.read(\u0026#34;manifest.json\u0026#34;) Passing ReadOnlyArchiveStorage as the sink argument is now reported by a static type checker before the program runs, and no exception handler is required anywhere.\nNote that the original single-parameter signature was itself part of the problem. A function that reads from and writes to the same storage object had conflated two roles, source and destination, that the corrected signature separates. Liskov violations frequently expose modelling errors of this kind in the consumers as well as in the hierarchy.\nAnswer to question 4. The original code also violates the Interface Segregation Principle, because FileStorage obliges every implementer to declare mutation capability regardless of whether it possesses any. This pairing is characteristic. An over-broad interface forces implementers to supply methods they cannot honour, and whichever way they discharge the obligation, by raising or by silently doing nothing, the result is a Liskov violation for the callers. Correcting the interface breadth is therefore usually the mechanism by which the substitution defect is removed, which is why the two principles are treated together in Section G.\nProblem D.3 (Level 3) Scenario. A retail bank offers three account products.\nA savings account permits deposits and withdrawals at any time, provided the resulting balance does not fall below zero.\nA current account permits deposits and withdrawals at any time and additionally permits the balance to fall to a negative overdraft limit agreed per customer.\nA recurring deposit account accepts a fixed monthly deposit and permits no withdrawal at all until a maturity date, after which the entire balance may be withdrawn once and the account is closed.\nThe bank runs an end-of-day sweep process that moves any balance above a configured ceiling from a customer\u0026rsquo;s operating accounts into a nominated investment account. It also runs a monthly interest posting process that applies to all three products, and a statement generation process that applies to all three products.\nRequired:\nDesign a hierarchy in which no operation raises an exception that its declared type does not anticipate. Present and justify a choice between the two available correction strategies. Implement the design and show the sweep process, which is the consumer whose correctness is most at risk. State the design decisions, including how the overdraft limit and the maturity rule are prevented from becoming special cases in the consumers. Solution D.3 Step 1: identifying what varies and what does not.\nThe requirement describes three consumers and three products. The consumers divide cleanly on the basis of which capability they require. Interest posting and statement generation require only that an account has a balance, an identifier, and a method of crediting interest, and all three products support all of these. The sweep process requires withdrawal, and only two of the three products support it at any given time.\nThis division is the substance of the design. The naive hierarchy, in which a single Account base declares withdraw and the recurring deposit refuses it, produces exactly the defect of Problem D.2: the sweep process is written against a contract the recurring deposit cannot honour, and it fails at runtime when the account mix changes.\nStep 2: choosing between the two correction strategies.\nTwo strategies are available whenever a subtype cannot honour a base operation, and selecting between them is a design judgement rather than a rule.\nThe first strategy is capability separation, in which the operation is moved to a narrower abstraction that only capable types implement. This is the strategy applied in Problem D.2. Its advantage is that misuse becomes a static error. Its cost is that the capability must be determinable from the type, because a static type cannot express a condition that changes at runtime.\nThe second strategy is contract widening, in which the base contract is changed so that refusal is a declared and expected outcome rather than a surprise. The operation returns a result value indicating success or refusal, or raises a documented domain exception that the base contract names, and every caller is then obliged to handle refusal because the signature says so. Its advantage is that it accommodates conditions evaluated at runtime. Its cost is that every caller must handle the refusal path, including callers dealing with types that never refuse.\nThe choice for this requirement is capability separation for the recurring deposit and contract widening for the withdrawal limit. The justification rests on when each condition is known.\nWhether a product supports withdrawal at all is a property of the product, fixed when the account is opened, and therefore expressible in the type. A recurring deposit account is not withdrawable, and this never changes during the period in which it is a recurring deposit account. Capability separation applies.\nWhether a specific withdrawal succeeds depends on the amount requested and the current balance, which are runtime values. No type system can prevent a request for more than the available balance, so refusal must be a declared outcome of the operation for every withdrawable account. Contract widening applies, and crucially it applies uniformly: the savings account and the current account both refuse under some conditions, they differ only in where the boundary lies, and therefore no caller needs to distinguish between them.\nTreating maturity as a runtime condition rather than a type property would be the common error here. It appears attractive because maturity does change over time. It is rejected because the transition at maturity is not a change in the account\u0026rsquo;s capability but the end of the account\u0026rsquo;s life: the requirement states that the entire balance is withdrawn once and the account closes. Modelling maturity as a withdraw that begins succeeding would oblige every caller of withdraw, including the sweep process, to reason about a case that occurs once per account and is handled by a dedicated closure process.\nStep 3: implementation.\nfrom __future__ import annotations from abc import ABC, abstractmethod from dataclasses import dataclass from datetime import date from typing import Sequence @dataclass(frozen=True) class WithdrawalResult: \u0026#34;\u0026#34;\u0026#34;The declared outcome of a withdrawal attempt. Refusal is part of the contract, so no caller is surprised by it.\u0026#34;\u0026#34;\u0026#34; succeeded: bool amount: float reason: str = \u0026#34;\u0026#34; @staticmethod def refused(reason: str) -\u0026gt; \u0026#34;WithdrawalResult\u0026#34;: return WithdrawalResult(succeeded=False, amount=0.0, reason=reason) class Account(ABC): \u0026#34;\u0026#34;\u0026#34;The widest abstraction. Everything every product can honour, and nothing else. Interest posting and statements depend only on this.\u0026#34;\u0026#34;\u0026#34; def __init__(self, account_no: str, balance: float = 0.0): self._account_no = account_no self._balance = balance @property def account_no(self) -\u0026gt; str: return self._account_no @property def balance(self) -\u0026gt; float: return self._balance def deposit(self, amount: float) -\u0026gt; None: if amount \u0026lt;= 0: raise ValueError(\u0026#34;Deposit amount must be positive\u0026#34;) self._balance += amount def credit_interest(self, amount: float) -\u0026gt; None: self._balance += amount @property @abstractmethod def annual_interest_rate(self) -\u0026gt; float: ... class WithdrawableAccount(Account): \u0026#34;\u0026#34;\u0026#34;Adds the withdrawal capability. Contract: withdraw never raises on grounds of insufficient funds or policy. It returns a WithdrawalResult whose succeeded flag reports the outcome.\u0026#34;\u0026#34;\u0026#34; @abstractmethod def withdraw(self, amount: float) -\u0026gt; WithdrawalResult: ... @property @abstractmethod def withdrawable_balance(self) -\u0026gt; float: \u0026#34;\u0026#34;\u0026#34;The largest amount a withdrawal would currently succeed with. Zero is a valid answer and is not an error.\u0026#34;\u0026#34;\u0026#34; class SavingsAccount(WithdrawableAccount): @property def annual_interest_rate(self) -\u0026gt; float: return 3.5 @property def withdrawable_balance(self) -\u0026gt; float: return max(self._balance, 0.0) def withdraw(self, amount: float) -\u0026gt; WithdrawalResult: if amount \u0026lt;= 0: raise ValueError(\u0026#34;Withdrawal amount must be positive\u0026#34;) if amount \u0026gt; self.withdrawable_balance: return WithdrawalResult.refused(\u0026#34;Insufficient funds\u0026#34;) self._balance -= amount return WithdrawalResult(succeeded=True, amount=amount) class CurrentAccount(WithdrawableAccount): def __init__(self, account_no: str, balance: float = 0.0, overdraft_limit: float = 0.0): super().__init__(account_no, balance) self._overdraft_limit = overdraft_limit @property def annual_interest_rate(self) -\u0026gt; float: return 0.0 @property def withdrawable_balance(self) -\u0026gt; float: return max(self._balance + self._overdraft_limit, 0.0) def withdraw(self, amount: float) -\u0026gt; WithdrawalResult: if amount \u0026lt;= 0: raise ValueError(\u0026#34;Withdrawal amount must be positive\u0026#34;) if amount \u0026gt; self.withdrawable_balance: return WithdrawalResult.refused(\u0026#34;Overdraft limit exceeded\u0026#34;) self._balance -= amount return WithdrawalResult(succeeded=True, amount=amount) class RecurringDepositAccount(Account): \u0026#34;\u0026#34;\u0026#34;Not withdrawable. There is no withdraw method to break, and the type system prevents this account from reaching the sweep process.\u0026#34;\u0026#34;\u0026#34; def __init__( self, account_no: str, monthly_instalment: float, maturity_date: date, balance: float = 0.0, ): super().__init__(account_no, balance) self._monthly_instalment = monthly_instalment self._maturity_date = maturity_date self._closed = False @property def annual_interest_rate(self) -\u0026gt; float: return 6.75 def pay_instalment(self) -\u0026gt; None: self.deposit(self._monthly_instalment) def is_matured(self, as_of: date) -\u0026gt; bool: return as_of \u0026gt;= self._maturity_date def close_on_maturity(self, as_of: date) -\u0026gt; float: \u0026#34;\u0026#34;\u0026#34;The single withdrawal permitted by the product, modelled as closure rather than as a withdrawal, because that is what it is.\u0026#34;\u0026#34;\u0026#34; if not self.is_matured(as_of): raise ValueError(\u0026#34;Recurring deposit has not matured\u0026#34;) if self._closed: raise ValueError(\u0026#34;Account already closed\u0026#34;) proceeds, self._balance, self._closed = self._balance, 0.0, True return proceeds The consumers now declare exactly the capability they need.\nclass SweepProcess: \u0026#34;\u0026#34;\u0026#34;Depends on WithdrawableAccount. A RecurringDepositAccount cannot be passed to it, and this is reported statically rather than at runtime.\u0026#34;\u0026#34;\u0026#34; def __init__(self, ceiling: float): self._ceiling = ceiling def run( self, operating: Sequence[WithdrawableAccount], investment: Account ) -\u0026gt; list[str]: moved: list[str] = [] for account in operating: surplus = min(account.balance - self._ceiling, account.withdrawable_balance) if surplus \u0026lt;= 0: continue result = account.withdraw(surplus) if not result.succeeded: moved.append(f\u0026#34;{account.account_no}: skipped ({result.reason})\u0026#34;) continue investment.deposit(result.amount) moved.append(f\u0026#34;{account.account_no}: moved {result.amount:.2f}\u0026#34;) return moved class InterestPosting: \u0026#34;\u0026#34;\u0026#34;Depends only on Account, so it applies to all three products with no type inspection whatsoever.\u0026#34;\u0026#34;\u0026#34; def run(self, accounts: Sequence[Account]) -\u0026gt; None: for account in accounts: monthly = account.balance * (account.annual_interest_rate / 100.0) / 12.0 if monthly \u0026gt; 0: account.credit_interest(round(monthly, 2)) Step 4: design decisions.\nWithdrawal returns a result object rather than raising on refusal. Insufficient funds is an expected outcome of a withdrawal request, not an exceptional condition, and a sweep process iterating over thousands of accounts must be able to continue past a refusal without exception handling around every call. The result type places the refusal in the signature, which obliges callers to acknowledge it, and it carries a reason so that the caller can report without inferring. Note the contrast with the two remaining raise statements: a non-positive amount is a programming error rather than a business outcome, so it correctly remains an exception.\nA withdrawable_balance property exists alongside withdraw. The sweep process must decide how much to move before attempting the move, and without this property it would need to know that a current account can go negative up to a limit, which is precisely the product-specific knowledge the abstraction exists to hide. Exposing the maximum permissible withdrawal as a value lets each product answer the question in its own terms while the consumer remains uniform. This is the mechanism by which the overdraft limit is prevented from becoming a special case in the consumer, and it should be recognised as a general technique: when a consumer needs to make a decision that depends on a subtype-specific rule, expose the outcome of the rule rather than the rule.\nMaturity closure is a distinct method on the concrete class, not an override of withdraw. The maturity withdrawal happens once, transitions the account to a closed state, and is performed by a dedicated maturity process that legitimately knows it is dealing with recurring deposits. Modelling it as withdraw would require RecurringDepositAccount to implement WithdrawableAccount, which would place it back within reach of the sweep process and reintroduce the original defect for the sake of one operation performed by one caller.\nThe interest rate is an abstract property on Account rather than a field on each subclass or a lookup table in the posting process. Had the posting process contained a mapping from account type to rate, adding a product would require editing the posting process, which is an Open/Closed defect. Declaring the rate as part of the base contract means each product answers for itself and the posting process is closed against new products. The current account returning zero is a legitimate answer rather than a missing implementation.\nSubstitutability was verified by inspection of the consumers, not of the hierarchy. The check performed on this design was to take each consumer, list the guarantees it relies on, and confirm that every type reaching it honours all of them. InterestPosting relies on balance, annual_interest_rate, and credit_interest, all of which all three products honour unconditionally. SweepProcess relies additionally on withdraw and withdrawable_balance, which the two withdrawable products honour unconditionally, and it is unreachable by the third. This procedure is the practical test for the principle and is more reliable than examining the class diagram, because the principle is a statement about what consumers may assume.\nSection D review Problem Domain Level Central lesson D.1 Geometry 1 Validity of inheritance depends on the mutability and contract of the base, not on real-world taxonomy D.2 Storage library 2 Separate capabilities into abstractions so that misuse becomes a static error rather than a runtime exception D.3 Retail banking 3 Use capability separation for properties fixed by type and contract widening for conditions evaluated at runtime Correction strategy Use when Cost Capability separation The capability is a fixed property of the type and can be determined statically Two abstractions to maintain, and consumers must declare the narrower type Contract widening The outcome depends on runtime values, so refusal cannot be prevented by typing Every caller must handle the refusal path, including those using types that rarely refuse 7. Section E: Interface Segregation Principle The principle under examination states that no client should be forced to depend upon methods it does not use. The corollary that governs design is that interfaces should be defined by the requirements of their clients rather than by the capabilities of their implementers.\nThe distinction between a client-defined interface and an implementer-defined interface is the substance of this principle. An implementer-defined interface is produced by listing everything the most capable implementation can do, which yields a large interface that smaller implementations cannot honour. A client-defined interface is produced by asking what a specific consumer needs, which yields several small interfaces that each implementation can adopt selectively. These narrow interfaces are called role interfaces.\nPython enforces no interface declarations, which changes how the principle manifests but not whether it applies. In a codebase using abstract base classes, the violation appears as implementers supplying empty or raising method bodies. In a codebase using no declared interfaces at all, the violation appears as functions that accept an object and use two of its twenty methods, thereby making themselves unusable with any simpler object. The typing.Protocol mechanism allows role interfaces to be declared for the benefit of static analysis without requiring implementers to inherit from them, which is the idiomatic expression of the principle in modern Python.\nProblem E.1 (Level 1) from abc import ABC, abstractmethod class MultiFunctionPrinter(ABC): @abstractmethod def print_document(self, document) -\u0026gt; None: ... @abstractmethod def scan_document(self) -\u0026gt; bytes: ... @abstractmethod def fax_document(self, document, number: str) -\u0026gt; None: ... @abstractmethod def photocopy_document(self, copies: int) -\u0026gt; None: ... @abstractmethod def staple_output(self, sheets: int) -\u0026gt; None: ... class InkjetPrinter(MultiFunctionPrinter): def print_document(self, document) -\u0026gt; None: print(f\u0026#34;Printing {document} on inkjet\u0026#34;) def scan_document(self) -\u0026gt; bytes: raise NotImplementedError(\u0026#34;This model has no scanner\u0026#34;) def fax_document(self, document, number: str) -\u0026gt; None: raise NotImplementedError(\u0026#34;This model has no fax\u0026#34;) def photocopy_document(self, copies: int) -\u0026gt; None: raise NotImplementedError(\u0026#34;This model has no copier\u0026#34;) def staple_output(self, sheets: int) -\u0026gt; None: raise NotImplementedError(\u0026#34;This model has no stapler\u0026#34;) def print_payslips(printer: MultiFunctionPrinter, payslips: list) -\u0026gt; None: for slip in payslips: printer.print_document(slip) printer.staple_output(len(payslips)) Questions:\nWhich principle is violated and what is the evidence? Which client is harmed, and what harm does it suffer? Which second principle is also violated, and why is that not a coincidence? Solution E.1 Answer to question 1. The Interface Segregation Principle. The evidence is the four methods in InkjetPrinter whose entire body raises NotImplementedError. Each one exists solely to satisfy an obligation imposed by the abstraction, and each is a declaration of a capability the object does not possess.\nAnswer to question 2. Two clients are harmed, in different ways, and distinguishing between them is the point of the exercise.\nInkjetPrinter is harmed as an implementer. It is coupled to the fax and stapling parts of the interface despite having no involvement with either. A change to the signature of fax_document, requested by an office that owns a fax-capable device, forces a modification to the inkjet class. This is coupling with no corresponding benefit, and it is the harm the principle is named for.\nprint_payslips is harmed as a consumer. It requires printing and stapling, so it declares a parameter type promising all five capabilities in order to obtain two, and it therefore cannot accept a simple printer even for the printing portion of its work. When handed an InkjetPrinter, it prints every payslip successfully and then fails at the stapling call, leaving the work partially done. A consumer that had declared only the capabilities it uses would have been prevented statically from accepting an unsuitable device.\nAnswer to question 3. The Liskov Substitution Principle is also violated, because InkjetPrinter is a declared subtype of MultiFunctionPrinter that fails on four of five inherited operations with an exception the base contract does not declare. This is not a coincidence but a mechanical consequence. When an interface obliges an implementer to declare a capability it lacks, the implementer has only three available responses, and all three are defective. Raising an exception produces a Liskov violation. Returning a null or empty value silently produces incorrect behaviour in the consumer, which is a worse Liskov violation because it is undetectable. Implementing the capability by simulation, for example printing to a file and calling it a fax, produces behaviour the consumer did not request. The defect originates in the breadth of the interface, and the Liskov violation is its symptom, which is why the correction is applied to the interface.\nThe corrected shape is a set of role interfaces, Printable, Scannable, Faxable, Photocopiable, and Staplable, with InkjetPrinter implementing only Printable, and print_payslips declaring a parameter type that requires printing and stapling and nothing else.\nProblem E.2 (Level 2) A storage abstraction layer defines the following interface, which is implemented by three classes with markedly different needs.\nfrom abc import ABC, abstractmethod class DataManager(ABC): @abstractmethod def read(self, key: str): ... @abstractmethod def write(self, key: str, value) -\u0026gt; None: ... @abstractmethod def delete(self, key: str) -\u0026gt; None: ... @abstractmethod def search(self, query: str) -\u0026gt; list: ... @abstractmethod def backup(self, destination: str) -\u0026gt; None: ... @abstractmethod def restore(self, source: str) -\u0026gt; None: ... The three implementers are as follows. InMemoryCache genuinely supports read, write, and delete only. AnalyticsStore is append-only and supports write and search only. ArchiveStorage supports backup and restore only.\nRequired:\nState the concrete cost the current interface imposes, with a specific example of a change that propagates incorrectly. Decompose the interface into role interfaces and show each implementer adopting only what it honours. Show two consumers declaring narrow dependencies, and explain what the narrowing buys. Explain when the decomposition should stop, and identify the anti-pattern that results from applying this principle without limit. Solution E.2 Diagnosis.\nEach implementer honours between two and three of the six declared methods, so between half and two-thirds of every implementation consists of methods that exist only to satisfy the abstraction. The cost is best demonstrated by a specific change.\nSuppose the backup mechanism is extended to support incremental backups, changing the signature to backup(self, destination: str, since: datetime | None = None). The classes requiring modification are ArchiveStorage, which is the only class that actually backs anything up, and also InMemoryCache and AnalyticsStore, neither of which has ever performed a backup. Two of the three edits are pure overhead: they modify working code, appear in the diff, consume review attention, and carry a nonzero probability of introducing a defect, in exchange for no functional change whatsoever. This is the mechanism by which a fat interface converts a localised change into a distributed one.\nA second cost is the loss of information at the type level. A function declaring a DataManager parameter tells the reader nothing about what it does with the argument, so the reader must examine the body. A function declaring a Searchable parameter has documented its behaviour in its signature.\nDecomposition.\nfrom __future__ import annotations from datetime import datetime from typing import Any, Protocol, runtime_checkable class Readable(Protocol): def read(self, key: str) -\u0026gt; Any: ... class Writable(Protocol): def write(self, key: str, value: Any) -\u0026gt; None: ... class Deletable(Protocol): def delete(self, key: str) -\u0026gt; None: ... class Searchable(Protocol): def search(self, query: str) -\u0026gt; list: ... @runtime_checkable class Archivable(Protocol): \u0026#34;\u0026#34;\u0026#34;Backup and restore are kept together deliberately. See the discussion of when to stop decomposing, below.\u0026#34;\u0026#34;\u0026#34; def backup(self, destination: str, since: datetime | None = None) -\u0026gt; None: ... def restore(self, source: str) -\u0026gt; None: ... The implementers declare only what they honour. Because these are protocols, conformance is structural and no inheritance is required, which also means a third-party class can satisfy them without modification.\nclass InMemoryCache: \u0026#34;\u0026#34;\u0026#34;Satisfies Readable, Writable, Deletable.\u0026#34;\u0026#34;\u0026#34; def __init__(self) -\u0026gt; None: self._store: dict[str, Any] = {} def read(self, key: str) -\u0026gt; Any: return self._store.get(key) def write(self, key: str, value: Any) -\u0026gt; None: self._store[key] = value def delete(self, key: str) -\u0026gt; None: self._store.pop(key, None) class AnalyticsStore: \u0026#34;\u0026#34;\u0026#34;Satisfies Writable, Searchable. Append-only by design, so the absence of delete is a product decision expressed in the type.\u0026#34;\u0026#34;\u0026#34; def __init__(self) -\u0026gt; None: self._events: list[tuple[str, Any]] = [] def write(self, key: str, value: Any) -\u0026gt; None: self._events.append((key, value)) def search(self, query: str) -\u0026gt; list: return [v for k, v in self._events if query in k] class ArchiveStorage: \u0026#34;\u0026#34;\u0026#34;Satisfies Archivable.\u0026#34;\u0026#34;\u0026#34; def backup(self, destination: str, since: datetime | None = None) -\u0026gt; None: window = \u0026#34;incremental\u0026#34; if since else \u0026#34;full\u0026#34; print(f\u0026#34;Writing {window} backup to {destination}\u0026#34;) def restore(self, source: str) -\u0026gt; None: print(f\u0026#34;Restoring from {source}\u0026#34;) The incremental backup change now touches exactly one implementer and one protocol definition, which is the correct blast radius for a change to backup behaviour.\nConsumers with narrow dependencies.\nclass SessionResolver: \u0026#34;\u0026#34;\u0026#34;Needs to read and write. Cannot be handed an ArchiveStorage, and does not care whether the store supports search or backup.\u0026#34;\u0026#34;\u0026#34; def __init__(self, cache: Readable | Writable): self._cache = cache class ReportSearch: \u0026#34;\u0026#34;\u0026#34;Needs search only. Its signature documents its entire interaction with the store.\u0026#34;\u0026#34;\u0026#34; def __init__(self, store: Searchable): self._store = store def find(self, term: str) -\u0026gt; list: return self._store.search(term) class NightlyMaintenance: \u0026#34;\u0026#34;\u0026#34;Needs archival only, and is therefore unaffected by any change to reading, writing, deletion, or search.\u0026#34;\u0026#34;\u0026#34; def __init__(self, archive: Archivable): self._archive = archive def run(self, destination: str) -\u0026gt; None: self._archive.backup(destination, since=None) The narrowing buys three specific things. Substitution becomes safe, because a consumer can be handed any object satisfying its narrow requirement, including a two-line fake in a test. Change propagation is confined, because a consumer is only affected by changes to the methods it names. Signatures become documentation, because the parameter type states what the consumer will do with the argument.\nA note on the union type in SessionResolver is warranted, since the annotation Readable | Writable is a common error and is written here as it frequently appears in real code. A union means either one or the other, which is weaker than intended. When a consumer needs several capabilities simultaneously, the correct construction is a composed protocol.\nclass ReadWriteStore(Readable, Writable, Protocol): \u0026#34;\u0026#34;\u0026#34;Composition of role interfaces into the exact requirement of one client. Declared once and reused, rather than repeated at each site.\u0026#34;\u0026#34;\u0026#34; class SessionResolverCorrected: def __init__(self, cache: ReadWriteStore): self._cache = cache def resolve(self, session_id: str) -\u0026gt; Any: cached = self._cache.read(session_id) if cached is None: cached = {\u0026#34;session\u0026#34;: session_id, \u0026#34;fresh\u0026#34;: True} self._cache.write(session_id, cached) return cached Answer to question 4: when to stop decomposing.\nThe decomposition stops when each interface corresponds to a cohesive role that at least one client requires in full. Two tests apply.\nThe first test asks whether any client uses the methods separately. backup and restore were kept in one Archivable protocol rather than split into Backupable and Restorable because no client performs one without being conceptually responsible for the other, and because the two are meaningless in isolation: a backup that cannot be restored has no purpose. Methods that are always required together belong in one interface.\nThe second test asks whether the interface has a name that describes a role rather than a method. Searchable names a role. A hypothetical SearchableWithPagination splitting search from count names an implementation detail.\nThe anti-pattern produced by applying the principle without limit is interface explosion, in which every method becomes its own single-method interface. The symptoms are a codebase in which a moderately capable class declares conformance to nine protocols, consumers whose constructors take six abstractions that always arrive as the same object, and a reader who must consult nine definitions to understand one class. Interface explosion trades one form of coupling for a much larger navigational cost, and it is the failure mode to watch for when this principle is applied enthusiastically. The correct number of interfaces is determined by the number of distinct client roles, which in most systems is small.\nProblem E.3 (Level 3) Scenario. A two-dimensional game engine must support the following entity types.\nA player character is drawn on screen each frame, updated each frame in response to input, participates in collision detection, can take damage and be destroyed, and must be written to and read from a save file.\nA static wall is drawn each frame and participates in collision detection. It is never updated, never takes damage, and its position is part of the level definition rather than the save file.\nAn ambient particle effect, for example falling snow, is drawn each frame and updated each frame. It has no collision, no health, and is not saved.\nA trigger volume, for example the invisible region that opens a door, participates in collision detection and is updated each frame to evaluate its condition. It is never drawn and is not saved, but the state of the door it controls is saved.\nThe engine\u0026rsquo;s main loop must, each frame, update everything that needs updating, resolve collisions among everything collidable, and draw everything drawable, in that order. The save system must serialise everything that is persistent.\nRequired:\nDefine the role interfaces implied by the requirement. Show which entity adopts which interfaces, as a table. Implement the interfaces, two representative entities, and the main loop. Explain why a single GameObject base class with default no-op implementations is rejected, and state the design decisions. Solution E.3 Step 1: deriving the interfaces from the clients.\nThe requirement names four clients, and the correct procedure is to derive one interface per client rather than one interface per entity. The main loop performs three distinct passes and the save system performs a fourth, and each pass needs exactly one capability.\nClient Capability required Role interface Update pass of the main loop Advance state by a time delta Updatable Collision pass of the main loop Supply a bounding region and respond to contact Collidable Draw pass of the main loop Emit draw instructions Drawable Save system Convert to and from a serialisable state Persistable Combat system Accept damage and report liveness Damageable Damageable is derived from the requirement that the player can take damage and be destroyed, which implies a combat client distinct from the three passes of the main loop. It is listed separately because grouping it with Updatable, on the grounds that both concern changing state, would oblige walls and particles to declare health.\nStep 2: adoption.\nEntity Updatable Collidable Drawable Persistable Damageable PlayerCharacter Yes Yes Yes Yes Yes StaticWall No Yes Yes No No SnowEmitter Yes No Yes No No TriggerVolume Yes Yes No No No The table is the design. Every cell containing \u0026ldquo;No\u0026rdquo; is a method that would have been an empty implementation under a single fat base class, and there are eight of them across four entities.\nStep 3: implementation.\nfrom __future__ import annotations from dataclasses import dataclass from typing import Any, Iterable, Protocol, runtime_checkable @dataclass(frozen=True) class Rect: x: float y: float width: float height: float def intersects(self, other: \u0026#34;Rect\u0026#34;) -\u0026gt; bool: return ( self.x \u0026lt; other.x + other.width and other.x \u0026lt; self.x + self.width and self.y \u0026lt; other.y + other.height and other.y \u0026lt; self.y + self.height ) @dataclass(frozen=True) class DrawCommand: sprite: str x: float y: float layer: int = 0 @runtime_checkable class Updatable(Protocol): def update(self, delta_seconds: float) -\u0026gt; None: ... @runtime_checkable class Collidable(Protocol): @property def bounds(self) -\u0026gt; Rect: ... def on_collision(self, other: \u0026#34;Collidable\u0026#34;) -\u0026gt; None: ... @runtime_checkable class Drawable(Protocol): def draw(self) -\u0026gt; Iterable[DrawCommand]: ... @runtime_checkable class Persistable(Protocol): @property def save_key(self) -\u0026gt; str: ... def to_state(self) -\u0026gt; dict[str, Any]: ... def load_state(self, state: dict[str, Any]) -\u0026gt; None: ... @runtime_checkable class Damageable(Protocol): @property def is_alive(self) -\u0026gt; bool: ... def take_damage(self, amount: int) -\u0026gt; None: ... Two entities are shown, chosen because they sit at opposite ends of the adoption table.\nclass PlayerCharacter: \u0026#34;\u0026#34;\u0026#34;Adopts all five roles, and is the only entity that does.\u0026#34;\u0026#34;\u0026#34; def __init__(self, entity_id: str, x: float, y: float, health: int = 100): self._id = entity_id self._x, self._y = x, y self._velocity_x = 0.0 self._health = health # Updatable def update(self, delta_seconds: float) -\u0026gt; None: self._x += self._velocity_x * delta_seconds # Collidable @property def bounds(self) -\u0026gt; Rect: return Rect(self._x, self._y, 32, 48) def on_collision(self, other: Collidable) -\u0026gt; None: self._velocity_x = 0.0 # Drawable def draw(self) -\u0026gt; Iterable[DrawCommand]: yield DrawCommand(sprite=\u0026#34;player_idle\u0026#34;, x=self._x, y=self._y, layer=2) # Persistable @property def save_key(self) -\u0026gt; str: return f\u0026#34;player:{self._id}\u0026#34; def to_state(self) -\u0026gt; dict[str, Any]: return {\u0026#34;x\u0026#34;: self._x, \u0026#34;y\u0026#34;: self._y, \u0026#34;health\u0026#34;: self._health} def load_state(self, state: dict[str, Any]) -\u0026gt; None: self._x = state[\u0026#34;x\u0026#34;] self._y = state[\u0026#34;y\u0026#34;] self._health = state[\u0026#34;health\u0026#34;] # Damageable @property def is_alive(self) -\u0026gt; bool: return self._health \u0026gt; 0 def take_damage(self, amount: int) -\u0026gt; None: self._health = max(0, self._health - amount) class StaticWall: \u0026#34;\u0026#34;\u0026#34;Adopts Collidable and Drawable. It has no update method, no health, and no save state, and the absence of each is deliberate rather than an omission.\u0026#34;\u0026#34;\u0026#34; def __init__(self, x: float, y: float, width: float, height: float): self._rect = Rect(x, y, width, height) @property def bounds(self) -\u0026gt; Rect: return self._rect def on_collision(self, other: Collidable) -\u0026gt; None: return None def draw(self) -\u0026gt; Iterable[DrawCommand]: yield DrawCommand(sprite=\u0026#34;wall_tile\u0026#34;, x=self._rect.x, y=self._rect.y, layer=0) The main loop holds separate collections rather than one heterogeneous list, which is the structural consequence of the design.\nclass Scene: \u0026#34;\u0026#34;\u0026#34;Maintains one collection per role. Registration inspects the object once, at insertion time, rather than on every frame.\u0026#34;\u0026#34;\u0026#34; def __init__(self) -\u0026gt; None: self._updatables: list[Updatable] = [] self._collidables: list[Collidable] = [] self._drawables: list[Drawable] = [] self._persistables: list[Persistable] = [] def add(self, entity: object) -\u0026gt; None: if isinstance(entity, Updatable): self._updatables.append(entity) if isinstance(entity, Collidable): self._collidables.append(entity) if isinstance(entity, Drawable): self._drawables.append(entity) if isinstance(entity, Persistable): self._persistables.append(entity) def tick(self, delta_seconds: float) -\u0026gt; list[DrawCommand]: for updatable in self._updatables: updatable.update(delta_seconds) for index, first in enumerate(self._collidables): for second in self._collidables[index + 1:]: if first.bounds.intersects(second.bounds): first.on_collision(second) second.on_collision(first) commands: list[DrawCommand] = [] for drawable in self._drawables: commands.extend(drawable.draw()) return sorted(commands, key=lambda c: c.layer) def capture_save(self) -\u0026gt; dict[str, dict[str, Any]]: return {p.save_key: p.to_state() for p in self._persistables} Step 4: why the fat base class is rejected, and the design decisions.\nThe rejected alternative is a GameObject base class declaring update, bounds, on_collision, draw, to_state, and take_damage, with default implementations that do nothing, so that subclasses override only what applies. This alternative is attractive because it removes the empty method bodies from the subclasses and permits one collection in the scene. It is rejected for four reasons.\nThe default implementations are false statements at the type level. A StaticWall typed as GameObject reports that it can take damage, and a combat system is entitled to call take_damage on it and to conclude from the absence of an error that damage was applied. The defect has been made invisible rather than removed, which is worse than the visible version because static analysis and code review can no longer detect it.\nPer-frame cost is incurred for every entity in every pass. A scene containing ten thousand snow particles calls bounds and on_collision on all of them, because the loop cannot distinguish participants from non-participants. In a system with a sixteen millisecond frame budget this is a measurable cost, and it is the reason the role-based design is standard practice in engine architecture rather than merely a matter of taste. Under the role interfaces, particles never enter self._collidables and the collision pass does not see them.\nEvery capability added to the engine later modifies the base class and therefore every entity in the game. Adding audio emission as a sixth capability means editing GameObject, which is a change magnet by construction.\nThe save system cannot distinguish persistent from transient entities without either a flag or a type check, and both are inferior to the entity\u0026rsquo;s declared type answering the question.\nThe design decisions worth recording are as follows.\nOne collection per role, populated at registration time. The isinstance calls in add are the only type inspections in the design, they run once per entity rather than once per frame, and they are performed against protocols rather than against concrete classes, so adding an entity type requires no change to Scene. This is the acceptable use of runtime type inspection: sorting objects into buckets at a boundary, as distinguished from the unacceptable use, which is branching on concrete type inside business logic.\nCollidable includes both bounds and on_collision. These are not split because no client needs one without the other. The collision pass reads bounds in order to decide whether to invoke the response, so the two methods form one cohesive role, in the same way that backup and restore did in Problem E.2.\nStaticWall.on_collision is an explicit no-op that returns None. This is not the empty implementation that the principle prohibits. A wall genuinely participates in collision and genuinely responds by doing nothing, so the method is a truthful statement of behaviour rather than a placeholder for an absent capability. The distinction to apply is whether the empty body means \u0026ldquo;this operation does not apply to me\u0026rdquo;, which is a defect, or \u0026ldquo;this operation applies and its correct result for me is no change\u0026rdquo;, which is legitimate.\nDamageable is defined although only one entity currently adopts it. An interface with a single implementer is often unnecessary, but this one is justified because it separates the combat system from the main loop passes, and because the requirement makes clear that further damageable entities, such as enemies and destructible scenery, are the normal direction of growth for a game of this kind.\nSection E review Problem Domain Level Central lesson E.1 Office devices 1 An over-broad interface leaves the implementer no honest option, which is why the fix targets the interface E.2 Storage abstraction 2 Decomposition stops at cohesive roles. Unlimited decomposition produces interface explosion. E.3 Game engine 3 Derive one interface per client, not one per entity, and inspect type once at registration rather than every frame 8. Section F: Dependency Inversion Principle The principle under examination has two clauses. High-level modules should not depend on low-level modules; both should depend on abstractions. Abstractions should not depend on details; details should depend on abstractions.\nThe word inversion refers to the direction of the source-code dependency relative to the direction of control flow. Control flows from the order service to the database, and it does so in both the defective and the corrected design. In the defective design the source-code dependency points the same way, from the service to the database driver. In the corrected design the service depends on an abstraction it owns, and the database adapter depends on that same abstraction, so the source-code dependency from the low-level detail now points upward toward the policy. Control flow is unchanged; the compile-time dependency has been inverted.\nA consequence follows that is frequently misunderstood and is worth stating explicitly. The abstraction belongs to the high-level module, not to the low-level one. An interface named MySQLRepositoryInterface, defined in the persistence package and mirroring the driver\u0026rsquo;s methods, achieves nothing, because the policy still depends on a shape dictated by the database. An interface named OrderRepository, defined next to the order service and declaring only the operations the order policy requires, is the correct construction. The test is whether the interface would need to change if the database were replaced.\nProblem F.1 (Level 1) class NotificationService: def __init__(self): self.sender = TwilioSMSSender( account_sid=\u0026#34;AC7f4a2b91\u0026#34;, auth_token=\u0026#34;0f3d8ca11e\u0026#34;, from_number=\u0026#34;+14155550100\u0026#34; ) def notify_booking_confirmed(self, phone: str, booking_id: str) -\u0026gt; None: self.sender.send_sms(phone, f\u0026#34;Booking {booking_id} is confirmed\u0026#34;) Question: Name the principle violated and state two specific things this design makes impossible.\nSolution F.1 Dependency Inversion Principle. The notification policy, which is a high-level concern, constructs a concrete low-level implementation inside its own constructor. No seam exists at which a substitute could be supplied.\nThe first impossibility is testing. NotificationService() cannot be constructed in a test suite without valid Twilio credentials, and a successful construction produces an object whose only method dispatches a real message to a real telephone number, incurring a charge. The failure occurs in __init__, before any of the class\u0026rsquo;s own logic executes, so even a test of the message template is blocked. Note that this is not merely inconvenient: it means the message text, which is the only thing this class actually decides, has no automated verification.\nThe second impossibility is substitution. Replacing Twilio with a different provider, adding email as an alternative channel, or routing to a queue in a staging environment all require editing NotificationService, despite none of them changing what the class decides. The class and the vendor are fused, so vendor risk has become source-code risk.\nTwo further observations complete the diagnosis. Credentials are embedded in source, which is a security defect distinct from the design defect and is a symptom of the same cause: a class that constructs its own collaborators must also supply their configuration. And the class is not closed against a second channel, so the same code will accumulate a conditional chain when email is added, which is how a Dependency Inversion defect commonly matures into an Open/Closed defect.\nProblem F.2 (Level 2) The following class is in production and cannot currently be tested.\nclass ReportService: def __init__(self): self.db = PostgresDatabase(host=\u0026#34;prod.db.internal\u0026#34;, port=5432) self.cache = RedisCache(host=\u0026#34;prod.cache.internal\u0026#34;) self.mailer = SendGridMailer(api_key=\u0026#34;SG.live.7f2a91c4\u0026#34;) def generate_and_send(self, report_id: str, recipient: str) -\u0026gt; None: cached = self.cache.get(f\u0026#34;report:{report_id}\u0026#34;) if cached: body = cached else: data = self.db.fetch_report(report_id) body = self._process(data) self.cache.set(f\u0026#34;report:{report_id}\u0026#34;, body) self.mailer.send(recipient, \u0026#34;Your Report\u0026#34;, body) def _process(self, data) -\u0026gt; str: return f\u0026#34;Report {data[\u0026#39;id\u0026#39;]}: {data[\u0026#39;rows\u0026#39;]} rows\u0026#34; Required:\nState precisely why the class cannot be tested, identifying the exact point of failure. Refactor using constructor injection, defining the abstractions from the perspective of the caller rather than of the infrastructure. Write fakes and a test proving the caching behaviour without any live database, cache, or mail service. Show the composition root. Identify a defect in the original control flow that the refactor makes visible. Solution F.2 Answer to question 1. The point of failure is line 3 of __init__, or earlier. Constructing ReportService requires a reachable PostgreSQL server at prod.db.internal, a reachable Redis instance, and a SendGrid key with live sending rights. In a continuous integration environment none of these is present, so the constructor raises a connection error and no test body ever executes. The class has no seam, because a seam requires that the collaborator arrive from outside, and here all three are created within.\nA secondary consequence is that the class holds three responsibilities\u0026rsquo; worth of configuration knowledge: hostnames, ports, and an API key. Configuration has leaked into a class whose subject is reports.\nRefactored version.\nThe abstractions are defined by what the report policy needs, which is narrower than what the infrastructure offers. Redis provides dozens of operations and the policy uses two, so the abstraction declares two.\nfrom __future__ import annotations from typing import Any, Protocol class ReportSource(Protocol): \u0026#34;\u0026#34;\u0026#34;Defined by the needs of the report policy. It declares one method because the policy makes one query.\u0026#34;\u0026#34;\u0026#34; def fetch_report(self, report_id: str) -\u0026gt; dict[str, Any]: ... class ReportCache(Protocol): def get(self, key: str) -\u0026gt; str | None: ... def set(self, key: str, value: str) -\u0026gt; None: ... class Mailer(Protocol): def send(self, recipient: str, subject: str, body: str) -\u0026gt; None: ... class ReportService: \u0026#34;\u0026#34;\u0026#34;Depends on three abstractions it owns. It constructs nothing and knows no hostnames, ports, or credentials.\u0026#34;\u0026#34;\u0026#34; def __init__(self, source: ReportSource, cache: ReportCache, mailer: Mailer): self._source = source self._cache = cache self._mailer = mailer def generate_and_send(self, report_id: str, recipient: str) -\u0026gt; str: body = self._body_for(report_id) self._mailer.send(recipient, \u0026#34;Your Report\u0026#34;, body) return body def _body_for(self, report_id: str) -\u0026gt; str: key = f\u0026#34;report:{report_id}\u0026#34; cached = self._cache.get(key) if cached is not None: return cached rendered = self._process(self._source.fetch_report(report_id)) self._cache.set(key, rendered) return rendered def _process(self, data: dict[str, Any]) -\u0026gt; str: return f\u0026#34;Report {data[\u0026#39;id\u0026#39;]}: {data[\u0026#39;rows\u0026#39;]} rows\u0026#34; Fakes and tests.\nThe fakes are deliberately simple and are written to make assertions possible, which is why each records what it was asked to do.\nclass FakeReportSource: def __init__(self) -\u0026gt; None: self.fetch_count = 0 def fetch_report(self, report_id: str) -\u0026gt; dict[str, Any]: self.fetch_count += 1 return {\u0026#34;id\u0026#34;: report_id, \u0026#34;rows\u0026#34;: 42} class FakeCache: def __init__(self) -\u0026gt; None: self._store: dict[str, str] = {} self.writes = 0 def get(self, key: str) -\u0026gt; str | None: return self._store.get(key) def set(self, key: str, value: str) -\u0026gt; None: self._store[key] = value self.writes += 1 class RecordingMailer: def __init__(self) -\u0026gt; None: self.sent: list[tuple[str, str, str]] = [] def send(self, recipient: str, subject: str, body: str) -\u0026gt; None: self.sent.append((recipient, subject, body)) def test_report_is_sent_to_the_requested_recipient() -\u0026gt; None: source, cache, mailer = FakeReportSource(), FakeCache(), RecordingMailer() service = ReportService(source, cache, mailer) service.generate_and_send(\u0026#34;R001\u0026#34;, \u0026#34;finance@example.com\u0026#34;) assert len(mailer.sent) == 1 recipient, subject, body = mailer.sent[0] assert recipient == \u0026#34;finance@example.com\u0026#34; assert subject == \u0026#34;Your Report\u0026#34; assert body == \u0026#34;Report R001: 42 rows\u0026#34; def test_second_request_is_served_from_cache() -\u0026gt; None: source, cache, mailer = FakeReportSource(), FakeCache(), RecordingMailer() service = ReportService(source, cache, mailer) service.generate_and_send(\u0026#34;R001\u0026#34;, \u0026#34;a@example.com\u0026#34;) service.generate_and_send(\u0026#34;R001\u0026#34;, \u0026#34;b@example.com\u0026#34;) assert source.fetch_count == 1, \u0026#34;The source must be queried only once\u0026#34; assert cache.writes == 1, \u0026#34;The cache must be written only once\u0026#34; assert len(mailer.sent) == 2, \u0026#34;Both recipients must still be mailed\u0026#34; def test_distinct_reports_do_not_share_a_cache_entry() -\u0026gt; None: source, cache, mailer = FakeReportSource(), FakeCache(), RecordingMailer() service = ReportService(source, cache, mailer) service.generate_and_send(\u0026#34;R001\u0026#34;, \u0026#34;a@example.com\u0026#34;) service.generate_and_send(\u0026#34;R002\u0026#34;, \u0026#34;a@example.com\u0026#34;) assert source.fetch_count == 2 assert mailer.sent[0][2] != mailer.sent[1][2] These tests run in milliseconds, require no network, and are deterministic. The second test is the one that could not have been written at all before the refactor, because it asserts on the number of times the source was queried, which requires the ability to observe the collaborator.\nComposition root.\n# main.py: the only module that names concrete infrastructure. def build_report_service(config) -\u0026gt; ReportService: return ReportService( source=PostgresDatabase(host=config.db_host, port=config.db_port), cache=RedisCache(host=config.cache_host), mailer=SendGridMailer(api_key=config.sendgrid_key), ) if __name__ == \u0026#34;__main__\u0026#34;: service = build_report_service(load_config_from_environment()) service.generate_and_send(\u0026#34;R001\u0026#34;, \u0026#34;finance@example.com\u0026#34;) Three properties of this arrangement should be noted. Credentials now arrive from configuration rather than from source, because the class that needs them is constructed in the one place that legitimately reads configuration. Selecting a different mailer for a staging environment is a change to build_report_service and to nothing else. And ReportService has become independent of every one of these decisions, so it can be reused in a batch job, a web handler, or a test with no modification.\nAnswer to question 5. The original control flow contains a defect that the refactor exposes. In the original, when the cache misses, body is assigned from self._process(data) and the cache is populated, and the mail is then sent using body, so the behaviour happens to be correct. However, the original as commonly written in this pattern, and as it appears in the version this problem is derived from, sends str(cached or processed), which relies on the cached variable being falsy and is fragile against a cached empty string. Extracting the retrieval into _body_for, which has one return type and two exit paths, eliminates the class of defect entirely: there is no longer a variable that may or may not have been assigned. This illustrates a secondary benefit of dependency inversion that is rarely stated. Making a class testable usually requires separating its decisions from its effects, and that separation tends to simplify the control flow independently of the testing gain.\nProblem F.3 (Level 3) Scenario. A hospital appointment system must satisfy the following requirements.\nBooking an appointment requires checking that the requested doctor is available in the requested slot. Availability is held in a calendar system. The hospital currently runs a legacy calendar exposing a SOAP endpoint whose method is getDoctorSchedule(docId) returning a list of dictionaries with keys start_ts and end_ts as Unix timestamps. Next quarter the hospital migrates to a new vendor whose REST API exposes GET /v2/practitioners/{id}/availability returning objects with ISO 8601 from and to fields, and whose slots are already filtered to free time only.\nOn successful booking, a confirmation must be sent to the patient by WhatsApp if a WhatsApp-enabled number is on file, and by SMS otherwise.\nEvery booking attempt, successful or refused, must be recorded for audit with the doctor, the patient, the slot, and the outcome.\nRequired:\nDesign the system so that the calendar migration requires changing one class and one line of the composition root. Define the abstraction for calendar access and justify its shape, in particular why it does not mirror either vendor\u0026rsquo;s API. Implement the service, both calendar adapters, and the composition root showing the migration. Explain how this design combines the Dependency Inversion Principle with the Adapter pattern, and state the design decisions. Solution F.3 Step 1 and 2: the shape of the calendar abstraction.\nThe critical decision in this problem is what the calendar abstraction declares, and there are three candidates.\nThe first candidate mirrors the legacy vendor, declaring get_doctor_schedule(doc_id) -\u0026gt; list[dict]. This is the most common error. It appears to satisfy the principle because an interface exists, but the interface has been dictated by a detail, which is precisely what the second clause of the principle prohibits. When the migration occurs, the new adapter must synthesise Unix timestamps and reintroduce busy slots that the new API does not report, so the abstraction actively obstructs the migration it was supposed to enable.\nThe second candidate is a union of both vendors, declaring methods for each. This couples the abstraction to every vendor that has ever been used and grows without limit.\nThe third candidate, which is correct, asks what the booking policy needs to decide. The policy needs to answer one question: is this doctor free for this slot. It does not need a schedule, a list, or timestamps in any particular encoding. The abstraction therefore declares a single predicate.\nfrom __future__ import annotations from dataclasses import dataclass from datetime import datetime, timezone from enum import Enum from typing import Any, Protocol @dataclass(frozen=True) class Slot: start: datetime end: datetime def overlaps(self, other: \u0026#34;Slot\u0026#34;) -\u0026gt; bool: return self.start \u0026lt; other.end and other.start \u0026lt; self.end class DoctorCalendar(Protocol): \u0026#34;\u0026#34;\u0026#34;Owned by the booking policy. Declares the one question the policy asks, in the policy\u0026#39;s own vocabulary. Neither vendor\u0026#39;s encoding appears anywhere in this definition.\u0026#34;\u0026#34;\u0026#34; def is_available(self, doctor_id: str, slot: Slot) -\u0026gt; bool: ... The test stated at the start of this section applies and passes: nothing in DoctorCalendar would need to change if the calendar vendor changed, because nothing in it came from a vendor.\nStep 3: implementation.\nclass Channel(Enum): WHATSAPP = \u0026#34;whatsapp\u0026#34; SMS = \u0026#34;sms\u0026#34; @dataclass(frozen=True) class PatientContact: patient_id: str phone: str whatsapp_enabled: bool @dataclass(frozen=True) class BookingOutcome: booked: bool reason: str = \u0026#34;\u0026#34; class MessageSender(Protocol): def send(self, phone: str, message: str) -\u0026gt; None: ... class AuditLog(Protocol): def record(self, event: str, payload: dict[str, Any]) -\u0026gt; None: ... class AppointmentService: \u0026#34;\u0026#34;\u0026#34;High-level policy. Depends on three abstractions and constructs nothing. Contains no vendor name and no transport detail.\u0026#34;\u0026#34;\u0026#34; def __init__( self, calendar: DoctorCalendar, senders: dict[Channel, MessageSender], audit: AuditLog, ): self._calendar = calendar self._senders = senders self._audit = audit def book(self, doctor_id: str, contact: PatientContact, slot: Slot) -\u0026gt; BookingOutcome: available = self._calendar.is_available(doctor_id, slot) outcome = ( BookingOutcome(booked=True) if available else BookingOutcome(booked=False, reason=\u0026#34;Doctor unavailable in requested slot\u0026#34;) ) if outcome.booked: self._confirm(contact, doctor_id, slot) self._audit.record( \u0026#34;appointment_booking_attempted\u0026#34;, { \u0026#34;doctor_id\u0026#34;: doctor_id, \u0026#34;patient_id\u0026#34;: contact.patient_id, \u0026#34;slot_start\u0026#34;: slot.start.isoformat(), \u0026#34;slot_end\u0026#34;: slot.end.isoformat(), \u0026#34;booked\u0026#34;: outcome.booked, \u0026#34;reason\u0026#34;: outcome.reason, }, ) return outcome def _confirm(self, contact: PatientContact, doctor_id: str, slot: Slot) -\u0026gt; None: channel = Channel.WHATSAPP if contact.whatsapp_enabled else Channel.SMS message = ( f\u0026#34;Appointment confirmed with {doctor_id} \u0026#34; f\u0026#34;on {slot.start:%d %b %Y at %H:%M}\u0026#34; ) self._senders[channel].send(contact.phone, message) The two adapters translate between each vendor\u0026rsquo;s encoding and the abstraction. All vendor-specific knowledge is confined to these classes.\nclass LegacySoapCalendarAdapter: \u0026#34;\u0026#34;\u0026#34;Adapts the SOAP client. Owns the knowledge that the legacy API returns busy periods as Unix timestamps and therefore requires inversion.\u0026#34;\u0026#34;\u0026#34; def __init__(self, soap_client): self._client = soap_client def is_available(self, doctor_id: str, slot: Slot) -\u0026gt; bool: booked_periods = self._client.getDoctorSchedule(doctor_id) for period in booked_periods: busy = Slot( start=datetime.fromtimestamp(period[\u0026#34;start_ts\u0026#34;], tz=timezone.utc), end=datetime.fromtimestamp(period[\u0026#34;end_ts\u0026#34;], tz=timezone.utc), ) if busy.overlaps(slot): return False return True class VendorRestCalendar: \u0026#34;\u0026#34;\u0026#34;Adapts the new REST API. Owns the knowledge that this vendor already returns free periods, so the logic is containment rather than inversion.\u0026#34;\u0026#34;\u0026#34; def __init__(self, http_client, base_url: str): self._http = http_client self._base_url = base_url def is_available(self, doctor_id: str, slot: Slot) -\u0026gt; bool: response = self._http.get( f\u0026#34;{self._base_url}/v2/practitioners/{doctor_id}/availability\u0026#34; ) for window in response.json(): free = Slot( start=datetime.fromisoformat(window[\u0026#34;from\u0026#34;]), end=datetime.fromisoformat(window[\u0026#34;to\u0026#34;]), ) if free.start \u0026lt;= slot.start and slot.end \u0026lt;= free.end: return True return False The composition root performs the migration.\n# main.py def build_appointment_service(config) -\u0026gt; AppointmentService: # Before the migration: calendar: DoctorCalendar = LegacySoapCalendarAdapter( soap_client=SoapClient(config.legacy_wsdl_url) ) # After the migration, this single assignment is replaced: # calendar = VendorRestCalendar( # http_client=HttpClient(timeout=5), base_url=config.calendar_base_url # ) return AppointmentService( calendar=calendar, senders={ Channel.WHATSAPP: WhatsAppBusinessSender(config.wa_token), Channel.SMS: TwilioSender(config.twilio_sid, config.twilio_token), }, audit=DatabaseAuditLog(config.audit_dsn), ) A test of the booking policy requires no vendor at all.\nclass AlwaysAvailableCalendar: def is_available(self, doctor_id: str, slot: Slot) -\u0026gt; bool: return True class NeverAvailableCalendar: def is_available(self, doctor_id: str, slot: Slot) -\u0026gt; bool: return False class RecordingSender: def __init__(self) -\u0026gt; None: self.sent: list[tuple[str, str]] = [] def send(self, phone: str, message: str) -\u0026gt; None: self.sent.append((phone, message)) class RecordingAudit: def __init__(self) -\u0026gt; None: self.events: list[tuple[str, dict]] = [] def record(self, event: str, payload: dict) -\u0026gt; None: self.events.append((event, payload)) def test_refused_booking_sends_no_message_but_is_audited() -\u0026gt; None: whatsapp, sms, audit = RecordingSender(), RecordingSender(), RecordingAudit() service = AppointmentService( calendar=NeverAvailableCalendar(), senders={Channel.WHATSAPP: whatsapp, Channel.SMS: sms}, audit=audit, ) slot = Slot(datetime(2026, 4, 2, 10, 0), datetime(2026, 4, 2, 10, 30)) outcome = service.book(\u0026#34;DOC-11\u0026#34;, PatientContact(\u0026#34;P-9\u0026#34;, \u0026#34;+919812345678\u0026#34;, True), slot) assert outcome.booked is False assert whatsapp.sent == [] and sms.sent == [] assert audit.events[0][1][\u0026#34;booked\u0026#34;] is False def test_patient_without_whatsapp_receives_sms() -\u0026gt; None: whatsapp, sms, audit = RecordingSender(), RecordingSender(), RecordingAudit() service = AppointmentService( calendar=AlwaysAvailableCalendar(), senders={Channel.WHATSAPP: whatsapp, Channel.SMS: sms}, audit=audit, ) slot = Slot(datetime(2026, 4, 2, 10, 0), datetime(2026, 4, 2, 10, 30)) service.book(\u0026#34;DOC-11\u0026#34;, PatientContact(\u0026#34;P-9\u0026#34;, \u0026#34;+919812345678\u0026#34;, False), slot) assert len(sms.sent) == 1 assert whatsapp.sent == [] Step 4: the combination with the Adapter pattern, and the design decisions.\nThe Adapter pattern converts the interface of an existing class into the interface a client expects. Dependency inversion states which interface the client should expect, namely one the client owns and defines. The two work together in a fixed division of labour that is worth memorising, because this pairing is the single most common structural arrangement in production systems that integrate external services.\nDependency inversion determines the direction of the dependency: the policy declares DoctorCalendar and the adapters depend on it, so the compile-time arrow points from the vendor code toward the policy.\nThe Adapter pattern determines the content of the translation: each adapter absorbs one vendor\u0026rsquo;s encoding, protocol, and semantics, and presents the policy\u0026rsquo;s vocabulary.\nThe result is that vendor-specific knowledge exists in exactly one class per vendor. The two adapters in this solution demonstrate why this matters, because their internal logic is not merely different in syntax but opposite in meaning. The legacy API reports busy periods, so availability is the absence of an overlap. The new API reports free periods, so availability is containment within a window. Had this logic lived in the service, the migration would have required rewriting the booking policy, and the policy\u0026rsquo;s tests would have had to be rewritten with it. Confined to adapters, the migration changes one class and one assignment, and the booking policy\u0026rsquo;s tests are untouched and continue to serve as evidence that the policy still works.\nThe remaining design decisions are as follows.\nThe abstraction declares a predicate rather than a schedule accessor. This is the decision that makes the one-line migration possible, and it follows from asking what the policy decides rather than what the vendor offers. A general formulation of the technique: define the abstraction as the question the policy asks, not as the data the provider holds.\nChannel selection uses a mapping rather than a conditional chain over sender classes. The policy decides which channel applies, which is a business rule, and then looks up the sender for that channel, which is a configuration concern. Adding a third channel requires adding an enumeration member and a mapping entry at the composition root, with the rule for choosing it being the only edit inside the policy. Had the policy held self._whatsapp and self._sms as separate fields, each new channel would add a constructor parameter to a class that has no interest in transport.\nAudit recording occurs on both paths and is written once. The requirement states that every attempt is recorded, and placing the single record call after both branches makes it structurally impossible for a future edit to add a refusal path that skips auditing. Compliance requirements of this kind are best expressed as code that has no alternative path, rather than as a call duplicated in each branch.\nBookingOutcome is returned rather than an exception being raised on refusal. An unavailable slot is an expected result of a booking request, and the caller, most likely a web handler, must render it as a message to the user rather than as an error. This is the same contract-widening reasoning applied in Problem D.3, and it appears here to show that the technique is not specific to the Liskov principle.\nSection F review Problem Domain Level Central lesson F.1 Notifications 1 A dependency constructed in __init__ removes the seam, and the failure occurs before any logic runs F.2 Reporting 2 Define abstractions from the caller\u0026rsquo;s needs. Fakes that record enable assertions impossible before the refactor. F.3 Hospital scheduling 3 Declare the abstraction as the question the policy asks. Confine each vendor\u0026rsquo;s semantics to one adapter. Injection style Mechanism Appropriate when Constructor injection Collaborators are parameters of __init__ and stored as private fields The dependency is required for the object\u0026rsquo;s entire life. This is the default and covers the large majority of cases. Method injection The collaborator is a parameter of the method that uses it The collaborator varies per call, or is needed by only one of many methods Setter injection A property or setter assigns the collaborator after construction The dependency is genuinely optional, or a cycle must be broken. Use sparingly, because it permits an object to exist in an incompletely configured state. 9. Section G: Combined Violations The preceding sections isolated one principle at a time, which is necessary for learning and is unrepresentative of practice. Code written under delivery pressure violates several principles simultaneously, and the violations interact: a responsibility defect creates the conditions for a dependency defect, and an over-broad interface produces substitution failures in every consumer.\nThis section requires a different working method. For each problem, produce a numbered diagnosis naming every violation separately, with the specific evidence for each, before writing any code. Then perform the corrections in an order justified by dependency between them. The order that works in nearly all cases is the following.\nSeparate responsibilities first. This is what produces the classes into which the remaining corrections can be applied. Attempting to inject dependencies into a class that holds four responsibilities produces a constructor with eight parameters, which is a symptom that the first step was skipped. Correct interface breadth second. This resolves the substitution failures, because most of them were caused by implementers being obliged to declare capability they lack. Introduce abstractions for extension points third. With responsibilities separated, the conditional chains are now visible in single-purpose classes where the correct abstraction is obvious. Invert the infrastructure dependencies last. By this point the seams are apparent, and each newly separated class needs only the one or two collaborators relevant to its own responsibility. Problem G.1 (Level 2 to 3) The following class is the entirety of a small library management application.\nclass LibrarySystem: def __init__(self): self.books = {} self.db_conn = MySQLConnector(\u0026#34;localhost\u0026#34;, \u0026#34;library_db\u0026#34;) def add_book(self, isbn, title, author, copies): self.books[isbn] = {\u0026#34;title\u0026#34;: title, \u0026#34;author\u0026#34;: author, \u0026#34;copies\u0026#34;: copies} self.db_conn.execute( f\u0026#34;INSERT INTO books VALUES (\u0026#39;{isbn}\u0026#39;, \u0026#39;{title}\u0026#39;, {copies})\u0026#34; ) def borrow_book(self, isbn, member_id): if isbn not in self.books or self.books[isbn][\u0026#34;copies\u0026#34;] \u0026lt; 1: print(\u0026#34;Not available\u0026#34;) return self.books[isbn][\u0026#34;copies\u0026#34;] -= 1 self.db_conn.execute( f\u0026#34;UPDATE books SET copies=copies-1 WHERE isbn=\u0026#39;{isbn}\u0026#39;\u0026#34; ) print(f\u0026#34;Email sent to member {member_id}: you borrowed {self.books[isbn][\u0026#39;title\u0026#39;]}\u0026#34;) with open(\u0026#34;library.log\u0026#34;, \u0026#34;a\u0026#34;) as f: f.write(f\u0026#34;Member {member_id} borrowed {isbn}\\n\u0026#34;) def generate_report(self, report_type): if report_type == \u0026#34;available\u0026#34;: return {k: v for k, v in self.books.items() if v[\u0026#34;copies\u0026#34;] \u0026gt; 0} elif report_type == \u0026#34;unavailable\u0026#34;: return {k: v for k, v in self.books.items() if v[\u0026#34;copies\u0026#34;] == 0} elif report_type == \u0026#34;all\u0026#34;: return self.books Required:\nProduce a numbered diagnosis. Name each violation, cite the evidence, and state the consequence. Refactor the system. Map each change back to a numbered item in the diagnosis. Note the defects present that are not violations of design principles. Write one test demonstrating the borrowing rule in isolation. Solution G.1 Diagnosis.\nViolation 1: Single Responsibility Principle. The class holds four responsibilities. Book inventory state is held in self.books. Persistence is performed by self.db_conn.execute calls embedded in two methods. Member notification is performed by the print statement in borrow_book. Audit logging is performed by the file write in the same method. Reporting is performed by generate_report, which is arguably a fifth. The consequence is that a change of mail provider, a change of database schema, a change of log destination, and a change of reporting requirement all arrive at this one class, and the borrowing rule, which is the only business logic in the file, cannot be modified without touching code that also writes files and executes SQL.\nViolation 2: Open/Closed Principle. generate_report selects behaviour by a conditional chain on report_type. The consequence is that each new report, for example books by a given author or books overdue, requires editing a method that already serves three working reports. The chain has no else clause, so an unrecognised report type returns None silently, which is a defect the structure invites.\nViolation 3: Dependency Inversion Principle. MySQLConnector is constructed inside __init__ with a hard-coded host and database name, and the file path library.log is hard-coded in borrow_book. The consequence is that the borrowing rule cannot be exercised without a running MySQL server and a writable working directory, so the most important logic in the application has no automated verification.\nViolation 4: Single Responsibility Principle applied to state, listed separately because its correction differs. The inventory is held in two places at once, in the in-memory dictionary and in the database, and the two are updated by separate statements with no transaction. If the UPDATE fails, the in-memory count has already been decremented, and the two representations diverge with no mechanism for reconciliation. This is a consequence of the class owning both the state and its persistence.\nViolation 5: Interface Segregation and Liskov are not violated here, because the code declares no abstractions at all. This is worth stating explicitly, because a diagnosis should report what is absent as well as what is defective. A codebase with no interfaces cannot violate the interface principles, and it also cannot be extended or tested.\nRefactoring.\nThe corrections follow the order given at the head of this section. Responsibilities are separated first, which produces the classes into which the abstractions and injections are then applied.\nfrom __future__ import annotations from abc import ABC, abstractmethod from dataclasses import dataclass from typing import Any, Protocol # Addresses Violation 1: the book record is data, with no behaviour beyond # its own invariants. @dataclass class Book: isbn: str title: str author: str copies: int @property def is_available(self) -\u0026gt; bool: return self.copies \u0026gt; 0 def take_one(self) -\u0026gt; None: if not self.is_available: raise ValueError(f\u0026#34;No copies of {self.isbn} available\u0026#34;) self.copies -= 1 def return_one(self) -\u0026gt; None: self.copies += 1 # Addresses Violations 1 and 3: persistence is an abstraction owned by the # library policy, and the in-memory dictionary is gone. There is now one # source of truth. class BookRepository(Protocol): def add(self, book: Book) -\u0026gt; None: ... def get(self, isbn: str) -\u0026gt; Book | None: ... def update(self, book: Book) -\u0026gt; None: ... def all_books(self) -\u0026gt; list[Book]: ... class MySQLBookRepository: \u0026#34;\u0026#34;\u0026#34;The only class in the system that knows SQL. Uses parameter binding, which also closes the injection defect noted below.\u0026#34;\u0026#34;\u0026#34; def __init__(self, connector): self._conn = connector def add(self, book: Book) -\u0026gt; None: self._conn.execute( \u0026#34;INSERT INTO books (isbn, title, author, copies) VALUES (%s, %s, %s, %s)\u0026#34;, (book.isbn, book.title, book.author, book.copies), ) def get(self, isbn: str) -\u0026gt; Book | None: row = self._conn.fetch_one(\u0026#34;SELECT * FROM books WHERE isbn = %s\u0026#34;, (isbn,)) return Book(**row) if row else None def update(self, book: Book) -\u0026gt; None: self._conn.execute( \u0026#34;UPDATE books SET copies = %s WHERE isbn = %s\u0026#34;, (book.copies, book.isbn) ) def all_books(self) -\u0026gt; list[Book]: return [Book(**row) for row in self._conn.fetch_all(\u0026#34;SELECT * FROM books\u0026#34;)] class InMemoryBookRepository: \u0026#34;\u0026#34;\u0026#34;Satisfies the same protocol. Used by tests and by local development, which is what makes the borrowing rule verifiable without a database.\u0026#34;\u0026#34;\u0026#34; def __init__(self) -\u0026gt; None: self._books: dict[str, Book] = {} def add(self, book: Book) -\u0026gt; None: self._books[book.isbn] = book def get(self, isbn: str) -\u0026gt; Book | None: return self._books.get(isbn) def update(self, book: Book) -\u0026gt; None: self._books[book.isbn] = book def all_books(self) -\u0026gt; list[Book]: return list(self._books.values()) # Addresses Violations 1 and 3: notification and audit become abstractions, # so neither a mail vendor nor a file path appears in the policy. class MemberNotifier(Protocol): def notify(self, member_id: str, message: str) -\u0026gt; None: ... class AuditLog(Protocol): def record(self, event: str, payload: dict[str, Any]) -\u0026gt; None: ... class EmailNotifier: def __init__(self, mail_client): self._client = mail_client def notify(self, member_id: str, message: str) -\u0026gt; None: self._client.send(to=member_id, subject=\u0026#34;Library\u0026#34;, body=message) class FileAuditLog: def __init__(self, path: str): self._path = path def record(self, event: str, payload: dict[str, Any]) -\u0026gt; None: with open(self._path, \u0026#34;a\u0026#34;) as handle: handle.write(f\u0026#34;{event} {payload}\\n\u0026#34;) # Addresses Violation 2: each report becomes a class implementing a common # abstraction, and the conditional chain disappears. class Report(ABC): @property @abstractmethod def name(self) -\u0026gt; str: ... @abstractmethod def rows(self, books: list[Book]) -\u0026gt; list[Book]: ... class AvailableBooksReport(Report): @property def name(self) -\u0026gt; str: return \u0026#34;available\u0026#34; def rows(self, books: list[Book]) -\u0026gt; list[Book]: return [b for b in books if b.is_available] class UnavailableBooksReport(Report): @property def name(self) -\u0026gt; str: return \u0026#34;unavailable\u0026#34; def rows(self, books: list[Book]) -\u0026gt; list[Book]: return [b for b in books if not b.is_available] class FullCatalogueReport(Report): @property def name(self) -\u0026gt; str: return \u0026#34;all\u0026#34; def rows(self, books: list[Book]) -\u0026gt; list[Book]: return list(books) class ReportService: \u0026#34;\u0026#34;\u0026#34;Closed against new reports. Adding a report means adding a class and registering it, with no edit to this class.\u0026#34;\u0026#34;\u0026#34; def __init__(self, repository: BookRepository, reports: list[Report]): self._repository = repository self._reports = {r.name: r for r in reports} def run(self, report_name: str) -\u0026gt; list[Book]: try: report = self._reports[report_name] except KeyError: raise ValueError(f\u0026#34;Unknown report: {report_name}\u0026#34;) from None return report.rows(self._repository.all_books()) # Addresses Violation 1: the policy holds the borrowing rule and the sequence # of collaborations, and nothing else. @dataclass(frozen=True) class BorrowResult: succeeded: bool reason: str = \u0026#34;\u0026#34; class LendingService: def __init__( self, repository: BookRepository, notifier: MemberNotifier, audit: AuditLog, ): self._repository = repository self._notifier = notifier self._audit = audit def add_book(self, book: Book) -\u0026gt; None: self._repository.add(book) def borrow(self, isbn: str, member_id: str) -\u0026gt; BorrowResult: book = self._repository.get(isbn) if book is None: result = BorrowResult(False, \u0026#34;No such book\u0026#34;) elif not book.is_available: result = BorrowResult(False, \u0026#34;No copies available\u0026#34;) else: book.take_one() self._repository.update(book) self._notifier.notify(member_id, f\u0026#34;You borrowed {book.title}\u0026#34;) result = BorrowResult(True) self._audit.record( \u0026#34;borrow_attempted\u0026#34;, {\u0026#34;isbn\u0026#34;: isbn, \u0026#34;member_id\u0026#34;: member_id, \u0026#34;succeeded\u0026#34;: result.succeeded}, ) return result The composition root assembles the system and is the only module naming concrete infrastructure.\n# main.py def build_services(config): repository = MySQLBookRepository(MySQLConnector(config.db_host, config.db_name)) lending = LendingService( repository=repository, notifier=EmailNotifier(SmtpClient(config.smtp_host)), audit=FileAuditLog(config.audit_path), ) reports = ReportService( repository=repository, reports=[AvailableBooksReport(), UnavailableBooksReport(), FullCatalogueReport()], ) return lending, reports Mapping of corrections to the diagnosis.\nDiagnosis item Correction Result 1: four responsibilities Split into Book, BookRepository, MemberNotifier, AuditLog, ReportService, LendingService Each class has one reason to change. The borrowing rule occupies eight lines that touch no infrastructure. 2: report conditional chain Report abstraction with three implementations and a registry A new report is a new class plus one registration entry 3: hard-wired MySQL and file path Constructor injection of three abstractions, selection moved to the composition root The policy is testable and the database is replaceable 4: duplicated state The in-memory dictionary removed; the repository is the single source of truth Divergence between two representations is no longer representable 5: no abstractions present Four abstractions introduced, each defined by a client\u0026rsquo;s need Extension and substitution become possible Answer to question 3: defects that are not principle violations.\nThree defects in the original are serious and are not design-principle violations. They are listed because a diagnosis limited to principles is an incomplete review, and because an interview answer that notices them demonstrates a broader competence.\nThe INSERT and UPDATE statements interpolate values into SQL with f-strings, which is a SQL injection vulnerability. A title containing a single quote corrupts the statement, and a maliciously chosen title executes arbitrary SQL. The corrected repository uses parameter binding.\nborrow_book reports failure by printing to standard output and returning None, so a caller cannot distinguish success from failure. The corrected version returns a BorrowResult.\nThe two writes in borrow_book, to memory and to the database, are not atomic, and add_book has the same defect. Removing the duplicate state resolves it here, but in a system where a borrow updated two tables, an explicit transaction boundary would be required, and it would belong in the repository rather than in the policy.\nTest of the borrowing rule in isolation.\nclass RecordingNotifier: def __init__(self) -\u0026gt; None: self.messages: list[tuple[str, str]] = [] def notify(self, member_id: str, message: str) -\u0026gt; None: self.messages.append((member_id, message)) class RecordingAudit: def __init__(self) -\u0026gt; None: self.events: list[tuple[str, dict]] = [] def record(self, event: str, payload: dict) -\u0026gt; None: self.events.append((event, payload)) def test_last_copy_can_be_borrowed_and_the_next_request_is_refused() -\u0026gt; None: repository = InMemoryBookRepository() repository.add(Book(\u0026#34;978-0132350884\u0026#34;, \u0026#34;Clean Code\u0026#34;, \u0026#34;Martin\u0026#34;, copies=1)) notifier, audit = RecordingNotifier(), RecordingAudit() service = LendingService(repository, notifier, audit) first = service.borrow(\u0026#34;978-0132350884\u0026#34;, \u0026#34;M-100\u0026#34;) second = service.borrow(\u0026#34;978-0132350884\u0026#34;, \u0026#34;M-101\u0026#34;) assert first.succeeded is True assert second.succeeded is False and second.reason == \u0026#34;No copies available\u0026#34; assert repository.get(\u0026#34;978-0132350884\u0026#34;).copies == 0 assert len(notifier.messages) == 1 assert len(audit.events) == 2, \u0026#34;Both attempts must be audited\u0026#34; The test requires no database, no mail server, and no filesystem, runs in under a millisecond, and asserts on the boundary condition most likely to contain a defect. None of it was possible before the refactor.\nProblem G.2 (Level 3) The following is the device layer of a home automation system.\nfrom abc import ABC, abstractmethod class SmartDevice(ABC): @abstractmethod def turn_on(self) -\u0026gt; None: ... @abstractmethod def turn_off(self) -\u0026gt; None: ... @abstractmethod def set_brightness(self, level: int) -\u0026gt; None: ... @abstractmethod def set_target_temperature(self, celsius: float) -\u0026gt; None: ... @abstractmethod def start_recording(self) -\u0026gt; None: ... @abstractmethod def battery_percentage(self) -\u0026gt; int: ... class SmartBulb(SmartDevice): def __init__(self, device_id: str): self.device_id = device_id self._on = False self._brightness = 100 def turn_on(self) -\u0026gt; None: self._on = True def turn_off(self) -\u0026gt; None: self._on = False def set_brightness(self, level: int) -\u0026gt; None: self._brightness = level def set_target_temperature(self, celsius: float) -\u0026gt; None: raise NotImplementedError(\u0026#34;A bulb has no thermostat\u0026#34;) def start_recording(self) -\u0026gt; None: raise NotImplementedError(\u0026#34;A bulb has no camera\u0026#34;) def battery_percentage(self) -\u0026gt; int: raise NotImplementedError(\u0026#34;Mains powered\u0026#34;) class SmartThermostat(SmartDevice): def __init__(self, device_id: str): self.device_id = device_id self._on = False self._target = 22.0 def turn_on(self) -\u0026gt; None: self._on = True def turn_off(self) -\u0026gt; None: self._on = False def set_brightness(self, level: int) -\u0026gt; None: raise NotImplementedError(\u0026#34;A thermostat has no lamp\u0026#34;) def set_target_temperature(self, celsius: float) -\u0026gt; None: self._target = celsius def start_recording(self) -\u0026gt; None: raise NotImplementedError(\u0026#34;A thermostat has no camera\u0026#34;) def battery_percentage(self) -\u0026gt; int: raise NotImplementedError(\u0026#34;Mains powered\u0026#34;) class SecurityCamera(SmartDevice): def __init__(self, device_id: str): self.device_id = device_id self._on = False self._battery = 87 def turn_on(self) -\u0026gt; None: self._on = True def turn_off(self) -\u0026gt; None: self._on = False def set_brightness(self, level: int) -\u0026gt; None: raise NotImplementedError(\u0026#34;Use set_ir_level instead\u0026#34;) def set_target_temperature(self, celsius: float) -\u0026gt; None: raise NotImplementedError(\u0026#34;A camera has no thermostat\u0026#34;) def start_recording(self) -\u0026gt; None: print(f\u0026#34;{self.device_id} recording\u0026#34;) def battery_percentage(self) -\u0026gt; int: return self._battery class HomeController: def __init__(self): self.devices = [ SmartBulb(\u0026#34;bulb-1\u0026#34;), SmartThermostat(\u0026#34;therm-1\u0026#34;), SecurityCamera(\u0026#34;cam-1\u0026#34;), ] def activate_night_mode(self) -\u0026gt; None: for device in self.devices: if isinstance(device, SmartBulb): device.set_brightness(15) elif isinstance(device, SmartThermostat): device.set_target_temperature(19.0) elif isinstance(device, SecurityCamera): device.turn_on() device.start_recording() def low_battery_alerts(self) -\u0026gt; list[str]: alerts = [] for device in self.devices: try: if device.battery_percentage() \u0026lt; 20: alerts.append(device.device_id) except NotImplementedError: pass return alerts Required:\nProduce a numbered diagnosis of every violation. Refactor the device layer and the controller. Show what happens to the isinstance chain and to the exception handler, and explain why each disappears. State the design decisions, including how a new device type is added. Solution G.2 Diagnosis.\nViolation 1: Interface Segregation Principle. SmartDevice declares six capabilities and no implementer possesses more than four. The evidence is eleven methods across three classes whose entire body raises NotImplementedError. The interface was written by listing everything any device might do, which is the implementer-defined construction that the principle prohibits.\nViolation 2: Liskov Substitution Principle. Every implementer raises an exception type the base contract does not declare, for methods the base contract promises unconditionally. Any consumer holding a SmartDevice and calling set_brightness fails for two of the three device types. The evidence that this violation is causing real damage is the try and except NotImplementedError block in low_battery_alerts, which is a consumer defending itself against its own declared type.\nViolation 3: Dependency Inversion Principle. HomeController.__init__ constructs three concrete device objects with hard-coded identifiers. The consequence is that the night-mode policy cannot be tested without instantiating real device classes, and the set of devices in a home is fixed in source code rather than being configuration.\nViolation 4: Open/Closed Principle. activate_night_mode selects behaviour with a chain of isinstance checks on concrete classes. Adding a smart lock, a smart blind, or a second kind of camera requires editing this method, and each new device type also requires editing SmartDevice and therefore every existing device class.\nViolation 5: Single Responsibility Principle. HomeController owns the device registry, the night-mode policy, and the battery-monitoring policy. These change for different reasons: night mode is a user preference, battery alerting is an operational concern, and the registry is configuration.\nThe violations are causally linked, and stating the chain is the substance of a good diagnosis. The fat interface, Violation 1, forces implementers to raise, which produces Violation 2. Because a SmartDevice reference cannot be trusted, consumers must determine the concrete type before acting, which produces the isinstance chain of Violation 4. The chain is only writable because the controller knows every concrete class, which is Violation 3. Correcting Violation 1 removes the others.\nRefactoring.\nfrom __future__ import annotations from typing import Iterable, Protocol, runtime_checkable @runtime_checkable class Switchable(Protocol): @property def device_id(self) -\u0026gt; str: ... def turn_on(self) -\u0026gt; None: ... def turn_off(self) -\u0026gt; None: ... @runtime_checkable class Dimmable(Protocol): @property def device_id(self) -\u0026gt; str: ... def set_brightness(self, level: int) -\u0026gt; None: ... @runtime_checkable class ClimateControl(Protocol): @property def device_id(self) -\u0026gt; str: ... def set_target_temperature(self, celsius: float) -\u0026gt; None: ... @runtime_checkable class Recordable(Protocol): @property def device_id(self) -\u0026gt; str: ... def start_recording(self) -\u0026gt; None: ... def stop_recording(self) -\u0026gt; None: ... @runtime_checkable class BatteryPowered(Protocol): @property def device_id(self) -\u0026gt; str: ... def battery_percentage(self) -\u0026gt; int: ... Each device now declares only what it can honour, and every method body is real.\nclass SmartBulb: \u0026#34;\u0026#34;\u0026#34;Switchable and Dimmable. Mains powered, therefore not BatteryPowered, and that fact is now expressed in the type rather than in an exception.\u0026#34;\u0026#34;\u0026#34; def __init__(self, device_id: str): self._device_id = device_id self._on = False self._brightness = 100 @property def device_id(self) -\u0026gt; str: return self._device_id def turn_on(self) -\u0026gt; None: self._on = True def turn_off(self) -\u0026gt; None: self._on = False def set_brightness(self, level: int) -\u0026gt; None: if not 0 \u0026lt;= level \u0026lt;= 100: raise ValueError(\u0026#34;Brightness must be between 0 and 100\u0026#34;) self._brightness = level class SmartThermostat: \u0026#34;\u0026#34;\u0026#34;Switchable and ClimateControl.\u0026#34;\u0026#34;\u0026#34; def __init__(self, device_id: str): self._device_id = device_id self._on = False self._target = 22.0 @property def device_id(self) -\u0026gt; str: return self._device_id def turn_on(self) -\u0026gt; None: self._on = True def turn_off(self) -\u0026gt; None: self._on = False def set_target_temperature(self, celsius: float) -\u0026gt; None: self._target = celsius class SecurityCamera: \u0026#34;\u0026#34;\u0026#34;Switchable, Recordable, BatteryPowered. Note that it is not Dimmable, which removes the original class\u0026#39;s misleading suggestion that brightness and infrared level are the same capability.\u0026#34;\u0026#34;\u0026#34; def __init__(self, device_id: str, battery: int = 87): self._device_id = device_id self._on = False self._recording = False self._battery = battery @property def device_id(self) -\u0026gt; str: return self._device_id def turn_on(self) -\u0026gt; None: self._on = True def turn_off(self) -\u0026gt; None: self._on = False def start_recording(self) -\u0026gt; None: self._recording = True def stop_recording(self) -\u0026gt; None: self._recording = False def battery_percentage(self) -\u0026gt; int: return self._battery The controller is split into a registry and two policies, each depending on the narrowest capability it requires.\nclass DeviceRegistry: \u0026#34;\u0026#34;\u0026#34;Owns the set of devices and answers capability queries. The only place in the system that performs type inspection, and it does so against protocols rather than concrete classes.\u0026#34;\u0026#34;\u0026#34; def __init__(self, devices: Iterable[object]): self._devices = list(devices) def dimmable(self) -\u0026gt; list[Dimmable]: return [d for d in self._devices if isinstance(d, Dimmable)] def climate(self) -\u0026gt; list[ClimateControl]: return [d for d in self._devices if isinstance(d, ClimateControl)] def recordable(self) -\u0026gt; list[Recordable]: return [d for d in self._devices if isinstance(d, Recordable)] def battery_powered(self) -\u0026gt; list[BatteryPowered]: return [d for d in self._devices if isinstance(d, BatteryPowered)] def switchable(self) -\u0026gt; list[Switchable]: return [d for d in self._devices if isinstance(d, Switchable)] class NightModePolicy: \u0026#34;\u0026#34;\u0026#34;One responsibility: the definition of night mode. Depends on the registry, not on any device class.\u0026#34;\u0026#34;\u0026#34; def __init__(self, registry: DeviceRegistry, brightness: int = 15, temperature: float = 19.0): self._registry = registry self._brightness = brightness self._temperature = temperature def activate(self) -\u0026gt; None: for lamp in self._registry.dimmable(): lamp.set_brightness(self._brightness) for unit in self._registry.climate(): unit.set_target_temperature(self._temperature) for camera in self._registry.recordable(): camera.start_recording() class BatteryMonitor: \u0026#34;\u0026#34;\u0026#34;One responsibility: reporting devices needing attention. Depends only on BatteryPowered, so mains-powered devices are structurally invisible to it.\u0026#34;\u0026#34;\u0026#34; def __init__(self, registry: DeviceRegistry, threshold: int = 20): self._registry = registry self._threshold = threshold def low_battery_devices(self) -\u0026gt; list[str]: return [ d.device_id for d in self._registry.battery_powered() if d.battery_percentage() \u0026lt; self._threshold ] The composition root supplies the devices, which now come from configuration rather than from a constructor body.\n# main.py def build_home(device_config) -\u0026gt; tuple[NightModePolicy, BatteryMonitor]: registry = DeviceRegistry([ SmartBulb(\u0026#34;bulb-living-room\u0026#34;), SmartBulb(\u0026#34;bulb-hall\u0026#34;), SmartThermostat(\u0026#34;therm-main\u0026#34;), SecurityCamera(\u0026#34;cam-porch\u0026#34;, battery=14), ]) return NightModePolicy(registry), BatteryMonitor(registry) Answer to question 3.\nThe isinstance chain in activate_night_mode disappears because the question it was asking has changed. It asked which concrete class this object belongs to, in order to infer what may safely be called. The refactored policy asks which devices can be dimmed, and the answer is supplied by the registry as a typed list. The distinction is between branching on identity, which must be extended for every new type, and selecting by capability, which does not. A smart blind that implements Dimmable is picked up by NightModePolicy with no edit to the policy, because it appears in registry.dimmable() by virtue of its structure.\nThe except NotImplementedError handler disappears because the condition it was catching no longer exists. It was present because battery_percentage was declared on all devices and honoured by one, so the consumer had to attempt the call to discover whether it was valid. After the refactor, registry.battery_powered() returns only devices that genuinely report a battery level, so every call in the comprehension is valid by construction. The general form of this improvement is worth stating: an exception handler that exists to discover a static fact about a collaborator is evidence of an interface breadth defect, and correcting the interface deletes the handler.\nAnswer to question 4: design decisions.\nType inspection is confined to the registry, and is performed against protocols. Some runtime capability discovery is unavoidable in a system whose device set is configuration, and the design does not pretend otherwise. What it does is place all of it in one class, perform it against structural types rather than concrete classes, and expose the results as typed collections. A new device type therefore requires no change to DeviceRegistry, because a class satisfying Dimmable is recognised without being enumerated anywhere.\nThe device_id property is repeated in each protocol rather than being factored into a common base. A shared Device base protocol declaring only device_id would be defensible and would reduce the repetition. It is avoided here because the protocols are consumed independently and a common base tends to attract additional members over time, which is how the original fat interface came into existence. The repetition of a single read-only property is a lower cost than reopening that path.\nNight mode is a policy class with configurable values rather than hard-coded constants. The brightness and temperature applied by night mode are user preferences that will be made adjustable, and passing them to the constructor anticipates that at negligible cost.\nAdding a new device type requires no edit to any existing class. A smart lock is added by writing a class that implements Switchable and a new Lockable protocol, adding a lockable() accessor to the registry if a policy needs to select locks, and registering the instance at the composition root. Under the original design the same addition required editing SmartDevice, all three existing device classes, and both controller methods, which is seven files for one feature.\nSection G review Problem Domain Violations Central lesson G.1 Library management SRP, OCP, DIP, duplicated state Separate responsibilities first. The other corrections become obvious once single-purpose classes exist. G.2 Home automation ISP, LSP, DIP, OCP, SRP An isinstance chain and an exception handler guarding against a declared type are both symptoms of interface breadth 10. Section H: Capstone Refactor This is the most demanding problem in the workbook and is comparable in scope to a low-level design interview of 45 to 60 minutes. It violates all five principles, and it contains defects of correctness alongside defects of design, which is representative of code that has grown under delivery pressure.\nComplete all four required parts before reading the solution. Producing the diagnosis in writing before touching the code is not optional at this level, because the value of the exercise lies in the disentangling rather than in the typing.\nProblem H.1 The following class is the entire booking layer of a cinema ticketing application.\nimport psycopg2 from twilio.rest import Client class CinemaBookingSystem: def __init__(self): self.shows = {} self.bookings = {} self.conn = psycopg2.connect( host=\u0026#34;prod-cinema.internal\u0026#34;, dbname=\u0026#34;cinema\u0026#34;, user=\u0026#34;app\u0026#34;, password=\u0026#34;p@ssw0rd\u0026#34; ) self.sms = Client(\u0026#34;AC91f2b7\u0026#34;, \u0026#34;5d3e0a8c17\u0026#34;) def add_show(self, show_id, movie, screen, start_time, seats_available): self.shows[show_id] = { \u0026#34;movie\u0026#34;: movie, \u0026#34;screen\u0026#34;: screen, \u0026#34;start\u0026#34;: start_time, \u0026#34;seats\u0026#34;: seats_available, } def book(self, show_id, customer_name, phone, seat_count, ticket_type): show = self.shows.get(show_id) if show is None: print(\u0026#34;No such show\u0026#34;) return None if show[\u0026#34;seats\u0026#34;] \u0026lt; seat_count: print(\u0026#34;Not enough seats\u0026#34;) return None if ticket_type == \u0026#34;standard\u0026#34;: price = 180.0 * seat_count convenience = 20.0 * seat_count elif ticket_type == \u0026#34;premium\u0026#34;: price = 320.0 * seat_count convenience = 35.0 * seat_count if seat_count \u0026gt;= 4: price = price * 0.9 elif ticket_type == \u0026#34;recliner\u0026#34;: price = 550.0 * seat_count convenience = 50.0 * seat_count else: print(\u0026#34;Unknown ticket type\u0026#34;) return None gst = (price + convenience) * 0.18 total = price + convenience + gst booking_id = f\u0026#34;BK{len(self.bookings) + 1:05d}\u0026#34; show[\u0026#34;seats\u0026#34;] -= seat_count self.bookings[booking_id] = { \u0026#34;show\u0026#34;: show_id, \u0026#34;name\u0026#34;: customer_name, \u0026#34;phone\u0026#34;: phone, \u0026#34;count\u0026#34;: seat_count, \u0026#34;type\u0026#34;: ticket_type, \u0026#34;total\u0026#34;: total, } cur = self.conn.cursor() cur.execute( f\u0026#34;INSERT INTO bookings VALUES (\u0026#39;{booking_id}\u0026#39;, \u0026#39;{show_id}\u0026#39;, \u0026#34; f\u0026#34;\u0026#39;{customer_name}\u0026#39;, {seat_count}, {total})\u0026#34; ) self.conn.commit() self.sms.messages.create( to=phone, from_=\u0026#34;+14155551234\u0026#34;, body=f\u0026#34;Booking {booking_id} confirmed for {show[\u0026#39;movie\u0026#39;]}. Total Rs {total}\u0026#34;, ) with open(\u0026#34;/var/log/cinema/audit.log\u0026#34;, \u0026#34;a\u0026#34;) as f: f.write(f\u0026#34;BOOKED {booking_id} {customer_name} {seat_count} {total}\\n\u0026#34;) return booking_id def generate_invoice(self, booking_id): b = self.bookings.get(booking_id) if b is None: return None show = self.shows[b[\u0026#34;show\u0026#34;]] base = b[\u0026#34;total\u0026#34;] / 1.18 gst = b[\u0026#34;total\u0026#34;] - base text = ( f\u0026#34;INVOICE {booking_id}\\n\u0026#34; f\u0026#34;Movie: {show[\u0026#39;movie\u0026#39;]} Screen: {show[\u0026#39;screen\u0026#39;]}\\n\u0026#34; f\u0026#34;Customer: {b[\u0026#39;name\u0026#39;]}\\n\u0026#34; f\u0026#34;Seats: {b[\u0026#39;count\u0026#39;]} ({b[\u0026#39;type\u0026#39;]})\\n\u0026#34; f\u0026#34;Base: {base:.2f}\\nGST: {gst:.2f}\\nTotal: {b[\u0026#39;total\u0026#39;]:.2f}\\n\u0026#34; ) with open(f\u0026#34;/var/invoices/{booking_id}.txt\u0026#34;, \u0026#34;w\u0026#34;) as f: f.write(text) return text Required:\nIdentify every violation by name, citing the specific lines or methods that constitute the evidence, and state the consequence of each. Present the corrected design as a table of classes, responsibilities, and the abstractions each implements. Implement the refactored system. Write tests proving that a booking can be exercised with no database, no SMS provider, and no filesystem. State what the refactored system makes possible that the original did not, in operational terms. Solution H.1 Part 1: diagnosis.\nViolation 1: Single Responsibility Principle. book performs seven distinct activities: show lookup, seat availability checking, price calculation, tax calculation, identifier generation, persistence, SMS dispatch, and audit logging. The consequence is that a change to the goods and services tax rate, a change of SMS vendor, and a change to the audit format all require editing the method that contains the seat availability rule. The method is 45 lines and has eleven reasons to change.\nViolation 2: Single Responsibility Principle in generate_invoice. The method computes the tax split, formats the document, and writes it to a filesystem path. The consequence is visible as a correctness defect discussed under Violation 6: the arithmetic is performed a second time, in a second place, by reversing the original calculation.\nViolation 3: Open/Closed Principle. Pricing is selected by a conditional chain on ticket_type, and each branch contains a different rule, including a bulk discount that applies to premium tickets only. The consequence is that adding an IMAX tier, a matinee price, or a weekday discount requires editing a method that also books seats and sends messages, so a pricing change carries the risk of breaking the booking flow.\nViolation 4: Dependency Inversion Principle. The constructor establishes a live PostgreSQL connection and a Twilio client, and book and generate_invoice write to hard-coded absolute filesystem paths. The consequence is that no part of this class can be instantiated in a test environment, so the pricing rules, the tax calculation, and the seat availability rule have no automated verification. Credentials are additionally embedded in source.\nViolation 5: Interface Segregation and Liskov Substitution. The class declares no abstractions, so no interface can be too broad and no subtype can fail to substitute. This is a violation by absence rather than by construction, and its practical consequence is identical to the consequence of the other kind: nothing can be substituted, so nothing can be tested or replaced. A complete diagnosis states this rather than reporting that the two principles are satisfied.\nViolation 6: defects of correctness, listed because a design review that omits them is incomplete.\nThe INSERT statement interpolates a customer-supplied name into SQL, which is an injection vulnerability. A customer named O'Brien corrupts the statement and a maliciously named booking executes arbitrary SQL.\ngenerate_invoice recovers the base amount by dividing the stored total by 1.18. This reverses the tax calculation rather than recording it, and it produces a wrong answer for the premium bulk discount case because the discount was applied before tax and is not recoverable from the total alone. The convenience fee is also silently folded into the base line of the invoice. Storing a computed total without its components is what makes the error possible.\nSeat inventory is decremented in memory and separately inserted into the database with no transaction. If the INSERT fails, the seats are already gone from the in-memory count and the booking does not exist in the database.\nFailures are reported by printing and returning None, so a caller cannot distinguish an unknown show from insufficient seats from an unknown ticket type.\nThe booking identifier is derived from the length of the in-memory dictionary, which produces duplicates on restart and on any concurrent booking.\nPart 2: the corrected design.\nClass Responsibility Abstraction implemented Show Show data and the seat inventory invariant Data class with behaviour TicketType Enumeration of tiers Enumeration PriceBreakdown Immutable record of every pricing component Data class Booking Immutable record of a completed booking Data class PricingRule Contract for computing a price for a tier Abstraction StandardPricing, PremiumPricing, ReclinerPricing One tier\u0026rsquo;s rule each PricingRule TaxPolicy Applies the applicable tax to a subtotal Abstraction plus GstPolicy PriceCalculator Assembles a full breakdown from a rule and a tax policy Concrete, closed against new tiers BookingRepository Persistence of bookings and shows Abstraction plus PostgresBookingRepository and InMemoryBookingRepository BookingIdGenerator Produces unique identifiers Abstraction plus UuidBookingIdGenerator MessageSender Delivers a confirmation to a phone number Abstraction plus TwilioSender AuditLog Records booking events Abstraction plus FileAuditLog InvoiceRenderer Converts a booking and a breakdown into a document Abstraction plus TextInvoiceRenderer DocumentStore Persists a rendered document Abstraction plus FileDocumentStore BookingService Coordinates the booking sequence Concrete policy, depends only on abstractions InvoiceService Coordinates rendering and storage Concrete policy, depends only on abstractions Part 3: implementation.\nfrom __future__ import annotations import uuid from abc import ABC, abstractmethod from dataclasses import dataclass, replace from datetime import datetime from enum import Enum from typing import Any, Protocol class TicketType(Enum): STANDARD = \u0026#34;standard\u0026#34; PREMIUM = \u0026#34;premium\u0026#34; RECLINER = \u0026#34;recliner\u0026#34; @dataclass class Show: show_id: str movie: str screen: str start: datetime seats_available: int def can_accommodate(self, seat_count: int) -\u0026gt; bool: return 0 \u0026lt; seat_count \u0026lt;= self.seats_available def reserve(self, seat_count: int) -\u0026gt; None: if not self.can_accommodate(seat_count): raise ValueError(f\u0026#34;Cannot reserve {seat_count} seats on {self.show_id}\u0026#34;) self.seats_available -= seat_count @dataclass(frozen=True) class PriceBreakdown: \u0026#34;\u0026#34;\u0026#34;Every component is stored, so no downstream consumer needs to reverse the arithmetic. This closes the invoice defect at its source.\u0026#34;\u0026#34;\u0026#34; ticket_subtotal: float discount: float convenience_fee: float tax: float @property def taxable_amount(self) -\u0026gt; float: return self.ticket_subtotal - self.discount + self.convenience_fee @property def total(self) -\u0026gt; float: return round(self.taxable_amount + self.tax, 2) @dataclass(frozen=True) class Booking: booking_id: str show_id: str customer_name: str phone: str seat_count: int ticket_type: TicketType breakdown: PriceBreakdown @dataclass(frozen=True) class BookingOutcome: booking: Booking | None reason: str = \u0026#34;\u0026#34; @property def succeeded(self) -\u0026gt; bool: return self.booking is not None # ---------- Pricing: open for new tiers ---------- class PricingRule(ABC): @property @abstractmethod def ticket_type(self) -\u0026gt; TicketType: ... @abstractmethod def subtotal(self, seat_count: int) -\u0026gt; float: ... @abstractmethod def discount(self, seat_count: int, subtotal: float) -\u0026gt; float: ... @abstractmethod def convenience_fee(self, seat_count: int) -\u0026gt; float: ... class StandardPricing(PricingRule): @property def ticket_type(self) -\u0026gt; TicketType: return TicketType.STANDARD def subtotal(self, seat_count: int) -\u0026gt; float: return 180.0 * seat_count def discount(self, seat_count: int, subtotal: float) -\u0026gt; float: return 0.0 def convenience_fee(self, seat_count: int) -\u0026gt; float: return 20.0 * seat_count class PremiumPricing(PricingRule): @property def ticket_type(self) -\u0026gt; TicketType: return TicketType.PREMIUM def subtotal(self, seat_count: int) -\u0026gt; float: return 320.0 * seat_count def discount(self, seat_count: int, subtotal: float) -\u0026gt; float: return subtotal * 0.10 if seat_count \u0026gt;= 4 else 0.0 def convenience_fee(self, seat_count: int) -\u0026gt; float: return 35.0 * seat_count class ReclinerPricing(PricingRule): @property def ticket_type(self) -\u0026gt; TicketType: return TicketType.RECLINER def subtotal(self, seat_count: int) -\u0026gt; float: return 550.0 * seat_count def discount(self, seat_count: int, subtotal: float) -\u0026gt; float: return 0.0 def convenience_fee(self, seat_count: int) -\u0026gt; float: return 50.0 * seat_count class TaxPolicy(Protocol): def tax_on(self, taxable_amount: float) -\u0026gt; float: ... class GstPolicy: def __init__(self, rate_percent: float = 18.0): self._rate = rate_percent def tax_on(self, taxable_amount: float) -\u0026gt; float: return round(taxable_amount * self._rate / 100.0, 2) class PriceCalculator: \u0026#34;\u0026#34;\u0026#34;Closed against new tiers and against tax changes. Adding a tier means registering a rule; changing the tax rate means constructing a different TaxPolicy at the composition root.\u0026#34;\u0026#34;\u0026#34; def __init__(self, rules: list[PricingRule], tax: TaxPolicy): self._rules = {rule.ticket_type: rule for rule in rules} self._tax = tax def calculate(self, ticket_type: TicketType, seat_count: int) -\u0026gt; PriceBreakdown: try: rule = self._rules[ticket_type] except KeyError: raise ValueError(f\u0026#34;No pricing rule for {ticket_type.value}\u0026#34;) from None subtotal = rule.subtotal(seat_count) discount = rule.discount(seat_count, subtotal) fee = rule.convenience_fee(seat_count) taxable = subtotal - discount + fee return PriceBreakdown( ticket_subtotal=subtotal, discount=discount, convenience_fee=fee, tax=self._tax.tax_on(taxable), ) # ---------- Infrastructure abstractions, defined by the policy\u0026#39;s needs ---------- class BookingRepository(Protocol): def get_show(self, show_id: str) -\u0026gt; Show | None: ... def save_show(self, show: Show) -\u0026gt; None: ... def save_booking(self, booking: Booking) -\u0026gt; None: ... def get_booking(self, booking_id: str) -\u0026gt; Booking | None: ... class BookingIdGenerator(Protocol): def next_id(self) -\u0026gt; str: ... class MessageSender(Protocol): def send(self, phone: str, message: str) -\u0026gt; None: ... class AuditLog(Protocol): def record(self, event: str, payload: dict[str, Any]) -\u0026gt; None: ... class InvoiceRenderer(Protocol): def render(self, booking: Booking, show: Show) -\u0026gt; str: ... class DocumentStore(Protocol): def put(self, name: str, content: str) -\u0026gt; None: ... # ---------- Policy ---------- class BookingService: \u0026#34;\u0026#34;\u0026#34;The booking rule and the sequence of collaborations. Contains no SQL, no vendor, no filesystem path, and no pricing arithmetic.\u0026#34;\u0026#34;\u0026#34; def __init__( self, repository: BookingRepository, calculator: PriceCalculator, ids: BookingIdGenerator, sender: MessageSender, audit: AuditLog, ): self._repository = repository self._calculator = calculator self._ids = ids self._sender = sender self._audit = audit def book( self, show_id: str, customer_name: str, phone: str, seat_count: int, ticket_type: TicketType, ) -\u0026gt; BookingOutcome: show = self._repository.get_show(show_id) if show is None: return self._refuse(show_id, customer_name, \u0026#34;No such show\u0026#34;) if not show.can_accommodate(seat_count): return self._refuse(show_id, customer_name, \u0026#34;Not enough seats\u0026#34;) breakdown = self._calculator.calculate(ticket_type, seat_count) booking = Booking( booking_id=self._ids.next_id(), show_id=show_id, customer_name=customer_name, phone=phone, seat_count=seat_count, ticket_type=ticket_type, breakdown=breakdown, ) show.reserve(seat_count) self._repository.save_booking(booking) self._repository.save_show(show) self._sender.send( phone, f\u0026#34;Booking {booking.booking_id} confirmed for {show.movie}. \u0026#34; f\u0026#34;Total Rs {breakdown.total:.2f}\u0026#34;, ) self._audit.record( \u0026#34;booking_confirmed\u0026#34;, { \u0026#34;booking_id\u0026#34;: booking.booking_id, \u0026#34;show_id\u0026#34;: show_id, \u0026#34;seat_count\u0026#34;: seat_count, \u0026#34;total\u0026#34;: breakdown.total, }, ) return BookingOutcome(booking=booking) def _refuse(self, show_id: str, customer_name: str, reason: str) -\u0026gt; BookingOutcome: self._audit.record( \u0026#34;booking_refused\u0026#34;, {\u0026#34;show_id\u0026#34;: show_id, \u0026#34;customer_name\u0026#34;: customer_name, \u0026#34;reason\u0026#34;: reason}, ) return BookingOutcome(booking=None, reason=reason) class TextInvoiceRenderer: \u0026#34;\u0026#34;\u0026#34;Formatting only. Reads the stored components rather than reversing the total, so the premium discount appears as its own line.\u0026#34;\u0026#34;\u0026#34; def render(self, booking: Booking, show: Show) -\u0026gt; str: b = booking.breakdown return ( f\u0026#34;INVOICE {booking.booking_id}\\n\u0026#34; f\u0026#34;Movie: {show.movie} Screen: {show.screen}\\n\u0026#34; f\u0026#34;Customer: {booking.customer_name}\\n\u0026#34; f\u0026#34;Seats: {booking.seat_count} ({booking.ticket_type.value})\\n\u0026#34; f\u0026#34;Tickets: {b.ticket_subtotal:\u0026gt;10.2f}\\n\u0026#34; f\u0026#34;Discount: {-b.discount:\u0026gt;10.2f}\\n\u0026#34; f\u0026#34;Convenience: {b.convenience_fee:\u0026gt;10.2f}\\n\u0026#34; f\u0026#34;GST: {b.tax:\u0026gt;10.2f}\\n\u0026#34; f\u0026#34;Total: {b.total:\u0026gt;10.2f}\\n\u0026#34; ) class InvoiceService: def __init__( self, repository: BookingRepository, renderer: InvoiceRenderer, store: DocumentStore, ): self._repository = repository self._renderer = renderer self._store = store def issue(self, booking_id: str) -\u0026gt; str: booking = self._repository.get_booking(booking_id) if booking is None: raise ValueError(f\u0026#34;Unknown booking: {booking_id}\u0026#34;) show = self._repository.get_show(booking.show_id) if show is None: raise ValueError(f\u0026#34;Unknown show: {booking.show_id}\u0026#34;) document = self._renderer.render(booking, show) self._store.put(f\u0026#34;{booking_id}.txt\u0026#34;, document) return document class UuidBookingIdGenerator: def next_id(self) -\u0026gt; str: return f\u0026#34;BK{uuid.uuid4().hex[:10].upper()}\u0026#34; A representative concrete adapter, showing that all SQL knowledge is confined to one class and uses parameter binding:\nclass PostgresBookingRepository: def __init__(self, connection): self._conn = connection def get_show(self, show_id: str) -\u0026gt; Show | None: with self._conn.cursor() as cur: cur.execute( \u0026#34;SELECT show_id, movie, screen, start_time, seats_available \u0026#34; \u0026#34;FROM shows WHERE show_id = %s\u0026#34;, (show_id,), ) row = cur.fetchone() return Show(*row) if row else None def save_show(self, show: Show) -\u0026gt; None: with self._conn.cursor() as cur: cur.execute( \u0026#34;UPDATE shows SET seats_available = %s WHERE show_id = %s\u0026#34;, (show.seats_available, show.show_id), ) def save_booking(self, booking: Booking) -\u0026gt; None: b = booking.breakdown with self._conn.cursor() as cur: cur.execute( \u0026#34;INSERT INTO bookings (booking_id, show_id, customer_name, phone, \u0026#34; \u0026#34;seat_count, ticket_type, ticket_subtotal, discount, convenience_fee, \u0026#34; \u0026#34;tax) VALUES (%s, %s, %s, %s, %s, %s, %s, %s, %s, %s)\u0026#34;, ( booking.booking_id, booking.show_id, booking.customer_name, booking.phone, booking.seat_count, booking.ticket_type.value, b.ticket_subtotal, b.discount, b.convenience_fee, b.tax, ), ) def get_booking(self, booking_id: str) -\u0026gt; Booking | None: ... The composition root assembles the system.\n# main.py def build_services(config): connection = psycopg2.connect( host=config.db_host, dbname=config.db_name, user=config.db_user, password=config.db_password, ) repository = PostgresBookingRepository(connection) calculator = PriceCalculator( rules=[StandardPricing(), PremiumPricing(), ReclinerPricing()], tax=GstPolicy(rate_percent=config.gst_rate), ) booking = BookingService( repository=repository, calculator=calculator, ids=UuidBookingIdGenerator(), sender=TwilioSender(config.twilio_sid, config.twilio_token, config.from_number), audit=FileAuditLog(config.audit_path), ) invoices = InvoiceService( repository=repository, renderer=TextInvoiceRenderer(), store=FileDocumentStore(config.invoice_dir), ) return booking, invoices Part 4: tests.\nclass InMemoryBookingRepository: def __init__(self, shows: list[Show] | None = None): self._shows = {s.show_id: s for s in (shows or [])} self._bookings: dict[str, Booking] = {} def get_show(self, show_id: str) -\u0026gt; Show | None: return self._shows.get(show_id) def save_show(self, show: Show) -\u0026gt; None: self._shows[show.show_id] = show def save_booking(self, booking: Booking) -\u0026gt; None: self._bookings[booking.booking_id] = booking def get_booking(self, booking_id: str) -\u0026gt; Booking | None: return self._bookings.get(booking_id) class SequentialIds: def __init__(self) -\u0026gt; None: self._n = 0 def next_id(self) -\u0026gt; str: self._n += 1 return f\u0026#34;BK{self._n:05d}\u0026#34; class RecordingSender: def __init__(self) -\u0026gt; None: self.sent: list[tuple[str, str]] = [] def send(self, phone: str, message: str) -\u0026gt; None: self.sent.append((phone, message)) class RecordingAudit: def __init__(self) -\u0026gt; None: self.events: list[tuple[str, dict]] = [] def record(self, event: str, payload: dict) -\u0026gt; None: self.events.append((event, payload)) class InMemoryDocumentStore: def __init__(self) -\u0026gt; None: self.documents: dict[str, str] = {} def put(self, name: str, content: str) -\u0026gt; None: self.documents[name] = content def _build(seats: int = 100): show = Show(\u0026#34;SH1\u0026#34;, \u0026#34;Dune Part Two\u0026#34;, \u0026#34;Screen 3\u0026#34;, datetime(2026, 4, 2, 19, 30), seats) repository = InMemoryBookingRepository([show]) sender, audit = RecordingSender(), RecordingAudit() service = BookingService( repository=repository, calculator=PriceCalculator( rules=[StandardPricing(), PremiumPricing(), ReclinerPricing()], tax=GstPolicy(18.0), ), ids=SequentialIds(), sender=sender, audit=audit, ) return service, repository, sender, audit, show def test_standard_booking_prices_and_reserves_correctly() -\u0026gt; None: service, repository, sender, audit, show = _build(seats=100) outcome = service.book(\u0026#34;SH1\u0026#34;, \u0026#34;Asha\u0026#34;, \u0026#34;+919812345678\u0026#34;, 2, TicketType.STANDARD) assert outcome.succeeded breakdown = outcome.booking.breakdown assert breakdown.ticket_subtotal == 360.0 assert breakdown.convenience_fee == 40.0 assert breakdown.discount == 0.0 assert breakdown.tax == 72.0 assert breakdown.total == 472.0 assert show.seats_available == 98 assert len(sender.sent) == 1 assert audit.events[0][0] == \u0026#34;booking_confirmed\u0026#34; def test_premium_bulk_discount_applies_before_tax() -\u0026gt; None: service, *_ = _build() outcome = service.book(\u0026#34;SH1\u0026#34;, \u0026#34;Ravi\u0026#34;, \u0026#34;+919812345678\u0026#34;, 4, TicketType.PREMIUM) b = outcome.booking.breakdown assert b.ticket_subtotal == 1280.0 assert b.discount == 128.0 assert b.convenience_fee == 140.0 assert b.taxable_amount == 1292.0 assert b.tax == 232.56 assert b.total == 1524.56 def test_booking_is_refused_when_seats_are_insufficient() -\u0026gt; None: service, repository, sender, audit, show = _build(seats=3) outcome = service.book(\u0026#34;SH1\u0026#34;, \u0026#34;Meera\u0026#34;, \u0026#34;+919812345678\u0026#34;, 5, TicketType.RECLINER) assert outcome.succeeded is False assert outcome.reason == \u0026#34;Not enough seats\u0026#34; assert show.seats_available == 3, \u0026#34;A refused booking must not reserve seats\u0026#34; assert sender.sent == [], \u0026#34;A refused booking must send no message\u0026#34; assert audit.events[0][0] == \u0026#34;booking_refused\u0026#34; def test_invoice_reports_the_discount_as_its_own_line() -\u0026gt; None: service, repository, *_ = _build() outcome = service.book(\u0026#34;SH1\u0026#34;, \u0026#34;Ravi\u0026#34;, \u0026#34;+919812345678\u0026#34;, 4, TicketType.PREMIUM) store = InMemoryDocumentStore() invoices = InvoiceService(repository, TextInvoiceRenderer(), store) document = invoices.issue(outcome.booking.booking_id) assert \u0026#34;Discount: -128.00\u0026#34; in document assert \u0026#34;Total: 1524.56\u0026#34; in document assert f\u0026#34;{outcome.booking.booking_id}.txt\u0026#34; in store.documents def test_a_zero_rated_tax_jurisdiction_needs_no_code_change() -\u0026gt; None: calculator = PriceCalculator([StandardPricing()], GstPolicy(rate_percent=0.0)) breakdown = calculator.calculate(TicketType.STANDARD, 2) assert breakdown.tax == 0.0 assert breakdown.total == 400.0 Five tests cover the pricing arithmetic, the bulk discount ordering, the refusal path, the invoice contents, and the configurability of the tax rate. All five run without a database, an SMS provider, or a filesystem, and the entire suite completes in milliseconds. The second test is the one the original code could not have passed at all, because the discount was not recoverable from the stored total.\nPart 5: what the refactor makes possible.\nThe following statements are operational rather than aesthetic, and they are the terms in which a refactor should be justified to a team or an interviewer.\nRequirement Original Refactored The goods and services tax rate changes on a stated date Edit the multiplier inside a 45-line booking method, then manually verify that booking, SMS, and audit behaviour are unaffected Construct GstPolicy(rate_percent=...) from configuration. No code change, and the existing tests continue to prove correctness. An IMAX tier is introduced Add a branch to the pricing chain inside the booking method Add one PricingRule class and one registry entry. No existing pricing class, and no booking code, is edited. Migration from Twilio to a different provider Edit the constructor and the send call inside the booking method Write one class satisfying MessageSender and change one line at the composition root Verifying the premium bulk discount Not possible without a production database and a live SMS account, and the result is unverifiable because the discount is not stored A three-line assertion in a test that runs offline Invoices for a booking made before a tax change Wrong, because the invoice reverses the current rate against a historical total Correct, because the tax charged is stored with the booking rather than recomputed Concurrent bookings for the last seats Identifier collisions and lost updates Identifiers are collision-free, and the seat check and reservation are colocated in Show, which is the correct place for a transaction or a lock to be introduced A customer named O'Brien Corrupts the SQL statement Handled by parameter binding in the one class that issues SQL A final observation on scope. The refactored system contains sixteen classes where the original contained one, and this is the objection most frequently raised against work of this kind. The count is justified here because each class corresponds to an axis along which this system is known to change: tax rates change by legislation, tiers change by commercial decision, vendors change by procurement, and storage changes by platform migration. Had the requirement been a single-screen cinema with one ticket price and no tax, the same sixteen classes would be unjustifiable. The judgement of when structure is warranted is developed further in the module on advanced design topics; for the purposes of this workbook, the operative rule is that structure is warranted by demonstrated axes of change and not by the possibility of change in the abstract.\n11. Self-Review Checklist The following checklist is intended for application to one\u0026rsquo;s own code, in review, before a design is submitted. It is deliberately phrased as questions with unambiguous answers rather than as principles to consider.\nSINGLE RESPONSIBILITY [ ] Can I list more than one concrete requirement change that would edit this class? [ ] Does an honest name for this class require the word \u0026#34;And\u0026#34; or the suffix \u0026#34;Manager\u0026#34;? [ ] Does this class both decide something and perform input or output? [ ] Would two different teams need to change this file for unrelated reasons? OPEN / CLOSED [ ] Is there a conditional chain keyed on a type code, mode, or format string? [ ] Does adding the next obvious variant require editing working, tested code? [ ] Could a consumer of this module extend it without modifying its source? LISKOV SUBSTITUTION [ ] Does any override raise an exception the base contract does not declare? [ ] Does any override reject inputs the base accepts, or guarantee less than the base? [ ] Does any caller use isinstance to decide whether a method is safe to call? [ ] Does any override leave an invariant of the base broken? INTERFACE SEGREGATION [ ] Does any implementer supply a method body that is pass, None, or NotImplementedError? [ ] Does any consumer declare a type with ten methods in order to call two? [ ] Is any interface defined by what the largest implementer can do rather than by what a client needs? [ ] Conversely: does any class now declare conformance to more than four interfaces that are always required together? (Interface explosion.) DEPENDENCY INVERSION [ ] Does this class construct any collaborator inside __init__ or a method body? [ ] Does this class contain a hostname, port, path, API key, or vendor name? [ ] Can this class be instantiated in a test with no network and no filesystem? [ ] Was each abstraction defined by the needs of the caller, or copied from the shape of the provider\u0026#39;s API? [ ] Is there exactly one composition root, and does it contain every concrete infrastructure choice? The final question in each group is the one most often answered incorrectly, because each concerns a defect that looks like a correction. A class split into ten pieces along the wrong lines, an interface hierarchy that has exploded, an override defended by a caller\u0026rsquo;s type check, and an abstraction that mirrors a vendor\u0026rsquo;s API are all failures produced by applying a principle without understanding its purpose.\n12. Summary and Key Takeaways 12.1 What the workbook covered Section Problems Levels Domains A: Recognition drills 5 1 Hospital, e-commerce, banking, ride-hailing, retail B: Single Responsibility 3 1, 2, 3 Content platform, human resources, restaurant C: Open/Closed 3 1, 2, 3 Payments, logging, e-commerce promotions D: Liskov Substitution 3 1, 2, 3 Geometry, storage library, retail banking E: Interface Segregation 3 1, 2, 3 Office devices, storage abstraction, game engine F: Dependency Inversion 3 1, 2, 3 Notifications, reporting, hospital scheduling G: Combined violations 2 2, 3 Library management, home automation H: Capstone 1 3 Cinema ticketing Total 23 Fifteen distinct domains 12.2 Diagnostic signatures This table is the practical residue of the workbook and is worth committing to memory, because each row is a syntactic pattern detectable without domain knowledge.\nSignature observed in code Principle Direction of correction A class that computes, formats, stores, and transmits SRP Split by axis of change, then inject the infrastructure into the coordinator A conditional chain on a type code or mode string OCP One class per variant behind a common abstraction, selected by registry lookup An override raising an exception the base does not declare LSP Separate the capability, or widen the base contract so refusal is expected A caller using isinstance to decide what is safe to call LSP and OCP Select by capability rather than by identity An implementer with pass or NotImplementedError bodies ISP Decompose the interface into cohesive role interfaces A collaborator constructed inside __init__ DIP Constructor injection, with selection moved to the composition root An abstraction whose method names mirror a vendor\u0026rsquo;s API DIP Redefine the abstraction as the question the policy asks An exception handler that exists to discover a static fact ISP Correct the interface; the handler then has nothing to catch 12.3 Key takeaways Diagnose in a fixed order. Responsibilities first, then interface breadth, then extension points, then dependencies. A responsibility defect creates the conditions for the others, so correcting it first frequently causes the remaining problems to have obvious solutions. The reverse order produces constructors with eight parameters, which is the characteristic sign that the split was skipped.\nCount stakeholder roles, not lines. The Single Responsibility Principle is not a statement about class size. Four methods of three lines each can be four responsibilities, and a two-hundred-line class implementing one algorithm can be one.\nThe abstraction belongs to the caller. This single idea determines whether an application of the Dependency Inversion Principle achieves anything. An interface derived from what a provider offers leaves the policy coupled to the provider through the shape of the interface. An interface derived from what the policy needs to decide is what makes a vendor migration a one-line change.\nExtensibility is a risk-management technique. The reason to prefer adding a class over editing a method is that working code carries accumulated validation which editing partially discards. Stating the benefit in these terms, rather than as elegance, is what makes the case for a refactor persuasive to people who own delivery dates.\nSubstitutability is a property of consumers. A hierarchy cannot be judged in isolation. The test is to take each consumer, enumerate what it assumes, and verify that every type reaching it honours all of those assumptions unconditionally.\nChoose between capability separation and contract widening by asking when the condition is known. Conditions fixed by type belong in the type system, where misuse becomes a static error. Conditions determined by runtime values belong in the contract, expressed as a declared outcome that every caller must handle.\nEach principle has a failure mode that resembles compliance. Excessive splitting for SRP, speculative abstraction for OCP, defensive type checks for LSP, interface explosion for ISP, and container-driven indirection for DIP are all produced by applying a rule without its purpose. Structure is warranted by demonstrated axes of change, and a design that anticipates variation which never arrives has paid a cost for nothing.\nThe measurement that matters is the second pass. Repeat this workbook after seven to ten days with no reference material. Principles diagnosed correctly on the first attempt and incorrectly on the second were recalled rather than learned, and those are the sections to revisit.\n13. Appendix: Violation Signature Reference A single-page reference for use during review or interview preparation.\n# Signature Principle Standard correction Workbook problem 1 Data, formatting, storage, and transport in one class SRP Split by axis of change A.1, B.1, B.2 2 A class name requiring \u0026ldquo;And\u0026rdquo; to be honest SRP Split; the conjunction names the seam B.1 3 A method that both decides and performs output SRP Return a value; let a coordinator perform the output B.2 4 Conditional chain on a variant string OCP Abstraction plus one class per variant plus registry A.2, C.1, C.2, G.1 5 A caller-supplied mode parameter re-evaluated per call OCP Move the selection to construction time C.2 6 Engine inspecting variant type to label its output OCP Have the abstraction return a self-describing value C.3 7 Override raising an undeclared exception type LSP Capability separation or contract widening A.3, D.2 8 Override weakening a postcondition of an inherited setter LSP Reject the inheritance relationship D.1 9 isinstance guard protecting a method call LSP, OCP Select by capability, not identity G.2 10 Implementer with pass bodies ISP Decompose into role interfaces A.4, E.1 11 Implementer with NotImplementedError bodies ISP, LSP Decompose into role interfaces E.1, G.2 12 Change to one method propagating to unrelated implementers ISP Decompose; verify the blast radius afterwards E.2 13 Nine single-method interfaces on one class ISP applied without limit Recombine into cohesive roles E.2 14 Collaborator constructed in __init__ DIP Constructor injection A.5, F.1, F.2, G.1, H.1 15 Hostname, path, or credential in a policy class DIP Move to configuration, read at the composition root F.2, H.1 16 Abstraction mirroring a provider\u0026rsquo;s method names DIP Redefine as the question the policy asks F.3 17 Exception handler discovering a statically known fact ISP Correct the interface G.2 18 A computed total stored without its components Not a SOLID violation; a modelling defect Store the breakdown H.1 19 Values interpolated into SQL Not a SOLID violation; a security defect Parameter binding, in one repository class G.1, H.1 20 Failure reported by printing and returning None Not a SOLID violation; a contract defect Return an outcome value carrying the reason G.1, H.1 The last three rows are included deliberately. A design review that reports only principle violations is incomplete, and the ability to notice a security defect or a modelling error alongside the structural ones is what distinguishes a competent review from a recitation of the acronym.\nThis content is part of Codeverra, a platform for learning coding, data science, DSA, and AI from scratch. Explore more: https://codeverra.com\n","permalink":"https://blogs.codeverra.com/blog/low_level_design/02a_solid_practice/","summary":"\u003ch1 id=\"solid-principles-practice-workbook\"\u003eSOLID Principles: Practice Workbook\u003c/h1\u003e\n\u003cblockquote\u003e\n\u003cp\u003e\u003cstrong\u003ePrerequisite:\u003c/strong\u003e \u003ccode\u003e02_solid_principles.md\u003c/code\u003e. This workbook assumes the five principles have already been defined and demonstrated. It contains no new theory. Its purpose is to convert conceptual knowledge into diagnostic skill.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003chr\u003e\n\u003ch2 id=\"table-of-contents\"\u003eTable of Contents\u003c/h2\u003e\n\u003col\u003e\n\u003cli\u003e\u003ca href=\"#1-how-to-use-this-workbook\"\u003eHow to Use This Workbook\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#2-vocabulary-and-the-diagnostic-method\"\u003eVocabulary and the Diagnostic Method\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#3-section-a-recognition-drills\"\u003eSection A: Recognition Drills\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#4-section-b-single-responsibility-principle\"\u003eSection B: Single Responsibility Principle\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#5-section-c-openclosed-principle\"\u003eSection C: Open/Closed Principle\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#6-section-d-liskov-substitution-principle\"\u003eSection D: Liskov Substitution Principle\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#7-section-e-interface-segregation-principle\"\u003eSection E: Interface Segregation Principle\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#8-section-f-dependency-inversion-principle\"\u003eSection F: Dependency Inversion Principle\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#9-section-g-combined-violations\"\u003eSection G: Combined Violations\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#10-section-h-capstone-refactor\"\u003eSection H: Capstone Refactor\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#11-self-review-checklist\"\u003eSelf-Review Checklist\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#12-summary-and-key-takeaways\"\u003eSummary and Key Takeaways\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#13-appendix-violation-signature-reference\"\u003eAppendix: Violation Signature Reference\u003c/a\u003e\u003c/li\u003e\n\u003c/ol\u003e\n\u003chr\u003e\n\u003ch2 id=\"1-how-to-use-this-workbook\"\u003e1. How to Use This Workbook\u003c/h2\u003e\n\u003ch3 id=\"11-what-this-document-is-for\"\u003e1.1 What this document is for\u003c/h3\u003e\n\u003cp\u003eKnowing the definition of the Single Responsibility Principle and being able to detect a Single Responsibility violation in unfamiliar code are two different competencies. The first is declarative knowledge and can be acquired by reading. The second is a diagnostic skill and can only be acquired by repeated exposure to examples, followed by correction.\u003c/p\u003e","tags":["lld","solid","python","oop"],"title":"SOLID Principles: A Complete Deep Dive"},{"content":"UML Deep Dive: Class Diagrams and Sequence Diagrams Who this is for: Anyone who has been designing classes and systems (perhaps through OOP, SOLID, and design patterns) and now wants a standardized visual language to communicate those designs to teammates, interviewers, and their own future self.\nTable of Contents Why UML Exists The 14 UML Diagram Types: A Complete Map Class Diagrams: Anatomy of a Class Box Abstract Classes vs. Interfaces: Are They the Same Thing? Class Diagrams: Relationships Between Classes Class Diagrams: Advanced Concepts Sequence Diagrams: Why We Need a Second Diagram Sequence Diagrams: Anatomy and Core Concepts Sequence Diagrams: Control Flow (alt, opt, loop, par) 9.5 More Sequence Diagram Examples Putting It All Together: A Simple E-Commerce System 10.1 The Complete Class Diagram 10.2 The Complete Sequence Diagram Summary and Key Takeaways Practice Questions with Solutions 1. Why UML Exists When a design exists only in someone\u0026rsquo;s head, or scattered across a codebase, it is expensive to communicate. Explaining a system in prose forces the listener to reconstruct a graph of classes and relationships from a linear sequence of sentences, and details get lost or misinterpreted along the way. Two people reading the same paragraph of design prose regularly walk away with two different mental models of the same system, particularly around exactly how strong a relationship between two entities is.\nReading raw code has the opposite problem. Code is precise and complete, but it is also verbose. Understanding that a system has six classes and five relationships between them by reading four hundred lines of Python, spread across multiple files, imports, and boilerplate, is slow. Code answers \u0026ldquo;how is this built,\u0026rdquo; not \u0026ldquo;what is the shape of this.\u0026rdquo;\nUML (Unified Modeling Language) solves this by providing a small, standardized set of symbols for structure and behavior. A class diagram or a sequence diagram, once drawn, is read the same way by any engineer who has learned the notation, regardless of which programming language they use day to day. This is why UML is used constantly in system design interviews, architecture reviews, and onboarding documentation: it compresses a design into a form that can be scanned and verified quickly, and it exposes ambiguity (such as whether one object owns another\u0026rsquo;s lifecycle) that prose tends to hide.\nIt is worth being direct about what UML is not. It is not a substitute for code, and it is not meant to document every class of a large system in exhaustive detail; that kind of documentation goes stale the moment the code changes and nobody updates the diagram. UML earns its value in design discussions, reviews, and communication, where a diagram is drawn, discussed, and then discarded or kept only as a high-level reference, not as a live specification.\nApproach Strength Weakness Prose / documentation Explains reasoning behind decisions Ambiguous about precise structure and relationship strength Reading the code directly Ground truth, always accurate Slow to build a mental model, too much low-level detail Informal whiteboard boxes and arrows Fast for brainstorming No shared vocabulary; an arrow can mean different things to different people UML diagrams Standardized, precise, language-agnostic Can go stale if treated as permanent documentation instead of a communication tool 2. The 14 UML Diagram Types UML 2.x defines fourteen diagram types, split into two families: structural diagrams, which describe what a system looks like at a fixed point in time, and behavioral diagrams, which describe how a system behaves as it runs.\ngraph TD UML[UML Diagrams - 14 Types] --\u0026gt; S[Structural Diagrams - 7] UML --\u0026gt; B[Behavioral Diagrams - 7] S --\u0026gt; S1[Class Diagram] S --\u0026gt; S2[Object Diagram] S --\u0026gt; S3[Component Diagram] S --\u0026gt; S4[Composite Structure Diagram] S --\u0026gt; S5[Package Diagram] S --\u0026gt; S6[Deployment Diagram] S --\u0026gt; S7[Profile Diagram] B --\u0026gt; B1[Use Case Diagram] B --\u0026gt; B2[Activity Diagram] B --\u0026gt; B3[State Machine Diagram] B --\u0026gt; B4[Sequence Diagram] B --\u0026gt; B5[Communication Diagram] B --\u0026gt; B6[Interaction Overview Diagram] B --\u0026gt; B7[Timing Diagram] This deep dive focuses on Class Diagrams and Sequence Diagrams, because they are by far the most commonly used diagram in each family: class diagrams because most system designs are ultimately expressed as classes, and sequence diagrams because \u0026ldquo;walk me through what happens when a user does X\u0026rdquo; is the single most common question in a design discussion. The remaining twelve diagram types are summarized below so you know what each one is for and can recognize it when you see it.\nNote: Mermaid, the diagramming syntax used throughout this material, has native, purpose-built syntax only for a subset of these types (class, state machine, and sequence diagrams). For the remaining types, the diagrams below are approximated using Mermaid\u0026rsquo;s generic flowchart syntax to give you a visual sense of the layout; in professional UML tooling (e.g., PlantUML, Enterprise Architect, Visual Paradigm) each of these has its own dedicated notation.\n2.1 Structural Diagrams Class Diagram. Shows classes, their attributes and methods, and the relationships between them. Covered in full depth from Section 3 onward.\nObject Diagram. A snapshot of actual object instances and the links between them at one moment in time, rather than the general class-level rules. Useful for illustrating a specific, concrete scenario that would otherwise be confusing at the class level (for example, a many-to-many relationship is easier to sanity-check by looking at three real objects and their actual links).\nclassDiagram class order101[\u0026#34;order101 : Order\u0026#34;] class cust55[\u0026#34;cust55 : Customer\u0026#34;] class item1[\u0026#34;item1 : OrderItem\u0026#34;] class item2[\u0026#34;item2 : OrderItem\u0026#34;] cust55 --\u0026gt; order101 order101 --\u0026gt; item1 order101 --\u0026gt; item2 Component Diagram. Shows how a system is divided into larger, independently deployable or replaceable components (such as an \u0026ldquo;Auth Service\u0026rdquo; or a \u0026ldquo;Payment Gateway Client\u0026rdquo;) and the interfaces they expose to or require from each other. Used one level above class diagrams, when the discussion is about system architecture rather than individual classes.\ngraph LR UI[Web UI Component] --\u0026gt;|uses| API[Order API Component] API --\u0026gt;|uses| PAY[Payment Component] API --\u0026gt;|uses| INV[Inventory Component] API --\u0026gt;|reads/writes| DB[(Order Database)] Composite Structure Diagram. Shows the internal structure of a single class or component: the parts it is made of and how those parts are wired together internally. Used when a single class is complex enough that its internal collaborators need their own diagram, separate from the system-wide class diagram.\ngraph TD subgraph OrderProcessor Validator[Validator] Calculator[PriceCalculator] Dispatcher[EventDispatcher] Validator --\u0026gt; Calculator Calculator --\u0026gt; Dispatcher end Package Diagram. Groups classes into higher-level packages or modules and shows the dependencies between those packages. Used to check for problems like circular dependencies between modules before they show up as import errors.\ngraph TD subgraph pkg_orders[\u0026#34;package: orders\u0026#34;] Order OrderItem end subgraph pkg_payments[\u0026#34;package: payments\u0026#34;] PaymentMethod end subgraph pkg_notifications[\u0026#34;package: notifications\u0026#34;] NotificationService end pkg_orders --\u0026gt; pkg_payments pkg_orders --\u0026gt; pkg_notifications Deployment Diagram. Shows the physical or virtual infrastructure a system runs on: servers, containers, devices, and the network connections between them, along with which software artifacts run where. Used by infrastructure and DevOps-adjacent discussions rather than by application-level design work.\ngraph TD subgraph Client Device Browser end subgraph Cloud Load Balancer LB[Load Balancer] end subgraph App Server Cluster App1[App Server 1] App2[App Server 2] end subgraph DB Server DB[(PostgreSQL)] end Browser --\u0026gt; LB --\u0026gt; App1 LB --\u0026gt; App2 App1 --\u0026gt; DB App2 --\u0026gt; DB Profile Diagram. Defines custom stereotypes, tagged values, and constraints that extend UML itself for a specific domain (for example, a company might define a \u0026lt;\u0026lt;microservice\u0026gt;\u0026gt; stereotype with its own custom rules). This is the least commonly used of the fourteen diagrams in everyday engineering work and is mentioned here for completeness rather than demonstrated in depth.\n2.2 Behavioral Diagrams Use Case Diagram. Shows the actors (users or external systems) interacting with a system and the goals (use cases) they can accomplish, without describing how those goals are achieved internally. Used early in requirements gathering, before any class or method has been designed.\ngraph LR Customer((Customer)) Admin((Admin)) Customer --\u0026gt; UC1[Browse Products] Customer --\u0026gt; UC2[Place Order] Customer --\u0026gt; UC3[Track Order] Admin --\u0026gt; UC4[Manage Inventory] Admin --\u0026gt; UC5[Process Refunds] Activity Diagram. Similar to a flowchart: shows the flow of control or data through a process, including decision points, parallel branches, and start/end points. Used to model a business process or algorithm at a level above any single class, often before deciding which classes will implement each step.\nflowchart TD Start([Start]) --\u0026gt; Check{Item in stock?} Check --\u0026gt;|Yes| Charge[Charge payment] Check --\u0026gt;|No| Notify[Notify out of stock] Charge --\u0026gt; Success{Payment succeeded?} Success --\u0026gt;|Yes| Confirm[Confirm order] Success --\u0026gt;|No| Decline[Show payment declined] Confirm --\u0026gt; End([End]) Decline --\u0026gt; End Notify --\u0026gt; End State Machine Diagram. Shows the distinct states a single object can be in, and the events that trigger transitions between those states. This is the diagram version of the State design pattern, and it has native Mermaid support.\nstateDiagram-v2 [*] --\u0026gt; Pending Pending --\u0026gt; Paid : payment succeeds Pending --\u0026gt; Cancelled : customer cancels Paid --\u0026gt; Shipped : warehouse ships item Shipped --\u0026gt; Delivered : courier confirms delivery Delivered --\u0026gt; [*] Cancelled --\u0026gt; [*] Sequence Diagram. Shows a set of objects and the ordered messages passed between them for one specific scenario, with time flowing from top to bottom. Covered in full depth from Section 7 onward.\nCommunication Diagram (formerly Collaboration Diagram). Shows the same information as a sequence diagram (which objects send which messages to accomplish a scenario) but arranges the objects freely in space and numbers the messages to indicate order, instead of using vertical lifelines and top-to-bottom time. Used when the emphasis is on the network of relationships between objects rather than the precise timing.\ngraph LR Customer --\u0026gt;|\u0026#34;1: checkout()\u0026#34;| Order Order --\u0026gt;|\u0026#34;2: pay()\u0026#34;| PaymentMethod Order --\u0026gt;|\u0026#34;3: reduceStock()\u0026#34;| Inventory Order --\u0026gt;|\u0026#34;4: send()\u0026#34;| NotificationService Interaction Overview Diagram. A high-level flowchart where each node is itself a small sequence or communication diagram, used to show how several different scenarios connect to and branch from one another. Used for complex workflows made up of many smaller interactions, rarely needed outside large enterprise systems.\nflowchart LR A[Interaction: Login] --\u0026gt; B{Login successful?} B --\u0026gt;|Yes| C[Interaction: Load Dashboard] B --\u0026gt;|No| D[Interaction: Show Error] Timing Diagram. Shows how the state or value of one or more objects changes across a precise timeline, typically used in embedded systems and hardware-adjacent software where exact timing between state changes matters (for example, verifying that a sensor\u0026rsquo;s state changes within a required number of milliseconds of a trigger event). Rarely used in typical backend or web application design.\nWith the map of all fourteen types established, the rest of this material goes deep on the two diagrams used in the overwhelming majority of day-to-day design work.\n3. Class Diagrams: Anatomy The Class Box A class in UML is drawn as a box with up to three compartments:\nclassDiagram class BankAccount { -owner_name: str -balance: float +deposit(amount: float) void +withdraw(amount: float) bool +get_balance() float } The top compartment holds the class name. The middle compartment holds attributes, written as visibility name: type. The bottom compartment holds methods, written as visibility name(params): return_type.\nVisibility Symbols Symbol Meaning Python equivalent + Public: accessible from anywhere self.balance - Private: accessible only within the class self.__balance # Protected: accessible within the class and subclasses self._balance ~ Package/internal: accessible within the same module Convention only in Python Visibility on a diagram is a design decision, not decoration. Marking balance as - tells every reader that external code should not touch it directly and must go through deposit() or withdraw() instead. This is UML expressing encapsulation visually, the same idea covered in OOP fundamentals.\nThe Python this diagram maps to:\nclass BankAccount: def __init__(self, owner_name: str, balance: float = 0.0): self.__owner_name = owner_name # -owner_name self.__balance = balance # -balance def deposit(self, amount: float) -\u0026gt; None: self.__balance += amount def withdraw(self, amount: float) -\u0026gt; bool: if amount \u0026gt; self.__balance: return False self.__balance -= amount return True def get_balance(self) -\u0026gt; float: return self.__balance The diagram and the code say the same thing in two notations. The diagram is a compressed, faster-to-scan summary of the code\u0026rsquo;s structure, useful precisely because it omits implementation detail.\n4. Abstract Classes vs. Interfaces A common point of confusion: are an abstract class and an interface the same thing? They are related but distinct, and the difference matters for how you design a system.\nThe Core Difference An abstract class is a class that cannot be instantiated directly and is meant to be subclassed. It can contain a mix of:\nFully implemented methods (with a body, ready to be inherited and used as-is). Abstract methods (declared but with no implementation, forcing subclasses to provide one). State (instance attributes shared by all subclasses). An interface is a pure contract. It declares a set of methods that implementing classes must provide, but it supplies no implementation and no state at all. It answers only the question \u0026ldquo;what must this class be able to do,\u0026rdquo; never \u0026ldquo;how does it do it\u0026rdquo; or \u0026ldquo;what data does it carry.\u0026rdquo;\nclassDiagram class Vehicle { \u0026lt;\u0026lt;abstract\u0026gt;\u0026gt; #fuel_level: float +refuel(amount: float) void +drive()* } class Drivable { \u0026lt;\u0026lt;interface\u0026gt;\u0026gt; +drive() +stop() } Vehicle \u0026lt;|-- Car Drivable \u0026lt;|.. Car In this diagram, Vehicle is an abstract class: it has real state (fuel_level) and a fully implemented method (refuel), alongside one abstract method (drive) that subclasses must fill in. Car inherits from Vehicle using a solid line, meaning it receives that shared state and implementation for free. Drivable is an interface: it defines no state and no implementation at all, only a promise that any implementing class has drive() and stop() methods. Car realizes Drivable using a dashed line, meaning it agrees to the contract but writes its own implementation from scratch.\nWhy the Distinction Exists The distinction exists because it separates two different design questions. \u0026ldquo;What common code and data can be shared and reused\u0026rdquo; is answered by an abstract class. \u0026ldquo;What behavior must be guaranteed, regardless of how it is implemented, possibly across completely unrelated classes\u0026rdquo; is answered by an interface.\nA concrete illustration: Duck and Airplane share almost no implementation or state, and forcing them into the same abstract class hierarchy would be artificial. But both can independently implement a Flyable interface, because both can be asked to fly(), even though a duck flaps wings and a plane uses engines. Interfaces let unrelated classes agree to a shared contract without being forced into a shared inheritance tree.\nMost object-oriented languages also enforce a practical rule that reinforces this distinction: a class can inherit from only one abstract (or concrete) class, but it can implement many interfaces. This is because inheriting implementation and state from multiple parents creates ambiguity (if two parent classes both define a method or field with the same name, which one wins), while agreeing to multiple contracts creates no such ambiguity, since a contract carries no implementation to conflict.\nHow This Maps to Python Python does not have a dedicated interface keyword, but the same distinction is expressed in two common ways:\nfrom abc import ABC, abstractmethod # Abstract class: has real state and a partially implemented method class Vehicle(ABC): def __init__(self, fuel_level: float): self.fuel_level = fuel_level # shared state def refuel(self, amount: float) -\u0026gt; None: # shared, fully implemented self.fuel_level += amount @abstractmethod def drive(self) -\u0026gt; None: ... # must be overridden # Interface-style class: no state, every method abstract class Drivable(ABC): @abstractmethod def drive(self) -\u0026gt; None: ... @abstractmethod def stop(self) -\u0026gt; None: ... class Car(Vehicle, Drivable): def drive(self) -\u0026gt; None: print(\u0026#34;Car is driving\u0026#34;) def stop(self) -\u0026gt; None: print(\u0026#34;Car has stopped\u0026#34;) Python also offers typing.Protocol as a more direct interface mechanism: a class does not even need to explicitly inherit from a Protocol to satisfy it, it only needs to have matching method signatures (this is called structural typing, sometimes summarized as \u0026ldquo;if it walks like a duck and quacks like a duck\u0026rdquo;).\nfrom typing import Protocol class Flyable(Protocol): def fly(self) -\u0026gt; None: ... class Duck: def fly(self) -\u0026gt; None: print(\u0026#34;Duck flaps wings\u0026#34;) class Airplane: def fly(self) -\u0026gt; None: print(\u0026#34;Airplane engines engage\u0026#34;) # Both Duck and Airplane satisfy the Flyable protocol without inheriting from it. def take_off(flyer: Flyable) -\u0026gt; None: flyer.fly() Quick Reference Aspect Abstract Class Interface Can hold state (attributes) Yes No Can provide method implementations Some methods can be implemented No, all methods are abstract A class can inherit/implement how many One (in most languages, including Python\u0026rsquo;s class hierarchy for shared implementation) Many Answers the question \u0026ldquo;What can be shared across a family of related classes?\u0026rdquo; \u0026ldquo;What must this class be able to do, regardless of what it is?\u0026rdquo; UML relationship used Inheritance (solid line, hollow triangle) Realization (dashed line, hollow triangle) 5. Class Diagrams: Relationships This section covers the relationships that connect classes on a diagram, ordered from weakest to strongest coupling. Each one is shown with more than one example so the notation is recognizable across different contexts, not just one memorized case.\n5.1 Dependency: Briefly Uses, No Stored Reference A dependency exists when one class uses another only momentarily, typically as a method parameter or a local variable, without holding a permanent reference to it as a field. It is drawn as a dashed line with an open arrowhead.\nclassDiagram OrderService ..\u0026gt; InvoiceGenerator : uses ReportBuilder ..\u0026gt; Logger : uses class InvoiceGenerator: def generate(self, order) -\u0026gt; str: return f\u0026#34;Invoice for order total: {order.total}\u0026#34; class OrderService: def checkout(self, order, generator: InvoiceGenerator) -\u0026gt; str: # InvoiceGenerator is used only inside this method. OrderService # does not store it as a field, so this is a dependency, not an association. return generator.generate(order) The test to distinguish a dependency from an association: does the class hold onto the other one as a field that lives as long as the object itself? If not, and it is only used within a single method\u0026rsquo;s parameters or body, it is a dependency.\n5.2 Association: Holds a Reference An association exists when one class holds a reference to another as a field, meaning the relationship persists across the lifetime of the object, not just for the duration of one method call.\nclassDiagram Professor \u0026#34;1\u0026#34; --\u0026gt; \u0026#34;*\u0026#34; Course : teaches Student \u0026#34;*\u0026#34; --\u0026gt; \u0026#34;*\u0026#34; Course : enrolls in Multiplicity (the 1, *, 0..1, and so on) specifies how many instances participate on each end:\nSymbol Meaning 1 Exactly one 0..1 Zero or one (optional) * or 0..* Zero or more 1..* One or more 3..5 Between 3 and 5 class Course: def __init__(self, title: str): self.title = title class Professor: def __init__(self, name: str): self.name = name self.courses: list[Course] = [] # 1 Professor --\u0026gt; * Course def teach(self, course: Course) -\u0026gt; None: self.courses.append(course) If both classes need to navigate to each other (a Course also needs to look up its Professor), the association becomes bidirectional, drawn as a plain line without an arrowhead or with arrowheads on both ends. Bidirectional associations mean both classes must be kept in sync whenever the relationship changes, which is a common source of bugs, so a one-way association is preferred by default, upgraded to bidirectional only when both sides genuinely need to navigate to the other.\n5.3 Aggregation: Has-a, Part Can Outlive the Whole Aggregation is a whole-part relationship where the part can exist independently of the whole. It is drawn with a hollow diamond at the \u0026ldquo;whole\u0026rdquo; end.\nclassDiagram Department o-- Professor : has Playlist o-- Song : contains A Department has Professors, but if the Department is dissolved, the professors do not cease to exist; they can transfer elsewhere. A Playlist contains Songs, but deleting the playlist does not delete the underlying songs, which may belong to other playlists too.\nclass Professor: def __init__(self, name: str): self.name = name class Department: def __init__(self, name: str, professors: list[Professor]): self.name = name self.professors = professors # aggregation: professors passed in, can exist without this Department The tell in code: Professor objects are created outside Department and simply handed to it. Department does not control their creation or destruction.\n5.4 Composition: Has-a, Part Cannot Outlive the Whole Composition is a stricter whole-part relationship where the part\u0026rsquo;s lifecycle is entirely owned by the whole. It is drawn with a filled diamond.\nclassDiagram Order *-- OrderItem : contains House *-- Room : consists of An Order contains OrderItems; deleting the Order deletes its items with it, since an OrderItem has no independent meaning outside the order it belongs to. A House consists of Rooms that only make sense as part of that specific house.\nclass OrderItem: def __init__(self, product_name: str, quantity: int, price: float): self.product_name = product_name self.quantity = quantity self.price = price class Order: def __init__(self): self.items: list[OrderItem] = [] # composition: created and owned entirely by Order def add_item(self, product_name: str, quantity: int, price: float) -\u0026gt; None: # OrderItem is created here, inside Order, and has no life outside an Order. self.items.append(OrderItem(product_name, quantity, price)) The test to distinguish aggregation from composition: if the whole is destroyed, does the part still make sense on its own? If yes, it is aggregation (hollow diamond). If no, it is composition (filled diamond).\n5.5 Inheritance (Generalization): Is-a, Shares Contract and Implementation Inheritance means a subclass is a specialized version of a superclass, receiving both its data and its method implementations. It is drawn with a solid line and a hollow triangle arrowhead pointing to the parent.\nclassDiagram Employee \u0026lt;|-- Manager Employee \u0026lt;|-- Engineer class Employee { +name: str +salary: float +work() } class Manager { +approve_leave() } class Engineer { +write_code() } class Employee: def __init__(self, name: str, salary: float): self.name = name self.salary = salary def work(self) -\u0026gt; str: return f\u0026#34;{self.name} is working.\u0026#34; class Manager(Employee): # Employee \u0026lt;|-- Manager def approve_leave(self) -\u0026gt; str: return f\u0026#34;{self.name} approved leave.\u0026#34; class Engineer(Employee): # Employee \u0026lt;|-- Engineer def write_code(self) -\u0026gt; str: return f\u0026#34;{self.name} is writing code.\u0026#34; 5.6 Realization: Implements an Interface\u0026rsquo;s Contract Only Realization means a class implements an interface or abstract contract without inheriting any shared implementation. It is drawn with a dashed line and a hollow triangle arrowhead, distinguishing it visually from inheritance\u0026rsquo;s solid line.\nclassDiagram class PaymentMethod { \u0026lt;\u0026lt;interface\u0026gt;\u0026gt; +pay(amount: float) bool } PaymentMethod \u0026lt;|.. CreditCardPayment PaymentMethod \u0026lt;|.. UpiPayment from abc import ABC, abstractmethod class PaymentMethod(ABC): @abstractmethod def pay(self, amount: float) -\u0026gt; bool: ... class CreditCardPayment(PaymentMethod): # PaymentMethod \u0026lt;|.. CreditCardPayment def pay(self, amount: float) -\u0026gt; bool: print(f\u0026#34;Charging {amount} to credit card\u0026#34;) return True class UpiPayment(PaymentMethod): # PaymentMethod \u0026lt;|.. UpiPayment def pay(self, amount: float) -\u0026gt; bool: print(f\u0026#34;Charging {amount} via UPI\u0026#34;) return True Section 4 above covers this distinction between inheritance and realization, and between abstract classes and interfaces, in full detail.\n5.7 Relationship Strength: The Full Spectrum classDiagram direction LR class A1 class B1 A1 ..\u0026gt; B1 : Dependency, weakest class A2 class B2 A2 --\u0026gt; B2 : Association class A3 class B3 A3 o-- B3 : Aggregation class A4 class B4 A4 *-- B4 : Composition class A5 class B5 A5 \u0026lt;|.. B5 : Realization class A6 class B6 A6 \u0026lt;|-- B6 : Inheritance, strongest Relationship Line style Lifecycle coupling Dependency Dashed, open arrow None Association Solid, open arrow None Aggregation Solid, hollow diamond Part can outlive whole Composition Solid, filled diamond Part dies with whole Realization Dashed, hollow triangle Shares contract only Inheritance Solid, hollow triangle Shares contract and implementation 6. Class Diagrams: Advanced Concepts Multiplicity in Both Directions Real associations often have multiplicity on both ends:\nclassDiagram Student \u0026#34;many\u0026#34; -- \u0026#34;many\u0026#34; Course : enrolls in A Student can enroll in many Courses, and a Course can have many Students: a many-to-many relationship. In a relational database, this requires a join or association table, since a many-to-many relationship cannot be modeled with a single foreign key on either side.\nclass Student: def __init__(self, name: str): self.name = name self.courses: list[\u0026#34;Course\u0026#34;] = [] class Course: def __init__(self, title: str): self.title = title self.students: list[Student] = [] def enroll(student: Student, course: Course) -\u0026gt; None: student.courses.append(course) course.students.append(student) Generic / Template Classes classDiagram class Stack~T~ { -items: List~T~ +push(item: T) void +pop() T } from typing import Generic, TypeVar T = TypeVar(\u0026#34;T\u0026#34;) class Stack(Generic[T]): def __init__(self): self._items: list[T] = [] def push(self, item: T) -\u0026gt; None: self._items.append(item) def pop(self) -\u0026gt; T: return self._items.pop() Static Members Strict UML notation underlines static (class-level) attributes and methods. Mermaid does not render underlining, so a $ suffix is used instead as a convention:\nclassDiagram class IdGenerator { -counter$: int +next_id()$ int } class IdGenerator: counter: int = 0 # class-level (static) attribute @staticmethod def next_id() -\u0026gt; int: IdGenerator.counter += 1 return IdGenerator.counter Enumerations classDiagram class OrderStatus { \u0026lt;\u0026lt;enumeration\u0026gt;\u0026gt; PENDING CONFIRMED SHIPPED DELIVERED CANCELLED } from enum import Enum class OrderStatus(Enum): PENDING = \u0026#34;pending\u0026#34; CONFIRMED = \u0026#34;confirmed\u0026#34; SHIPPED = \u0026#34;shipped\u0026#34; DELIVERED = \u0026#34;delivered\u0026#34; CANCELLED = \u0026#34;cancelled\u0026#34; 7. Sequence Diagrams: Why We Need a Second Diagram A class diagram describes what exists and how classes relate structurally at a fixed point in time. It deliberately leaves out order and timing. Looking at the Order, OrderItem, and PaymentMethod classes from earlier, the class diagram alone cannot answer whether inventory is checked before or after the customer is charged, what happens if pay() returns False, or the exact sequence of calls that occurs when a \u0026ldquo;place order\u0026rdquo; button is clicked.\nA sequence diagram fills this gap. It picks one specific scenario, such as \u0026ldquo;customer places an order,\u0026rdquo; and shows the objects involved as vertical lifelines, with time flowing from top to bottom, and arrows indicating exactly which object calls which method, in what order.\n8. Sequence Diagrams: Anatomy and Core Concepts Before going through each concept in detail, here is a one-glance summary of the components that make up a sequence diagram. Each one is expanded with a diagram and explanation in the rest of this section and the next.\nComponent Description Actor A human or external system that initiates or receives interaction with the system, drawn as a stick figure. Used at the edges of a diagram, not for internal objects. Participant An object, class, or service involved in the scenario, drawn as a labeled box at the top (and sometimes bottom) of the diagram. Lifeline The vertical line beneath a participant or actor, representing that entity\u0026rsquo;s existence through the duration of the scenario, with time flowing top to bottom. Message An arrow between two lifelines representing a method call, a request, or a piece of communication, labeled with the method name or content being sent. Can be synchronous (solid, filled arrowhead, sender waits) or asynchronous (solid, open arrowhead, sender does not wait); see the Message Types section below. Activation bar A thin rectangle on a lifeline showing the span of time during which that participant is actively executing (processing a call it has received). Return message A dashed arrow, drawn from a participant back to its caller, representing the value or result handed back once processing finishes. Conditional flow An alt (or opt) fragment, a labeled box drawn around part of the diagram, showing that different messages occur depending on a condition, equivalent to if/else or if in code. Loop A loop fragment, a labeled box showing that the messages inside it repeat, equivalent to a for or while loop in code. Object creation A message pointing at a lifeline that begins partway down the diagram instead of at the top, showing that the object did not exist before this point in the scenario. Object destruction The point, marked with an X in strict UML (or a shortened lifeline in Mermaid), where a participant\u0026rsquo;s lifeline ends and the object is discarded. Lifelines and Activation Bars sequenceDiagram participant C as Customer participant O as OrderService C-\u0026gt;\u0026gt;O: placeOrder(items) activate O O--\u0026gt;\u0026gt;C: orderConfirmation deactivate O A lifeline is the vertical line beneath each participant box, representing that object\u0026rsquo;s existence through time, with the top of the diagram being the start of the scenario and downward motion representing time passing. An activation bar is the thin rectangle drawn on a lifeline while that object is actively executing, created and removed with activate/deactivate (or Mermaid\u0026rsquo;s shorthand of +/- on the message arrow).\nA note on notation: in strict UML notation, a lifeline is drawn as a dashed vertical line. Mermaid, the tool used to render every diagram in this material, draws lifelines as solid vertical lines instead, and does not currently offer a setting to change this. This is a rendering choice made by the Mermaid library, not an error in any of the diagrams here. If you draw a sequence diagram by hand, in PlantUML, or in a dedicated UML tool, use a dashed line for the lifeline; when reading Mermaid-rendered diagrams (including every one in this document), a solid vertical line under a participant box should be read as that participant\u0026rsquo;s lifeline.\nElement Strict UML Mermaid\u0026rsquo;s rendering Lifeline Dashed vertical line Solid vertical line Activation bar Thin solid rectangle on the lifeline Thin solid rectangle on the lifeline (matches) Object destruction An X at the bottom of the lifeline A shortened lifeline ending where destroy is called Message Types The most important distinction among message types is synchronous versus asynchronous, because it changes the arrowhead used and it carries real meaning about how the call behaves at runtime.\nA synchronous message is a call where the sender blocks and waits until the receiver finishes processing before the sender does anything else. This is the default, ordinary function or method call: result = obj.method() does not move to the next line of code until method() has fully returned. It is drawn as a solid line with a solid, filled arrowhead.\nAn asynchronous message is a call where the sender fires the message and immediately continues its own work, without waiting for the receiver to process it or respond. This corresponds to publishing an event to a queue, triggering a background job, or asyncio.create_task(obj.method()) in Python, where control returns to the caller right away. It is drawn as a solid line with an open (unfilled) arrowhead, visually thinner and less \u0026ldquo;solid-looking\u0026rdquo; than a synchronous call\u0026rsquo;s arrowhead, which is the visual cue that the sender is not going to sit and wait.\nA return message is the reply that eventually comes back from a synchronous call, once the receiver has finished. It is drawn as a dashed line with a filled arrowhead, distinguishing it from the solid line of the original request. Asynchronous calls typically do not have a matching return message at all in the same diagram, since the whole point is that the sender does not wait for one; if a reply does eventually happen, it is drawn as its own separate asynchronous message later on, not as a \u0026ldquo;return\u0026rdquo; tied to the original call.\nsequenceDiagram participant A as ClientCode participant B as InventoryService participant Q as EventQueue A-\u0026gt;\u0026gt;B: checkStock(productId) B--\u0026gt;\u0026gt;A: stockCount A-)Q: publishStockCheckedEvent(productId) A-\u0026gt;\u0026gt;A: logRequest() Arrow syntax Line and arrowhead Meaning Python equivalent -\u0026gt;\u0026gt; Solid line, filled (solid) arrowhead Synchronous call: the sender waits for this to complete before continuing result = obj.method() --\u0026gt;\u0026gt; Dashed line, filled (solid) arrowhead Return message: the reply to a synchronous call, sent once processing finishes The value produced by a return statement -) Solid line, open (unfilled) arrowhead Asynchronous call: the sender does not wait, and continues immediately asyncio.create_task(obj.method()), or publishing a message to a queue A-\u0026gt;\u0026gt;A Solid line, filled arrowhead, same participant on both ends Self-message: an object calling its own method self.some_other_method() inside a class In the diagram above, checkStock(productId) is synchronous because ClientCode genuinely needs the stockCount value before it can proceed, so it makes sense for it to wait. publishStockCheckedEvent(productId) is asynchronous because ClientCode has no reason to wait around for whatever downstream system consumes that event; it fires the message and moves straight on to logRequest().\nThe question to ask when choosing between the two for your own diagrams: does the sender need something back before it can continue, or is the sender just informing another part of the system and moving on regardless of what happens next? The former is synchronous; the latter is asynchronous. Choosing correctly matters beyond the diagram too, since it usually reflects a real architectural decision, such as whether an operation is a blocking network call or a message published to a queue.\nWhere synchronous and asynchronous messages appear across this material\u0026rsquo;s examples:\nExample Message types used Why Section 8, Lifelines and Activation Bars Synchronous only placeOrder needs a confirmation before the customer\u0026rsquo;s interaction is complete Section 8, Object Creation and Destruction Synchronous only Each step (addItem, finalize) depends on the previous one having completed Section 9, alt/opt/loop/par Synchronous only Kept simple deliberately, so the fragment being taught is the only new concept in each diagram Section 10.2, the e-commerce checkout Synchronous only Every step, checking stock, charging payment, reducing stock, genuinely needs to happen in order, with each step\u0026rsquo;s result determining the next one Section 9.5, Example 1 (login) Synchronous only The login attempt cannot proceed or respond to the user until the database lookup returns Section 9.5, Example 2 (password reset) Both Generating a token and confirming the request back to the user is synchronous, since the user is waiting on screen; queuing and later delivering the email is asynchronous, since the user should not be kept waiting for an email to actually send Section 9.5, Example 3 (ATM withdrawal) Synchronous only Every step (balance check, cash check, debit, dispense) must complete before the next one can safely happen Notice that most real system flows lean synchronous when a decision downstream depends on the result of the current step, and asynchronous when a step is closer to \u0026ldquo;notify and move on.\u0026rdquo; The password reset example (Section 9.5) is the clearest illustration of both appearing side by side in one diagram, precisely because it contains one of each kind of step.\nObject Creation and Destruction sequenceDiagram participant S as OrderService create participant O as Order S-\u0026gt;\u0026gt;O: new Order() S-\u0026gt;\u0026gt;O: addItem(product, qty) S-\u0026gt;\u0026gt;O: finalize() O--\u0026gt;\u0026gt;S: total destroy O S-\u0026gt;\u0026gt;O: discard() Two Mermaid keywords do the actual work here, and both are easy to miss: create participant O as Order (instead of a plain participant O as Order in the header) is what makes Order\u0026rsquo;s lifeline start partway down the diagram, at the new Order() message, rather than at the top alongside OrderService. Simply naming a participant in an arrow is not enough to shorten its lifeline; without the create keyword, Mermaid draws every participant\u0026rsquo;s lifeline at full height from the very top, regardless of when a \u0026ldquo;creation\u0026rdquo; message is drawn pointing at it. Likewise, destroy O, placed on the line immediately before Order\u0026rsquo;s final message, is what causes Mermaid to end Order\u0026rsquo;s lifeline right after that message, so it stops short instead of running to the bottom of the diagram like OrderService\u0026rsquo;s does. The destroy statement is placed before the last message involving that object, not after, since it marks that the object\u0026rsquo;s upcoming message is its last.\nNotice that total (a normal return value) and discard() (the act of destruction) are kept as two separate messages here, rather than combining them into one arrow. This is a deliberate change from an earlier version of this example, and it is worth calling out why: total is ordinary business logic, a value Order had reason to hand back regardless of what happens to it afterward, while discard() is a lifecycle event, OrderService deciding it no longer needs this Order object. Folding both meanings onto a single arrow blurs that distinction for a reader, and in practice it can also trigger rendering issues in some Mermaid versions, since the arrow is being asked to represent a return value and a lifeline termination at the same time. Giving destruction its own dedicated message keeps both the diagram\u0026rsquo;s meaning and its rendering clean. The exact visual marker Mermaid draws at the point of destruction (a small X, or simply a shortened lifeline with no further activity) varies slightly by renderer and theme; what to look for reliably is that the object\u0026rsquo;s lifeline visibly stops at that point rather than continuing to the bottom of the diagram.\nStep From To Message What it means 1 OrderService Order new Order() A new Order object is created; its lifeline begins here 2 OrderService Order addItem(product, qty) An item is added to the newly created order 3 OrderService Order finalize() The order is marked complete, no more items can be added 4 Order OrderService total (return) An ordinary business return value, unrelated to the object\u0026rsquo;s lifecycle 5 (n/a) Order destroy Marks that Order\u0026rsquo;s next message is its last 6 OrderService Order discard() The message at which Order\u0026rsquo;s lifeline actually ends 9. Sequence Diagrams: Control Flow Real scenarios branch, repeat, and sometimes run in parallel. UML sequence diagrams use combined fragments to express this.\nalt: Alternative Paths sequenceDiagram participant C as Customer participant P as PaymentService C-\u0026gt;\u0026gt;P: pay(amount) alt payment succeeds P--\u0026gt;\u0026gt;C: paymentConfirmed else payment fails P--\u0026gt;\u0026gt;C: paymentDeclined end Step From To Message What it means 1 Customer PaymentService pay(amount) Customer requests a payment 2a PaymentService Customer paymentConfirmed Runs only if the payment succeeds 2b PaymentService Customer paymentDeclined Runs only if the payment fails; mutually exclusive with 2a opt: Optional Step, No Else Branch sequenceDiagram participant O as OrderService participant N as NotificationService O-\u0026gt;\u0026gt;O: finalizeOrder() opt customer opted into SMS updates O-\u0026gt;\u0026gt;N: sendSms(orderId) end Step From To Message What it means 1 OrderService OrderService finalizeOrder() Self-message, the order is finalized 2 OrderService NotificationService sendSms(orderId) Runs only if the condition in the opt box is true; skipped entirely otherwise, with no alternative path loop: Repetition sequenceDiagram participant O as OrderService participant Inv as InventoryService loop for each item in cart O-\u0026gt;\u0026gt;Inv: reserve(item) Inv--\u0026gt;\u0026gt;O: reserved end Step From To Message What it means 1 OrderService InventoryService reserve(item) Repeated once for every item in the cart 2 InventoryService OrderService reserved Confirmation returned for each reservation, one pair of messages per iteration par: Parallel or Concurrent Steps sequenceDiagram participant O as OrderService participant E as EmailService participant S as SmsService par send email O-\u0026gt;\u0026gt;E: sendConfirmationEmail() and send sms O-\u0026gt;\u0026gt;S: sendConfirmationSms() end Step From To Message What it means 1a OrderService EmailService sendConfirmationEmail() Runs concurrently with 1b, neither waits for the other 1b OrderService SmsService sendConfirmationSms() Runs concurrently with 1a These four fragments correspond directly to if/else, if, for, and concurrent execution in code. Learning to draw them means any control flow a method contains, not only its happy path, can be modeled visually.\n9.5 More Sequence Diagram Examples The examples above each isolate a single concept. Real scenarios usually combine several of them at once. The three examples below walk through different kinds of systems, each combining multiple message types and control-flow fragments, to build fluency in reading and drawing more realistic diagrams.\nExample 1: User Login with Failed Attempts A user submits a username and password. The system checks the credentials against the database. If they are correct, a session token is created and returned. If they are incorrect, an error is returned, and after three consecutive failures the account is locked.\nsequenceDiagram actor U as User participant Auth as AuthService participant DB as UserDatabase U-\u0026gt;\u0026gt;+Auth: login(username, password) Auth-\u0026gt;\u0026gt;+DB: findUser(username) DB--\u0026gt;\u0026gt;-Auth: userRecord alt credentials valid Auth-\u0026gt;\u0026gt;Auth: generateSessionToken() Auth--\u0026gt;\u0026gt;U: loginSuccess(token) else credentials invalid Auth-\u0026gt;\u0026gt;Auth: incrementFailedAttempts() alt failedAttempts \u0026gt;= 3 Auth-\u0026gt;\u0026gt;DB: lockAccount(username) Auth--\u0026gt;\u0026gt;U: accountLocked else failedAttempts \u0026lt; 3 Auth--\u0026gt;\u0026gt;U: invalidCredentials end end deactivate Auth Step From To Message What it means 1 User AuthService login(username, password) Login attempt begins; AuthService activation starts 2 AuthService UserDatabase findUser(username) Look up the stored record for this username 3 UserDatabase AuthService userRecord (return) The stored record, including the correct password hash, is returned 4a AuthService AuthService generateSessionToken() Self-message, runs only if credentials are valid 5a AuthService User loginSuccess(token) Runs only in the valid-credentials branch 4b AuthService AuthService incrementFailedAttempts() Runs only if credentials are invalid 5b AuthService UserDatabase lockAccount(username) Runs only in the nested branch where failed attempts have reached 3 6b AuthService User accountLocked or invalidCredentials Whichever nested branch applies determines the exact message returned This example shows a nested alt inside another alt, the same pattern used later in the e-commerce diagram: the outer branch decides between success and failure, and the inner branch further decides what kind of failure response to send.\nExample 2: Asynchronous Password Reset A user requests a password reset. The system does not make the user wait for an email to be sent; it queues the email job and immediately confirms the request was received. The email itself is sent by a separate worker process sometime later.\nsequenceDiagram actor U as User participant API as ResetPasswordAPI participant Q as EmailQueue participant W as EmailWorker U-\u0026gt;\u0026gt;+API: requestPasswordReset(email) API-\u0026gt;\u0026gt;API: generateResetToken() API-)Q: enqueueEmailJob(email, token) API--\u0026gt;\u0026gt;-U: \u0026#34;Reset link sent, check your inbox\u0026#34; Note over Q,W: Some time later, independent of the user\u0026#39;s request Q-)W: deliverJob(email, token) W-\u0026gt;\u0026gt;W: sendEmail(email, token) Step From To Message What it means 1 User ResetPasswordAPI requestPasswordReset(email) User initiates the password reset flow 2 ResetPasswordAPI ResetPasswordAPI generateResetToken() Self-message, a one-time token is generated 3 ResetPasswordAPI EmailQueue enqueueEmailJob(email, token) An asynchronous message (open arrowhead), the API does not wait for this to complete 4 ResetPasswordAPI User Confirmation text (return) The user gets an immediate response, well before any email is actually sent 5 EmailQueue EmailWorker deliverJob(email, token) Happens independently, at a later and unpredictable time 6 EmailWorker EmailWorker sendEmail(email, token) Self-message, the actual email is sent by the worker, not by the original API call The -) arrow style marks an asynchronous message, drawn with an open arrowhead instead of a filled one, signaling that the sender does not block waiting for the receiver. This is the diagram-level equivalent of publishing a job to a queue in code, rather than calling a function directly and waiting for its result. Notice also that the API\u0026rsquo;s activation bar closes right after step 4; the API\u0026rsquo;s own work is done at that point, even though the email has not been sent yet.\nExample 3: ATM Cash Withdrawal A customer requests a withdrawal at an ATM. The ATM checks the account balance. If funds are sufficient, it also checks whether the machine physically has enough cash on hand before dispensing it.\nsequenceDiagram actor C as Customer participant ATM as ATMMachine participant Bank as BankServer C-\u0026gt;\u0026gt;+ATM: withdraw(amount) ATM-\u0026gt;\u0026gt;+Bank: checkBalance(accountId) Bank--\u0026gt;\u0026gt;-ATM: balance alt balance \u0026gt;= amount ATM-\u0026gt;\u0026gt;ATM: checkCashAvailable(amount) alt cash available in machine ATM-\u0026gt;\u0026gt;+Bank: debitAccount(accountId, amount) Bank--\u0026gt;\u0026gt;-ATM: debitConfirmed ATM-\u0026gt;\u0026gt;ATM: dispenseCash(amount) ATM--\u0026gt;\u0026gt;C: cash and receipt else cash not available in machine ATM--\u0026gt;\u0026gt;C: \u0026#34;Unable to dispense, try a smaller amount\u0026#34; end else balance \u0026lt; amount ATM--\u0026gt;\u0026gt;C: \u0026#34;Insufficient funds\u0026#34; end deactivate ATM Step From To Message What it means 1 Customer ATMMachine withdraw(amount) Customer requests a withdrawal 2 ATMMachine BankServer checkBalance(accountId) ATM asks the bank for the current balance 3 BankServer ATMMachine balance (return) Balance is returned to the ATM 4 ATMMachine ATMMachine checkCashAvailable(amount) Self-message, runs only if the balance check passed 5 ATMMachine BankServer debitAccount(accountId, amount) Runs only if both the balance and physical cash checks passed 6 BankServer ATMMachine debitConfirmed Confirms the account has been debited 7 ATMMachine ATMMachine dispenseCash(amount) Self-message, physical cash is released 8 ATMMachine Customer Cash and receipt, or an error message The exact outcome depends on which of the three branches was taken This example demonstrates two independent decision points modeled as a nested alt: the outer check is a business rule (does the account have enough money), and the inner check is a physical constraint (does the machine have enough cash). Separating them into nested fragments, rather than one large combined condition, keeps each decision traceable to a single, specific reason for failure.\n10. Putting It All Together: A Simple E-Commerce System The following combines the concepts above into one small e-commerce system: a Customer places an Order made of OrderItems, each referencing a Product. The order is charged through a PaymentMethod, and once paid, Inventory is updated and a Notification is sent. This is intentionally minimal so the concepts working together remain visible, rather than buried under production-scale complexity.\n10.1 The Complete Class Diagram classDiagram class Customer { -customer_id: str -name: str -email: str +place_order(items: List) Order } class Order { -order_id: str -status: OrderStatus +add_item(product: Product, qty: int) void +calculate_total() float +checkout(payment: PaymentMethod) bool } class OrderItem { -quantity: int -unit_price: float +subtotal() float } class Product { -product_id: str -name: str -price: float } class OrderStatus { \u0026lt;\u0026lt;enumeration\u0026gt;\u0026gt; PENDING PAID SHIPPED CANCELLED } class PaymentMethod { \u0026lt;\u0026lt;interface\u0026gt;\u0026gt; +pay(amount: float) bool } class CreditCardPayment { +pay(amount: float) bool } class UpiPayment { +pay(amount: float) bool } class Inventory { +check_stock(product: Product) bool +reduce_stock(product: Product, qty: int) void } class NotificationService { +send(customer: Customer, message: str) void } Customer \u0026#34;1\u0026#34; --\u0026gt; \u0026#34;*\u0026#34; Order : places Order \u0026#34;1\u0026#34; *-- \u0026#34;1..*\u0026#34; OrderItem : contains OrderItem \u0026#34;*\u0026#34; --\u0026gt; \u0026#34;1\u0026#34; Product : refers to Order --\u0026gt; OrderStatus : has Order ..\u0026gt; PaymentMethod : uses PaymentMethod \u0026lt;|.. CreditCardPayment PaymentMethod \u0026lt;|.. UpiPayment Order ..\u0026gt; Inventory : uses Order ..\u0026gt; NotificationService : uses Reading this diagram: a Customer places one or more Orders, an association with 1-to-many multiplicity. An Order is composed of OrderItems, drawn with a filled diamond, because an OrderItem has no meaning without its Order and is deleted along with it. Each OrderItem refers to a Product through a plain association, since the same Product is referenced by many orders over time and is not owned by any single one of them. Order has an OrderStatus enum. Order uses a PaymentMethod only during checkout, passed in and used rather than stored, which is why it is drawn as a dependency with a dashed arrow. CreditCardPayment and UpiPayment each realize the PaymentMethod interface, letting Order.checkout() accept any payment method without knowing which concrete one it received. Order also has brief dependencies on Inventory and NotificationService, used during checkout and not held onto afterward.\nCorresponding Python skeleton, trimmed for clarity:\nfrom abc import ABC, abstractmethod from enum import Enum from typing import List class OrderStatus(Enum): PENDING = \u0026#34;pending\u0026#34; PAID = \u0026#34;paid\u0026#34; SHIPPED = \u0026#34;shipped\u0026#34; CANCELLED = \u0026#34;cancelled\u0026#34; class Product: def __init__(self, product_id: str, name: str, price: float): self.product_id = product_id self.name = name self.price = price class OrderItem: def __init__(self, product: Product, quantity: int): self.product = product # Association: OrderItem --\u0026gt; Product self.quantity = quantity self.unit_price = product.price def subtotal(self) -\u0026gt; float: return self.unit_price * self.quantity class PaymentMethod(ABC): # \u0026lt;\u0026lt;interface\u0026gt;\u0026gt; @abstractmethod def pay(self, amount: float) -\u0026gt; bool: ... class CreditCardPayment(PaymentMethod): def pay(self, amount: float) -\u0026gt; bool: print(f\u0026#34;Charged {amount} to credit card\u0026#34;) return True class UpiPayment(PaymentMethod): def pay(self, amount: float) -\u0026gt; bool: print(f\u0026#34;Charged {amount} via UPI\u0026#34;) return True class Inventory: def check_stock(self, product: Product) -\u0026gt; bool: return True # simplified def reduce_stock(self, product: Product, qty: int) -\u0026gt; None: print(f\u0026#34;Reduced stock for {product.name} by {qty}\u0026#34;) class NotificationService: def send(self, customer: \u0026#34;Customer\u0026#34;, message: str) -\u0026gt; None: print(f\u0026#34;Notifying {customer.name}: {message}\u0026#34;) class Order: def __init__(self, order_id: str): self.order_id = order_id self.items: List[OrderItem] = [] # Composition: Order *-- OrderItem self.status = OrderStatus.PENDING def add_item(self, product: Product, qty: int) -\u0026gt; None: self.items.append(OrderItem(product, qty)) def calculate_total(self) -\u0026gt; float: return sum(item.subtotal() for item in self.items) def checkout( self, payment: PaymentMethod, # Dependency: used, not stored inventory: Inventory, notifier: NotificationService, customer: \u0026#34;Customer\u0026#34;, ) -\u0026gt; bool: total = self.calculate_total() for item in self.items: if not inventory.check_stock(item.product): return False if not payment.pay(total): return False for item in self.items: inventory.reduce_stock(item.product, item.quantity) self.status = OrderStatus.PAID notifier.send(customer, f\u0026#34;Order {self.order_id} confirmed!\u0026#34;) return True class Customer: def __init__(self, customer_id: str, name: str, email: str): self.customer_id = customer_id self.name = name self.email = email self.orders: List[Order] = [] # Association: Customer --\u0026gt; Order def place_order(self, order: Order) -\u0026gt; None: self.orders.append(order) 10.2 The Complete Sequence Diagram The behavioral view of one scenario from the class diagram above: a customer checks out an order.\nsequenceDiagram actor Cust as Customer participant O as Order participant Inv as Inventory participant P as PaymentMethod participant N as NotificationService Cust-\u0026gt;\u0026gt;+O: checkout(payment, inventory, notifier) O-\u0026gt;\u0026gt;O: calculate_total() loop for each item in order O-\u0026gt;\u0026gt;Inv: check_stock(product) Inv--\u0026gt;\u0026gt;O: inStock (bool) end alt all items in stock O-\u0026gt;\u0026gt;+P: pay(total) alt payment succeeds P--\u0026gt;\u0026gt;O: true loop for each item in order O-\u0026gt;\u0026gt;Inv: reduce_stock(product, qty) end O-\u0026gt;\u0026gt;O: status = PAID O-\u0026gt;\u0026gt;N: send(customer, \u0026#34;Order confirmed\u0026#34;) N--\u0026gt;\u0026gt;Cust: order confirmation email or SMS O--\u0026gt;\u0026gt;Cust: checkoutSuccess else payment fails P--\u0026gt;\u0026gt;O: false O--\u0026gt;\u0026gt;Cust: checkoutFailed(\u0026#34;payment declined\u0026#34;) end deactivate P else an item is out of stock O--\u0026gt;\u0026gt;Cust: checkoutFailed(\u0026#34;out of stock\u0026#34;) end deactivate O A note on this diagram\u0026rsquo;s structure: notice that PaymentMethod is activated once, with +P attached to the pay(total) message, and deactivated exactly once, right after the nested alt block ends, rather than once inside each of the two branches. This matters because Mermaid (and UML tooling generally) reads activation and deactivation as a simple, linear count for each participant across the entire diagram source, not as a runtime simulation of \u0026ldquo;only one branch actually happens.\u0026rdquo; Placing a deactivate P inside both the \u0026ldquo;payment succeeds\u0026rdquo; branch and the \u0026ldquo;payment fails\u0026rdquo; branch would attempt to deactivate PaymentMethod twice for a single activation, which some renderers (including GitHub\u0026rsquo;s) will reject with an error. Whenever a participant is activated before a branching fragment and stays relevant across more than one branch, close its activation once, after the fragment ends, rather than once per branch.\nReading this diagram: the Customer calls checkout() on Order, a synchronous call, so Order\u0026rsquo;s activation bar begins. Order checks stock for every item inside a loop, matching the for item in self.items: inventory.check_stock(...) line in the Python code above. An alt fragment branches on whether all items are in stock. Inside the \u0026ldquo;in stock\u0026rdquo; branch, a nested alt branches on whether payment.pay() succeeds, matching the if not payment.pay(total): return False line in the code. On success, stock is reduced in another loop, the order status changes through a self-message (since Order is updating its own field), and a message is sent through NotificationService. Every arrow in this diagram corresponds to a method call that exists in the Python checkout() method above: a sequence diagram is a method\u0026rsquo;s control flow, drawn as a conversation between objects instead of as code.\nThe table below traces every step of the diagram in order, so the flow can be followed independently of the visual layout.\nStep From To Message Fragment context 1 Customer Order checkout(payment, inventory, notifier) Entry point, Order\u0026rsquo;s activation begins 2 Order Order calculate_total() Self-message, computed once before any branching 3 Order Inventory check_stock(product) Inside loop, repeated once per item in the order 4 Inventory Order inStock (return) Return value for each iteration of the stock-check loop 5 Order PaymentMethod pay(total) Runs only inside the \u0026ldquo;all items in stock\u0026rdquo; branch of the outer alt 6a PaymentMethod Order true (return) Runs only inside the \u0026ldquo;payment succeeds\u0026rdquo; branch of the inner alt 7a Order Inventory reduce_stock(product, qty) Inside a loop, nested within the \u0026ldquo;payment succeeds\u0026rdquo; branch, repeated per item 8a Order Order status = PAID Self-message, runs only after stock has been reduced 9a Order NotificationService send(customer, \u0026quot;Order confirmed\u0026quot;) Runs only in the success path 10a NotificationService Customer Order confirmation (return) The customer receives the confirmation 11a Order Customer checkoutSuccess Final response to the customer for the success path 6b PaymentMethod Order false (return) Runs only inside the \u0026ldquo;payment fails\u0026rdquo; branch of the inner alt 7b Order Customer checkoutFailed(\u0026quot;payment declined\u0026quot;) Final response to the customer for the payment-failure path 6c Order Customer checkoutFailed(\u0026quot;out of stock\u0026quot;) Runs only inside the \u0026ldquo;an item is out of stock\u0026rdquo; branch of the outer alt, skipping payment entirely Only one of the three final rows (11a, 7b, or 6c) is ever reached in a single execution; which one depends on which branch of each alt fragment is taken.\n11. Summary and Key Takeaways UML provides a standardized, language-agnostic way to communicate design, addressing the ambiguity of prose and the excessive detail of raw code. UML defines 14 diagram types across two families: structural diagrams (Class, Object, Component, Composite Structure, Package, Deployment, Profile) describe what a system looks like, and behavioral diagrams (Use Case, Activity, State Machine, Sequence, Communication, Interaction Overview, Timing) describe how it behaves over time. Class diagrams and sequence diagrams are the two most heavily used diagrams in day-to-day design work, which is why this material focuses on them in depth. Class diagram anatomy consists of name, attributes, and methods, with +/-/#/~ visibility symbols mapping directly to Python\u0026rsquo;s public, __private, and _protected conventions. An abstract class can hold state and provide some implemented methods, and a class can inherit from only one. An interface holds no state and provides no implementation at all, only a contract, and a class can implement many of them. This distinction is why inheritance uses a solid line and realization uses a dashed line. The seven relationships between classes, ordered from weakest to strongest coupling: Dependency (..\u0026gt;, briefly uses, no stored reference), Association (--\u0026gt;, holds a reference, with multiplicity), Aggregation (o--, has-a, part survives without the whole), Composition (*--, has-a, part dies with the whole), Realization (\u0026lt;|.., implements an interface\u0026rsquo;s contract, no shared code), and Inheritance (\u0026lt;|--, is-a, shares contract and implementation). Multiplicity (1, 0..1, *, 1..*) states precisely how many objects participate on each end of a relationship. Sequence diagram anatomy consists of lifelines (objects over time), activation bars (when an object is executing), and four message types: synchronous call, return, asynchronous call, and self-message. Combined fragments (alt, opt, loop, par) are the visual equivalent of if/else, if, for, and concurrent execution, letting a sequence diagram model any control flow a method contains. A class diagram is the structural blueprint of a system\u0026rsquo;s nouns; a sequence diagram is the behavioral record of its verbs for one specific scenario. Strong design communication uses both together. 12. Practice Questions Attempt each question before checking the solution. Draw the diagram by hand or in Mermaid before reading further.\nQuestion 1 (Level 1: Relationship Identification) For each pair below, name the correct UML relationship and explain the reasoning in one sentence:\na) A Car and its Engine (the engine is built specifically for that car and is scrapped with it). b) A Library and its Books (books continue to exist even if the library closes down and donates them elsewhere). c) A Driver and a Car they are currently renting (the driver calls car.start() a few times during a trip; no field is stored). d) A Shape abstract class and its Circle/Square subclasses, which inherit a describe() method. e) A Drawable interface with only abstract methods, implemented by Circle and Square.\nSolution a) Composition. The Engine\u0026rsquo;s lifecycle is entirely bound to the Car; it cannot meaningfully outlive it in this scenario. Filled diamond on the Car end.\nb) Aggregation. Books can exist independently of any particular Library. Hollow diamond on the Library end.\nc) Dependency. The Driver uses the Car only within the scope of a call or trip and does not hold a permanent reference to it. Dashed arrow.\nd) Inheritance (Generalization). Circle/Square are specialized Shapes and inherit actual implementation (describe()). Solid line, hollow triangle.\ne) Realization. Circle/Square implement the Drawable contract but share no implementation with it, since there is none to share; it is pure abstract methods. Dashed line, hollow triangle.\nQuestion 2 (Level 1: Multiplicity) A Playlist can contain zero or more Songs, and the same Song can appear in many different Playlists. Draw the association with correct multiplicity on both ends, and explain what real-world implementation detail this multiplicity implies.\nSolution classDiagram Playlist \u0026#34;0..*\u0026#34; -- \u0026#34;0..*\u0026#34; Song : contains This is a many-to-many relationship. In a relational database, many-to-many relationships cannot be modeled with a simple foreign key on either table; they require a join or association table, for example PlaylistSong(playlist_id, song_id, position). In Python, both classes typically hold a list referencing the other:\nclass Song: def __init__(self, title: str): self.title = title class Playlist: def __init__(self, name: str): self.name = name self.songs: list[Song] = [] Question 3 (Level 2: Draw a Class Diagram) Design a simple class diagram for a library book-lending system with these rules:\nA Library has many Books. A Member can borrow multiple Books, and a Book can be borrowed by different Members over time, not simultaneously. Borrowing creates a Loan record that tracks the borrow date and due date, and has no meaning outside the borrowing event. Book has a BookStatus (AVAILABLE, BORROWED, LOST). Identify at least one association, one aggregation or composition, and one enum.\nSolution classDiagram class Library { -name: str +add_book(book: Book) void } class Book { -isbn: str -title: str -status: BookStatus } class BookStatus { \u0026lt;\u0026lt;enumeration\u0026gt;\u0026gt; AVAILABLE BORROWED LOST } class Member { -member_id: str -name: str +borrow(book: Book) Loan } class Loan { -borrow_date: str -due_date: str +is_overdue() bool } Library \u0026#34;1\u0026#34; o-- \u0026#34;*\u0026#34; Book : has Book --\u0026gt; BookStatus : has Member \u0026#34;1\u0026#34; --\u0026gt; \u0026#34;*\u0026#34; Loan : creates Loan \u0026#34;*\u0026#34; --\u0026gt; \u0026#34;1\u0026#34; Book : refers to Library o-- Book is aggregation: books can be transferred to another library or exist in a catalog independent of one specific Library instance. Member --\u0026gt; Loan is an association: a member creates loans over time, but a Loan\u0026rsquo;s existence is tied to the borrowing event rather than the Member\u0026rsquo;s own lifecycle (a case for composition here is also defensible with reasoning). Loan --\u0026gt; Book is a plain association: a Loan refers to a Book without owning it. BookStatus is an enumeration.\nThere is rarely one single correct answer in class design. What matters is being able to justify each relationship choice against the question of whether the part survives the whole.\nQuestion 4 (Level 2: Draw a Sequence Diagram) Using the library system from Question 3, draw a sequence diagram for the scenario: a Member tries to borrow a Book. If the book is available, a Loan is created and the book\u0026rsquo;s status changes to BORROWED. If the book is not available, the member is told it is unavailable.\nSolution sequenceDiagram actor M as Member participant B as Book participant L as Library M-\u0026gt;\u0026gt;L: borrow(book) activate L L-\u0026gt;\u0026gt;B: check status B--\u0026gt;\u0026gt;L: status alt status == AVAILABLE L-\u0026gt;\u0026gt;L: create Loan L-\u0026gt;\u0026gt;B: status = BORROWED L--\u0026gt;\u0026gt;M: loanConfirmation else status != AVAILABLE L--\u0026gt;\u0026gt;M: \u0026#34;Book unavailable\u0026#34; end deactivate L The alt fragment mirrors an if/else in the eventual Python implementation, and the self-message L-\u0026gt;\u0026gt;L: create Loan shows Library instantiating a new Loan object as part of its own method body.\nQuestion 5 (Level 3: Spot the Mistake) A junior developer drew this diagram for a blogging platform:\nclassDiagram Author \u0026#34;1\u0026#34; *-- \u0026#34;*\u0026#34; BlogPost : writes BlogPost \u0026#34;*\u0026#34; *-- \u0026#34;*\u0026#34; Tag : has Identify what is wrong with both relationships and propose the correct notation.\nSolution Problem 1: Author *-- BlogPost (Composition). This states that a BlogPost cannot exist without its Author and is destroyed the moment the Author is deleted. That is usually too strong; most blogging platforms preserve posts, perhaps reassigned to a \u0026ldquo;deleted user,\u0026rdquo; even after an author\u0026rsquo;s account is removed. This should generally be aggregation (o--) or a plain association (--\u0026gt;), not composition, unless the specific system genuinely cascades deletion of posts with the author, which is a business decision to confirm rather than a default choice.\nProblem 2: BlogPost *-- Tag (Composition) with * -- * multiplicity. Composition represents exclusive ownership: one whole owns each part. Combining composition with a many-to-many multiplicity on the part side is a modeling error, because the same Tag (for example, \u0026ldquo;python\u0026rdquo;) is almost certainly shared across many BlogPosts and cannot be exclusively owned by all of them at once. This should be a plain association (--\u0026gt;) with * -- * multiplicity.\nCorrected diagram:\nclassDiagram Author \u0026#34;1\u0026#34; o-- \u0026#34;*\u0026#34; BlogPost : writes BlogPost \u0026#34;*\u0026#34; -- \u0026#34;*\u0026#34; Tag : has The general lesson: composition should be reserved for genuine exclusive, lifecycle-bound parts, such as Order/OrderItem. When uncertain, default to association or aggregation, and upgrade to composition only when the \u0026ldquo;does the part survive the whole\u0026rdquo; test clearly supports it.\nQuestion 6 (Level 3: Design Discussion) In the e-commerce example from Section 10, Order.checkout() takes payment, inventory, and notifier as parameters rather than storing them as fields on Order, making them dependencies rather than associations. A teammate suggests storing them as fields instead, for convenience. What is the correct response, referencing both UML and design principles?\nSolution Storing payment, inventory, and notifier as permanent fields on Order would upgrade three dependencies into three associations, and this is a design regression rather than an improvement, for three reasons:\nEvery Order object would be forced to carry references to services it needs only during checkout, bloating the object and creating a misleading class diagram, since a reader would assume Order needs PaymentMethod for its entire lifetime, including before and after checkout, which is not true.\nIt couples Order\u0026rsquo;s construction to the availability of these services, so a lightweight Order object could not be created in a test or a script without also providing a working Inventory and NotificationService, even when only testing calculate_total().\nThis is precisely the Dependency Injection pattern, and it maps to the Dependency Inversion Principle from SOLID: pass in what a method needs, when it needs it, rather than baking dependencies into the object\u0026rsquo;s permanent structure. The dashed dependency arrows in the diagram directly signal that this coupling is intentionally temporary and lightweight, a property worth protecting.\nThe short answer for the teammate: keeping them as parameters keeps Order a lightweight data-and-rules object; storing them as fields would make every Order permanently depend on services it only needs for a few seconds during checkout, so the current diagram correctly reflects the intended design.\nThis content is part of Codeverra, a platform for learning coding, data science, DSA, and AI from scratch. Explore more: https://codeverra.com\n","permalink":"https://blogs.codeverra.com/blog/low_level_design/08_uml_deep_dive/","summary":"\u003ch1 id=\"uml-deep-dive-class-diagrams-and-sequence-diagrams\"\u003eUML Deep Dive: Class Diagrams and Sequence Diagrams\u003c/h1\u003e\n\u003cblockquote\u003e\n\u003cp\u003e\u003cstrong\u003eWho this is for:\u003c/strong\u003e Anyone who has been designing classes and systems (perhaps through OOP, SOLID, and design patterns) and now wants a standardized visual language to communicate those designs to teammates, interviewers, and their own future self.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003chr\u003e\n\u003ch2 id=\"table-of-contents\"\u003eTable of Contents\u003c/h2\u003e\n\u003col\u003e\n\u003cli\u003e\u003ca href=\"#1-why-uml-exists\"\u003eWhy UML Exists\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#2-the-14-uml-diagram-types\"\u003eThe 14 UML Diagram Types: A Complete Map\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#3-class-diagrams-anatomy\"\u003eClass Diagrams: Anatomy of a Class Box\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#4-abstract-classes-vs-interfaces\"\u003eAbstract Classes vs. Interfaces: Are They the Same Thing?\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#5-class-diagrams-relationships\"\u003eClass Diagrams: Relationships Between Classes\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#6-class-diagrams-advanced\"\u003eClass Diagrams: Advanced Concepts\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#7-sequence-diagrams-why\"\u003eSequence Diagrams: Why We Need a Second Diagram\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#8-sequence-diagrams-anatomy\"\u003eSequence Diagrams: Anatomy and Core Concepts\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#9-sequence-diagrams-control-flow\"\u003eSequence Diagrams: Control Flow (alt, opt, loop, par)\u003c/a\u003e\n\u003cul\u003e\n\u003cli\u003e\u003ca href=\"#95-more-sequence-diagram-examples\"\u003e9.5 More Sequence Diagram Examples\u003c/a\u003e\u003c/li\u003e\n\u003c/ul\u003e\n\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#10-putting-it-together\"\u003ePutting It All Together: A Simple E-Commerce System\u003c/a\u003e\n\u003cul\u003e\n\u003cli\u003e\u003ca href=\"#101-the-complete-class-diagram\"\u003e10.1 The Complete Class Diagram\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#102-the-complete-sequence-diagram\"\u003e10.2 The Complete Sequence Diagram\u003c/a\u003e\u003c/li\u003e\n\u003c/ul\u003e\n\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#11-summary\"\u003eSummary and Key Takeaways\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#12-practice-questions\"\u003ePractice Questions with Solutions\u003c/a\u003e\u003c/li\u003e\n\u003c/ol\u003e\n\u003chr\u003e\n\u003ch2 id=\"1-why-uml-exists\"\u003e1. Why UML Exists\u003c/h2\u003e\n\u003cp\u003eWhen a design exists only in someone\u0026rsquo;s head, or scattered across a codebase, it is expensive to communicate. Explaining a system in prose forces the listener to reconstruct a graph of classes and relationships from a linear sequence of sentences, and details get lost or misinterpreted along the way. Two people reading the same paragraph of design prose regularly walk away with two different mental models of the same system, particularly around exactly how strong a relationship between two entities is.\u003c/p\u003e","tags":["lld","uml","design-patterns"],"title":"UML Deep Dive: Class Diagrams and Sequence Diagrams"},{"content":"🗺️ Array Patterns - The Complete Study Guide Welcome This is a structured, beginner-friendly study guide for mastering array and string patterns - the building blocks of coding interviews. Each pattern has a dedicated deep-dive document with theory, intuition, templates, and 10 LeetCode problems solved step by step.\nWho is this for?\nYou\u0026rsquo;re preparing for coding interviews and want a systematic approach You can solve easy problems but struggle to recognize which technique to apply You want to understand why an approach works, not just memorize solutions How to use this guide:\nFollow the patterns in order - they build on each other Read the theory and templates first Try each problem yourself for 15-20 minutes before reading the solution If stuck, read only the approach section (not the code) and try again After solving, read the dry run and edge cases to solidify understanding The Patterns # Pattern Document Key Idea Problems 1 Two Pointers 01_Two_Pointers.md Two indices moving through an array - toward each other or in the same direction - to find pairs, partition, or rearrange 10 2 Binary Search 02_Binary_Search.md Halving the search space each step by exploiting sorted order or a monotonic condition 10 3 Sliding Window 03_Sliding_Window.md A moving range between two pointers, tracking a running state (sum, frequency, etc.) to avoid redundant computation 10 4 Prefix Sum 04_Prefix_Sum.md Precomputing cumulative sums so any range sum can be answered in O(1) via subtraction 10 5 HashMap / Frequency Counting 05_HashMap_Frequency_Counting.md Trading O(n) space for O(1) lookups - complement search, frequency counting, grouping, and index tracking 10 6 Kadane\u0026rsquo;s Algorithm 06_Kadanes_Algorithm.md Finding the maximum (or minimum) subarray sum in one pass using a local vs global max decision Coming soon 7 Sorting-Based Patterns 07_Sorting_Based.md Sorting as a preprocessing step to simplify pair-finding, interval merging, and greedy decisions Coming soon 8 Interval / Merge Intervals 08_Intervals.md Sorting intervals by start time and merging or processing overlaps Coming soon 9 Monotonic Stack 09_Monotonic_Stack.md Maintaining a stack in increasing/decreasing order to efficiently find \u0026ldquo;next greater/smaller\u0026rdquo; elements Coming soon Total: 90 problems across 9 patterns when complete.\nHow the Patterns Connect These patterns don\u0026rsquo;t exist in isolation. Understanding how they relate helps you pick the right tool.\n┌──────────────┐ │ Two Pointers │ └──────┬───────┘ │ ┌───────────────┼───────────────┐ │ │ │ ▼ ▼ ▼ ┌────────────┐ ┌─────────────┐ ┌──────────────┐ │ Binary │ │ Sliding │ │ Sorting │ │ Search │ │ Window │ │ Based │ └────────────┘ └──────┬──────┘ └──────┬───────┘ │ │ │ ▼ │ ┌──────────────┐ │ │ Intervals │ │ └──────────────┘ ▼ ┌─────────────┐ │ Prefix Sum │ └──────┬──────┘ │ ▼ ┌─────────────┐ │ HashMap / │ │ Frequency │ └─────────────┘ ┌──────────────┐ ┌─────────────────┐ │ Kadane\u0026#39;s │ │ Monotonic Stack │ │ Algorithm │ │ │ └──────────────┘ └─────────────────┘ (builds on prefix sum (often combined with and sliding window) sliding window) Key relationships:\nTwo Pointers → Binary Search: Binary search uses two pointers (low, high) that converge on an answer by halving the search space. It\u0026rsquo;s the \u0026ldquo;sorted array\u0026rdquo; specialization of two pointers. Two Pointers → Sliding Window: Sliding window is two same-direction pointers with a \u0026ldquo;window state\u0026rdquo; tracked between them. Sliding Window → Prefix Sum: Both avoid recomputation over ranges. Sliding window updates incrementally; prefix sum precomputes everything upfront. Prefix Sum → HashMap: Many prefix sum problems use a hashmap to store and look up previous prefix values (e.g., \u0026ldquo;subarray sum equals k\u0026rdquo;). Sorting → Intervals: Nearly all interval problems start by sorting by start time. Kadane\u0026rsquo;s: Combines ideas from prefix sum (running sum) and sliding window (expand/reset decision). Monotonic Stack: Often paired with sliding window for problems like \u0026ldquo;sliding window maximum.\u0026rdquo; Pattern Selection Cheat Sheet When you see a new problem, use these signals to narrow down the pattern:\nIs the input SORTED (or should you sort it)? │ ├── YES, and looking for a TARGET value or boundary │ └── Binary Search │ ├── YES, and looking for PAIRS or comparing ends │ └── Two Pointers (opposite direction) │ ├── SORT FIRST, then process │ ├── Intervals / overlaps → Merge Intervals │ ├── Pairs / triplets → Sort + Two Pointers │ └── Greedy decisions → Sorting-Based │ └── NO (unsorted), what are you looking for? │ ├── CONTIGUOUS subarray / substring with some property? │ ├── Fixed size or longest/shortest → Sliding Window │ ├── Range sum queries → Prefix Sum │ ├── Count subarrays with sum = k → Prefix Sum + HashMap │ └── Maximum sum subarray → Kadane\u0026#39;s Algorithm │ ├── Finding PAIRS, complements, or duplicates? │ └── HashMap │ ├── Rearranging IN-PLACE? │ └── Two Pointers (same direction / partitioning) │ ├── \u0026#34;Next greater / smaller element\u0026#34;? │ └── Monotonic Stack │ └── Grouping or frequency analysis? └── HashMap / Frequency Counting Difficulty Progression Across All Patterns If you\u0026rsquo;re just starting out, here\u0026rsquo;s a suggested order of problems across all patterns, grouped by difficulty:\nPhase 1: Foundations (Easy) Problem Pattern LeetCode # Binary Search Binary Search 704 Two Sum HashMap 1 Valid Palindrome Two Pointers 125 Running Sum of 1D Array Prefix Sum 1480 Move Zeroes Two Pointers 283 Maximum Average Subarray I Sliding Window 643 Search Insert Position Binary Search 35 Squares of a Sorted Array Two Pointers 977 First Unique Character HashMap 387 Merge Sorted Array Two Pointers 88 Phase 2: Core Patterns (Easy-Medium) Problem Pattern LeetCode # Two Sum II Two Pointers 167 Find First and Last Position Binary Search 34 Remove Duplicates Two Pointers 26 Valid Anagram HashMap 242 Range Sum Query Prefix Sum 303 Find Pivot Index Prefix Sum 724 Contains Duplicate II HashMap 219 Search a 2D Matrix Binary Search 74 Minimum Size Subarray Sum Sliding Window 209 Intersection of Two Arrays II HashMap 350 Phase 3: Pattern Mastery (Medium) Problem Pattern LeetCode # Container With Most Water Two Pointers 11 3Sum Two Pointers 15 Search in Rotated Sorted Array Binary Search 33 Longest Substring Without Repeating Sliding Window 3 Subarray Sum Equals K Prefix Sum + HashMap 560 Group Anagrams HashMap 49 Top K Frequent Elements HashMap 347 Longest Consecutive Sequence HashMap/Set 128 Find Minimum in Rotated Sorted Array Binary Search 153 Koko Eating Bananas Binary Search on Answer 875 Phase 4: Advanced (Medium-Hard) Problem Pattern LeetCode # Trapping Rain Water Two Pointers 42 Minimum Window Substring Sliding Window 76 Sliding Window Maximum Sliding Window + Deque 239 4Sum II HashMap 454 Range Sum Query 2D 2D Prefix Sum 304 Median of Two Sorted Arrays Binary Search 4 Split Array Largest Sum Binary Search on Answer 410 Capacity to Ship Packages Binary Search on Answer 1011 Each Document Follows This Structure Every pattern document is organized the same way so you always know what to expect:\nWhat is [Pattern]? - Concept explained from scratch with a concrete example When to Use It? - Signals and triggers to recognize in problem statements Types / Variations - Sub-patterns within the main technique Template Code - Reusable Python templates with comments Problem Set (10 problems) - Each with: Problem statement \u0026amp; LeetCode link Clarifying questions \u0026amp; constraints Multiple approaches (brute force → optimal) with complexity analysis Well-documented Python code Edge case discussion Step-by-step dry run Key Takeaways \u0026amp; Summary - Tables, decision trees, and cheat sheets Getting Started Open Two Pointer Guide and begin. Good luck, and remember that the struggle is where the learning happens. Don\u0026rsquo;t rush to the solutions.\nThis guide is a work in progress. Patterns 6-9 are coming soon.\n","permalink":"https://blogs.codeverra.com/blog/dsa-patterns-00-index/","summary":"\u003ch1 id=\"-array-patterns-----the-complete-study-guide\"\u003e🗺️ Array Patterns  -  The Complete Study Guide\u003c/h1\u003e\n\u003ch2 id=\"welcome\"\u003eWelcome\u003c/h2\u003e\n\u003cp\u003eThis is a structured, beginner-friendly study guide for mastering \u003cstrong\u003earray and string patterns\u003c/strong\u003e  -  the building blocks of coding interviews. Each pattern has a dedicated deep-dive document with theory, intuition, templates, and 10 LeetCode problems solved step by step.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eWho is this for?\u003c/strong\u003e\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003eYou\u0026rsquo;re preparing for coding interviews and want a systematic approach\u003c/li\u003e\n\u003cli\u003eYou can solve easy problems but struggle to recognize \u003cem\u003ewhich technique\u003c/em\u003e to apply\u003c/li\u003e\n\u003cli\u003eYou want to understand \u003cem\u003ewhy\u003c/em\u003e an approach works, not just memorize solutions\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003e\u003cstrong\u003eHow to use this guide:\u003c/strong\u003e\u003c/p\u003e","tags":["dsa","dsa-patterns"],"title":"Array Patterns  -  The Complete Study Guide"},{"content":"Arrays Table of Contents What You Will Learn in This Blog How Data is Stored in Computer Memory What is an Array Arrays in Python Python Lists The array Module NumPy Arrays Comparison of the Three Referential Arrays vs Value Arrays Dynamic Arrays \u0026ndash; How Python Lists Grow Time and Space Complexity of Array Operations Two-Dimensional Arrays Patterns and Techniques Pattern 1 \u0026ndash; Two Pointers Pattern 2 \u0026ndash; Sliding Window Pattern 3 \u0026ndash; Prefix Sum Summary Key Takeaways Practice Problems Next Steps What You Will Learn in This Blog How data is stored in computer memory and why it matters What an array is and how it uses contiguous memory How Python represents arrays \u0026ndash; lists, the array module, and NumPy The difference between referential arrays and value arrays How dynamic arrays work and how Python lists resize themselves Time and space complexity of every common array operation Three core patterns that come from arrays: Two Pointers, Sliding Window, and Prefix Sum Solved LeetCode problems for each pattern A practice problem table for further study How Data is Stored in Computer Memory Before we talk about arrays, we need to understand the environment arrays live in \u0026ndash; computer memory.\nWhen your program runs, it gets access to RAM (Random Access Memory). You can think of RAM as a very long row of small storage boxes. Each box:\nHolds exactly one byte (8 bits) of data Has a unique address \u0026ndash; a number that identifies its position When you store a value, the computer places it at some address in memory. When you want to read it back, the computer goes directly to that address. This direct access by address is extremely fast \u0026ndash; it is O(1) regardless of how large the memory is or where in memory the value lives.\nHere is the key insight: if you know the starting address and the size of each element, you can calculate the address of any element instantly using simple arithmetic.\naddress of element at index i = starting_address + (i * size_of_each_element) This formula is why arrays are so powerful. And it only works because of one critical requirement: the elements must be stored in a contiguous block of memory \u0026ndash; one right after the other, with no gaps.\nWhat is an Array An array is a collection of elements stored in contiguous memory locations, where every element is of the same size.\nConsider an array of integers: [10, 20, 30, 40, 50]\nIf each integer takes 4 bytes and the array starts at memory address 1000, here is how it looks in memory:\nIndex Value Memory Address 0 10 1000 1 20 1004 2 30 1008 3 40 1012 4 50 1016 To access arr[3], the computer calculates:\naddress = 1000 + (3 * 4) = 1012 It goes directly to address 1012 and reads the value. This is one arithmetic operation and one memory read \u0026ndash; O(1) regardless of array size. It does not scan from the beginning. It jumps straight there.\nThis is the fundamental reason array access by index is O(1), and it is one of the most important ideas in all of computer science.\nArrays in Python Python gives you three ways to work with arrays, and they are quite different from each other.\nPython Lists Python\u0026rsquo;s built-in list is what you will use for almost everything in this guide. It is a dynamic array \u0026ndash; it can grow and shrink as needed.\n# Creating a list nums = [10, 20, 30, 40, 50] # Accessing by index -- O(1) print(nums[0]) # 10 print(nums[2]) # 30 print(nums[-1]) # 50 (negative index counts from the end) # Modifying an element -- O(1) nums[1] = 99 print(nums) # [10, 99, 30, 40, 50] # Length -- O(1) print(len(nums)) # 5 Python lists are incredibly flexible but they come with an important caveat that we will cover shortly \u0026ndash; they do not store values directly. They store references to objects.\nThe array Module Python\u0026rsquo;s built-in array module gives you a true value array \u0026ndash; it stores actual values of a fixed type, not references. This uses less memory than a list when you have large amounts of numeric data.\nimport array # Create an array of signed integers (\u0026#39;i\u0026#39; is the type code for int) nums = array.array(\u0026#39;i\u0026#39;, [10, 20, 30, 40, 50]) print(nums[0]) # 10 print(nums[2]) # 30 # Common type codes # \u0026#39;i\u0026#39; -- signed int (4 bytes) # \u0026#39;f\u0026#39; -- float (4 bytes) # \u0026#39;d\u0026#39; -- double / float64 (8 bytes) # \u0026#39;b\u0026#39; -- signed char / int8 (1 byte) In DSA problems you will almost never need the array module \u0026ndash; Python lists are used universally. But it is worth knowing it exists and why it is different.\nNumPy Arrays NumPy is a third-party library built for numerical computing. NumPy arrays are true value arrays and support fast vectorised operations on large datasets.\nimport numpy as np # Create a NumPy array nums = np.array([10, 20, 30, 40, 50]) print(nums[0]) # 10 print(nums.dtype) # int64 -- all elements are the same type # Vectorised operations -- applied to every element at once print(nums * 2) # [20 40 60 80 100] print(nums + 10) # [20 30 40 50 60] # 2D arrays (matrices) matrix = np.array([[1, 2, 3], [4, 5, 6], [7, 8, 9]]) print(matrix[1][2]) # 6 -- row 1, column 2 For DSA problems, NumPy is rarely needed. It matters more in data science and machine learning. We mention it here for completeness.\nComparison of the Three Feature Python list array module NumPy array Stores References to objects Actual values Actual values Element types Mixed (any type) Fixed type Fixed type Memory usage Higher Lower Lower Speed (numeric ops) Slower Faster Fastest Flexibility Highest Medium Medium Use in DSA Always Rarely Rarely Referential Arrays vs Value Arrays This is one of the most important things to understand about Python lists, and it is something that surprises many beginners.\nIn languages like C, an array of integers stores the actual integer values side by side in memory. Python lists do not work this way.\nA Python list stores references (memory addresses pointing to objects), not the values themselves. Each element in the list is a pointer to a Python object somewhere else in memory.\nnums = [10, 20, 30] # What Python actually stores in the list: # [ pointer_to_10, pointer_to_20, pointer_to_30 ] # Each pointer is 8 bytes on a 64-bit system # The actual integer objects live elsewhere in memory This has several consequences:\n1. A Python list can hold mixed types because it stores references, not values\nmixed = [42, \u0026#34;hello\u0026#34;, 3.14, True, [1, 2, 3]] # All valid -- each element is just a pointer to a different type of object 2. Assignment copies the reference, not the value\na = [1, 2, 3] b = a # b points to the SAME list as a, not a copy b[0] = 99 print(a) # [99, 2, 3] -- a is also changed! print(b) # [99, 2, 3] # To make a true copy, use slicing or list() c = a[:] # or list(a) or a.copy() c[0] = 0 print(a) # [99, 2, 3] -- a is unchanged print(c) # [0, 2, 3] 3. Equality vs identity\na = [1, 2, 3] b = [1, 2, 3] c = a print(a == b) # True -- same values print(a is b) # False -- different objects in memory print(a is c) # True -- same object in memory Understanding this distinction prevents many subtle bugs, especially when you pass lists to functions or try to make copies.\nDynamic Arrays \u0026ndash; How Python Lists Grow A regular array has a fixed size \u0026ndash; you declare it with a certain capacity and it cannot grow. Python lists, however, can grow dynamically. How does this work?\nUnder the hood, Python allocates more memory than the list currently needs. When the list fills up, Python allocates a new, larger block of memory, copies everything over, and releases the old block.\nHere is the key detail: Python does not just add one slot at a time. It roughly doubles the capacity each time. This is called the doubling strategy.\nLet us trace through what happens as you append elements:\nAppend call List size Allocated capacity Action Start 0 0 \u0026ndash; append(1) 1 4 Allocate space for 4 append(2) 2 4 No resize needed append(3) 3 4 No resize needed append(4) 4 4 No resize needed append(5) 5 8 Resize: copy 4 elements to new block of 8 append(6) 6 8 No resize needed append(7) 7 8 No resize needed append(8) 8 8 No resize needed append(9) 9 16 Resize: copy 8 elements to new block of 16 Most appends are O(1). Occasionally one append triggers a resize, which takes O(n) to copy everything. But because resizes happen less and less frequently as the list grows, the amortised cost per append is still O(1).\nAmortised means: averaged over a long sequence of operations. A single append might take O(n), but if you do n appends total, the total work is O(n), so each append costs O(1) on average.\nimport sys # You can see Python allocating extra capacity nums = [] for i in range(10): nums.append(i) # sys.getsizeof gives the memory size of the list object in bytes print(f\u0026#34;Length: {len(nums)}, Memory: {sys.getsizeof(nums)} bytes\u0026#34;) # Output (approximate, varies by Python version): # Length: 1, Memory: 88 bytes # Length: 2, Memory: 88 bytes # Length: 3, Memory: 88 bytes # Length: 4, Memory: 88 bytes # Length: 5, Memory: 120 bytes \u0026lt;-- resize happened # Length: 6, Memory: 120 bytes # ... Time and Space Complexity of Array Operations Now that you understand how arrays work internally, the complexity of each operation should make intuitive sense.\nOperation Code Time Complexity Why Access by index nums[i] O(1) Direct address calculation Update by index nums[i] = x O(1) Direct address calculation Append to end nums.append(x) O(1) amortised Occasional O(n) resize Pop from end nums.pop() O(1) No shifting needed Insert at index i nums.insert(i, x) O(n) Must shift all elements after i Delete at index i del nums[i] O(n) Must shift all elements after i Search for value x in nums O(n) Must scan the entire list Get length len(nums) O(1) Stored as a property Slice [i:j] nums[i:j] O(k) k = j - i, copies k elements Concatenate a + b O(n + m) Copies both lists Sort nums.sort() O(n log n) Timsort Reverse nums.reverse() O(n) Visits every element once Copy nums.copy() O(n) Copies every element The most important thing to remember: inserting or deleting at the beginning or middle of a list is O(n) because every element after that position must shift. This is the main weakness of arrays compared to linked lists.\nTwo-Dimensional Arrays A 2D array is an array of arrays. It is used to represent grids, matrices, and boards.\n# Creating a 2D list (3 rows, 4 columns) matrix = [ [1, 2, 3, 4], [5, 6, 7, 8], [9, 10, 11, 12] ] # Accessing an element -- O(1) print(matrix[1][2]) # 7 -- row 1, column 2 print(matrix[0][0]) # 1 -- top left print(matrix[2][3]) # 12 -- bottom right # Iterating over a 2D array for row in matrix: for val in row: print(val, end=\u0026#34; \u0026#34;) print() # Creating an n x m grid filled with zeros n, m = 3, 4 grid = [[0] * m for _ in range(n)] print(grid) # [[0, 0, 0, 0], # [0, 0, 0, 0], # [0, 0, 0, 0]] # WARNING -- do NOT do this bad_grid = [[0] * m] * n # This creates n references to the SAME row # Modifying one row modifies all of them bad_grid[0][0] = 99 print(bad_grid) # [[99, 0, 0, 0], # [99, 0, 0, 0], \u0026lt;-- all rows changed! # [99, 0, 0, 0]] Patterns and Techniques Now we get to the techniques that make arrays so powerful in problem solving. Each of these patterns comes directly from the properties of arrays \u0026ndash; indexed access, contiguity, and ordering.\nPattern 1 \u0026ndash; Two Pointers The core idea: Use two index variables that move through the array, usually from opposite ends toward the middle or in the same direction at different speeds. This turns many O(n^2) problems into O(n).\nWhen to use it:\nThe array is sorted (or you can sort it) You are looking for a pair of elements that satisfy some condition You need to remove or partition elements in place The template:\nleft, right = 0, len(nums) - 1 while left \u0026lt; right: # use nums[left] and nums[right] # move left forward, right backward, or both # based on the condition of the problem if some_condition: left += 1 elif some_other_condition: right -= 1 else: left += 1 right -= 1 Solved Problem \u0026ndash; Valid Palindrome LeetCode 125 \u0026ndash; Valid Palindrome\nProblem: A phrase is a palindrome if, after converting all uppercase letters to lowercase and removing all non-alphanumeric characters, it reads the same forward and backward. Given a string s, return True if it is a palindrome, otherwise return False.\nInput: s = \u0026#34;A man, a plan, a canal: Panama\u0026#34; Output: True Input: s = \u0026#34;race a car\u0026#34; Output: False Thinking through it:\nA brute force approach would be to clean the string first, then check if it equals its reverse. That works in O(n) time and O(n) space (for the cleaned string).\nBut we can do better on space. Instead of building a new string, use two pointers \u0026ndash; one at the start, one at the end. Skip non-alphanumeric characters and compare characters as you go. If they ever differ, it is not a palindrome.\ndef is_palindrome(s: str) -\u0026gt; bool: \u0026#34;\u0026#34;\u0026#34; Uses two pointers starting from both ends, moving inward. Skip any character that is not alphanumeric. Compare the characters at both pointers -- if they ever differ, return False. Time complexity: O(n) -- each character is visited at most once Space complexity: O(1) -- no extra data structures, just two index variables \u0026#34;\u0026#34;\u0026#34; left, right = 0, len(s) - 1 while left \u0026lt; right: # Move left pointer forward past non-alphanumeric characters while left \u0026lt; right and not s[left].isalnum(): left += 1 # Move right pointer backward past non-alphanumeric characters while left \u0026lt; right and not s[right].isalnum(): right -= 1 # Compare characters (case insensitive) if s[left].lower() != s[right].lower(): return False # mismatch -- not a palindrome # Both matched, move both pointers inward left += 1 right -= 1 return True # all characters matched # Test cases print(is_palindrome(\u0026#34;A man, a plan, a canal: Panama\u0026#34;)) # True print(is_palindrome(\u0026#34;race a car\u0026#34;)) # False print(is_palindrome(\u0026#34; \u0026#34;)) # True (empty after cleaning) Solved Problem \u0026ndash; Container With Most Water LeetCode 11 \u0026ndash; Container With Most Water\nProblem: You are given an integer array height of length n. There are n vertical lines drawn such that the two endpoints of the ith line are (i, 0) and (i, height[i]). Find two lines that together with the x-axis form a container that holds the most water. Return the maximum amount of water a container can store.\nInput: height = [1, 8, 6, 2, 5, 4, 8, 3, 7] Output: 49 Thinking through it:\nThe brute force is to check every pair of lines \u0026ndash; O(n^2). Can we do better?\nThe water between lines at index left and right is:\nwater = min(height[left], height[right]) * (right - left) Start with the widest possible container (left = 0, right = n-1). To try to find more water, we need to increase the height. Moving the pointer at the taller side inward cannot help \u0026ndash; the height is still limited by the shorter side and the width decreases. So we always move the pointer at the shorter side inward, hoping to find a taller line.\ndef max_area(height: list) -\u0026gt; int: \u0026#34;\u0026#34;\u0026#34; Start with the widest container. At each step, move the pointer at the shorter side inward -- this is the only move that could possibly find more water. Time complexity: O(n) -- each element is visited at most once Space complexity: O(1) -- only two pointers and a max variable \u0026#34;\u0026#34;\u0026#34; left, right = 0, len(height) - 1 max_water = 0 while left \u0026lt; right: # Calculate water in current container width = right - left h = min(height[left], height[right]) current_water = width * h # Update maximum max_water = max(max_water, current_water) # Move the pointer at the shorter side inward if height[left] \u0026lt; height[right]: left += 1 else: right -= 1 return max_water # Test print(max_area([1, 8, 6, 2, 5, 4, 8, 3, 7])) # 49 print(max_area([1, 1])) # 1 Pattern 2 \u0026ndash; Sliding Window The core idea: Maintain a window (a contiguous subarray) that slides across the array. Instead of recomputing the result from scratch for each window position, you add the new element entering the window and remove the element leaving it. This turns many O(n^2) subarray problems into O(n).\nWhen to use it:\nYou need to find a subarray or substring that satisfies some condition The condition involves something about a contiguous range (sum, count, unique elements) The problem involves \u0026ldquo;maximum\u0026rdquo;, \u0026ldquo;minimum\u0026rdquo;, or \u0026ldquo;longest\u0026rdquo; subarray with a constraint Two types of sliding window:\nFixed size window \u0026ndash; the window size k is given:\n# Template: fixed window of size k def fixed_window(nums, k): # Build the first window window_sum = sum(nums[:k]) result = window_sum # Slide the window: add right element, remove left element for i in range(k, len(nums)): window_sum += nums[i] # add element entering window window_sum -= nums[i - k] # remove element leaving window result = max(result, window_sum) return result Variable size window \u0026ndash; find the longest/shortest window satisfying a condition:\n# Template: variable window def variable_window(nums): left = 0 result = 0 for right in range(len(nums)): # Expand: include nums[right] in the window while # window condition is violated: # Shrink: remove nums[left] from the window left += 1 # Window is now valid -- update result result = max(result, right - left + 1) return result Solved Problem \u0026ndash; Best Time to Buy and Sell Stock LeetCode 121 \u0026ndash; Best Time to Buy and Sell Stock\nProblem: You are given an array prices where prices[i] is the price of a stock on day i. You want to buy on one day and sell on a later day to maximise profit. Return the maximum profit. If no profit is possible, return 0.\nInput: prices = [7, 1, 5, 3, 6, 4] Output: 5 (buy on day 1 at price 1, sell on day 4 at price 6) Input: prices = [7, 6, 4, 3, 1] Output: 0 (prices only go down, no profit possible) Thinking through it:\nThe brute force checks every pair of days \u0026ndash; O(n^2). The key observation is: the best profit on any given day is price[today] - minimum_price_seen_so_far. So we only need one pass, tracking the minimum price seen so far and the maximum profit seen so far.\nThis is a sliding window in spirit \u0026ndash; the window is from the minimum price day to the current day.\ndef max_profit(prices: list) -\u0026gt; int: \u0026#34;\u0026#34;\u0026#34; Track the minimum price seen so far. At each day, compute the profit if we sold today. Update the maximum profit if this is better. Time complexity: O(n) -- single pass through prices Space complexity: O(1) -- only two variables \u0026#34;\u0026#34;\u0026#34; min_price = float(\u0026#39;inf\u0026#39;) # smallest price seen so far max_profit = 0 # best profit seen so far for price in prices: if price \u0026lt; min_price: min_price = price # found a better day to buy else: profit = price - min_price # profit if we sell today max_profit = max(max_profit, profit) return max_profit # Test print(max_profit([7, 1, 5, 3, 6, 4])) # 5 print(max_profit([7, 6, 4, 3, 1])) # 0 print(max_profit([2, 4, 1])) # 2 Solved Problem \u0026ndash; Maximum Average Subarray I LeetCode 643 \u0026ndash; Maximum Average Subarray I\nProblem: You are given an integer array nums consisting of n elements, and an integer k. Find a contiguous subarray of length exactly k that has the maximum average value and return this value.\nInput: nums = [1, 12, -5, -6, 50, 3], k = 4 Output: 12.75 (subarray [12, -5, -6, 50], average = 51/4 = 12.75) Thinking through it:\nThis is a fixed-size sliding window. Build the first window of size k, then slide it across, adding the new element and removing the old one. Track the maximum sum seen.\ndef find_max_average(nums: list, k: int) -\u0026gt; float: \u0026#34;\u0026#34;\u0026#34; Fixed sliding window of size k. Compute the sum of the first window, then slide it across by adding the incoming element and removing the outgoing one. Time complexity: O(n) -- one pass after the initial window Space complexity: O(1) -- only store the current and max sum \u0026#34;\u0026#34;\u0026#34; # Build the first window window_sum = sum(nums[:k]) # O(k) but k \u0026lt;= n so overall still O(n) max_sum = window_sum # Slide the window from position k to end for i in range(k, len(nums)): window_sum += nums[i] # element entering the window window_sum -= nums[i - k] # element leaving the window max_sum = max(max_sum, window_sum) return max_sum / k # Test print(find_max_average([1, 12, -5, -6, 50, 3], 4)) # 12.75 print(find_max_average([5], 1)) # 5.0 Pattern 3 \u0026ndash; Prefix Sum The core idea: Precompute a running total of the array so that the sum of any subarray nums[i:j] can be computed in O(1) instead of O(n).\nThe formula:\nprefix[0] = 0 prefix[i] = nums[0] + nums[1] + ... + nums[i-1] sum of nums[left..right] = prefix[right + 1] - prefix[left] When to use it:\nMultiple queries asking for the sum of a subarray Problems involving subarray sums equal to a target Any problem where you need to quickly compute the sum of a range The template:\ndef build_prefix(nums): n = len(nums) prefix = [0] * (n + 1) # prefix[0] = 0 by convention for i in range(n): prefix[i + 1] = prefix[i] + nums[i] return prefix # Query: sum of nums[left..right] inclusive def range_sum(prefix, left, right): return prefix[right + 1] - prefix[left] Solved Problem \u0026ndash; Range Sum Query - Immutable LeetCode 303 \u0026ndash; Range Sum Query - Immutable\nProblem: Given an integer array nums, handle multiple queries of the form: calculate the sum of elements between indices left and right inclusive.\nInput: nums = [-2, 0, 3, -5, 2, -1] sumRange(0, 2) --\u0026gt; 1 (-2 + 0 + 3) sumRange(2, 5) --\u0026gt; -1 (3 + -5 + 2 + -1) sumRange(0, 5) --\u0026gt; -3 (-2 + 0 + 3 + -5 + 2 + -1) Thinking through it:\nIf we compute the sum naively for each query, each query costs O(n) and with q queries the total is O(n * q). With a prefix sum array, each query costs O(1) after an O(n) preprocessing step.\nclass NumArray: \u0026#34;\u0026#34;\u0026#34; Precompute prefix sums once at initialisation. Answer each range query in O(1). Time complexity: __init__: O(n) -- build the prefix sum array sumRange: O(1) -- subtract two prefix values Space complexity: O(n) -- the prefix sum array \u0026#34;\u0026#34;\u0026#34; def __init__(self, nums: list): n = len(nums) # prefix[i] = sum of nums[0..i-1] # prefix[0] = 0 (sum of empty prefix) self.prefix = [0] * (n + 1) for i in range(n): self.prefix[i + 1] = self.prefix[i] + nums[i] def sumRange(self, left: int, right: int) -\u0026gt; int: # sum of nums[left..right] # = (sum of nums[0..right]) - (sum of nums[0..left-1]) # = prefix[right + 1] - prefix[left] return self.prefix[right + 1] - self.prefix[left] # Test obj = NumArray([-2, 0, 3, -5, 2, -1]) print(obj.sumRange(0, 2)) # 1 print(obj.sumRange(2, 5)) # -1 print(obj.sumRange(0, 5)) # -3 Solved Problem \u0026ndash; Subarray Sum Equals K LeetCode 560 \u0026ndash; Subarray Sum Equals K\nProblem: Given an array of integers nums and an integer k, return the total number of subarrays whose sum equals k.\nInput: nums = [1, 1, 1], k = 2 Output: 2 Input: nums = [1, 2, 3], k = 3 Output: 2 Thinking through it:\nThe brute force is O(n^2): try every subarray. Can we do better?\nThe sum of subarray nums[i..j] equals prefix[j+1] - prefix[i]. We want this to equal k, which means we want prefix[i] = prefix[j+1] - k.\nSo as we compute the prefix sum, we check: have we seen a prefix sum equal to current_prefix - k before? If yes, we have found subarrays that sum to k. We use a hash map to track prefix sums seen so far.\ndef subarray_sum(nums: list, k: int) -\u0026gt; int: \u0026#34;\u0026#34;\u0026#34; As we build the prefix sum, we ask at each position j: how many previous prefix sums equal (current_prefix - k)? Each such prefix sum corresponds to a valid subarray ending at j. Time complexity: O(n) -- single pass Space complexity: O(n) -- hash map stores up to n prefix sums \u0026#34;\u0026#34;\u0026#34; count = 0 prefix_sum = 0 # Maps prefix_sum value to how many times we have seen it seen = {0: 1} # prefix sum of 0 exists once before we start for num in nums: prefix_sum += num # update running prefix sum # How many previous positions had prefix sum = prefix_sum - k? # Each one gives a subarray ending here that sums to k complement = prefix_sum - k count += seen.get(complement, 0) # O(1) lookup # Record this prefix sum seen[prefix_sum] = seen.get(prefix_sum, 0) + 1 return count # Test print(subarray_sum([1, 1, 1], 2)) # 2 print(subarray_sum([1, 2, 3], 3)) # 2 print(subarray_sum([1, -1, 1], 0)) # 3 Summary What We Covered Topic Key Point Computer memory RAM is a sequence of addressed bytes. Knowing the start address and element size lets you reach any index in O(1) Arrays Contiguous memory block. Fixed element size. O(1) access by index Python lists Dynamic array. Stores references, not values. Resizes by doubling array module True value array. Fixed type. Less memory than a list NumPy arrays High-performance value array. Used in data science, rarely in DSA Referential arrays Python lists store pointers. Assignment copies the pointer, not the value Dynamic arrays Extra capacity allocated. Amortised O(1) append. Occasional O(n) resize 2D arrays Array of arrays. Create with list comprehension, not [[0]*m]*n Operation Complexity at a Glance Operation Complexity Access / update by index O(1) Append to end O(1) amortised Pop from end O(1) Insert at position i O(n) Delete at position i O(n) Search (unsorted) O(n) Sort O(n log n) Slice [i:j] O(k) where k = j - i Patterns Introduced Pattern Core idea Time complexity gain Use when Two Pointers Two indices moving through the array O(n^2) to O(n) Pair problems, sorted arrays, in-place operations Sliding Window Maintain a window, add right, remove left O(n^2) to O(n) Subarray/substring with a constraint Prefix Sum Precompute cumulative sums O(n) per query to O(1) Range sum queries, subarray sum problems Key Takeaways Array access is O(1) because of contiguous memory and direct address calculation. This is the most fundamental property of arrays. Python lists are dynamic and store references, not values. Assigning a list to another variable does not copy it. Insert and delete in the middle of an array are O(n) because elements must shift. This is the main weakness of arrays. The doubling strategy gives Python lists amortised O(1) append. Two Pointers, Sliding Window, and Prefix Sum are patterns that exploit the indexed, ordered nature of arrays to solve problems efficiently. Always create 2D arrays with [[0]*m for _ in range(n)], not [[0]*m]*n. Practice Problems Work through these on your own. For each problem, think about which pattern applies before writing any code.\nProblem Link Difficulty Pattern Two Sum LC 1 Easy Hash Map Remove Duplicates from Sorted Array LC 26 Easy Two Pointers Move Zeroes LC 283 Easy Two Pointers Squares of a Sorted Array LC 977 Easy Two Pointers Maximum Subarray LC 53 Medium Sliding Window / Kadane\u0026rsquo;s Minimum Size Subarray Sum LC 209 Medium Sliding Window 3Sum LC 15 Medium Two Pointers Product of Array Except Self LC 238 Medium Prefix Sum Contiguous Array LC 525 Medium Prefix Sum + Hash Map Trapping Rain Water LC 42 Hard Two Pointers / Prefix Sum Next Steps Next blog: [Strings] \u0026ndash; strings are closely related to arrays and share several of the same patterns Reference: Codeverra DSA Sheet ","permalink":"https://blogs.codeverra.com/blog/dsa-04-arrays/","summary":"\u003ch1 id=\"arrays\"\u003eArrays\u003c/h1\u003e\n\u003chr\u003e\n\u003ch2 id=\"table-of-contents\"\u003eTable of Contents\u003c/h2\u003e\n\u003cul\u003e\n\u003cli\u003e\u003ca href=\"#what-you-will-learn-in-this-blog\"\u003eWhat You Will Learn in This Blog\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#how-data-is-stored-in-computer-memory\"\u003eHow Data is Stored in Computer Memory\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#what-is-an-array\"\u003eWhat is an Array\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#arrays-in-python\"\u003eArrays in Python\u003c/a\u003e\n\u003cul\u003e\n\u003cli\u003e\u003ca href=\"#python-lists\"\u003ePython Lists\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#the-array-module\"\u003eThe array Module\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#numpy-arrays\"\u003eNumPy Arrays\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#comparison-of-the-three\"\u003eComparison of the Three\u003c/a\u003e\u003c/li\u003e\n\u003c/ul\u003e\n\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#referential-arrays-vs-value-arrays\"\u003eReferential Arrays vs Value Arrays\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#dynamic-arrays----how-python-lists-grow\"\u003eDynamic Arrays \u0026ndash; How Python Lists Grow\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#time-and-space-complexity-of-array-operations\"\u003eTime and Space Complexity of Array Operations\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#two-dimensional-arrays\"\u003eTwo-Dimensional Arrays\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#patterns-and-techniques\"\u003ePatterns and Techniques\u003c/a\u003e\n\u003cul\u003e\n\u003cli\u003e\u003ca href=\"#pattern-1----two-pointers\"\u003ePattern 1 \u0026ndash; Two Pointers\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#pattern-2----sliding-window\"\u003ePattern 2 \u0026ndash; Sliding Window\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#pattern-3----prefix-sum\"\u003ePattern 3 \u0026ndash; Prefix Sum\u003c/a\u003e\u003c/li\u003e\n\u003c/ul\u003e\n\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#summary\"\u003eSummary\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#key-takeaways\"\u003eKey Takeaways\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#practice-problems\"\u003ePractice Problems\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#next-steps\"\u003eNext Steps\u003c/a\u003e\u003c/li\u003e\n\u003c/ul\u003e\n\u003chr\u003e\n\u003ch2 id=\"what-you-will-learn-in-this-blog\"\u003eWhat You Will Learn in This Blog\u003c/h2\u003e\n\u003cul\u003e\n\u003cli\u003eHow data is stored in computer memory and why it matters\u003c/li\u003e\n\u003cli\u003eWhat an array is and how it uses contiguous memory\u003c/li\u003e\n\u003cli\u003eHow Python represents arrays \u0026ndash; lists, the array module, and NumPy\u003c/li\u003e\n\u003cli\u003eThe difference between referential arrays and value arrays\u003c/li\u003e\n\u003cli\u003eHow dynamic arrays work and how Python lists resize themselves\u003c/li\u003e\n\u003cli\u003eTime and space complexity of every common array operation\u003c/li\u003e\n\u003cli\u003eThree core patterns that come from arrays: Two Pointers, Sliding Window, and Prefix Sum\u003c/li\u003e\n\u003cli\u003eSolved LeetCode problems for each pattern\u003c/li\u003e\n\u003cli\u003eA practice problem table for further study\u003c/li\u003e\n\u003c/ul\u003e\n\u003chr\u003e\n\u003ch2 id=\"how-data-is-stored-in-computer-memory\"\u003eHow Data is Stored in Computer Memory\u003c/h2\u003e\n\u003cp\u003eBefore we talk about arrays, we need to understand the environment arrays live in \u0026ndash; computer memory.\u003c/p\u003e","tags":["dsa"],"title":"Arrays in Python  -  Complete Guide"},{"content":"🔰 Pattern 2: Binary Search Table of Contents What is Binary Search? The Core Idea - Why Halving Works The Three Binary Search Templates Binary Search on Answer - The Hidden Pattern Common Pitfalls Problem Set Problem 1: Binary Search Problem 2: Search Insert Position Problem 3: Find First and Last Position of Element in Sorted Array Problem 4: Search a 2D Matrix Problem 5: Search in Rotated Sorted Array Problem 6: Find Minimum in Rotated Sorted Array Problem 7: Koko Eating Bananas Problem 8: Capacity to Ship Packages Within D Days Problem 9: Split Array Largest Sum Problem 10: Median of Two Sorted Arrays Key Takeaways \u0026amp; Summary What is Binary Search? In the Two Pointers pattern, we used two indices to narrow down a search space. Binary search takes that idea to its most powerful form: instead of narrowing one step at a time, we cut the search space in half at every step.\nLet\u0026rsquo;s see the difference with a concrete example.\nProblem: Find the number 7 in a sorted array of 1 million elements.\nLinear search (one by one): Check index 0, then 1, then 2\u0026hellip; up to 1,000,000 in the worst case.\nBinary search (halving):\nStep 1: Check middle of 1,000,000 elements → eliminate 500,000 Step 2: Check middle of 500,000 elements → eliminate 250,000 Step 3: Check middle of 250,000 elements → eliminate 125,000 ... Step 20: Check middle of 1 element → found (or not) 20 steps to search 1 million elements. That\u0026rsquo;s the power of halving - each step eliminates half the remaining possibilities.\nBinary Search is a technique where you repeatedly divide the search space in half, using a condition to decide which half to keep, until you find your target or narrow down to a single candidate.\nConnection to Two Pointers Binary search uses two pointers (low and high) that converge, just like opposite-direction two pointers. The difference:\nTwo Pointers Binary Search Movement One step at a time Jump to the middle Elimination One element per step Half the space per step Time O(n) O(log n) Requirement Usually sorted Must have a condition that splits the space The Core Idea - Why Halving Works Binary search works whenever you have a monotonic condition - a property that is False for one portion of the search space and True for the rest (or vice versa), with a clean boundary between them.\nIndex: 0 1 2 3 4 5 6 7 8 9 Condition: F F F F T T T T T T ^ boundary (first True) At any point, if we check the middle and find:\nTrue → the boundary is at or to the left → search left half False → the boundary is to the right → search right half Each check eliminates half the candidates. After log₂(n) checks, we\u0026rsquo;ve found the boundary.\nThis is more general than \u0026ldquo;finding a number in a sorted array.\u0026rdquo; The sorted-array case is just one instance where the condition is arr[mid] \u0026gt;= target. Binary search applies anytime you can define such a condition:\n\u0026ldquo;Is this speed fast enough to finish the task?\u0026rdquo; (Binary Search on Answer) \u0026ldquo;Is this capacity sufficient to ship all packages?\u0026rdquo; (Binary Search on Answer) \u0026ldquo;Is arr[mid] in the left or right half of a rotated array?\u0026rdquo; (Rotated Array Search) The Three Binary Search Templates Different problems need slightly different templates. The variations come down to: what exactly are you looking for?\nTemplate 1: Find Exact Target def binary_search_exact(arr, target): \u0026#34;\u0026#34;\u0026#34; Find the index of \u0026#39;target\u0026#39; in a sorted array. Returns -1 if not found. Loop invariant: if target exists, it\u0026#39;s in arr[low..high]. We stop when low \u0026gt; high (search space is empty → not found). Time Complexity: O(log n) Space Complexity: O(1) \u0026#34;\u0026#34;\u0026#34; low, high = 0, len(arr) - 1 while low \u0026lt;= high: mid = low + (high - low) // 2 # Avoids integer overflow if arr[mid] == target: return mid # Found it elif arr[mid] \u0026lt; target: low = mid + 1 # Target is in the right half else: high = mid - 1 # Target is in the left half return -1 # Not found When to use: You need the exact position of a specific value.\nKey detail: low \u0026lt;= high (inclusive) - the loop runs while the search space has at least one element.\nTemplate 2: Find First/Left Boundary def binary_search_left(arr, target): \u0026#34;\u0026#34;\u0026#34; Find the FIRST (leftmost) position where arr[pos] \u0026gt;= target. Also known as: lower_bound, insertion point. This is the most versatile template. It finds the boundary where the condition arr[mid] \u0026gt;= target flips from False to True. Returns: index of the first element \u0026gt;= target. If all elements \u0026lt; target, returns len(arr). Time Complexity: O(log n) Space Complexity: O(1) \u0026#34;\u0026#34;\u0026#34; low, high = 0, len(arr) # Note: high = len(arr), not len(arr) - 1 while low \u0026lt; high: # Note: strict \u0026lt;, not \u0026lt;= mid = low + (high - low) // 2 if arr[mid] \u0026lt; target: low = mid + 1 # mid is too small, search right else: high = mid # mid could be the answer, search left (including mid) return low # low == high == first position where arr[pos] \u0026gt;= target When to use: Find the first occurrence, insertion point, or leftmost boundary.\nKey details:\nhigh = len(arr) - allows returning past-the-end when all elements are smaller. low \u0026lt; high (strict) - loop ends when low == high, which is our answer. high = mid (not mid - 1) - because mid itself might be the answer. Template 3: Find Last/Right Boundary def binary_search_right(arr, target): \u0026#34;\u0026#34;\u0026#34; Find the LAST (rightmost) position where arr[pos] \u0026lt;= target. Also known as: upper_bound - 1. Returns: index of the last element \u0026lt;= target. If all elements \u0026gt; target, returns -1. Time Complexity: O(log n) Space Complexity: O(1) \u0026#34;\u0026#34;\u0026#34; low, high = 0, len(arr) - 1 result = -1 while low \u0026lt;= high: mid = low + (high - low) // 2 if arr[mid] \u0026lt;= target: result = mid # mid is a valid candidate, but there might be a later one low = mid + 1 # Search right for a later occurrence else: high = mid - 1 # mid is too large, search left return result When to use: Find the last occurrence, rightmost boundary, or \u0026ldquo;latest valid\u0026rdquo; position.\nWhich Template to Use? What are you looking for? \u0026#34;Find target\u0026#34; → Template 1 (exact match) \u0026#34;First position where...\u0026#34; → Template 2 (left boundary) \u0026#34;Last position where...\u0026#34; → Template 3 (right boundary) \u0026#34;Where would target go?\u0026#34; → Template 2 (insertion point) \u0026#34;First and last position\u0026#34; → Template 2 + Template 3 combined A Visual Comparison arr = [1, 3, 3, 3, 5, 7, 9] Template 1: find 3 → returns index 2 (any of the 3s - typically middle) Template 2: find first ≥ 3 → returns index 1 (leftmost 3) Template 3: find last ≤ 3 → returns index 3 (rightmost 3) Template 2: find first ≥ 4 → returns index 4 (first element after all 3s → insertion point for 4) Binary Search on Answer - The Hidden Pattern This is where binary search gets really interesting. Instead of searching for a value in an array, you search for the answer itself across a range of possible values.\nThe setup:\nYou\u0026rsquo;re looking for the minimum (or maximum) value that satisfies some condition. The condition is monotonic: once it becomes True, it stays True for all larger (or smaller) values. You can write a function is_feasible(x) that checks if a given value x works. The structure:\ndef binary_search_on_answer(problem_input): \u0026#34;\u0026#34;\u0026#34; Search for the minimum valid answer in the range [lo, hi]. \u0026#34;\u0026#34;\u0026#34; lo, hi = minimum_possible_answer, maximum_possible_answer while lo \u0026lt; hi: mid = lo + (hi - lo) // 2 if is_feasible(mid): hi = mid # mid works, but maybe something smaller also works else: lo = mid + 1 # mid doesn\u0026#39;t work, need something bigger return lo # Smallest value that satisfies is_feasible Example: \u0026ldquo;Koko has piles of bananas. She can eat at speed k bananas/hour. What\u0026rsquo;s the minimum k to finish all bananas in h hours?\u0026rdquo;\nSearch space: k can range from 1 to max(piles). Condition: At speed k, can she finish in ≤ h hours? (Monotonic - if speed 5 works, speed 6 definitely works.) Binary search on answer: Find the smallest k where the condition is True. This pattern shows up in a huge number of problems disguised as optimization questions. The trick is recognizing that the answer has a monotonic feasibility condition.\nCommon Pitfalls 1. Off-by-one errors The #1 source of bugs. The differences between templates matter:\nTemplate 1 Template 2 Template 3 high init len(arr) - 1 len(arr) len(arr) - 1 Loop condition low \u0026lt;= high low \u0026lt; high low \u0026lt;= high On match return mid high = mid result = mid; low = mid + 1 2. Infinite loops If low = mid (without + 1) when low == mid, the loop never progresses. This happens when:\nYou use low \u0026lt; high with low = mid (should be low = mid + 1) You forget that integer division rounds down: (0 + 1) // 2 = 0 Rule of thumb: If low might equal mid, always use low = mid + 1.\n3. Integer overflow for mid In languages with fixed-size integers (C++, Java), (low + high) can overflow. Always use:\nmid = low + (high - low) // 2 # Safe # NOT: mid = (low + high) // 2 # Can overflow in some languages Python handles big integers natively, so this isn\u0026rsquo;t an issue in Python, but it\u0026rsquo;s good practice.\n4. Deciding which half to search When stuck, ask: \u0026ldquo;If arr[mid] has this value, can the answer be at mid, to the left, or to the right?\u0026rdquo;\nIf the answer could be at mid → don\u0026rsquo;t exclude it (high = mid, not high = mid - 1) If the answer cannot be at mid → exclude it (low = mid + 1 or high = mid - 1) Problem Set Difficulty Progression # Problem Difficulty Key Concept 1 Binary Search Easy Basic exact match 2 Search Insert Position Easy Left boundary / insertion point 3 Find First and Last Position Medium Left + right boundary combined 4 Search a 2D Matrix Medium Treating 2D as 1D 5 Search in Rotated Sorted Array Medium Modified binary search with rotation 6 Find Minimum in Rotated Sorted Array Medium Boundary in rotated array 7 Koko Eating Bananas Medium Binary search on answer 8 Capacity to Ship Packages Medium Binary search on answer 9 Split Array Largest Sum Hard Binary search on answer (advanced) 10 Median of Two Sorted Arrays Hard Binary search on partition Problem 1: Binary Search LeetCode Link: https://leetcode.com/problems/binary-search/\nProblem Statement Given a sorted array nums and a target, return the index of target. If not found, return -1.\nExample:\nInput: nums = [-1, 0, 3, 5, 9, 12], target = 9 Output: 4 Clarifying Questions \u0026amp; Constraints Array is sorted in ascending order. All elements are unique. Must be O(log n). Approach Discussion Approach 1: Linear Scan\nWalk through the array one by one. Time: O(n) ❌ - doesn\u0026rsquo;t use the sorted property. Approach 2: Binary Search (Optimal) ✅\nClassic Template 1: check the middle, go left or right. Time: O(log n), Space: O(1) This is the simplest binary search problem - the one to get the fundamentals right.\nCode (Optimal Solution) def search(nums: list[int], target: int) -\u0026gt; int: \u0026#34;\u0026#34;\u0026#34; Classic binary search for an exact target in a sorted array. At each step: - If nums[mid] == target → found it - If nums[mid] \u0026lt; target → target must be in the right half → low = mid + 1 - If nums[mid] \u0026gt; target → target must be in the left half → high = mid - 1 Time Complexity: O(log n) - halving the search space each step Space Complexity: O(1) - only three variables \u0026#34;\u0026#34;\u0026#34; low, high = 0, len(nums) - 1 while low \u0026lt;= high: mid = low + (high - low) // 2 if nums[mid] == target: return mid elif nums[mid] \u0026lt; target: low = mid + 1 else: high = mid - 1 return -1 Edge Cases Target at start: [1, 2, 3], target=1 → 0. Target at end: [1, 2, 3], target=3 → 2. Target not present: [1, 2, 3], target=4 → -1. Single element: [5], target=5 → 0; [5], target=3 → -1. Two elements: [1, 3], target=3 → 1. Dry Run Input: nums = [-1, 0, 3, 5, 9, 12], target = 9 Step 1: low=0, high=5, mid=2 nums[2]=3 \u0026lt; 9 → low = 3 Step 2: low=3, high=5, mid=4 nums[4]=9 == 9 → return 4 ✅ Output: 4 Problem 2: Search Insert Position LeetCode Link: https://leetcode.com/problems/search-insert-position/\nProblem Statement Given a sorted array of distinct integers and a target, return the index if the target is found. If not, return the index where it would be inserted to keep the array sorted.\nExample:\nInput: nums = [1, 3, 5, 6], target = 5 → Output: 2 Input: nums = [1, 3, 5, 6], target = 2 → Output: 1 (insert between 1 and 3) Input: nums = [1, 3, 5, 6], target = 7 → Output: 4 (insert at end) Clarifying Questions \u0026amp; Constraints Array has distinct elements sorted in ascending order. Must be O(log n). This is equivalent to finding the first index where nums[i] \u0026gt;= target. Approach Discussion Approach 1: Linear Scan\nWalk until you find target or a value greater. Time: O(n) Approach 2: Binary Search - Left Boundary (Optimal) ✅\nThis is exactly Template 2: find the first index where nums[mid] \u0026gt;= target. If target exists, that\u0026rsquo;s its index. If not, that\u0026rsquo;s where it would be inserted. Time: O(log n), Space: O(1) Code (Optimal Solution) def searchInsert(nums: list[int], target: int) -\u0026gt; int: \u0026#34;\u0026#34;\u0026#34; Find the insertion point for target in a sorted array. This is the index of the first element \u0026gt;= target. Uses Template 2 (left boundary): - If nums[mid] \u0026lt; target: mid is too small, search right - If nums[mid] \u0026gt;= target: mid could be the answer, search left (including mid) Time Complexity: O(log n) Space Complexity: O(1) \u0026#34;\u0026#34;\u0026#34; low, high = 0, len(nums) # high = len(nums) for \u0026#34;insert at end\u0026#34; case while low \u0026lt; high: mid = low + (high - low) // 2 if nums[mid] \u0026lt; target: low = mid + 1 # Too small, search right else: high = mid # Could be the answer, search left return low # First index where nums[low] \u0026gt;= target Edge Cases Target smaller than all: [2, 4, 6], target=1 → 0. Target larger than all: [2, 4, 6], target=8 → 3. Target exists: [1, 3, 5], target=3 → 1. Empty array: [], target=5 → 0. Dry Run Input: nums = [1, 3, 5, 6], target = 2 Step 1: low=0, high=4, mid=2 nums[2]=5 \u0026gt;= 2 → high = 2 Step 2: low=0, high=2, mid=1 nums[1]=3 \u0026gt;= 2 → high = 1 Step 3: low=0, high=1, mid=0 nums[0]=1 \u0026lt; 2 → low = 1 low == high == 1 → return 1 ✅ (2 would be inserted between 1 and 3) Problem 3: Find First and Last Position of Element in Sorted Array LeetCode Link: https://leetcode.com/problems/find-first-and-last-position-of-element-in-sorted-array/\nProblem Statement Given a sorted array nums and a target, find the starting and ending position of target. If not found, return [-1, -1]. Must be O(log n).\nExample:\nInput: nums = [5, 7, 7, 8, 8, 10], target = 8 Output: [3, 4] Clarifying Questions \u0026amp; Constraints Array is sorted in non-decreasing order (duplicates allowed). Must be O(log n) - can\u0026rsquo;t scan linearly after finding one occurrence. Return [-1, -1] if target doesn\u0026rsquo;t exist. Approach Discussion Approach 1: Find One + Expand\nBinary search to find any occurrence, then expand left and right. Time: O(n) in worst case (all elements are target) ❌ Approach 2: Two Binary Searches (Optimal) ✅\nRun binary search twice: Find the first (leftmost) occurrence of target. Find the last (rightmost) occurrence of target. Time: O(log n), Space: O(1) This combines Templates 2 and 3.\nCode (Optimal Solution) def searchRange(nums: list[int], target: int) -\u0026gt; list[int]: \u0026#34;\u0026#34;\u0026#34; Find the first and last positions of target using two binary searches. Search 1: Find the leftmost index where nums[i] == target. Search 2: Find the rightmost index where nums[i] == target. Time Complexity: O(log n) - two binary searches, each O(log n) Space Complexity: O(1) \u0026#34;\u0026#34;\u0026#34; def find_first(nums, target): \u0026#34;\u0026#34;\u0026#34;Find the leftmost index of target (Template 2 style).\u0026#34;\u0026#34;\u0026#34; low, high = 0, len(nums) - 1 result = -1 while low \u0026lt;= high: mid = low + (high - low) // 2 if nums[mid] == target: result = mid # Found, but there might be an earlier one high = mid - 1 # Keep searching LEFT elif nums[mid] \u0026lt; target: low = mid + 1 else: high = mid - 1 return result def find_last(nums, target): \u0026#34;\u0026#34;\u0026#34;Find the rightmost index of target (Template 3 style).\u0026#34;\u0026#34;\u0026#34; low, high = 0, len(nums) - 1 result = -1 while low \u0026lt;= high: mid = low + (high - low) // 2 if nums[mid] == target: result = mid # Found, but there might be a later one low = mid + 1 # Keep searching RIGHT elif nums[mid] \u0026lt; target: low = mid + 1 else: high = mid - 1 return result first = find_first(nums, target) # If target doesn\u0026#39;t exist at all, no need to search for last if first == -1: return [-1, -1] last = find_last(nums, target) return [first, last] Edge Cases Target not present: [1, 3, 5], target=2 → [-1, -1]. Single occurrence: [1, 3, 5], target=3 → [1, 1]. All same: [7, 7, 7, 7], target=7 → [0, 3]. Empty array: [], target=5 → [-1, -1]. Dry Run Input: nums = [5, 7, 7, 8, 8, 10], target = 8 find_first(target=8): low=0, high=5, mid=2: nums[2]=7 \u0026lt; 8 → low=3 low=3, high=5, mid=4: nums[4]=8 == 8 → result=4, high=3 low=3, high=3, mid=3: nums[3]=8 == 8 → result=3, high=2 low=3 \u0026gt; high=2 → return 3 find_last(target=8): low=0, high=5, mid=2: nums[2]=7 \u0026lt; 8 → low=3 low=3, high=5, mid=4: nums[4]=8 == 8 → result=4, low=5 low=5, high=5, mid=5: nums[5]=10 \u0026gt; 8 → high=4 low=5 \u0026gt; high=4 → return 4 Output: [3, 4] ✅ Problem 4: Search a 2D Matrix LeetCode Link: https://leetcode.com/problems/search-a-2d-matrix/\nProblem Statement Write an efficient algorithm to search for a value in an m × n matrix with these properties:\nEach row is sorted left to right. The first integer of each row is greater than the last integer of the previous row. Example:\nMatrix: [1, 3, 5, 7] [10, 11, 16, 20] [23, 30, 34, 60] target = 3 → True Clarifying Questions \u0026amp; Constraints The entire matrix, read row by row, forms one sorted sequence. Must be O(log(m × n)). Approach Discussion Approach 1: Two Binary Searches\nBinary search to find the correct row, then binary search within that row. Time: O(log m + log n) = O(log(m × n)), Space: O(1) Approach 2: Treat as a Flat Sorted Array (Optimal) ✅\nSince the matrix is one sorted sequence, treat it as a 1D array of size m × n. Convert 1D index to 2D: row = index // n, col = index % n. Run a single binary search. Time: O(log(m × n)), Space: O(1) Both approaches have the same time complexity, but Approach 2 is cleaner.\nCode (Both Approaches) # ============================================================ # APPROACH 1: Two Binary Searches - O(log m + log n) # ============================================================ def searchMatrix_two(matrix: list[list[int]], target: int) -\u0026gt; bool: \u0026#34;\u0026#34;\u0026#34; First find the row, then search within it. Time Complexity: O(log m + log n) Space Complexity: O(1) \u0026#34;\u0026#34;\u0026#34; m, n = len(matrix), len(matrix[0]) # Binary search for the correct row # The target row is the last row where matrix[row][0] \u0026lt;= target top, bottom = 0, m - 1 while top \u0026lt;= bottom: mid = top + (bottom - top) // 2 if matrix[mid][0] \u0026lt;= target \u0026lt;= matrix[mid][n - 1]: # Target could be in this row break elif matrix[mid][0] \u0026gt; target: bottom = mid - 1 else: top = mid + 1 if top \u0026gt; bottom: return False # No valid row found row = top + (bottom - top) // 2 # Binary search within the row low, high = 0, n - 1 while low \u0026lt;= high: mid = low + (high - low) // 2 if matrix[row][mid] == target: return True elif matrix[row][mid] \u0026lt; target: low = mid + 1 else: high = mid - 1 return False # ============================================================ # APPROACH 2: Flat 1D Binary Search - O(log(m × n)) ✅ # ============================================================ def searchMatrix(matrix: list[list[int]], target: int) -\u0026gt; bool: \u0026#34;\u0026#34;\u0026#34; Treat the 2D matrix as a flat 1D sorted array. The matrix read row by row is a sorted sequence. A 1D index maps to 2D as: row = index // num_columns col = index % num_columns Time Complexity: O(log(m × n)) Space Complexity: O(1) \u0026#34;\u0026#34;\u0026#34; m, n = len(matrix), len(matrix[0]) low, high = 0, m * n - 1 while low \u0026lt;= high: mid = low + (high - low) // 2 # Convert 1D index to 2D coordinates row = mid // n col = mid % n value = matrix[row][col] if value == target: return True elif value \u0026lt; target: low = mid + 1 else: high = mid - 1 return False Edge Cases Target is the smallest element: target = matrix[0][0]. Target is the largest element: target = matrix[m-1][n-1]. Target not in matrix: Any value between existing values. 1×1 matrix: [[5]], target=5 → True. Dry Run Matrix: target = 3 [1, 3, 5, 7] [10, 11, 16, 20] [23, 30, 34, 60] m=3, n=4, flat size = 12 Step 1: low=0, high=11, mid=5 row=5//4=1, col=5%4=1 → matrix[1][1]=11 11 \u0026gt; 3 → high=4 Step 2: low=0, high=4, mid=2 row=2//4=0, col=2%4=2 → matrix[0][2]=5 5 \u0026gt; 3 → high=1 Step 3: low=0, high=1, mid=0 row=0//4=0, col=0%4=0 → matrix[0][0]=1 1 \u0026lt; 3 → low=1 Step 4: low=1, high=1, mid=1 row=1//4=0, col=1%4=1 → matrix[0][1]=3 3 == 3 → return True ✅ Problem 5: Search in Rotated Sorted Array LeetCode Link: https://leetcode.com/problems/search-in-rotated-sorted-array/\nProblem Statement A sorted array has been rotated at some pivot (e.g., [0,1,2,4,5,6,7] → [4,5,6,7,0,1,2]). Given the rotated array and a target, return the index of target or -1.\nExample:\nInput: nums = [4, 5, 6, 7, 0, 1, 2], target = 0 Output: 4 Clarifying Questions \u0026amp; Constraints All elements are unique. Must be O(log n). The array was sorted, then rotated - so one half is always sorted. Approach Discussion Approach 1: Find Pivot + Two Binary Searches\nFind the rotation point, then binary search the correct half. Time: O(log n), Space: O(1) Approach 2: Modified Single Binary Search (Optimal) ✅\nThe key insight: when you split a rotated sorted array at any point, at least one half is always sorted. We can check which half is sorted and then decide whether the target falls in that sorted half.\n[4, 5, 6, 7, 0, 1, 2] ^ mid=3 (value=7) Left: [4, 5, 6, 7] ← sorted ✅ Right: [0, 1, 2] ← sorted ✅ (both can be sorted) [6, 7, 0, 1, 2, 4, 5] ^ mid=3 (value=1) Left: [6, 7, 0, 1] ← NOT sorted (rotation point is here) Right: [2, 4, 5] ← sorted ✅ Decision logic:\nIf the left half is sorted (nums[low] \u0026lt;= nums[mid]): Is the target in this sorted range? (nums[low] \u0026lt;= target \u0026lt; nums[mid]) If yes → search left. If no → search right. If the right half is sorted: Is the target in this sorted range? (nums[mid] \u0026lt; target \u0026lt;= nums[high]) If yes → search right. If no → search left. Time: O(log n), Space: O(1) Code (Optimal Solution) def search(nums: list[int], target: int) -\u0026gt; int: \u0026#34;\u0026#34;\u0026#34; Binary search in a rotated sorted array. Key insight: At any mid point, at least one half (left or right) is sorted. We identify the sorted half, check if the target falls in it, and decide. Time Complexity: O(log n) Space Complexity: O(1) \u0026#34;\u0026#34;\u0026#34; low, high = 0, len(nums) - 1 while low \u0026lt;= high: mid = low + (high - low) // 2 if nums[mid] == target: return mid # Determine which half is sorted if nums[low] \u0026lt;= nums[mid]: # LEFT half [low..mid] is sorted if nums[low] \u0026lt;= target \u0026lt; nums[mid]: # Target is in the sorted left half high = mid - 1 else: # Target is in the right half low = mid + 1 else: # RIGHT half [mid..high] is sorted if nums[mid] \u0026lt; target \u0026lt;= nums[high]: # Target is in the sorted right half low = mid + 1 else: # Target is in the left half high = mid - 1 return -1 Edge Cases No rotation: [1, 2, 3, 4, 5], target=3 → works as normal binary search. Rotated by 1: [5, 1, 2, 3, 4], target=5 → 0. Two elements: [3, 1], target=1 → 1. Target not present: [4, 5, 6, 7, 0, 1, 2], target=3 → -1. Dry Run Input: nums = [4, 5, 6, 7, 0, 1, 2], target = 0 Step 1: low=0, high=6, mid=3 nums[3]=7, not target nums[0]=4 \u0026lt;= nums[3]=7 → LEFT half [4,5,6,7] is sorted Is 4 \u0026lt;= 0 \u0026lt; 7? NO → search right: low=4 Step 2: low=4, high=6, mid=5 nums[5]=1, not target nums[4]=0 \u0026lt;= nums[5]=1 → LEFT half [0,1] is sorted Is 0 \u0026lt;= 0 \u0026lt; 1? YES → search left: high=4 Step 3: low=4, high=4, mid=4 nums[4]=0 == 0 → return 4 ✅ Problem 6: Find Minimum in Rotated Sorted Array LeetCode Link: https://leetcode.com/problems/find-minimum-in-rotated-sorted-array/\nProblem Statement Find the minimum element in a rotated sorted array. All elements are unique.\nExample:\nInput: nums = [3, 4, 5, 1, 2] Output: 1 Clarifying Questions \u0026amp; Constraints All elements are unique. Must be O(log n). The minimum is the rotation point - the place where the sorted order \u0026ldquo;breaks.\u0026rdquo; Approach Discussion Approach 1: Linear Scan\nFind the minimum in one pass. Time: O(n) ❌ Approach 2: Binary Search on Boundary (Optimal) ✅\nThe minimum is at the point where the array \u0026ldquo;drops.\u0026rdquo; We can binary search for this boundary.\nKey condition: Compare nums[mid] with nums[high]:\nIf nums[mid] \u0026gt; nums[high] → the rotation point (minimum) is in the right half. If nums[mid] \u0026lt;= nums[high] → mid is in the sorted right portion, minimum is at mid or to the left. This is Template 2 (left boundary) applied to a rotated array.\nTime: O(log n), Space: O(1) Code (Optimal Solution) def findMin(nums: list[int]) -\u0026gt; int: \u0026#34;\u0026#34;\u0026#34; Find the minimum in a rotated sorted array. The minimum is at the rotation point where the array \u0026#34;drops.\u0026#34; We binary search for this boundary using nums[mid] vs nums[high]. Why compare with nums[high] and not nums[low]? Comparing with nums[low] fails when the array isn\u0026#39;t rotated at all (e.g., [1,2,3,4,5]). Comparing with nums[high] works universally. Time Complexity: O(log n) Space Complexity: O(1) \u0026#34;\u0026#34;\u0026#34; low, high = 0, len(nums) - 1 while low \u0026lt; high: mid = low + (high - low) // 2 if nums[mid] \u0026gt; nums[high]: # The drop (minimum) is somewhere in (mid, high] # mid itself can\u0026#39;t be the minimum since it\u0026#39;s bigger than something to its right low = mid + 1 else: # nums[mid] \u0026lt;= nums[high]: the right side is sorted # mid COULD be the minimum, so don\u0026#39;t exclude it high = mid return nums[low] # low == high, pointing to the minimum Edge Cases Not rotated: [1, 2, 3, 4] → 1 (minimum is the first element). Rotated by 1: [4, 1, 2, 3] → 1. Fully rotated (back to original): Same as not rotated. Two elements: [2, 1] → 1. Dry Run Input: nums = [3, 4, 5, 1, 2] Step 1: low=0, high=4, mid=2 nums[2]=5 \u0026gt; nums[4]=2 → minimum is in right half low = 3 Step 2: low=3, high=4, mid=3 nums[3]=1 \u0026lt;= nums[4]=2 → minimum could be at mid or left high = 3 low == high == 3 → return nums[3] = 1 ✅ Problem 7: Koko Eating Bananas LeetCode Link: https://leetcode.com/problems/koko-eating-bananas/\nProblem Statement Koko has n piles of bananas. She can eat at most k bananas per hour. Each hour, she picks a pile and eats k bananas from it (if the pile has fewer than k, she eats all of it and waits). She has h hours to eat all bananas. Find the minimum k (eating speed) such that she can eat all bananas within h hours.\nExample:\nInput: piles = [3, 6, 7, 11], h = 8 Output: 4 (at speed 4: ceil(3/4)+ceil(6/4)+ceil(7/4)+ceil(11/4) = 1+2+2+3 = 8 ≤ 8) Clarifying Questions \u0026amp; Constraints h \u0026gt;= len(piles) (she has at least one hour per pile). 1 \u0026lt;= k (minimum speed is 1). Maximum useful speed is max(piles) (finishes any pile in one hour). At speed k, time for one pile = ceil(pile_size / k). Approach Discussion Approach 1: Linear Search\nTry every speed from 1 to max(piles), pick the first one that works. Time: O(max(piles) × n), Space: O(1) ❌ Too slow. Approach 2: Binary Search on Answer (Optimal) ✅\nThis is a classic \u0026ldquo;binary search on answer\u0026rdquo; problem.\nSearch space: k ranges from 1 to max(piles).\nFeasibility check: At speed k, total hours = sum(ceil(pile/k) for each pile). Is this ≤ h?\nMonotonic condition: If speed k works, then speed k+1 definitely works (faster = fewer hours). So the feasibility is monotonic.\nWe want: The smallest k where feasible is True → Template 2 (left boundary).\nTime: O(n × log(max(piles))), Space: O(1)\nCode (Optimal Solution) import math def minEatingSpeed(piles: list[int], h: int) -\u0026gt; int: \u0026#34;\u0026#34;\u0026#34; Find the minimum eating speed to finish all bananas in h hours. Binary search on the answer (speed k): - Search space: [1, max(piles)] - Feasibility: at speed k, can Koko finish in ≤ h hours? - Monotonic: higher speed → fewer hours (always feasible if lower speed was) Time Complexity: O(n × log(max(piles))) - Binary search: O(log(max(piles))) iterations - Each feasibility check: O(n) to sum over all piles Space Complexity: O(1) \u0026#34;\u0026#34;\u0026#34; def hours_needed(speed): \u0026#34;\u0026#34;\u0026#34;Calculate total hours to eat all piles at the given speed.\u0026#34;\u0026#34;\u0026#34; total = 0 for pile in piles: total += math.ceil(pile / speed) # or: (pile + speed - 1) // speed return total # Binary search for the minimum feasible speed low, high = 1, max(piles) while low \u0026lt; high: mid = low + (high - low) // 2 if hours_needed(mid) \u0026lt;= h: # This speed works! But maybe a slower speed also works. high = mid else: # Too slow - need to eat faster. low = mid + 1 return low Edge Cases h == len(piles): Must eat each pile in exactly 1 hour → k = max(piles). h very large: Can eat very slowly → k = 1 if h \u0026gt;= sum(piles). Single pile: piles=[10], h=5 → k = 2 (ceil(10/2)=5 hours). All piles size 1: piles=[1,1,1], h=3 → k = 1. Dry Run Input: piles = [3, 6, 7, 11], h = 8 Search space: low=1, high=11 Step 1: mid=6 hours = ceil(3/6)+ceil(6/6)+ceil(7/6)+ceil(11/6) = 1+1+2+2 = 6 6 ≤ 8 → works! high=6 Step 2: mid=3 hours = ceil(3/3)+ceil(6/3)+ceil(7/3)+ceil(11/3) = 1+2+3+4 = 10 10 \u0026gt; 8 → too slow. low=4 Step 3: mid=5 hours = ceil(3/5)+ceil(6/5)+ceil(7/5)+ceil(11/5) = 1+2+2+3 = 8 8 ≤ 8 → works! high=5 Step 4: mid=4 hours = ceil(3/4)+ceil(6/4)+ceil(7/4)+ceil(11/4) = 1+2+2+3 = 8 8 ≤ 8 → works! high=4 low == high == 4 → return 4 ✅ Problem 8: Capacity to Ship Packages Within D Days LeetCode Link: https://leetcode.com/problems/capacity-to-ship-packages-within-d-days/\nProblem Statement Packages on a conveyor belt have weights weights[i]. A ship loads packages in order (left to right). Find the minimum ship capacity to ship all packages within days days.\nExample:\nInput: weights = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10], days = 5 Output: 15 Clarifying Questions \u0026amp; Constraints Packages must be shipped in order (can\u0026rsquo;t rearrange). A day\u0026rsquo;s load is a contiguous group of packages. Minimum capacity ≥ max(weights) (must fit the heaviest package). Maximum capacity = sum(weights) (ship everything in one day). Approach Discussion This is another \u0026ldquo;binary search on answer\u0026rdquo; problem, almost identical in structure to Koko Eating Bananas.\nSearch space: capacity ranges from max(weights) to sum(weights).\nFeasibility check: With this capacity, how many days are needed? Count by greedily filling each day until the capacity is reached.\nMonotonic: Larger capacity → fewer days (always feasible if smaller capacity was).\nWe want: Minimum capacity where days needed ≤ days.\nTime: O(n × log(sum(weights))), Space: O(1)\nCode (Optimal Solution) def shipWithinDays(weights: list[int], days: int) -\u0026gt; int: \u0026#34;\u0026#34;\u0026#34; Find the minimum ship capacity to deliver all packages within \u0026#39;days\u0026#39; days. Binary search on the answer (capacity): - Search space: [max(weights), sum(weights)] - Feasibility: can we ship everything in ≤ days using this capacity? - Monotonic: bigger ship → fewer days needed Time Complexity: O(n × log(sum(weights) - max(weights))) - Binary search: O(log(range)) iterations - Each feasibility check: O(n) to simulate loading Space Complexity: O(1) \u0026#34;\u0026#34;\u0026#34; def days_needed(capacity): \u0026#34;\u0026#34;\u0026#34;Simulate shipping: how many days to ship all packages at this capacity?\u0026#34;\u0026#34;\u0026#34; current_load = 0 num_days = 1 # Start with day 1 for w in weights: if current_load + w \u0026gt; capacity: # This package doesn\u0026#39;t fit today - start a new day num_days += 1 current_load = w else: current_load += w return num_days # Binary search for the minimum feasible capacity low = max(weights) # Must fit the heaviest package high = sum(weights) # Ship everything in one day while low \u0026lt; high: mid = low + (high - low) // 2 if days_needed(mid) \u0026lt;= days: high = mid # This capacity works, try smaller else: low = mid + 1 # Not enough capacity, need bigger return low Edge Cases days == len(weights): One package per day → capacity = max(weights). days == 1: Ship everything in one day → capacity = sum(weights). All same weight: [5,5,5,5], days=2 → capacity = 10. Dry Run Input: weights = [1,2,3,4,5,6,7,8,9,10], days = 5 low = max = 10, high = sum = 55 mid=32: days_needed = simulate → 2 days (1+2+...+8=36\u0026gt;32, so day1=[1..7]=28, day2=[8,9,10]=27) Actually let me be precise: day1: 1+2+3+4+5+6+7=28≤32, +8=36\u0026gt;32 → new day day2: 8+9=17≤32, +10=27≤32 → fits days=2, 2≤5 → high=32 mid=21: 1+2+3+4+5+6=21≤21, +7\u0026gt;21 → day2: 7+8=15≤21, +9=24\u0026gt;21 → day3: 9+10=19≤21 days=3, 3≤5 → high=21 mid=15: 1+2+3+4+5=15≤15, +6\u0026gt;15 → day2: 6+7=13≤15, +8\u0026gt;15 → day3: 8≤15, +9\u0026gt;15 → day4: 9≤15, +10\u0026gt;15 → day5: 10 days=5, 5≤5 → high=15 mid=12: 1+2+3+4=10≤12, +5=15\u0026gt;12 → day2: 5+6=11≤12, +7\u0026gt;12 → day3: 7≤12, +8\u0026gt;12 → day4: 8≤12, +9\u0026gt;12 → day5: 9≤12, +10\u0026gt;12 → day6: 10 days=6, 6\u0026gt;5 → low=13 mid=14: 1+2+3+4=10≤14, +5=15\u0026gt;14 → day2: 5+6+7=18\u0026gt;14... 5+6=11≤14, +7=18\u0026gt;14 → day3: 7≤14, +8=15\u0026gt;14 → day4: 8≤14, +9\u0026gt;14 → day5: 9≤14, +10\u0026gt;14 → day6: 10 days=6, 6\u0026gt;5 → low=15 low == high == 15 → return 15 ✅ Problem 9: Split Array Largest Sum LeetCode Link: https://leetcode.com/problems/split-array-largest-sum/\nProblem Statement Given an integer array nums and an integer k, split nums into k non-empty contiguous subarrays such that the largest sum among the subarrays is minimized. Return this minimized largest sum.\nExample:\nInput: nums = [7, 2, 5, 10, 8], k = 2 Output: 18 Explanation: Split as [7,2,5] and [10,8]. Largest sum = max(14, 18) = 18. This is the minimum possible largest sum for any 2-way split. Clarifying Questions \u0026amp; Constraints Must split into exactly k contiguous subarrays. Each subarray must be non-empty. We\u0026rsquo;re minimizing the maximum subarray sum (a minimax problem). Approach Discussion Approach 1: Dynamic Programming\ndp[i][j] = minimum largest sum when splitting nums[0..i] into j parts. Time: O(n² × k), Space: O(n × k) Works but is complex and slower. Approach 2: Binary Search on Answer (Optimal) ✅\nThis is the same structure as the previous two problems.\nSearch space: The largest subarray sum can range from max(nums) (each element is its own subarray, limited by the biggest) to sum(nums) (one subarray takes everything).\nFeasibility check: If the maximum allowed subarray sum is limit, how many subarrays (splits) do we need? Greedily fill subarrays up to limit, count how many we need.\nMonotonic: Larger limit → fewer splits needed. If limit allows ≤ k splits, it\u0026rsquo;s feasible.\nWe want: The smallest limit where splits needed ≤ k.\nTime: O(n × log(sum - max)), Space: O(1)\nConnection to \u0026ldquo;Capacity to Ship Packages\u0026rdquo;: This is the exact same problem with different words! \u0026ldquo;Split into k subarrays, minimize the largest sum\u0026rdquo; = \u0026ldquo;Ship in k days, minimize the capacity.\u0026rdquo;\nCode (Optimal Solution) def splitArray(nums: list[int], k: int) -\u0026gt; int: \u0026#34;\u0026#34;\u0026#34; Minimize the largest subarray sum when splitting into k parts. Binary search on the answer (maximum allowed sum per subarray): - Search space: [max(nums), sum(nums)] - Feasibility: with this max allowed sum, can we split into ≤ k parts? - Monotonic: larger allowed sum → fewer parts needed The feasibility check is identical to the \u0026#34;ship packages\u0026#34; problem: greedily fill each part up to the limit, count parts. Time Complexity: O(n × log(sum(nums) - max(nums))) Space Complexity: O(1) \u0026#34;\u0026#34;\u0026#34; def parts_needed(max_sum): \u0026#34;\u0026#34;\u0026#34;How many parts are needed if each part\u0026#39;s sum is at most max_sum?\u0026#34;\u0026#34;\u0026#34; current_sum = 0 parts = 1 for num in nums: if current_sum + num \u0026gt; max_sum: parts += 1 current_sum = num else: current_sum += num return parts low = max(nums) # At minimum, must handle the largest element high = sum(nums) # At maximum, put everything in one part while low \u0026lt; high: mid = low + (high - low) // 2 if parts_needed(mid) \u0026lt;= k: high = mid # This limit works, try smaller else: low = mid + 1 # Need a larger limit return low Edge Cases k == 1: Entire array in one subarray → return sum(nums). k == len(nums): Each element is its own subarray → return max(nums). All equal: [5,5,5,5], k=2 → 10. Dry Run Input: nums = [7, 2, 5, 10, 8], k = 2 low = 10 (max), high = 32 (sum) mid=21: parts? 7+2+5+10=24\u0026gt;21 → 7+2+5=14≤21, new part: 10+8=18≤21 → 2 parts 2 ≤ 2 → high=21 mid=15: 7+2+5=14≤15, +10=24\u0026gt;15 → new part: 10+8=18\u0026gt;15 → new part: 8 → 3 parts 3 \u0026gt; 2 → low=16 mid=18: 7+2+5=14≤18, +10=24\u0026gt;18 → new part: 10+8=18≤18 → 2 parts 2 ≤ 2 → high=18 mid=17: 7+2+5=14≤17, +10=24\u0026gt;17 → new part: 10+8=18\u0026gt;17 → new part: 8 → 3 parts 3 \u0026gt; 2 → low=18 low == high == 18 → return 18 ✅ Problem 10: Median of Two Sorted Arrays LeetCode Link: https://leetcode.com/problems/median-of-two-sorted-arrays/\nProblem Statement Given two sorted arrays nums1 and nums2, return the median of the two sorted arrays. The overall run time complexity should be O(log(m+n)).\nExample:\nInput: nums1 = [1, 3], nums2 = [2] Output: 2.0 (merged: [1, 2, 3], median = 2) Input: nums1 = [1, 2], nums2 = [3, 4] Output: 2.5 (merged: [1, 2, 3, 4], median = (2+3)/2 = 2.5) Clarifying Questions \u0026amp; Constraints Must be O(log(m+n)) - cannot merge and sort (that\u0026rsquo;s O(m+n)). Arrays can have different lengths. If total length is even, median = average of the two middle elements. Approach Discussion Approach 1: Merge and Find Middle\nMerge both arrays, find the middle. Time: O(m+n) ❌ - violates the O(log) requirement. Approach 2: Binary Search on Partition (Optimal) ✅\nThis is the hardest binary search problem in this set. The idea:\nThe median splits the combined elements into two equal halves. We need to find the correct \u0026ldquo;partition point\u0026rdquo; in both arrays such that:\nAll elements on the left side ≤ all elements on the right side. Left side has exactly (m+n+1) / 2 elements. We binary search on the partition position in the smaller array (to minimize search space). The partition in the larger array is determined automatically.\nPartition concept:\nnums1: [... left1 ...] | [... right1 ...] nums2: [... left2 ...] | [... right2 ...] Combined left side: left1 + left2 (size = half of total) Combined right side: right1 + right2 Valid partition: max(left1, left2) \u0026lt;= min(right1, right2) Time: O(log(min(m, n))), Space: O(1) Code (Optimal Solution) def findMedianSortedArrays(nums1: list[int], nums2: list[int]) -\u0026gt; float: \u0026#34;\u0026#34;\u0026#34; Find the median of two sorted arrays using binary search on partition. Concept: We want to partition both arrays into left and right halves such that: 1. len(left_total) == len(right_total) (or differ by 1) 2. max(left_total) \u0026lt;= min(right_total) Binary search on the partition index of the SMALLER array. The partition of the larger array is derived: partition2 = half - partition1. We check if the partition is valid (condition 2). If not, adjust. Time Complexity: O(log(min(m, n))) - binary search on the smaller array Space Complexity: O(1) \u0026#34;\u0026#34;\u0026#34; # Always binary search on the smaller array if len(nums1) \u0026gt; len(nums2): return findMedianSortedArrays(nums2, nums1) m, n = len(nums1), len(nums2) half = (m + n + 1) // 2 # Size of the left half (ceiling for odd total) low, high = 0, m # We can take 0 to m elements from nums1 while low \u0026lt;= high: # partition1: how many elements we take from nums1 for the left half partition1 = low + (high - low) // 2 # partition2: how many elements we take from nums2 for the left half partition2 = half - partition1 # Edge values (use -inf and +inf for boundaries) # left1 = largest element on the left side from nums1 # right1 = smallest element on the right side from nums1 left1 = nums1[partition1 - 1] if partition1 \u0026gt; 0 else float(\u0026#39;-inf\u0026#39;) right1 = nums1[partition1] if partition1 \u0026lt; m else float(\u0026#39;inf\u0026#39;) left2 = nums2[partition2 - 1] if partition2 \u0026gt; 0 else float(\u0026#39;-inf\u0026#39;) right2 = nums2[partition2] if partition2 \u0026lt; n else float(\u0026#39;inf\u0026#39;) # Check if the partition is valid if left1 \u0026lt;= right2 and left2 \u0026lt;= right1: # Valid partition! Calculate the median. if (m + n) % 2 == 1: # Odd total: median is the max of the left half return max(left1, left2) else: # Even total: median is the average of max(left) and min(right) return (max(left1, left2) + min(right1, right2)) / 2 elif left1 \u0026gt; right2: # Too many elements from nums1 on the left → move partition1 left high = partition1 - 1 else: # Too few elements from nums1 on the left → move partition1 right low = partition1 + 1 return 0.0 # Should never reach here with valid input Edge Cases One empty array: nums1=[], nums2=[1] → 1.0. One element each: [1], [2] → 1.5. Very different sizes: [1], [2,3,4,5,6] → 3.5. Same arrays: [1,2], [1,2] → 1.5. No overlap: [1,2], [3,4] → 2.5. Dry Run Input: nums1 = [1, 3, 8, 9, 15], nums2 = [7, 11, 18, 19, 21, 25] m=5 \u0026gt; n=6? No, but let\u0026#39;s swap to use the smaller array. Actually m=5, n=6, so nums1 is already smaller. half = (5+6+1)//2 = 6 (left half has 6 elements) low=0, high=5 Step 1: partition1=2, partition2=6-2=4 left1=nums1[1]=3, right1=nums1[2]=8 left2=nums2[3]=19, right2=nums2[4]=21 left1=3 ≤ right2=21 ✅ left2=19 ≤ right1=8? ❌ (19 \u0026gt; 8) → Too few from nums1 on left. low = 3 Step 2: partition1=4, partition2=6-4=2 left1=nums1[3]=9, right1=nums1[4]=15 left2=nums2[1]=11, right2=nums2[2]=18 left1=9 ≤ right2=18 ✅ left2=11 ≤ right1=15 ✅ → Valid partition! Total = 11 (odd), so median = max(left1, left2) = max(9, 11) = 11 Output: 11 ✅ Verify: merged = [1,3,7,8,9,11,15,18,19,21,25], median at index 5 = 11 ✅ Key Takeaways \u0026amp; Summary Quick Reference Table Problem BS Type Time Space Core Trick Binary Search Exact match O(log n) O(1) Template 1: low \u0026lt;= high Search Insert Position Left boundary O(log n) O(1) Template 2: first \u0026gt;= target First and Last Position Left + right boundary O(log n) O(1) Two searches: leftmost and rightmost Search a 2D Matrix Exact match (flattened) O(log(mn)) O(1) row = mid//n, col = mid%n Rotated Sorted Array Modified match O(log n) O(1) One half is always sorted Find Min in Rotated Boundary O(log n) O(1) Compare mid with high Koko Eating Bananas BS on answer O(n log M) O(1) Minimize speed, check feasibility Ship Packages BS on answer O(n log S) O(1) Same as Koko - minimize capacity Split Array Largest Sum BS on answer O(n log S) O(1) Same structure - minimize max sum Median of Two Arrays BS on partition O(log min(m,n)) O(1) Partition both arrays, validate Decision Framework Is the search space SORTED or does it have a MONOTONIC property? │ ├── Searching for an EXACT VALUE in a sorted array? │ └── Template 1 (exact match) │ ├── Searching for a BOUNDARY (first/last occurrence, insertion point)? │ ├── First occurrence / insertion point → Template 2 (left boundary) │ └── Last occurrence → Template 3 (right boundary) │ ├── Sorted array with a TWIST (rotated, 2D)? │ └── Modified binary search (identify which part is \u0026#34;normal\u0026#34;) │ └── Optimization problem - \u0026#34;minimum X such that condition holds\u0026#34;? └── Binary Search on Answer 1. Define the search space: [minimum possible, maximum possible] 2. Write is_feasible(x) - can check in O(n) or similar 3. Binary search for the boundary where feasibility changes The Binary Search on Answer Checklist When you suspect a problem uses binary search on answer, verify these three things:\nCan you define a clear search range? There should be a minimum and maximum possible answer.\nIs the feasibility condition monotonic? If answer x works, does x+1 also work (or vice versa)? If yes, binary search applies.\nCan you write a feasibility check efficiently? The is_feasible(x) function should run in O(n) or O(n log n), not O(n²).\nIf all three are yes, the solution structure is always the same:\nlow, high = min_answer, max_answer while low \u0026lt; high: mid = low + (high - low) // 2 if is_feasible(mid): high = mid # (or low = mid + 1 if looking for max) else: low = mid + 1 # (or high = mid - 1) return low What\u0026rsquo;s Next? Binary search and two pointers form the foundation of \u0026ldquo;searching\u0026rdquo; in arrays. We\u0026rsquo;ve now covered five patterns: Two Pointers, Binary Search, Sliding Window, Prefix Sum, and HashMap. Next up is Pattern 6: Kadane\u0026rsquo;s Algorithm - the classic technique for maximum subarray problems. Stay tuned!\n💡 Practice Tip: The three \u0026ldquo;binary search on answer\u0026rdquo; problems (7, 8, 9) are almost identical in structure. Once you solve one, the others become straightforward. The real skill is recognizing that a problem fits this pattern - look for optimization with a monotonic feasibility condition.\n","permalink":"https://blogs.codeverra.com/blog/dsa-patterns-02-binary-search/","summary":"\u003ch1 id=\"-pattern-2-binary-search\"\u003e🔰 Pattern 2: Binary Search\u003c/h1\u003e\n\u003ch2 id=\"table-of-contents\"\u003eTable of Contents\u003c/h2\u003e\n\u003col\u003e\n\u003cli\u003e\u003ca href=\"#what-is-binary-search\"\u003eWhat is Binary Search?\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#the-core-idea--why-halving-works\"\u003eThe Core Idea  -  Why Halving Works\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#the-three-binary-search-templates\"\u003eThe Three Binary Search Templates\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#binary-search-on-answer--the-hidden-pattern\"\u003eBinary Search on Answer  -  The Hidden Pattern\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#common-pitfalls\"\u003eCommon Pitfalls\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#problem-set\"\u003eProblem Set\u003c/a\u003e\n\u003cul\u003e\n\u003cli\u003e\u003ca href=\"#problem-1-binary-search\"\u003eProblem 1: Binary Search\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#problem-2-search-insert-position\"\u003eProblem 2: Search Insert Position\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#problem-3-find-first-and-last-position-of-element-in-sorted-array\"\u003eProblem 3: Find First and Last Position of Element in Sorted Array\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#problem-4-search-a-2d-matrix\"\u003eProblem 4: Search a 2D Matrix\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#problem-5-search-in-rotated-sorted-array\"\u003eProblem 5: Search in Rotated Sorted Array\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#problem-6-find-minimum-in-rotated-sorted-array\"\u003eProblem 6: Find Minimum in Rotated Sorted Array\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#problem-7-koko-eating-bananas\"\u003eProblem 7: Koko Eating Bananas\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#problem-8-capacity-to-ship-packages-within-d-days\"\u003eProblem 8: Capacity to Ship Packages Within D Days\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#problem-9-split-array-largest-sum\"\u003eProblem 9: Split Array Largest Sum\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#problem-10-median-of-two-sorted-arrays\"\u003eProblem 10: Median of Two Sorted Arrays\u003c/a\u003e\u003c/li\u003e\n\u003c/ul\u003e\n\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#key-takeaways--summary\"\u003eKey Takeaways \u0026amp; Summary\u003c/a\u003e\u003c/li\u003e\n\u003c/ol\u003e\n\u003chr\u003e\n\u003ch2 id=\"what-is-binary-search\"\u003eWhat is Binary Search?\u003c/h2\u003e\n\u003cp\u003eIn the Two Pointers pattern, we used two indices to narrow down a search space. Binary search takes that idea to its most powerful form: instead of narrowing one step at a time, we \u003cstrong\u003ecut the search space in half\u003c/strong\u003e at every step.\u003c/p\u003e","tags":["dsa","dsa-patterns"],"title":"Binary Search Pattern  -  Complete Guide"},{"content":"Complexity Analysis \u0026ndash; Time and Space Table of Contents What You Will Learn in This Blog Why Do We Need Complexity Analysis Building Intuition First How We Evaluate Algorithms \u0026ndash; A Brief History Approach 1 \u0026ndash; Experimental Analysis Approach 2 \u0026ndash; Counting Every Operation Exactly Approach 3 \u0026ndash; Asymptotic Analysis Primitive Operations \u0026ndash; The Foundation of the Model What is Big O Notation The Common Complexity Classes O(1) \u0026ndash; Constant Time O(log n) \u0026ndash; Logarithmic Time O(n) \u0026ndash; Linear Time O(n log n) \u0026ndash; Linearithmic Time O(n^2) \u0026ndash; Quadratic Time O(2^n) \u0026ndash; Exponential Time O(n!) \u0026ndash; Factorial Time Comparing the Complexity Classes How to Analyse Code Step by Step Space Complexity Common Misconceptions Complexity of Python Built-ins Practice Problems Summary Key Takeaways Next Steps What You Will Learn in This Blog Why complexity analysis exists and what problem it solves How to think about the efficiency of code intuitively before using any formula What Big O notation is and what it actually means The most common complexity classes with examples and code for each How to analyse any piece of code step by step Space complexity and how to think about memory usage Common misconceptions that trip up beginners Practice problems to test your understanding Why Do We Need Complexity Analysis Before anything else, let us answer the most important question: why does this topic exist at all?\nConsider this situation. You write a function that searches for a number in a list. It works correctly. Your friend writes a different function that also searches for a number in a list. It also works correctly. How do you know which one is better?\nYou could time them both. Run them, measure the seconds, compare. But that approach has serious problems:\nThe result depends on your hardware. A faster computer will make both functions look faster. The result depends on the input. Both functions might be fast on a list of 10 elements, but what happens with 10 million elements? You cannot test every possible input size. What you really want to know is: how does the running time of this code grow as the input gets larger?\nThat is exactly what complexity analysis gives you. It is a way of describing the efficiency of an algorithm in a way that is independent of hardware, language, and specific input values. It answers the question: if I double the size of the input, what happens to the running time?\nBuilding Intuition First Before introducing any notation, let us build the intuition with a real example.\nImagine you have a list of 1000 names and you want to find a specific name.\nApproach 1: Go through every name one by one from the start until you find it.\nIn the worst case, the name is the last one or not there at all, and you check all 1000 names. If the list had 2000 names, you would check up to 2000. If it had 1 million names, you would check up to 1 million. The work grows directly with the size of the list.\nApproach 2: The list is sorted alphabetically. Open to the middle. If your name comes before the middle name alphabetically, ignore the second half. If it comes after, ignore the first half. Repeat with the remaining half.\nWith 1000 names, you find the answer in at most 10 steps. With 2000 names, at most 11 steps. With 1 million names, at most 20 steps. The work barely grows even as the input explodes in size.\nBoth approaches solve the same problem. The difference in efficiency is enormous. Complexity analysis is the tool that lets you see and describe that difference clearly.\nHow We Evaluate Algorithms \u0026ndash; A Brief History Before Big O became the standard, people evaluated algorithms in other ways. Understanding those approaches and why they fall short helps you appreciate why asymptotic analysis is the tool we use today.\nApproach 1 \u0026ndash; Experimental Analysis (Just Run It and Time It) The most intuitive approach: implement the algorithm, run it on various inputs, measure the time, and plot the results.\nimport time def measure_time(func, input_data): start = time.time() func(input_data) end = time.time() return end - start Pros:\nSimple and concrete Gives you real-world numbers Easy to understand Cons:\nResults depend on the hardware. The same algorithm runs faster on a faster machine. You cannot compare results across machines. Results depend on the programming language and how the code is compiled or interpreted. Results depend on what else is running on the machine at the time. You can only test inputs you actually run. You cannot reason about inputs you have not tried. You have to implement the algorithm before you can evaluate it. Sometimes you want to compare two approaches before writing either one. Approach 2 \u0026ndash; Counting Every Operation Exactly A more rigorous approach: sit down with the code and count every single operation. If the function does 3n + 7 operations, write that down. Compare two algorithms by comparing their exact operation counts.\nPros:\nMore precise than timing Does not depend on hardware Can be done without running the code Cons:\nWhat counts as one operation? Is a + b one operation or several? Does reading a from memory count separately from adding? The exact count depends on the hardware architecture. On some machines, certain operations are faster than others. The analysis becomes very tedious for even moderately complex code. The exact constants (like the 3 in 3n + 7) are not meaningful across different machines or languages. Approach 3 \u0026ndash; Asymptotic Analysis (Big O) This is the approach we use. Instead of counting every operation or timing the code, we ask: how does the running time grow as the input size grows?\nWe express this growth as a function of n (the input size), and we drop constants and lower-order terms because they become irrelevant at large n.\nPros:\nHardware independent \u0026ndash; the analysis holds regardless of the machine Language independent \u0026ndash; it describes the algorithm, not the implementation You do not need to run the code to analyse it. You can reason about efficiency on paper. Focuses on what actually matters: behaviour at scale Gives you a common vocabulary to compare algorithms unambiguously Cons:\nIgnores constant factors, which can matter in practice for small inputs Two O(n) algorithms can have very different real-world performance Does not capture cache behaviour, memory access patterns, or hardware-specific optimisations The benefit you should internalise: asymptotic analysis lets you evaluate an algorithm by reading the code, without running it. This is enormously powerful. You can look at a function, reason about its complexity in a few seconds, and know whether it will scale.\nPrimitive Operations \u0026ndash; The Foundation of the Model Before we can say that a loop is O(n) or a function is O(1), we need to agree on what counts as a single unit of work. This is the concept of a primitive operation.\nA primitive operation is a basic computational step that runs in constant time, regardless of the values involved. These are the atomic actions that we count as O(1):\nPrimitive Operation Example Assigning a value to a variable x = 5 Reading a variable print(x) Arithmetic operation x + y, x * y, x % 2 Comparison x \u0026gt; y, x == y Accessing an element by index nums[i] Calling a function (not counting what the function does) len(nums) Returning from a function return x The key assumption of this model is that all primitive operations take the same amount of time, and that time does not depend on the values involved.\nThis means:\nx = 2 + 2 # one arithmetic operation -- O(1) x = 2000 + 2000 # still one arithmetic operation -- O(1) x = 2000000 + 2000000 # still one arithmetic operation -- O(1) It does not matter whether you are adding small numbers or large numbers. The addition itself is one primitive operation and costs O(1).\nSimilarly:\nx = 5 # one assignment -- O(1) y = x # one read, one assignment -- O(1) z = x + y # two reads, one addition, one assignment -- O(1) Even though z = x + y involves multiple primitive operations, it is still a fixed, constant number of operations. So we call it O(1).\nThis is why we can look at a loop body and say \u0026ldquo;the work inside is O(1)\u0026rdquo; \u0026ndash; we mean it does a fixed, constant number of primitive operations per iteration, regardless of the values in the loop.\nImportant note: This model assumes we are working with fixed-size integers (like 32-bit or 64-bit integers). For arbitrarily large integers (like Python\u0026rsquo;s arbitrary precision integers), arithmetic is no longer truly O(1) because the number of bits grows. For DSA purposes, we always assume fixed-size values and treat arithmetic as O(1).\nWhat is Big O Notation Big O notation is a mathematical way of describing how the running time (or memory usage) of an algorithm grows relative to the size of its input.\nWhen we write O(n), we are saying: as the input size n grows, the running time grows proportionally to n.\nA few important things to understand about Big O before we look at examples:\n1. Big O describes the worst case\nUnless stated otherwise, Big O refers to the worst-case scenario. When you search a list of n elements, the worst case is that the element is at the very end or not there at all, so you check all n elements.\n2. Big O drops constants and lower-order terms\nIf an algorithm does 3n + 5 operations, we write O(n), not O(3n + 5). The constant 3 and the extra 5 become irrelevant as n grows large. We only care about the dominant term.\nIf an algorithm does n^2 + n operations, we write O(n^2). At large values of n, the n term is negligible compared to n^2.\n3. Big O is about the shape of growth, not the exact count\nWe are not counting every operation. We are describing the pattern of how the work scales.\nThe graph above shows how different complexity classes grow as input size increases. Notice how O(1) stays flat, O(log n) barely rises, O(n) is a straight line, and O(n^2) shoots upward steeply.\nThe Common Complexity Classes Let us go through each complexity class from best to worst, with concrete examples and code.\nO(1) \u0026ndash; Constant Time The algorithm takes the same amount of time regardless of the size of the input.\nReal world analogy: Looking up a word in a dictionary if you already know the exact page number. It does not matter how thick the dictionary is.\ndef get_first_element(nums): \u0026#34;\u0026#34;\u0026#34; Returns the first element of a list. No matter how long the list is, this is always one operation. Time complexity: O(1) Space complexity: O(1) \u0026#34;\u0026#34;\u0026#34; return nums[0] def is_even(n): \u0026#34;\u0026#34;\u0026#34; Checks if a number is even using the modulo operator. One operation regardless of how large n is. Time complexity: O(1) Space complexity: O(1) \u0026#34;\u0026#34;\u0026#34; return n % 2 == 0 Accessing a list by index, dictionary lookup by key, push/pop on a stack \u0026ndash; these are all O(1).\nO(log n) \u0026ndash; Logarithmic Time The algorithm cuts the problem in half (or by some fraction) at each step. As the input doubles, the work increases by only one extra step.\nReal world analogy: Finding a word in a physical dictionary. You open to the middle, decide which half the word is in, open to the middle of that half, and repeat. Doubling the size of the dictionary only adds one more step.\ndef binary_search(nums, target): \u0026#34;\u0026#34;\u0026#34; Searches for target in a sorted list by repeatedly halving the search space. After each step, we eliminate half the remaining elements. - 1000 elements: at most 10 steps (2^10 = 1024) - 1 million elements: at most 20 steps (2^20 = 1,048,576) - 1 billion elements: at most 30 steps (2^30 = 1,073,741,824) Time complexity: O(log n) Space complexity: O(1) \u0026#34;\u0026#34;\u0026#34; left, right = 0, len(nums) - 1 while left \u0026lt;= right: mid = (left + right) // 2 # find the middle index if nums[mid] == target: return mid # found it elif nums[mid] \u0026lt; target: left = mid + 1 # target is in the right half else: right = mid - 1 # target is in the left half return -1 # not found # Example nums = [1, 3, 5, 7, 9, 11, 13, 15] print(binary_search(nums, 7)) # 3 (index of 7) print(binary_search(nums, 6)) # -1 (not found) The key intuition: every time you double the input size, you only add one more step. That is the power of O(log n).\nO(n) \u0026ndash; Linear Time The work grows directly in proportion to the input size. Double the input, double the work.\nReal world analogy: Reading every page of a book to find a specific sentence. The longer the book, the longer it takes, directly proportionally.\ndef find_max(nums): \u0026#34;\u0026#34;\u0026#34; Finds the maximum value in an unsorted list. We have no choice but to look at every element at least once. Time complexity: O(n) -- we visit each element exactly once Space complexity: O(1) -- we only store the current max \u0026#34;\u0026#34;\u0026#34; max_val = float(\u0026#39;-inf\u0026#39;) # start with negative infinity for num in nums: # visit each element once -- this is the O(n) part if num \u0026gt; max_val: max_val = num return max_val def linear_search(nums, target): \u0026#34;\u0026#34;\u0026#34; Searches for target by checking every element one by one. In the worst case, we check all n elements. Time complexity: O(n) Space complexity: O(1) \u0026#34;\u0026#34;\u0026#34; for i, num in enumerate(nums): if num == target: return i return -1 O(n log n) \u0026ndash; Linearithmic Time More than linear but much better than quadratic. This is the complexity of efficient sorting algorithms.\nIntuition: Think of it as doing O(log n) work for each of the n elements. Or equivalently, splitting and merging the list O(log n) times, with each split/merge touching all n elements.\ndef merge_sort(nums): \u0026#34;\u0026#34;\u0026#34; Splits the list in half recursively, sorts each half, then merges the sorted halves back together. - Splitting happens log n times (we halve each time) - Each level of merging touches all n elements - Total: O(n log n) Time complexity: O(n log n) Space complexity: O(n) -- we create new lists during merging \u0026#34;\u0026#34;\u0026#34; if len(nums) \u0026lt;= 1: return nums # Split into two halves mid = len(nums) // 2 left = merge_sort(nums[:mid]) # sort left half right = merge_sort(nums[mid:]) # sort right half # Merge the two sorted halves return merge(left, right) def merge(left, right): \u0026#34;\u0026#34;\u0026#34;Merges two sorted lists into one sorted list.\u0026#34;\u0026#34;\u0026#34; result = [] i = j = 0 while i \u0026lt; len(left) and j \u0026lt; len(right): if left[i] \u0026lt;= right[j]: result.append(left[i]) i += 1 else: result.append(right[j]) j += 1 # Append any remaining elements result.extend(left[i:]) result.extend(right[j:]) return result # Example nums = [5, 2, 8, 1, 9, 3] print(merge_sort(nums)) # [1, 2, 3, 5, 8, 9] Python\u0026rsquo;s built-in sort() and sorted() run in O(n log n). This is generally the best you can achieve for a comparison-based sort.\nO(n^2) \u0026ndash; Quadratic Time The work grows with the square of the input size. Double the input, quadruple the work. This usually comes from nested loops where each loop runs n times.\nReal world analogy: Comparing every person in a room to every other person. If there are 10 people, that is 100 comparisons. If there are 100 people, that is 10,000 comparisons.\ndef bubble_sort(nums): \u0026#34;\u0026#34;\u0026#34; Repeatedly steps through the list and swaps adjacent elements that are in the wrong order. For each of the n elements (outer loop), we potentially compare it against all other n elements (inner loop). Time complexity: O(n^2) Space complexity: O(1) -- sorting is done in place \u0026#34;\u0026#34;\u0026#34; n = len(nums) for i in range(n): # outer loop runs n times for j in range(n - i - 1): # inner loop runs roughly n times if nums[j] \u0026gt; nums[j + 1]: # swap adjacent elements nums[j], nums[j + 1] = nums[j + 1], nums[j] return nums def has_duplicate_brute_force(nums): \u0026#34;\u0026#34;\u0026#34; Checks for duplicates by comparing every pair of elements. Time complexity: O(n^2) -- nested loops, each running n times Space complexity: O(1) \u0026#34;\u0026#34;\u0026#34; n = len(nums) for i in range(n): for j in range(i + 1, n): # compare nums[i] with every element after it if nums[i] == nums[j]: return True return False def has_duplicate_efficient(nums): \u0026#34;\u0026#34;\u0026#34; The same problem solved with a hash set. Time complexity: O(n) -- one pass through the list Space complexity: O(n) -- the set can grow up to n elements \u0026#34;\u0026#34;\u0026#34; seen = set() for num in nums: if num in seen: # O(1) lookup return True seen.add(num) return False The duplicate detection example above is a perfect illustration of why complexity matters. The brute force O(n^2) solution and the O(n) solution both give the correct answer, but on a list of 1 million elements the O(n) solution will be roughly 1 million times faster.\nO(2^n) \u0026ndash; Exponential Time The work doubles with each addition to the input. These algorithms become impractically slow very quickly.\nWhen does this appear? Recursion that branches into two calls at each step, like a naive recursive Fibonacci or generating all subsets.\ndef fibonacci_naive(n): \u0026#34;\u0026#34;\u0026#34; Computes the nth Fibonacci number by recursively computing fibonacci(n-1) and fibonacci(n-2). The problem: each call spawns two more calls, and many subproblems are recomputed over and over. fibonacci(5) calls fibonacci(4) and fibonacci(3) fibonacci(4) calls fibonacci(3) and fibonacci(2) fibonacci(3) is computed multiple times -- wasteful Time complexity: O(2^n) -- exponential, very slow for large n Space complexity: O(n) -- call stack depth \u0026#34;\u0026#34;\u0026#34; if n \u0026lt;= 1: return n return fibonacci_naive(n - 1) + fibonacci_naive(n - 2) def fibonacci_memo(n, memo={}): \u0026#34;\u0026#34;\u0026#34; Same computation but we store results we have already computed. This is called memoisation and is a core DP technique. Time complexity: O(n) -- each value computed exactly once Space complexity: O(n) -- memo dictionary stores n values \u0026#34;\u0026#34;\u0026#34; if n in memo: return memo[n] if n \u0026lt;= 1: return n memo[n] = fibonacci_memo(n - 1, memo) + fibonacci_memo(n - 2, memo) return memo[n] O(n!) \u0026ndash; Factorial Time The slowest common complexity class. Generates all permutations of n elements.\ndef generate_permutations(nums): \u0026#34;\u0026#34;\u0026#34; Generates all possible orderings of a list. For n elements there are n! permutations. n=1: 1 permutation n=2: 2 permutations n=3: 6 permutations n=10: 3,628,800 permutations n=20: 2,432,902,008,176,640,000 permutations Time complexity: O(n!) Space complexity: O(n!) \u0026#34;\u0026#34;\u0026#34; if len(nums) == 0: return [[]] result = [] for i, num in enumerate(nums): rest = nums[:i] + nums[i+1:] # all elements except nums[i] for perm in generate_permutations(rest): # permutations of the rest result.append([num] + perm) return result print(generate_permutations([1, 2, 3])) # [[1,2,3], [1,3,2], [2,1,3], [2,3,1], [3,1,2], [3,2,1]] Comparing the Complexity Classes Here is what the same input size looks like across different complexity classes. This table makes the differences visceral.\nn (input size) O(1) O(log n) O(n) O(n log n) O(n^2) O(2^n) 10 1 3 10 33 100 1,024 100 1 7 100 664 10,000 way too large 1,000 1 10 1,000 9,966 1,000,000 impossible 1,000,000 1 20 1,000,000 19,931,568 10^12 impossible Notice: O(n^2) with 1 million elements requires 1 trillion operations. At 10^9 operations per second, that is 1000 seconds. An O(n log n) solution does the same job in about 20 seconds. An O(n) solution does it in about 1 second.\nHow to Analyse Code Step by Step Now that you know the complexity classes, here is a systematic process for analysing any piece of code.\nRule 1: A simple statement is O(1)\nx = 5 # O(1) y = x + 3 # O(1) z = nums[i] # O(1) Rule 2: A loop that runs n times is O(n)\nfor i in range(n): # runs n times print(i) # O(1) work inside # Total: O(n) Rule 3: Nested loops multiply\nfor i in range(n): # runs n times for j in range(n): # runs n times for each i print(i, j) # O(1) work inside # Total: O(n * n) = O(n^2) Rule 4: Sequential blocks add, and we keep only the dominant term\nfor i in range(n): # O(n) print(i) for i in range(n): # O(n) for j in range(n): print(i, j) # O(n^2) # Total: O(n) + O(n^2) = O(n^2) -- we drop the lower order term Rule 5: If the loop variable halves each time, it is O(log n)\ni = n while i \u0026gt; 0: print(i) i = i // 2 # i halves each iteration # Total: O(log n) Putting it all together \u0026ndash; a real example:\ndef example(nums): \u0026#34;\u0026#34;\u0026#34; Let\u0026#39;s analyse this function step by step. nums has n elements. \u0026#34;\u0026#34;\u0026#34; # Step 1: Find the maximum -- one pass through the list max_val = float(\u0026#39;-inf\u0026#39;) # O(1) for num in nums: # loop runs n times max_val = max(max_val, num) # O(1) work # This block: O(n) # Step 2: Find all pairs that sum to max_val result = [] # O(1) for i in range(len(nums)): # loop runs n times for j in range(i+1, len(nums)): # loop runs up to n times if nums[i] + nums[j] == max_val: # O(1) result.append((nums[i], nums[j])) # O(1) # This block: O(n^2) return result # Total: O(n) + O(n^2) = O(n^2) # The O(n^2) block dominates, so the overall complexity is O(n^2) Space Complexity Space complexity measures how much extra memory an algorithm uses relative to the input size. The rules are similar to time complexity.\nNote: we do not count the input itself \u0026ndash; we count the additional memory the algorithm allocates.\ndef sum_list(nums): \u0026#34;\u0026#34;\u0026#34; Adds up all elements. Only stores the running total. Time complexity: O(n) Space complexity: O(1) -- only one extra variable regardless of input size \u0026#34;\u0026#34;\u0026#34; total = 0 for num in nums: total += num return total def double_all(nums): \u0026#34;\u0026#34;\u0026#34; Creates a new list with each element doubled. Time complexity: O(n) Space complexity: O(n) -- the new list grows with the input \u0026#34;\u0026#34;\u0026#34; result = [] for num in nums: result.append(num * 2) return result def two_sum(nums, target): \u0026#34;\u0026#34;\u0026#34; Finds two indices that add up to target using a hash map. Time complexity: O(n) -- one pass through nums Space complexity: O(n) -- the seen dictionary can hold up to n entries \u0026#34;\u0026#34;\u0026#34; seen = {} # maps value to its index for i, num in enumerate(nums): complement = target - num if complement in seen: return [seen[complement], i] seen[num] = i return [] Space complexity of recursion: Each recursive call adds a frame to the call stack. If a function calls itself n times deep, the space complexity is at least O(n).\ndef factorial(n): \u0026#34;\u0026#34;\u0026#34; Computes n! recursively. Time complexity: O(n) -- n recursive calls Space complexity: O(n) -- n frames on the call stack at maximum depth \u0026#34;\u0026#34;\u0026#34; if n == 0: return 1 return n * factorial(n - 1) Common Misconceptions Misconception 1: Faster computer means better algorithm\nA faster computer makes every algorithm faster by a constant factor. It does not change the complexity class. An O(n^2) algorithm on a computer 1000x faster is still O(n^2) \u0026ndash; it will still be slower than an O(n) algorithm at large enough inputs.\nMisconception 2: O(n^2) is always bad\nFor small inputs, an O(n^2) algorithm can be perfectly fine. If you know n will never exceed 100, the difference between O(n) and O(n^2) is negligible. Complexity analysis matters most at scale.\nMisconception 3: The best complexity is always the goal\nSometimes an O(n log n) solution is harder to write and maintain than an O(n^2) solution, and the input is small enough that it does not matter. Always consider the constraints. For n \u0026lt;= 100, almost anything works. For n \u0026gt;= 10^6, you need to think carefully.\nMisconception 4: Big O is the whole story\nTwo algorithms can both be O(n log n) but one can be twice as fast in practice because of constant factors, cache behaviour, or simpler code. Big O tells you the shape of growth, not the exact performance.\nMisconception 5: Space and time are independent\nOften you trade one for the other. The duplicate detection example earlier used O(n) space to get O(n) time instead of O(1) space with O(n^2) time. This trade-off comes up constantly.\nComplexity of Python Built-ins (Reference) Now that you understand complexity, the table from the Python Refresher blog will make complete sense.\nOperation Time Complexity list[i] O(1) list.append(x) O(1) amortised list.insert(i, x) O(n) x in list O(n) x in dict O(1) average x in set O(1) average list.sort() O(n log n) len(list) O(1) dict[key] O(1) average heapq.heappush O(log n) heapq.heappop O(log n) Practice Problems Work through these in order. For each one, state the time and space complexity of your solution before looking at the answer.\nProblem 1 \u0026ndash; Count the Steps (Easy) What is the time complexity of the following function? Explain your reasoning.\ndef mystery(n): count = 0 i = 1 while i \u0026lt; n: count += 1 i *= 2 return count Think about it: What does i *= 2 do to i on each iteration? How many times can you double i before it reaches n?\nAnswer:\nThe variable i starts at 1 and doubles each iteration: 1, 2, 4, 8, 16 ... It reaches n after log2(n) steps. Time complexity: O(log n) Space complexity: O(1) Problem 2 \u0026ndash; Two Sum (Easy) Given a list of integers and a target, return the indices of two numbers that add up to the target. Each input has exactly one solution.\n# Example nums = [2, 7, 11, 15] target = 9 # Output: [0, 1] because nums[0] + nums[1] = 2 + 7 = 9 Approach 1 \u0026ndash; Brute Force:\ndef two_sum_brute(nums, target): \u0026#34;\u0026#34;\u0026#34; Check every pair of elements. Time complexity: O(n^2) -- nested loops Space complexity: O(1) -- no extra data structures \u0026#34;\u0026#34;\u0026#34; n = len(nums) for i in range(n): for j in range(i + 1, n): # j starts after i to avoid using same element twice if nums[i] + nums[j] == target: return [i, j] return [] Approach 2 \u0026ndash; Hash Map:\ndef two_sum_optimal(nums, target): \u0026#34;\u0026#34;\u0026#34; For each number, check if its complement (target - num) has been seen before. Store each number and its index in a dictionary as we go. Time complexity: O(n) -- single pass through the list Space complexity: O(n) -- dictionary can hold up to n entries \u0026#34;\u0026#34;\u0026#34; seen = {} # maps number to its index for i, num in enumerate(nums): complement = target - num # what do we need to pair with num? if complement in seen: # O(1) lookup -- have we seen the complement? return [seen[complement], i] seen[num] = i # store current number and its index return [] # Test print(two_sum_optimal([2, 7, 11, 15], 9)) # [0, 1] print(two_sum_optimal([3, 2, 4], 6)) # [1, 2] Problem 3 \u0026ndash; Count Unique Values (Easy) Given a sorted list, count the number of unique values without using a set.\n# Example nums = [1, 1, 2, 3, 3, 4, 4, 4, 5] # Output: 5 (values 1, 2, 3, 4, 5) def count_unique(nums): \u0026#34;\u0026#34;\u0026#34; Since the list is sorted, duplicates are adjacent. We just count how many times the value changes. Time complexity: O(n) -- single pass Space complexity: O(1) -- only store the count and last seen value \u0026#34;\u0026#34;\u0026#34; if not nums: return 0 count = 1 # first element is always unique for i in range(1, len(nums)): if nums[i] != nums[i - 1]: # value changed -- new unique element count += 1 return count print(count_unique([1, 1, 2, 3, 3, 4, 4, 4, 5])) # 5 print(count_unique([])) # 0 print(count_unique([7])) # 1 Problem 4 \u0026ndash; Analyse This Function (Medium) What is the time and space complexity of the following function?\ndef process(nums): n = len(nums) result = [] for i in range(n): total = 0 for j in range(i, n): total += nums[j] result.append(total) return result Answer:\nOuter loop runs n times. Inner loop runs n - i times for each i, which on average is n/2 times. Total iterations: roughly n * n/2 = n^2/2 operations. We drop the constant: O(n^2) For space: the result list. How many elements does it hold? i=0: n elements appended i=1: n-1 elements appended ... i=n-1: 1 element appended Total: n + (n-1) + ... + 1 = n(n+1)/2 elements Space complexity: O(n^2) Summary Complexity Name Example How it grows O(1) Constant Array index access, dict lookup Does not grow O(log n) Logarithmic Binary search Grows by 1 step when input doubles O(n) Linear Linear search, single loop Doubles when input doubles O(n log n) Linearithmic Merge sort, heapsort Slightly more than linear O(n^2) Quadratic Bubble sort, nested loops Quadruples when input doubles O(2^n) Exponential Naive recursion, subsets Doubles with each new element O(n!) Factorial Permutations Grows faster than any of the above Key Takeaways Complexity analysis describes how an algorithm scales, independent of hardware or language. Big O notation captures the dominant term and drops constants. O(3n + 5) is O(n). Nested loops usually mean O(n^2). A loop that halves its range each time is O(log n). Space complexity measures extra memory used, not the input itself. You often trade space for time. Using a hash map turns an O(n^2) problem into O(n) at the cost of O(n) space. Complexity is most important at scale. For tiny inputs, any solution works. Always state the time and space complexity of every solution you write. Make it a habit from day one. Next Steps Next blog: [Arrays] \u0026ndash; the first data structure, covered in full depth, with all operations and patterns Reference: Codeverra DSA Sheet ","permalink":"https://blogs.codeverra.com/blog/dsa-03-complexity-analysis/","summary":"\u003ch1 id=\"complexity-analysis--time-and-space\"\u003eComplexity Analysis \u0026ndash; Time and Space\u003c/h1\u003e\n\u003chr\u003e\n\u003ch2 id=\"table-of-contents\"\u003eTable of Contents\u003c/h2\u003e\n\u003cul\u003e\n\u003cli\u003e\u003ca href=\"#what-you-will-learn-in-this-blog\"\u003eWhat You Will Learn in This Blog\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#why-do-we-need-complexity-analysis\"\u003eWhy Do We Need Complexity Analysis\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#building-intuition-first\"\u003eBuilding Intuition First\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#how-we-evaluate-algorithms----a-brief-history\"\u003eHow We Evaluate Algorithms \u0026ndash; A Brief History\u003c/a\u003e\n\u003cul\u003e\n\u003cli\u003e\u003ca href=\"#approach-1----experimental-analysis-just-run-it-and-time-it\"\u003eApproach 1 \u0026ndash; Experimental Analysis\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#approach-2----counting-every-operation-exactly\"\u003eApproach 2 \u0026ndash; Counting Every Operation Exactly\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#approach-3----asymptotic-analysis-big-o\"\u003eApproach 3 \u0026ndash; Asymptotic Analysis\u003c/a\u003e\u003c/li\u003e\n\u003c/ul\u003e\n\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#primitive-operations----the-foundation-of-the-model\"\u003ePrimitive Operations \u0026ndash; The Foundation of the Model\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#what-is-big-o-notation\"\u003eWhat is Big O Notation\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#the-common-complexity-classes\"\u003eThe Common Complexity Classes\u003c/a\u003e\n\u003cul\u003e\n\u003cli\u003e\u003ca href=\"#o1----constant-time\"\u003eO(1) \u0026ndash; Constant Time\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#olog-n----logarithmic-time\"\u003eO(log n) \u0026ndash; Logarithmic Time\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#on----linear-time\"\u003eO(n) \u0026ndash; Linear Time\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#on-log-n----linearithmic-time\"\u003eO(n log n) \u0026ndash; Linearithmic Time\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#on2----quadratic-time\"\u003eO(n^2) \u0026ndash; Quadratic Time\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#o2n----exponential-time\"\u003eO(2^n) \u0026ndash; Exponential Time\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#on----factorial-time\"\u003eO(n!) \u0026ndash; Factorial Time\u003c/a\u003e\u003c/li\u003e\n\u003c/ul\u003e\n\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#comparing-the-complexity-classes\"\u003eComparing the Complexity Classes\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#how-to-analyse-code-step-by-step\"\u003eHow to Analyse Code Step by Step\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#space-complexity\"\u003eSpace Complexity\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#common-misconceptions\"\u003eCommon Misconceptions\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#complexity-of-python-built-ins-reference\"\u003eComplexity of Python Built-ins\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#practice-problems\"\u003ePractice Problems\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#summary\"\u003eSummary\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#key-takeaways\"\u003eKey Takeaways\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#next-steps\"\u003eNext Steps\u003c/a\u003e\u003c/li\u003e\n\u003c/ul\u003e\n\u003chr\u003e\n\u003ch2 id=\"what-you-will-learn-in-this-blog\"\u003eWhat You Will Learn in This Blog\u003c/h2\u003e\n\u003cul\u003e\n\u003cli\u003eWhy complexity analysis exists and what problem it solves\u003c/li\u003e\n\u003cli\u003eHow to think about the efficiency of code intuitively before using any formula\u003c/li\u003e\n\u003cli\u003eWhat Big O notation is and what it actually means\u003c/li\u003e\n\u003cli\u003eThe most common complexity classes with examples and code for each\u003c/li\u003e\n\u003cli\u003eHow to analyse any piece of code step by step\u003c/li\u003e\n\u003cli\u003eSpace complexity and how to think about memory usage\u003c/li\u003e\n\u003cli\u003eCommon misconceptions that trip up beginners\u003c/li\u003e\n\u003cli\u003ePractice problems to test your understanding\u003c/li\u003e\n\u003c/ul\u003e\n\u003chr\u003e\n\u003ch2 id=\"why-do-we-need-complexity-analysis\"\u003eWhy Do We Need Complexity Analysis\u003c/h2\u003e\n\u003cp\u003eBefore anything else, let us answer the most important question: why does this topic exist at all?\u003c/p\u003e","tags":["dsa"],"title":"Complexity Analysis  -  Time and Space"},{"content":"Control Flow in Python Making Decisions in Your Code \u0026ndash; if, elif, else, and Beyond Before We Begin \u0026ndash; Why Control Flow is Everything Every program you have ever used makes decisions.\nWhen you log into a website: if the password is correct, show the dashboard, otherwise show an error. When your food delivery app calculates the delivery fee: if the order is above Rs.200, delivery is free, else charge Rs.30. When a data pipeline processes records: if the record is valid, process it, else log an error and skip it.\nThese decisions \u0026ndash; branching based on conditions \u0026ndash; are called control flow. Without it, every program would execute the same instructions in the same order every time, regardless of the data or the situation. That is not a program. That is a calculator.\nControl flow is what makes programs intelligent.\nYou already know how to write conditions from the Python Basics guide: comparison operators, logical operators, truthiness. This guide is about using those conditions to direct your program\u0026rsquo;s execution.\nTable of Contents The if Statement if-else \u0026ndash; Two Paths if-elif-else \u0026ndash; Multiple Conditions Nested Conditions The Ternary Operator \u0026ndash; One-Line if-else Truthiness \u0026ndash; What Python Considers True or False Comparison and Logical Operators in Depth Common Patterns \u0026ndash; Guard Clauses and Early Returns match-case \u0026ndash; Structural Pattern Matching (Python 3.10+) Common Mistakes and How to Avoid Them Practice Questions Solutions Summary and Key Takeaways 1. The if Statement The if statement is the most fundamental control flow tool. It runs a block of code only when a condition is True.\n# Syntax if condition: # this block runs only when condition is True # it can be as many lines as you need # Simple example temperature = 38.5 if temperature \u0026gt; 37.5: print(\u0026#34;You have a fever.\u0026#34;) print(\u0026#34;Please rest and drink water.\u0026#34;) You have a fever. Please rest and drink water. If the condition is False, the block is skipped entirely:\ntemperature = 36.8 if temperature \u0026gt; 37.5: print(\u0026#34;You have a fever.\u0026#34;) # not printed -- condition is False print(\u0026#34;Condition check complete.\u0026#34;) # always runs Condition check complete. Indentation is not optional The indented block is how Python knows what belongs to the if statement. Four spaces is the standard. Never use tabs mixed with spaces.\nscore = 85 if score \u0026gt;= 60: print(\u0026#34;Passed\u0026#34;) # inside the if block (4 spaces) print(\u0026#34;Congratulations\u0026#34;) # also inside (same indentation) print(\u0026#34;End of check\u0026#34;) # outside the if block (no indentation) 2. if-else \u0026ndash; Two Paths else provides an alternative block that runs when the condition is False. One of the two blocks always runs. Never both, never neither.\nif condition: # runs when condition is True else: # runs when condition is False # Student grade check marks = 45 if marks \u0026gt;= 60: print(\u0026#34;Passed\u0026#34;) print(f\u0026#34;Your marks: {marks}/100\u0026#34;) else: print(\u0026#34;Failed\u0026#34;) print(f\u0026#34;You need {60 - marks} more marks to pass.\u0026#34;) Failed You need 15 more marks to pass. # Zomato delivery fee calculation order_amount = 180 if order_amount \u0026gt;= 200: delivery_fee = 0 else: delivery_fee = 30 print(f\u0026#34;Delivery fee: Rs.{delivery_fee}\u0026#34;) # Delivery fee: Rs.30 Every if-else is a choice, not a sequence A very common beginner mistake is writing two separate if statements when they mean if-else:\n# WRONG -- both conditions can be checked # even though they are mutually exclusive marks = 45 if marks \u0026gt;= 60: result = \u0026#34;Passed\u0026#34; if marks \u0026lt; 60: result = \u0026#34;Failed\u0026#34; # RIGHT -- exactly one block runs if marks \u0026gt;= 60: result = \u0026#34;Passed\u0026#34; else: result = \u0026#34;Failed\u0026#34; The second version is more efficient (only evaluates one condition), clearer to read, and correctly expresses the intent.\n3. if-elif-else \u0026ndash; Multiple Conditions When you have more than two possible outcomes, elif (short for \u0026ldquo;else if\u0026rdquo;) lets you chain conditions. Python evaluates them top to bottom and runs the first block whose condition is True. Once a match is found, all remaining elif and else blocks are skipped.\nif condition1: # runs if condition1 is True elif condition2: # runs if condition1 is False AND condition2 is True elif condition3: # runs if condition1 and condition2 are both False AND condition3 is True else: # runs if ALL conditions above are False # Grade classification marks = 82 if marks \u0026gt;= 90: grade = \u0026#34;A+\u0026#34; elif marks \u0026gt;= 80: grade = \u0026#34;A\u0026#34; elif marks \u0026gt;= 70: grade = \u0026#34;B\u0026#34; elif marks \u0026gt;= 60: grade = \u0026#34;C\u0026#34; elif marks \u0026gt;= 40: grade = \u0026#34;D\u0026#34; else: grade = \u0026#34;F\u0026#34; print(f\u0026#34;Marks: {marks} -- Grade: {grade}\u0026#34;) # Marks: 82 -- Grade: A Order matters Python stops at the first True condition. This means:\nPut the most specific conditions first Put the broadest conditions last # WRONG -- will always match the first condition # because \u0026gt;= 40 is True for 82 marks = 82 if marks \u0026gt;= 40: grade = \u0026#34;D\u0026#34; # this matches first! elif marks \u0026gt;= 60: grade = \u0026#34;C\u0026#34; elif marks \u0026gt;= 80: grade = \u0026#34;A\u0026#34; # never reached for marks=82 # RIGHT -- most specific (highest threshold) first if marks \u0026gt;= 80: grade = \u0026#34;A\u0026#34; # matches correctly elif marks \u0026gt;= 60: grade = \u0026#34;C\u0026#34; elif marks \u0026gt;= 40: grade = \u0026#34;D\u0026#34; Real example \u0026ndash; ticket pricing # Indian Railways ticket pricing by age age = 67 if age \u0026lt; 5: fare_category = \u0026#34;Free\u0026#34; discount = 100 elif age \u0026lt; 12: fare_category = \u0026#34;Child\u0026#34; discount = 50 elif age \u0026gt;= 60: fare_category = \u0026#34;Senior Citizen\u0026#34; discount = 40 else: fare_category = \u0026#34;Adult\u0026#34; discount = 0 base_fare = 850 final_fare = base_fare * (1 - discount / 100) print(f\u0026#34;Category : {fare_category}\u0026#34;) print(f\u0026#34;Discount : {discount}%\u0026#34;) print(f\u0026#34;Fare : Rs.{final_fare:.0f}\u0026#34;) Category : Senior Citizen Discount : 40% Fare : Rs.510 There can be many elifs but only one else # Day of week classifier day_num = 6 # Saturday if day_num == 1: day_name = \u0026#34;Monday\u0026#34; elif day_num == 2: day_name = \u0026#34;Tuesday\u0026#34; elif day_num == 3: day_name = \u0026#34;Wednesday\u0026#34; elif day_num == 4: day_name = \u0026#34;Thursday\u0026#34; elif day_num == 5: day_name = \u0026#34;Friday\u0026#34; elif day_num == 6: day_name = \u0026#34;Saturday\u0026#34; elif day_num == 7: day_name = \u0026#34;Sunday\u0026#34; else: day_name = \u0026#34;Invalid day number\u0026#34; print(day_name) # Saturday Note: for this specific case, match-case (covered in Section 9) or a dictionary lookup is cleaner than a long if-elif chain.\n4. Nested Conditions You can place if statements inside other if statements. This is called nesting. Use it when a second decision only makes sense if the first condition is True.\n# Login system is_registered = True entered_password = \u0026#34;india@123\u0026#34; correct_password = \u0026#34;india@123\u0026#34; is_active = True if is_registered: if entered_password == correct_password: if is_active: print(\u0026#34;Login successful. Welcome!\u0026#34;) else: print(\u0026#34;Account is deactivated. Contact support.\u0026#34;) else: print(\u0026#34;Wrong password. Please try again.\u0026#34;) else: print(\u0026#34;Account not found. Please register.\u0026#34;) Nesting depth \u0026ndash; when to stop Deep nesting is hard to read and maintain. As a rule of thumb: if you are nesting more than two levels deep, consider refactoring.\nThe guard clause pattern (covered in Section 8) is the standard way to reduce nesting.\n# Deep nesting -- hard to follow def process_order(order): if order is not None: if order[\u0026#34;status\u0026#34;] == \u0026#34;pending\u0026#34;: if order[\u0026#34;amount\u0026#34;] \u0026gt; 0: if order[\u0026#34;customer_id\u0026#34;] in active_customers: # finally do the actual work return process_payment(order) else: return \u0026#34;Invalid customer\u0026#34; else: return \u0026#34;Invalid amount\u0026#34; else: return \u0026#34;Order not pending\u0026#34; else: return \u0026#34;No order\u0026#34; # Flatter -- same logic, easier to read (covered properly in Section 8) def process_order(order): if order is None: return \u0026#34;No order\u0026#34; if order[\u0026#34;status\u0026#34;] != \u0026#34;pending\u0026#34;: return \u0026#34;Order not pending\u0026#34; if order[\u0026#34;amount\u0026#34;] \u0026lt;= 0: return \u0026#34;Invalid amount\u0026#34; if order[\u0026#34;customer_id\u0026#34;] not in active_customers: return \u0026#34;Invalid customer\u0026#34; return process_payment(order) 5. The Ternary Operator \u0026ndash; One-Line if-else Python has a one-line form of if-else called the conditional expression (commonly called the ternary operator). It is useful for simple assignments where both branches produce a single value.\n# Syntax value = expression_if_true if condition else expression_if_false # Traditional if-else marks = 72 if marks \u0026gt;= 60: result = \u0026#34;Pass\u0026#34; else: result = \u0026#34;Fail\u0026#34; # Ternary -- same result, one line result = \u0026#34;Pass\u0026#34; if marks \u0026gt;= 60 else \u0026#34;Fail\u0026#34; print(result) # Pass # Practical examples age = 22 category = \u0026#34;Adult\u0026#34; if age \u0026gt;= 18 else \u0026#34;Minor\u0026#34; temperature = 35 weather = \u0026#34;Hot\u0026#34; if temperature \u0026gt; 30 else \u0026#34;Comfortable\u0026#34; # With function calls name = \u0026#34; Aarav \u0026#34; clean_name = name.strip() if name else \u0026#34;Unknown\u0026#34; # In f-strings score = 88 print(f\u0026#34;Result: {\u0026#39;Pass\u0026#39; if score \u0026gt;= 60 else \u0026#39;Fail\u0026#39;}\u0026#34;) # Absolute value without abs() x = -42 absolute = x if x \u0026gt;= 0 else -x # Default value pattern config = {} timeout = config.get(\u0026#34;timeout\u0026#34;) if config else 30 When NOT to use ternary # Acceptable -- simple, readable status = \u0026#34;Active\u0026#34; if is_logged_in else \u0026#34;Inactive\u0026#34; # Too complex -- use regular if-else # This is hard to read result = (process_high(x) if x \u0026gt; 100 else process_low(x)) if x \u0026gt; 0 else handle_negative(x) # Much better as if-elif-else if x \u0026gt; 100: result = process_high(x) elif x \u0026gt; 0: result = process_low(x) else: result = handle_negative(x) 6. Truthiness \u0026ndash; What Python Considers True or False Python\u0026rsquo;s if statement does not require a strictly True or False value. Any value can be used as a condition. Python converts it to a boolean internally using the rules of truthiness.\nFalsy values -- evaluate to False in a boolean context: False None 0 (int zero) 0.0 (float zero) 0j (complex zero) \u0026#34;\u0026#34; (empty string) [] (empty list) () (empty tuple) {} (empty dict) set() (empty set) Everything else is Truthy -- evaluates to True. # You can use truthiness to check for empty/None values directly name = \u0026#34;\u0026#34; if name: print(f\u0026#34;Hello, {name}!\u0026#34;) else: print(\u0026#34;No name provided.\u0026#34;) # No name provided. -- because \u0026#34;\u0026#34; is falsy # Check if a list has items students = [] if students: print(f\u0026#34;Processing {len(students)} students\u0026#34;) else: print(\u0026#34;No students to process\u0026#34;) # Check if a value is not None result = None if result: print(f\u0026#34;Got result: {result}\u0026#34;) else: print(\u0026#34;No result yet\u0026#34;) Truthiness with numbers count = 0 if count: print(\u0026#34;Has items\u0026#34;) else: print(\u0026#34;Empty\u0026#34;) # 0 is falsy count = 5 if count: print(f\u0026#34;Has {count} items\u0026#34;) # 5 is truthy Be careful with 0 and False Truthiness can cause subtle bugs when you specifically need to allow 0:\n# BUG -- treats 0 as \u0026#34;no value\u0026#34; def set_timeout(seconds=None): timeout = seconds if seconds else 30 # BUG: if seconds=0, uses 30 return timeout print(set_timeout(0)) # 30 -- wrong! Should be 0. print(set_timeout(None)) # 30 -- correct # CORRECT -- check for None specifically def set_timeout(seconds=None): timeout = seconds if seconds is not None else 30 return timeout print(set_timeout(0)) # 0 -- correct print(set_timeout(None)) # 30 -- correct 7. Comparison and Logical Operators in Depth You covered these in Python Basics. Here we look at the less obvious behaviour that matters in real control flow.\nChained comparisons Python supports chaining comparisons in a way that is both readable and correct:\nmarks = 75 # Python (clean and correct) if 60 \u0026lt;= marks \u0026lt;= 100: print(\u0026#34;Valid passing marks\u0026#34;) # Other languages need this (verbose, also has a subtle bug) if marks \u0026gt;= 60 and marks \u0026lt;= 100: print(\u0026#34;Valid passing marks\u0026#34;) # The chained version is exactly equivalent to the \u0026#39;and\u0026#39; version # but more readable and more Pythonic # Multi-step range check temperature = 22 if 15 \u0026lt;= temperature \u0026lt;= 25: print(\u0026#34;Comfortable temperature\u0026#34;) elif 25 \u0026lt; temperature \u0026lt;= 35: print(\u0026#34;Warm\u0026#34;) elif temperature \u0026gt; 35: print(\u0026#34;Hot\u0026#34;) else: print(\u0026#34;Cold\u0026#34;) Short-circuit evaluation and and or stop evaluating as soon as the result is known. This has practical consequences:\n# \u0026#39;and\u0026#39; stops at the first False # \u0026#39;or\u0026#39; stops at the first True user = None # Safe -- if user is None (falsy), the second condition is never evaluated # so user[\u0026#34;name\u0026#34;] never raises a TypeError or KeyError if user and user[\u0026#34;name\u0026#34;] == \u0026#34;Aarav\u0026#34;: print(\u0026#34;Found Aarav\u0026#34;) # This uses short-circuit to provide defaults name = \u0026#34;\u0026#34; or \u0026#34;Anonymous\u0026#34; # \u0026#34;\u0026#34; is falsy, so \u0026#39;or\u0026#39; continues to \u0026#34;Anonymous\u0026#34; print(name) # Anonymous value = 42 or \u0026#34;default\u0026#34; # 42 is truthy, so \u0026#39;or\u0026#39; stops here print(value) # 42 # Common pattern: default value using \u0026#39;or\u0026#39; config_timeout = None timeout = config_timeout or 30 # if config_timeout is None/0/\u0026#34;\u0026#34;, use 30 is vs == # == checks VALUE equality # is checks IDENTITY (same object in memory) a = [1, 2, 3] b = [1, 2, 3] c = a print(a == b) # True -- same values print(a is b) # False -- different objects print(a is c) # True -- same object # Always use \u0026#39;is\u0026#39; for None checks result = None if result is None: # correct print(\u0026#34;No result\u0026#34;) if result == None: # works but not recommended (PEP 8) print(\u0026#34;No result\u0026#34;) # \u0026#39;is not\u0026#39; for the negative check if result is not None: print(f\u0026#34;Result: {result}\u0026#34;) not, and, or \u0026ndash; operator precedence # Precedence (highest to lowest): not, then and, then or x = True y = False z = True # This is evaluated as: (not x) and y or z # = False and False or True # = False or True # = True print(not x and y or z) # True # When in doubt, use parentheses to be explicit print((not x) and (y or z)) # False -- (not True) and (False or True) # = False and True = False in and not in # Membership testing -- works on lists, tuples, sets, strings, dicts (checks keys) cities = [\u0026#34;Delhi\u0026#34;, \u0026#34;Mumbai\u0026#34;, \u0026#34;Bangalore\u0026#34;, \u0026#34;Chennai\u0026#34;] city = \u0026#34;Mumbai\u0026#34; if city in cities: print(f\u0026#34;{city} is in our list\u0026#34;) if \u0026#34;Kolkata\u0026#34; not in cities: print(\u0026#34;Kolkata is not in our list\u0026#34;) # Checking against multiple values -- cleaner than multiple == comparisons status = \u0026#34;pending\u0026#34; # Instead of: if status == \u0026#34;pending\u0026#34; or status == \u0026#34;processing\u0026#34; or status == \u0026#34;queued\u0026#34;: print(\u0026#34;Order is in progress\u0026#34;) # Write: if status in {\u0026#34;pending\u0026#34;, \u0026#34;processing\u0026#34;, \u0026#34;queued\u0026#34;}: print(\u0026#34;Order is in progress\u0026#34;) # Use a set (curly braces) instead of a list for membership checks -- # sets have O(1) lookup while lists have O(n) lookup 8. Common Patterns \u0026ndash; Guard Clauses and Early Returns The guard clause pattern A guard clause is an early return (or raise) at the top of a function that handles edge cases before the main logic. It eliminates deep nesting and makes the happy path clear.\n# WITHOUT guard clauses -- deeply nested def process_payment(order, user, payment_method): if order is not None: if user is not None: if user[\u0026#34;is_active\u0026#34;]: if payment_method in [\u0026#34;UPI\u0026#34;, \u0026#34;NetBanking\u0026#34;, \u0026#34;Card\u0026#34;]: if order[\u0026#34;amount\u0026#34;] \u0026gt; 0: # actual work is buried five levels deep return charge_payment(order, payment_method) else: return {\u0026#34;error\u0026#34;: \u0026#34;Invalid amount\u0026#34;} else: return {\u0026#34;error\u0026#34;: \u0026#34;Invalid payment method\u0026#34;} else: return {\u0026#34;error\u0026#34;: \u0026#34;User account inactive\u0026#34;} else: return {\u0026#34;error\u0026#34;: \u0026#34;User not found\u0026#34;} else: return {\u0026#34;error\u0026#34;: \u0026#34;Order not found\u0026#34;} # WITH guard clauses -- flat and readable def process_payment(order, user, payment_method): # Guard clauses: handle all invalid cases first, return early if order is None: return {\u0026#34;error\u0026#34;: \u0026#34;Order not found\u0026#34;} if user is None: return {\u0026#34;error\u0026#34;: \u0026#34;User not found\u0026#34;} if not user[\u0026#34;is_active\u0026#34;]: return {\u0026#34;error\u0026#34;: \u0026#34;User account inactive\u0026#34;} if payment_method not in {\u0026#34;UPI\u0026#34;, \u0026#34;NetBanking\u0026#34;, \u0026#34;Card\u0026#34;}: return {\u0026#34;error\u0026#34;: \u0026#34;Invalid payment method\u0026#34;} if order[\u0026#34;amount\u0026#34;] \u0026lt;= 0: return {\u0026#34;error\u0026#34;: \u0026#34;Invalid amount\u0026#34;} # Happy path: all validations passed, do the actual work return charge_payment(order, payment_method) The second version is:\nEasier to read (the main logic is not buried) Easier to test (each guard can be tested independently) Easier to extend (add a new validation without touching indentation) Using guard clauses in loops # Processing a list of student records students = [ {\u0026#34;name\u0026#34;: \u0026#34;Aarav\u0026#34;, \u0026#34;marks\u0026#34;: 88, \u0026#34;active\u0026#34;: True}, {\u0026#34;name\u0026#34;: None, \u0026#34;marks\u0026#34;: 75, \u0026#34;active\u0026#34;: True}, # missing name {\u0026#34;name\u0026#34;: \u0026#34;Rohan\u0026#34;, \u0026#34;marks\u0026#34;: -5, \u0026#34;active\u0026#34;: True}, # invalid marks {\u0026#34;name\u0026#34;: \u0026#34;Sneha\u0026#34;, \u0026#34;marks\u0026#34;: 91, \u0026#34;active\u0026#34;: False}, # inactive {\u0026#34;name\u0026#34;: \u0026#34;Karan\u0026#34;, \u0026#34;marks\u0026#34;: 72, \u0026#34;active\u0026#34;: True}, ] results = [] for student in students: # Guard clauses -- skip invalid records with continue if not student.get(\u0026#34;name\u0026#34;): print(\u0026#34;Skipping record with missing name\u0026#34;) continue if student[\u0026#34;marks\u0026#34;] \u0026lt; 0 or student[\u0026#34;marks\u0026#34;] \u0026gt; 100: print(f\u0026#34;Invalid marks for {student[\u0026#39;name\u0026#39;]}: {student[\u0026#39;marks\u0026#39;]}\u0026#34;) continue if not student[\u0026#34;active\u0026#34;]: print(f\u0026#34;Skipping inactive student: {student[\u0026#39;name\u0026#39;]}\u0026#34;) continue # Happy path grade = \u0026#34;Pass\u0026#34; if student[\u0026#34;marks\u0026#34;] \u0026gt;= 60 else \u0026#34;Fail\u0026#34; results.append({\u0026#34;name\u0026#34;: student[\u0026#34;name\u0026#34;], \u0026#34;grade\u0026#34;: grade}) print(results) 9. match-case \u0026ndash; Structural Pattern Matching (Python 3.10+) match-case was introduced in Python 3.10. It is Python\u0026rsquo;s version of the switch-case statement found in other languages, but significantly more powerful because it can match on structure, not just values.\nBasic value matching # The old way -- long if-elif chain for simple value matching command = \u0026#34;quit\u0026#34; if command == \u0026#34;start\u0026#34;: print(\u0026#34;Starting...\u0026#34;) elif command == \u0026#34;stop\u0026#34;: print(\u0026#34;Stopping...\u0026#34;) elif command == \u0026#34;quit\u0026#34;: print(\u0026#34;Quitting...\u0026#34;) else: print(f\u0026#34;Unknown command: {command}\u0026#34;) # match-case -- cleaner for this kind of logic command = \u0026#34;quit\u0026#34; match command: case \u0026#34;start\u0026#34;: print(\u0026#34;Starting...\u0026#34;) case \u0026#34;stop\u0026#34;: print(\u0026#34;Stopping...\u0026#34;) case \u0026#34;quit\u0026#34;: print(\u0026#34;Quitting...\u0026#34;) case _: # _ is the wildcard -- matches anything print(f\u0026#34;Unknown command: {command}\u0026#34;) Matching multiple values in one case status_code = 404 match status_code: case 200 | 201 | 204: print(\u0026#34;Success\u0026#34;) case 400 | 422: print(\u0026#34;Client error -- check your request\u0026#34;) case 401 | 403: print(\u0026#34;Authentication/authorisation error\u0026#34;) case 404: print(\u0026#34;Resource not found\u0026#34;) case 500 | 502 | 503: print(\u0026#34;Server error -- try again later\u0026#34;) case _: print(f\u0026#34;Unexpected status: {status_code}\u0026#34;) Matching with a guard (if condition inside case) score = 72 match score: case n if n \u0026gt;= 90: grade = \u0026#34;A+\u0026#34; case n if n \u0026gt;= 80: grade = \u0026#34;A\u0026#34; case n if n \u0026gt;= 70: grade = \u0026#34;B\u0026#34; case n if n \u0026gt;= 60: grade = \u0026#34;C\u0026#34; case _: grade = \u0026#34;F\u0026#34; print(f\u0026#34;Score: {score} -- Grade: {grade}\u0026#34;) Matching sequences (lists and tuples) # match-case can destructure sequences point = (3, 4) match point: case (0, 0): print(\u0026#34;Origin\u0026#34;) case (x, 0): print(f\u0026#34;On X-axis at {x}\u0026#34;) case (0, y): print(f\u0026#34;On Y-axis at {y}\u0026#34;) case (x, y): print(f\u0026#34;Point at ({x}, {y})\u0026#34;) # Matching commands with arguments command = [\u0026#34;move\u0026#34;, \u0026#34;north\u0026#34;, \u0026#34;10\u0026#34;] match command: case [\u0026#34;quit\u0026#34;]: print(\u0026#34;Quitting game\u0026#34;) case [\u0026#34;move\u0026#34;, direction, steps]: print(f\u0026#34;Moving {direction} by {steps} steps\u0026#34;) case [\u0026#34;attack\u0026#34;, target]: print(f\u0026#34;Attacking {target}\u0026#34;) case [\u0026#34;help\u0026#34;]: print(\u0026#34;Showing help\u0026#34;) case _: print(f\u0026#34;Unknown command: {command}\u0026#34;) Matching dictionaries # Matching based on the presence and values of dict keys event = {\u0026#34;type\u0026#34;: \u0026#34;order_placed\u0026#34;, \u0026#34;amount\u0026#34;: 5000, \u0026#34;city\u0026#34;: \u0026#34;Mumbai\u0026#34;} match event: case {\u0026#34;type\u0026#34;: \u0026#34;order_placed\u0026#34;, \u0026#34;amount\u0026#34;: amount} if amount \u0026gt; 10000: print(f\u0026#34;Large order placed: Rs.{amount}\u0026#34;) case {\u0026#34;type\u0026#34;: \u0026#34;order_placed\u0026#34;, \u0026#34;amount\u0026#34;: amount}: print(f\u0026#34;Order placed: Rs.{amount}\u0026#34;) case {\u0026#34;type\u0026#34;: \u0026#34;order_cancelled\u0026#34;, \u0026#34;reason\u0026#34;: reason}: print(f\u0026#34;Order cancelled: {reason}\u0026#34;) case {\u0026#34;type\u0026#34;: event_type}: print(f\u0026#34;Unknown event type: {event_type}\u0026#34;) Matching class instances from dataclasses import dataclass @dataclass class Point: x: float y: float @dataclass class Circle: center: Point radius: float shape = Circle(center=Point(0, 0), radius=5) match shape: case Circle(center=Point(x=0, y=0), radius=r): print(f\u0026#34;Circle centred at origin with radius {r}\u0026#34;) case Circle(center=Point(x=cx, y=cy), radius=r): print(f\u0026#34;Circle at ({cx}, {cy}) with radius {r}\u0026#34;) case Point(x=x, y=y): print(f\u0026#34;A point at ({x}, {y})\u0026#34;) When to use match-case vs if-elif Use match-case when: - Matching a single value against multiple specific values - Matching the structure of sequences or dicts - Destructuring data as part of the match Use if-elif when: - Conditions involve complex expressions or multiple variables - You need range checks (\u0026gt;= 60 and \u0026lt; 80) - Running Python \u0026lt; 3.10 - The conditions are fundamentally different in type 10. Common Mistakes and How to Avoid Them Mistake 1 \u0026ndash; Assignment inside a condition (= instead of ==) score = 85 # WRONG -- assigns 0 to score, condition is always False (0 is falsy) if score = 0: # SyntaxError in Python -- Python catches this one print(\u0026#34;Zero\u0026#34;) # RIGHT if score == 0: print(\u0026#34;Zero\u0026#34;) # Note: Python 3.8+ introduced the walrus operator := for intentional # assignment inside a condition -- covered briefly below Mistake 2 \u0026ndash; Checking a boolean variable redundantly is_logged_in = True # Redundant -- already a boolean if is_logged_in == True: print(\u0026#34;Welcome\u0026#34;) # Correct -- just use the boolean directly if is_logged_in: print(\u0026#34;Welcome\u0026#34;) # For False check if not is_logged_in: print(\u0026#34;Please log in\u0026#34;) Mistake 3 \u0026ndash; Forgetting that elif stops checking score = 72 # WRONG -- this prints two things because there are two separate if statements if score \u0026gt;= 60: print(\u0026#34;Passed\u0026#34;) if score \u0026gt;= 70: print(\u0026#34;Good performance\u0026#34;) # also prints! two separate conditions # RIGHT -- use elif so only one branch runs if score \u0026gt;= 70: print(\u0026#34;Good performance\u0026#34;) elif score \u0026gt;= 60: print(\u0026#34;Passed\u0026#34;) else: print(\u0026#34;Failed\u0026#34;) Mistake 4 \u0026ndash; Confusing \u0026lsquo;is\u0026rsquo; and \u0026lsquo;==\u0026rsquo; # == for value comparison a = [1, 2, 3] b = [1, 2, 3] print(a == b) # True -- same values # is for identity (same object) print(a is b) # False -- different objects # is for None (always correct) result = None print(result is None) # True -- correct print(result == None) # True -- works but not recommended Mistake 5 \u0026ndash; Not handling the else case # Can lead to NameError if no condition matches score = 105 # invalid if score \u0026gt;= 90: grade = \u0026#34;A+\u0026#34; elif score \u0026gt;= 80: grade = \u0026#34;A\u0026#34; # No else -- if score is 105, \u0026#39;grade\u0026#39; is never set print(grade) # NameError: name \u0026#39;grade\u0026#39; is not defined # Always handle the fallthrough case if score \u0026gt;= 90: grade = \u0026#34;A+\u0026#34; elif score \u0026gt;= 80: grade = \u0026#34;A\u0026#34; else: grade = \u0026#34;Invalid\u0026#34; # or raise ValueError(f\u0026#34;Invalid score: {score}\u0026#34;) The walrus operator := (Python 3.8+) The walrus operator assigns a value AND uses it in the same expression. Useful for avoiding redundant function calls.\nimport re # Without walrus -- calls match() twice if re.match(r\u0026#34;\\d{10}\u0026#34;, phone_number): match = re.match(r\u0026#34;\\d{10}\u0026#34;, phone_number) print(f\u0026#34;Valid number: {match.group()}\u0026#34;) # With walrus -- assign and check in one step if match := re.match(r\u0026#34;\\d{10}\u0026#34;, phone_number): print(f\u0026#34;Valid number: {match.group()}\u0026#34;) # Another common use -- reading from a file or stream with open(\u0026#34;data.txt\u0026#34;) as f: while line := f.readline(): # assign and check in one step print(line.strip()) # loop ends when readline() returns \u0026#34;\u0026#34; (empty string = falsy) 11. Practice Questions Try each question before reading the solution. Questions are ordered: Easy (1-5), Medium (6-10), Hard (11-15).\nEasy Q1 \u0026ndash; Basic if-else Write a function check_eligibility(age) that returns:\n\u0026quot;Eligible to vote\u0026quot; if age is 18 or above \u0026quot;Not eligible to vote\u0026quot; if age is below 18 Test with ages: 15, 18, 25, 17.\nQ2 \u0026ndash; if-elif-else Write a function bmi_category(bmi) that returns the category:\nBelow 18.5: \u0026quot;Underweight\u0026quot; 18.5 to 24.9: \u0026quot;Normal\u0026quot; 25.0 to 29.9: \u0026quot;Overweight\u0026quot; 30 and above: \u0026quot;Obese\u0026quot; Test with: 17.2, 22.5, 27.8, 35.1\nQ3 \u0026ndash; Ternary operator Rewrite the following using a ternary expression:\nnumber = -7 if number \u0026gt;= 0: sign = \u0026#34;positive\u0026#34; else: sign = \u0026#34;negative\u0026#34; Then use it to print: \u0026quot;-7 is negative\u0026quot;\nQ4 \u0026ndash; Truthiness Given the dictionary below, use truthiness (not explicit == None or == \u0026quot;\u0026quot; checks) to print only the fields that have values:\nprofile = { \u0026#34;name\u0026#34;: \u0026#34;Aarav Sharma\u0026#34;, \u0026#34;email\u0026#34;: \u0026#34;\u0026#34;, \u0026#34;phone\u0026#34;: None, \u0026#34;city\u0026#34;: \u0026#34;Bangalore\u0026#34;, \u0026#34;website\u0026#34;: \u0026#34;\u0026#34;, \u0026#34;company\u0026#34;: \u0026#34;Codeverra\u0026#34;, } Expected output:\nname : Aarav Sharma city : Bangalore company : Codeverra Q5 \u0026ndash; Nested conditions Write a function classify_triangle(a, b, c) that:\nFirst checks if the three sides form a valid triangle (each side must be less than the sum of the other two) If valid, classifies it as: \u0026quot;Equilateral\u0026quot; if all sides are equal \u0026quot;Isosceles\u0026quot; if exactly two sides are equal \u0026quot;Scalene\u0026quot; if all sides are different classify_triangle(3, 3, 3) # Equilateral classify_triangle(5, 5, 8) # Isosceles classify_triangle(3, 4, 5) # Scalene (also a right triangle) classify_triangle(1, 2, 10) # Not a valid triangle Medium Q6 \u0026ndash; Guard clauses Rewrite the following function using guard clauses to eliminate the nesting:\ndef process_student(student): if student is not None: if isinstance(student, dict): if \u0026#34;name\u0026#34; in student: if \u0026#34;marks\u0026#34; in student: if 0 \u0026lt;= student[\u0026#34;marks\u0026#34;] \u0026lt;= 100: return f\u0026#34;{student[\u0026#39;name\u0026#39;]}: {\u0026#39;Pass\u0026#39; if student[\u0026#39;marks\u0026#39;] \u0026gt;= 60 else \u0026#39;Fail\u0026#39;}\u0026#34; else: return \u0026#34;Invalid marks\u0026#34; else: return \u0026#34;Missing marks\u0026#34; else: return \u0026#34;Missing name\u0026#34; else: return \u0026#34;Invalid type\u0026#34; else: return \u0026#34;No student\u0026#34; Q7 \u0026ndash; Membership and logical operators Write a function validate_password(password) that checks all of:\nAt least 8 characters long Contains at least one uppercase letter Contains at least one lowercase letter Contains at least one digit Contains at least one special character from !@#$%^\u0026amp;*() Return a list of all failed requirements. If all pass, return an empty list.\nvalidate_password(\u0026#34;abc\u0026#34;) # [\u0026#34;Too short (min 8 chars)\u0026#34;, \u0026#34;No uppercase letter\u0026#34;, \u0026#34;No digit\u0026#34;, \u0026#34;No special character\u0026#34;] validate_password(\u0026#34;Codeverra1!\u0026#34;) # [] (all requirements met) Q8 \u0026ndash; match-case Write a function process_command(command) using match-case that handles a list-based command format:\nprocess_command([\u0026#34;help\u0026#34;]) # \u0026#34;Showing available commands\u0026#34; process_command([\u0026#34;add\u0026#34;, \u0026#34;Aarav\u0026#34;, \u0026#34;88\u0026#34;]) # \u0026#34;Added student Aarav with marks 88\u0026#34; process_command([\u0026#34;update\u0026#34;, \u0026#34;Aarav\u0026#34;, \u0026#34;92\u0026#34;]) # \u0026#34;Updated Aarav\u0026#39;s marks to 92\u0026#34; process_command([\u0026#34;delete\u0026#34;, \u0026#34;Aarav\u0026#34;]) # \u0026#34;Deleted student Aarav\u0026#34; process_command([\u0026#34;search\u0026#34;, \u0026#34;Priya\u0026#34;]) # \u0026#34;Searching for Priya\u0026#34; process_command([\u0026#34;export\u0026#34;, \u0026#34;csv\u0026#34;, \u0026#34;output.csv\u0026#34;]) # \u0026#34;Exporting to CSV: output.csv\u0026#34; process_command([\u0026#34;unknown\u0026#34;]) # \u0026#34;Unknown command: unknown\u0026#34; Q9 \u0026ndash; Short-circuit evaluation Explain what each of the following prints and why:\nx = 0 y = 10 print(x and y) print(x or y) print(not x) print(x or \u0026#34;default\u0026#34;) print(y and \u0026#34;found\u0026#34;) print(None or [] or 0 or \u0026#34;fallback\u0026#34;) Write your predictions before running the code. Then verify.\nQ10 \u0026ndash; Complete control flow Write a function calculate_electricity_bill(units) that calculates a monthly electricity bill based on units consumed, using these slabs (similar to Indian utility billing):\nFirst 100 units : Rs.3.50 per unit 101 to 200 units : Rs.4.50 per unit (for units above 100) 201 to 300 units : Rs.6.00 per unit (for units above 200) Above 300 units : Rs.8.00 per unit (for units above 300) Fixed charge : Rs.50 If total \u0026gt; Rs.500 : add 10% surcharge Test with: 80, 150, 250, 400 units.\nHard Q11 \u0026ndash; Walrus operator Rewrite the following loop using the walrus operator:\nimport re messages = [ \u0026#34;Call me at 9876543210\u0026#34;, \u0026#34;No number here\u0026#34;, \u0026#34;Contact: 8765432109\u0026#34;, \u0026#34;Email only\u0026#34;, \u0026#34;Reach out: 7654321098\u0026#34;, ] for msg in messages: match = re.search(r\u0026#34;\\b[6-9]\\d{9}\\b\u0026#34;, msg) if match: print(f\u0026#34;Found number: {match.group()} in: {msg[:30]}\u0026#34;) Q12 \u0026ndash; match-case with data structures You receive API responses as dictionaries. Write a function handle_api_response(response) using match-case that:\n# Success with data {\u0026#34;status\u0026#34;: \u0026#34;success\u0026#34;, \u0026#34;data\u0026#34;: {\u0026#34;users\u0026#34;: [...]}, \u0026#34;count\u0026#34;: 42} # Output: \u0026#34;Success: 42 users returned\u0026#34; # Success but empty {\u0026#34;status\u0026#34;: \u0026#34;success\u0026#34;, \u0026#34;data\u0026#34;: {}, \u0026#34;count\u0026#34;: 0} # Output: \u0026#34;Success but no data returned\u0026#34; # Error with code {\u0026#34;status\u0026#34;: \u0026#34;error\u0026#34;, \u0026#34;code\u0026#34;: 404, \u0026#34;message\u0026#34;: \u0026#34;Not found\u0026#34;} # Output: \u0026#34;Error 404: Not found\u0026#34; # Error without code {\u0026#34;status\u0026#34;: \u0026#34;error\u0026#34;, \u0026#34;message\u0026#34;: \u0026#34;Server error\u0026#34;} # Output: \u0026#34;Error: Server error\u0026#34; # Unexpected structure anything_else # Output: \u0026#34;Unexpected response format\u0026#34; Q13 \u0026ndash; Full validation pipeline Write a function validate_order(order) that validates an e-commerce order dictionary and returns a tuple of (is_valid, list_of_errors).\n# Valid order order = { \u0026#34;order_id\u0026#34;: \u0026#34;ORD001\u0026#34;, \u0026#34;customer_id\u0026#34;: \u0026#34;CUST123\u0026#34;, \u0026#34;items\u0026#34;: [{\u0026#34;product\u0026#34;: \u0026#34;Laptop\u0026#34;, \u0026#34;qty\u0026#34;: 1, \u0026#34;price\u0026#34;: 65000}], \u0026#34;city\u0026#34;: \u0026#34;Bangalore\u0026#34;, \u0026#34;pincode\u0026#34;: \u0026#34;560001\u0026#34;, \u0026#34;payment\u0026#34;: \u0026#34;UPI\u0026#34;, } # Required checks: # - order_id must be non-empty string # - customer_id must be non-empty string # - items must be a non-empty list # - each item must have product (str), qty (int \u0026gt;= 1), price (float \u0026gt; 0) # - city must be a non-empty string # - pincode must be exactly 6 digits # - payment must be one of: UPI, Card, NetBanking, COD Q14 \u0026ndash; Dynamic condition builder Write a function filter_students(students, **criteria) that accepts a list of student dicts and any combination of filter criteria as keyword arguments. It should apply all provided criteria and return matching students.\nstudents = [ {\u0026#34;name\u0026#34;: \u0026#34;Aarav\u0026#34;, \u0026#34;city\u0026#34;: \u0026#34;Delhi\u0026#34;, \u0026#34;marks\u0026#34;: 88, \u0026#34;dept\u0026#34;: \u0026#34;CS\u0026#34;}, {\u0026#34;name\u0026#34;: \u0026#34;Priya\u0026#34;, \u0026#34;city\u0026#34;: \u0026#34;Mumbai\u0026#34;, \u0026#34;marks\u0026#34;: 92, \u0026#34;dept\u0026#34;: \u0026#34;IT\u0026#34;}, {\u0026#34;name\u0026#34;: \u0026#34;Rohan\u0026#34;, \u0026#34;city\u0026#34;: \u0026#34;Delhi\u0026#34;, \u0026#34;marks\u0026#34;: 65, \u0026#34;dept\u0026#34;: \u0026#34;CS\u0026#34;}, {\u0026#34;name\u0026#34;: \u0026#34;Sneha\u0026#34;, \u0026#34;city\u0026#34;: \u0026#34;Bangalore\u0026#34;, \u0026#34;marks\u0026#34;: 95, \u0026#34;dept\u0026#34;: \u0026#34;IT\u0026#34;}, ] filter_students(students, city=\u0026#34;Delhi\u0026#34;) # [Aarav, Rohan] filter_students(students, dept=\u0026#34;CS\u0026#34;, min_marks=80) # [Aarav] -- Rohan has CS but marks=65 \u0026lt; 80 filter_students(students, min_marks=90) # [Priya, Sneha] Q15 \u0026ndash; State machine Implement a simple order status state machine. An order can only transition between states in specific ways:\nValid transitions: Pending --\u0026gt; Confirmed Pending --\u0026gt; Cancelled Confirmed --\u0026gt; Processing Confirmed --\u0026gt; Cancelled Processing --\u0026gt; Shipped Shipped --\u0026gt; Delivered Shipped --\u0026gt; Returned Write a class Order with a method transition(new_status) that:\nAllows the transition if it is valid Raises ValueError with a clear message if not Records the full history of transitions 12. Solutions Q1 def check_eligibility(age): return \u0026#34;Eligible to vote\u0026#34; if age \u0026gt;= 18 else \u0026#34;Not eligible to vote\u0026#34; for age in [15, 18, 25, 17]: print(f\u0026#34;Age {age}: {check_eligibility(age)}\u0026#34;) Q2 def bmi_category(bmi): if bmi \u0026lt; 18.5: return \u0026#34;Underweight\u0026#34; elif bmi \u0026lt; 25.0: return \u0026#34;Normal\u0026#34; elif bmi \u0026lt; 30.0: return \u0026#34;Overweight\u0026#34; else: return \u0026#34;Obese\u0026#34; for bmi in [17.2, 22.5, 27.8, 35.1]: print(f\u0026#34;BMI {bmi}: {bmi_category(bmi)}\u0026#34;) Q3 number = -7 sign = \u0026#34;positive\u0026#34; if number \u0026gt;= 0 else \u0026#34;negative\u0026#34; print(f\u0026#34;{number} is {sign}\u0026#34;) Q4 profile = { \u0026#34;name\u0026#34;: \u0026#34;Aarav Sharma\u0026#34;, \u0026#34;email\u0026#34;: \u0026#34;\u0026#34;, \u0026#34;phone\u0026#34;: None, \u0026#34;city\u0026#34;: \u0026#34;Bangalore\u0026#34;, \u0026#34;website\u0026#34;: \u0026#34;\u0026#34;, \u0026#34;company\u0026#34;: \u0026#34;Codeverra\u0026#34;, } for field, value in profile.items(): if value: # truthiness check -- empty string and None are falsy print(f\u0026#34;{field:\u0026lt;10}: {value}\u0026#34;) Q5 def classify_triangle(a, b, c): # Guard clause: validity check first if not (a + b \u0026gt; c and b + c \u0026gt; a and a + c \u0026gt; b): return \u0026#34;Not a valid triangle\u0026#34; if a == b == c: return \u0026#34;Equilateral\u0026#34; elif a == b or b == c or a == c: return \u0026#34;Isosceles\u0026#34; else: return \u0026#34;Scalene\u0026#34; print(classify_triangle(3, 3, 3)) # Equilateral print(classify_triangle(5, 5, 8)) # Isosceles print(classify_triangle(3, 4, 5)) # Scalene print(classify_triangle(1, 2, 10)) # Not a valid triangle Q6 def process_student(student): if student is None: return \u0026#34;No student\u0026#34; if not isinstance(student, dict): return \u0026#34;Invalid type\u0026#34; if \u0026#34;name\u0026#34; not in student: return \u0026#34;Missing name\u0026#34; if \u0026#34;marks\u0026#34; not in student: return \u0026#34;Missing marks\u0026#34; if not (0 \u0026lt;= student[\u0026#34;marks\u0026#34;] \u0026lt;= 100): return \u0026#34;Invalid marks\u0026#34; result = \u0026#34;Pass\u0026#34; if student[\u0026#34;marks\u0026#34;] \u0026gt;= 60 else \u0026#34;Fail\u0026#34; return f\u0026#34;{student[\u0026#39;name\u0026#39;]}: {result}\u0026#34; Q7 def validate_password(password): errors = [] special = set(\u0026#34;!@#$%^\u0026amp;*()\u0026#34;) if len(password) \u0026lt; 8: errors.append(\u0026#34;Too short (min 8 chars)\u0026#34;) if not any(c.isupper() for c in password): errors.append(\u0026#34;No uppercase letter\u0026#34;) if not any(c.islower() for c in password): errors.append(\u0026#34;No lowercase letter\u0026#34;) if not any(c.isdigit() for c in password): errors.append(\u0026#34;No digit\u0026#34;) if not any(c in special for c in password): errors.append(\u0026#34;No special character\u0026#34;) return errors print(validate_password(\u0026#34;abc\u0026#34;)) print(validate_password(\u0026#34;Codeverra1!\u0026#34;)) Q8 def process_command(command): match command: case [\u0026#34;help\u0026#34;]: return \u0026#34;Showing available commands\u0026#34; case [\u0026#34;add\u0026#34;, name, marks]: return f\u0026#34;Added student {name} with marks {marks}\u0026#34; case [\u0026#34;update\u0026#34;, name, marks]: return f\u0026#34;Updated {name}\u0026#39;s marks to {marks}\u0026#34; case [\u0026#34;delete\u0026#34;, name]: return f\u0026#34;Deleted student {name}\u0026#34; case [\u0026#34;search\u0026#34;, name]: return f\u0026#34;Searching for {name}\u0026#34; case [\u0026#34;export\u0026#34;, fmt, filename]: return f\u0026#34;Exporting to {fmt.upper()}: {filename}\u0026#34; case [cmd, *_]: return f\u0026#34;Unknown command: {cmd}\u0026#34; Q9 x = 0 y = 10 print(x and y) # 0 -- x is falsy, \u0026#39;and\u0026#39; returns x print(x or y) # 10 -- x is falsy, \u0026#39;or\u0026#39; continues and returns y print(not x) # True -- not 0 is True print(x or \u0026#34;default\u0026#34;) # \u0026#34;default\u0026#34; -- x is falsy, \u0026#39;or\u0026#39; returns \u0026#34;default\u0026#34; print(y and \u0026#34;found\u0026#34;) # \u0026#34;found\u0026#34; -- y is truthy, \u0026#39;and\u0026#39; returns the second value print(None or [] or 0 or \u0026#34;fallback\u0026#34;) # \u0026#34;fallback\u0026#34; -- all before it are falsy Key insight: and and or do not return True/False \u0026ndash; they return one of their operands. and returns the first falsy value (or the last value if all are truthy). or returns the first truthy value (or the last value if all are falsy).\nQ10 def calculate_electricity_bill(units): if units \u0026lt; 0: raise ValueError(\u0026#34;Units cannot be negative\u0026#34;) cost = 50 # fixed charge if units \u0026lt;= 100: cost += units * 3.50 elif units \u0026lt;= 200: cost += 100 * 3.50 + (units - 100) * 4.50 elif units \u0026lt;= 300: cost += 100 * 3.50 + 100 * 4.50 + (units - 200) * 6.00 else: cost += 100 * 3.50 + 100 * 4.50 + 100 * 6.00 + (units - 300) * 8.00 if cost \u0026gt; 500: cost *= 1.10 # 10% surcharge return round(cost, 2) for units in [80, 150, 250, 400]: bill = calculate_electricity_bill(units) print(f\u0026#34;{units:\u0026gt;4} units: Rs.{bill:,.2f}\u0026#34;) Q11 import re messages = [ \u0026#34;Call me at 9876543210\u0026#34;, \u0026#34;No number here\u0026#34;, \u0026#34;Contact: 8765432109\u0026#34;, \u0026#34;Email only\u0026#34;, \u0026#34;Reach out: 7654321098\u0026#34;, ] for msg in messages: if match := re.search(r\u0026#34;\\b[6-9]\\d{9}\\b\u0026#34;, msg): print(f\u0026#34;Found number: {match.group()} in: {msg[:30]}\u0026#34;) Q12 def handle_api_response(response): match response: case {\u0026#34;status\u0026#34;: \u0026#34;success\u0026#34;, \u0026#34;count\u0026#34;: count} if count \u0026gt; 0: return f\u0026#34;Success: {count} users returned\u0026#34; case {\u0026#34;status\u0026#34;: \u0026#34;success\u0026#34;}: return \u0026#34;Success but no data returned\u0026#34; case {\u0026#34;status\u0026#34;: \u0026#34;error\u0026#34;, \u0026#34;code\u0026#34;: code, \u0026#34;message\u0026#34;: msg}: return f\u0026#34;Error {code}: {msg}\u0026#34; case {\u0026#34;status\u0026#34;: \u0026#34;error\u0026#34;, \u0026#34;message\u0026#34;: msg}: return f\u0026#34;Error: {msg}\u0026#34; case _: return \u0026#34;Unexpected response format\u0026#34; Q13 def validate_order(order): errors = [] valid_payments = {\u0026#34;UPI\u0026#34;, \u0026#34;Card\u0026#34;, \u0026#34;NetBanking\u0026#34;, \u0026#34;COD\u0026#34;} if not order.get(\u0026#34;order_id\u0026#34;) or not isinstance(order[\u0026#34;order_id\u0026#34;], str): errors.append(\u0026#34;order_id must be a non-empty string\u0026#34;) if not order.get(\u0026#34;customer_id\u0026#34;) or not isinstance(order[\u0026#34;customer_id\u0026#34;], str): errors.append(\u0026#34;customer_id must be a non-empty string\u0026#34;) items = order.get(\u0026#34;items\u0026#34;, []) if not items or not isinstance(items, list): errors.append(\u0026#34;items must be a non-empty list\u0026#34;) else: for i, item in enumerate(items): if not isinstance(item.get(\u0026#34;product\u0026#34;), str) or not item[\u0026#34;product\u0026#34;]: errors.append(f\u0026#34;Item {i}: product must be a non-empty string\u0026#34;) if not isinstance(item.get(\u0026#34;qty\u0026#34;), int) or item.get(\u0026#34;qty\u0026#34;, 0) \u0026lt; 1: errors.append(f\u0026#34;Item {i}: qty must be an integer \u0026gt;= 1\u0026#34;) if not isinstance(item.get(\u0026#34;price\u0026#34;), (int, float)) or item.get(\u0026#34;price\u0026#34;, 0) \u0026lt;= 0: errors.append(f\u0026#34;Item {i}: price must be a positive number\u0026#34;) if not order.get(\u0026#34;city\u0026#34;) or not isinstance(order[\u0026#34;city\u0026#34;], str): errors.append(\u0026#34;city must be a non-empty string\u0026#34;) pincode = str(order.get(\u0026#34;pincode\u0026#34;, \u0026#34;\u0026#34;)) if not pincode.isdigit() or len(pincode) != 6: errors.append(\u0026#34;pincode must be exactly 6 digits\u0026#34;) if order.get(\u0026#34;payment\u0026#34;) not in valid_payments: errors.append(f\u0026#34;payment must be one of: {valid_payments}\u0026#34;) return (len(errors) == 0, errors) is_valid, errors = validate_order({\u0026#34;order_id\u0026#34;: \u0026#34;ORD001\u0026#34;, \u0026#34;customer_id\u0026#34;: \u0026#34;C1\u0026#34;, \u0026#34;items\u0026#34;: [{\u0026#34;product\u0026#34;: \u0026#34;Laptop\u0026#34;, \u0026#34;qty\u0026#34;: 1, \u0026#34;price\u0026#34;: 65000}], \u0026#34;city\u0026#34;: \u0026#34;Bangalore\u0026#34;, \u0026#34;pincode\u0026#34;: \u0026#34;560001\u0026#34;, \u0026#34;payment\u0026#34;: \u0026#34;UPI\u0026#34;}) print(is_valid, errors) Q14 def filter_students(students, **criteria): results = [] for student in students: match = True for key, value in criteria.items(): if key == \u0026#34;min_marks\u0026#34;: if student.get(\u0026#34;marks\u0026#34;, 0) \u0026lt; value: match = False break elif key == \u0026#34;max_marks\u0026#34;: if student.get(\u0026#34;marks\u0026#34;, 0) \u0026gt; value: match = False break else: if student.get(key) != value: match = False break if match: results.append(student) return results Q15 class Order: TRANSITIONS = { \u0026#34;Pending\u0026#34;: {\u0026#34;Confirmed\u0026#34;, \u0026#34;Cancelled\u0026#34;}, \u0026#34;Confirmed\u0026#34;: {\u0026#34;Processing\u0026#34;, \u0026#34;Cancelled\u0026#34;}, \u0026#34;Processing\u0026#34;: {\u0026#34;Shipped\u0026#34;}, \u0026#34;Shipped\u0026#34;: {\u0026#34;Delivered\u0026#34;, \u0026#34;Returned\u0026#34;}, \u0026#34;Delivered\u0026#34;: set(), \u0026#34;Cancelled\u0026#34;: set(), \u0026#34;Returned\u0026#34;: set(), } def __init__(self, order_id): self.order_id = order_id self.status = \u0026#34;Pending\u0026#34; self.history = [\u0026#34;Pending\u0026#34;] def transition(self, new_status): allowed = self.TRANSITIONS.get(self.status, set()) if new_status not in allowed: raise ValueError( f\u0026#34;Cannot transition from \u0026#39;{self.status}\u0026#39; to \u0026#39;{new_status}\u0026#39;. \u0026#34; f\u0026#34;Allowed: {allowed if allowed else \u0026#39;none (terminal state)\u0026#39;}\u0026#34; ) self.status = new_status self.history.append(new_status) print(f\u0026#34;Order {self.order_id}: {\u0026#39; -\u0026gt; \u0026#39;.join(self.history)}\u0026#34;) order = Order(\u0026#34;ORD001\u0026#34;) order.transition(\u0026#34;Confirmed\u0026#34;) order.transition(\u0026#34;Processing\u0026#34;) order.transition(\u0026#34;Shipped\u0026#34;) order.transition(\u0026#34;Delivered\u0026#34;) try: order.transition(\u0026#34;Cancelled\u0026#34;) # cannot cancel a delivered order except ValueError as e: print(f\u0026#34;Error: {e}\u0026#34;) 13. Summary and Key Takeaways What you learned if, elif, and else are how Python\u0026rsquo;s execution branches. Only one branch runs. Once a condition matches, the rest are skipped.\nTernary expressions (x if condition else y) are clean for simple one-line assignments but should not be nested.\nEvery value in Python has a boolean meaning. Empty strings, empty collections, zero, and None are all falsy. Everything else is truthy. Use this directly in conditions instead of explicit == None checks.\nGuard clauses \u0026ndash; returning early at the top of a function for invalid cases \u0026ndash; eliminate nesting and make the happy path clear. This is one of the most important patterns in clean code.\nmatch-case (Python 3.10+) is cleaner than long if-elif chains when matching values, sequences, or dictionary structures. Use it when you are matching a single subject against multiple patterns.\nThe rules to follow Put more specific conditions before broader ones in if-elif chains. Use \u0026#39;is\u0026#39; for None checks, \u0026#39;==\u0026#39; for value comparisons. Use \u0026#39;in\u0026#39; with a set for membership checks against multiple values. Use guard clauses to keep functions flat instead of deeply nested. Use \u0026#39;is not None\u0026#39; explicitly when 0 or False are valid values. Always have an else or a default case to handle the unexpected. Made with care for Codeverra learners | codeverra.com\n","permalink":"https://blogs.codeverra.com/blog/python-02-control-flow/","summary":"\u003ch1 id=\"control-flow-in-python\"\u003eControl Flow in Python\u003c/h1\u003e\n\u003ch3 id=\"making-decisions-in-your-code--if-elif-else-and-beyond\"\u003eMaking Decisions in Your Code \u0026ndash; if, elif, else, and Beyond\u003c/h3\u003e\n\u003chr\u003e\n\u003ch2 id=\"before-we-begin--why-control-flow-is-everything\"\u003eBefore We Begin \u0026ndash; Why Control Flow is Everything\u003c/h2\u003e\n\u003cp\u003eEvery program you have ever used makes decisions.\u003c/p\u003e\n\u003cp\u003eWhen you log into a website: if the password is correct, show the dashboard,\notherwise show an error. When your food delivery app calculates the delivery fee:\nif the order is above Rs.200, delivery is free, else charge Rs.30.\nWhen a data pipeline processes records: if the record is valid, process it,\nelse log an error and skip it.\u003c/p\u003e","tags":["python"],"title":"Control Flow in Python"},{"content":"🔰 Pattern 4: HashMap / Frequency Counting Table of Contents What is the HashMap Pattern? When to Use HashMaps? Types of HashMap Patterns Template Code Problem Set Problem 1: Two Sum Problem 2: Valid Anagram Problem 3: Group Anagrams Problem 4: Top K Frequent Elements Problem 5: Longest Consecutive Sequence Problem 6: Contains Duplicate II Problem 7: Intersection of Two Arrays II Problem 8: First Unique Character in a String Problem 9: 4Sum II Problem 10: Encode and Decode TinyURL Key Takeaways \u0026amp; Summary What is the HashMap Pattern? In the previous patterns, we\u0026rsquo;ve already used hashmaps as a supporting tool - tracking character frequencies in sliding window, storing prefix sums for subarray problems. In this section, we focus on problems where the hashmap is the core of the solution, not just a helper.\nA hashmap (dictionary in Python) gives us two superpowers:\nO(1) lookup: \u0026ldquo;Have I seen this value before?\u0026rdquo; - answered instantly. O(1) counting: \u0026ldquo;How many times has this value appeared?\u0026rdquo; - tracked effortlessly. These two operations are at the heart of an enormous number of array and string problems.\nLet\u0026rsquo;s see why this matters with the most classic example.\nProblem: Given an array [2, 7, 11, 15] and target 9, find two numbers that add up to 9.\nWithout a hashmap (brute force): Check every pair - O(n²).\n(2,7) → 9 ✅ Found! But in the worst case, we check n×(n-1)/2 pairs. With a hashmap: For each number, ask \u0026ldquo;have I already seen target - number?\u0026rdquo;\nnum=2: need 9-2=7, seen={} → 7 not seen. Store 2. num=7: need 9-7=2, seen={2: idx 0} → 2 IS seen! ✅ Done in 2 steps. The hashmap turns \u0026ldquo;searching for a complement\u0026rdquo; from O(n) to O(1), reducing the overall time from O(n²) to O(n).\nThis trade-off - spending O(n) space to save O(n) time per operation - is the fundamental idea behind every problem in this section.\nWhen to Use HashMaps? Signal Example \u0026ldquo;Find if a complement/pair exists\u0026rdquo; \u0026ldquo;Two numbers that sum to target\u0026rdquo; \u0026ldquo;Count frequencies of elements\u0026rdquo; \u0026ldquo;Most frequent element\u0026rdquo;, \u0026ldquo;valid anagram\u0026rdquo; \u0026ldquo;Group elements by some property\u0026rdquo; \u0026ldquo;Group anagrams together\u0026rdquo; \u0026ldquo;Find duplicates or unique elements\u0026rdquo; \u0026ldquo;Contains duplicate\u0026rdquo;, \u0026ldquo;first unique character\u0026rdquo; \u0026ldquo;Check if something was seen before\u0026rdquo; \u0026ldquo;Have we visited this state/value?\u0026rdquo; \u0026ldquo;Need O(1) lookup\u0026rdquo; in a collection Any problem where repeated searching kills performance HashMap vs Sorting Many hashmap problems can also be solved by sorting. The trade-off:\nApproach Time Space When to prefer HashMap O(n) O(n) When you need O(n) time; when order doesn\u0026rsquo;t matter Sorting O(n log n) O(1)* When you can\u0026rsquo;t afford O(n) space; when you need order *Sorting may use O(n) or O(log n) space depending on the algorithm.\nIn interviews, the hashmap approach is usually preferred because it\u0026rsquo;s faster. But knowing both gives you flexibility.\nTypes of HashMap Patterns 1. Complement Lookup Store values you\u0026rsquo;ve seen, and for each new value check if its complement (the value that would complete the condition) exists in the map.\n# \u0026#34;Does target - num exist in what I\u0026#39;ve seen?\u0026#34; seen = {} for i, num in enumerate(arr): complement = target - num if complement in seen: # Found a pair! seen[num] = i Used in: Two Sum, 4Sum II, pair-finding problems.\n2. Frequency Counting Count how many times each element appears. Use the counts to answer questions about the data.\nfrom collections import Counter freq = Counter(arr) # {element: count} # or manually: freq = {} for x in arr: freq[x] = freq.get(x, 0) + 1 Used in: Anagram problems, top-k problems, duplicate detection.\n3. Grouping by Key Group elements that share a property. The \u0026ldquo;key\u0026rdquo; is derived from each element, and the \u0026ldquo;value\u0026rdquo; is a list of elements with that key.\nfrom collections import defaultdict groups = defaultdict(list) for item in items: key = compute_key(item) groups[key].append(item) Used in: Group Anagrams, grouping by frequency, bucket sort.\n4. Index Tracking Store the index (not just presence) of each element. Useful when you need to know where something was seen, not just if it was seen.\nlast_seen = {} for i, num in enumerate(arr): if num in last_seen: prev_index = last_seen[num] # Do something with the gap (i - prev_index) last_seen[num] = i Used in: Contains Duplicate II, finding distances between occurrences.\n5. Set as a Simple HashMap When you only need \u0026ldquo;seen or not seen\u0026rdquo; (no counting, no indices), a set is a cleaner choice. It\u0026rsquo;s still a hash-based structure with O(1) lookup.\nseen = set() for num in arr: if num in seen: # Duplicate found seen.add(num) Used in: Longest Consecutive Sequence, duplicate detection.\nTemplate Code Template 1: Complement Lookup (Two Sum Style) def find_complement(arr, target): \u0026#34;\u0026#34;\u0026#34; For each element, check if the complement (target - element) was seen before. Time Complexity: O(n) - single pass Space Complexity: O(n) - storing seen elements \u0026#34;\u0026#34;\u0026#34; seen = {} # value → index for i, num in enumerate(arr): complement = target - num if complement in seen: return [seen[complement], i] # Found the pair seen[num] = i return [] # No pair found Template 2: Frequency Counter from collections import Counter def frequency_analysis(arr): \u0026#34;\u0026#34;\u0026#34; Count occurrences of each element and use the counts. Time Complexity: O(n) - one pass to count Space Complexity: O(k) - where k is the number of distinct elements \u0026#34;\u0026#34;\u0026#34; freq = Counter(arr) # Common operations on frequency maps: # freq.most_common(k) → top k elements by frequency # freq[x] → count of x # len(freq) → number of distinct elements # sum(freq.values()) → total elements (should equal len(arr)) return freq Template 3: Grouping from collections import defaultdict def group_by_property(items): \u0026#34;\u0026#34;\u0026#34; Group elements that share some computed property. Time Complexity: O(n × key_cost) - one pass, key computation per element Space Complexity: O(n) - storing all elements in groups \u0026#34;\u0026#34;\u0026#34; groups = defaultdict(list) for item in items: key = compute_key(item) # Define based on problem groups[key].append(item) return list(groups.values()) Problem Set Difficulty Progression # Problem Difficulty Key Concept 1 Two Sum Easy Complement lookup 2 Valid Anagram Easy Frequency comparison 3 Group Anagrams Medium Grouping by sorted key 4 Top K Frequent Elements Medium Frequency + bucket sort 5 Longest Consecutive Sequence Medium Set for O(1) lookup 6 Contains Duplicate II Easy Index tracking 7 Intersection of Two Arrays II Easy Frequency intersection 8 First Unique Character Easy Frequency + order 9 4Sum II Medium Complement lookup (pair sums) 10 Encode and Decode TinyURL Medium Bidirectional mapping Problem 1: Two Sum LeetCode Link: https://leetcode.com/problems/two-sum/\nProblem Statement Given an array of integers nums and an integer target, return the indices of the two numbers such that they add up to target. Each input has exactly one solution, and you may not use the same element twice.\nExample:\nInput: nums = [2, 7, 11, 15], target = 9 Output: [0, 1] (because nums[0] + nums[1] = 2 + 7 = 9) Clarifying Questions \u0026amp; Constraints Exactly one solution exists. Cannot use the same element twice (same index). Return indices, not values. Array is not sorted (unlike Two Sum II from the Two Pointers pattern). Approach Discussion Approach 1: Brute Force\nCheck every pair. Time: O(n²), Space: O(1) Approach 2: Sort + Two Pointers\nSort the array, use two pointers. Problem: Sorting changes the indices. You\u0026rsquo;d need to track original indices. Time: O(n log n), Space: O(n) Approach 3: HashMap - Complement Lookup (Optimal) ✅\nFor each number, check if target - num is already in the map. If yes → return the two indices. If no → store num: index in the map for future lookups. Time: O(n), Space: O(n) Why is this the go-to approach for unsorted arrays? Two pointers needs a sorted array. Sorting costs O(n log n) and loses index information. The hashmap skips sorting entirely and gives us O(n).\nCode (Both Solutions) # ============================================================ # APPROACH 1: Brute Force - O(n²) # ============================================================ def twoSum_brute(nums: list[int], target: int) -\u0026gt; list[int]: \u0026#34;\u0026#34;\u0026#34; Check every pair of elements. Time Complexity: O(n²) - nested loops Space Complexity: O(1) \u0026#34;\u0026#34;\u0026#34; n = len(nums) for i in range(n): for j in range(i + 1, n): if nums[i] + nums[j] == target: return [i, j] return [] # ============================================================ # APPROACH 3: HashMap - O(n) ✅ # ============================================================ def twoSum(nums: list[int], target: int) -\u0026gt; list[int]: \u0026#34;\u0026#34;\u0026#34; Find two indices whose values sum to target using complement lookup. For each number, the complement is (target - number). If the complement is already in our map, we found the pair. Otherwise, store this number and its index for future lookups. Why single pass works: When we reach the second number of the pair, the first number is already in the map. We don\u0026#39;t need to look ahead - only behind. Time Complexity: O(n) - single pass, O(1) per lookup Space Complexity: O(n) - storing up to n elements in the map \u0026#34;\u0026#34;\u0026#34; seen = {} # value → index for i, num in enumerate(nums): complement = target - num if complement in seen: # The complement was seen earlier - return both indices return [seen[complement], i] # Store this number\u0026#39;s index for future lookups seen[num] = i return [] # Problem guarantees a solution, so this won\u0026#39;t execute Edge Cases First two elements: [3, 3], target=6 → [0, 1]. Negative numbers: [-1, -2, -3, -4, -5], target=-8 → [-3 + -5] → [2, 4]. Zero in array: [0, 4, 3, 0], target=0 → [0, 3]. Large array: HashMap handles it in O(n) regardless of size. Dry Run Input: nums = [2, 7, 11, 15], target = 9 seen = {} i=0, num=2: complement = 9-2 = 7 7 not in seen → seen = {2: 0} i=1, num=7: complement = 9-7 = 2 2 IS in seen (index 0)! → return [0, 1] ✅ Output: [0, 1] Problem 2: Valid Anagram LeetCode Link: https://leetcode.com/problems/valid-anagram/\nProblem Statement Given two strings s and t, return True if t is an anagram of s, and False otherwise. An anagram uses the same characters with the same frequencies.\nExample:\nInput: s = \u0026#34;anagram\u0026#34;, t = \u0026#34;nagaram\u0026#34; Output: True Clarifying Questions \u0026amp; Constraints Both strings consist of lowercase English letters. Same length is necessary but not sufficient (frequencies must also match). Follow-up: What if the inputs contain Unicode characters? Approach Discussion Approach 1: Sort Both Strings\nSort both strings and compare. Time: O(n log n), Space: O(n) for the sorted copies. Approach 2: Frequency Count with Two Maps\nCount character frequencies in both strings, compare the maps. Time: O(n), Space: O(1) - at most 26 lowercase letters. Approach 3: Single Frequency Count (Optimal) ✅\nUse one map. Increment for characters in s, decrement for characters in t. If all counts are zero at the end, they\u0026rsquo;re anagrams. Time: O(n), Space: O(1) - at most 26 entries. Code (All Three Approaches) # ============================================================ # APPROACH 1: Sort - O(n log n) # ============================================================ def isAnagram_sort(s: str, t: str) -\u0026gt; bool: \u0026#34;\u0026#34;\u0026#34; Sort both strings and compare. Time Complexity: O(n log n) Space Complexity: O(n) for sorted copies \u0026#34;\u0026#34;\u0026#34; return sorted(s) == sorted(t) # ============================================================ # APPROACH 2: Two Counters - O(n) # ============================================================ from collections import Counter def isAnagram_two_counters(s: str, t: str) -\u0026gt; bool: \u0026#34;\u0026#34;\u0026#34; Compare frequency maps of both strings. Time Complexity: O(n) Space Complexity: O(1) - at most 26 entries \u0026#34;\u0026#34;\u0026#34; return Counter(s) == Counter(t) # ============================================================ # APPROACH 3: Single Counter - O(n) ✅ # ============================================================ def isAnagram(s: str, t: str) -\u0026gt; bool: \u0026#34;\u0026#34;\u0026#34; Use a single frequency map: increment for s, decrement for t. If all counts end at zero, the strings are anagrams. Why a single map is slightly better: - One pass instead of building two separate maps. - Early exit possible if lengths differ. - Handles the Unicode follow-up naturally. Time Complexity: O(n) - one pass through each string Space Complexity: O(1) - at most 26 entries for lowercase English O(k) for Unicode where k is the charset size \u0026#34;\u0026#34;\u0026#34; # Quick check: different lengths can\u0026#39;t be anagrams if len(s) != len(t): return False freq = {} # Increment for characters in s for char in s: freq[char] = freq.get(char, 0) + 1 # Decrement for characters in t for char in t: freq[char] = freq.get(char, 0) - 1 # Early exit: if any count goes negative, t has a character # that s doesn\u0026#39;t have (or has more of it) if freq[char] \u0026lt; 0: return False # All counts should be zero (guaranteed by the length check + no negatives) return True Edge Cases Empty strings: \u0026quot;\u0026quot;, \u0026quot;\u0026quot; → True. Different lengths: \u0026quot;ab\u0026quot;, \u0026quot;abc\u0026quot; → False (caught by length check). Same characters, different counts: \u0026quot;aab\u0026quot;, \u0026quot;abb\u0026quot; → False. Single characters: \u0026quot;a\u0026quot;, \u0026quot;a\u0026quot; → True; \u0026quot;a\u0026quot;, \u0026quot;b\u0026quot; → False. Dry Run Input: s = \u0026#34;anagram\u0026#34;, t = \u0026#34;nagaram\u0026#34; len(s) = 7 == len(t) = 7 ✅ Increment for s = \u0026#34;anagram\u0026#34;: freq = {\u0026#39;a\u0026#39;: 3, \u0026#39;n\u0026#39;: 1, \u0026#39;g\u0026#39;: 1, \u0026#39;r\u0026#39;: 1, \u0026#39;m\u0026#39;: 1} Decrement for t = \u0026#34;nagaram\u0026#34;: \u0026#39;n\u0026#39;: 1→0, \u0026#39;a\u0026#39;: 3→2, \u0026#39;g\u0026#39;: 1→0, \u0026#39;a\u0026#39;: 2→1, \u0026#39;r\u0026#39;: 1→0, \u0026#39;a\u0026#39;: 1→0, \u0026#39;m\u0026#39;: 1→0 No negative counts → return True ✅ Problem 3: Group Anagrams LeetCode Link: https://leetcode.com/problems/group-anagrams/\nProblem Statement Given an array of strings strs, group the anagrams together. You can return the answer in any order.\nExample:\nInput: strs = [\u0026#34;eat\u0026#34;, \u0026#34;tea\u0026#34;, \u0026#34;tan\u0026#34;, \u0026#34;ate\u0026#34;, \u0026#34;nat\u0026#34;, \u0026#34;bat\u0026#34;] Output: [[\u0026#34;eat\u0026#34;, \u0026#34;tea\u0026#34;, \u0026#34;ate\u0026#34;], [\u0026#34;tan\u0026#34;, \u0026#34;nat\u0026#34;], [\u0026#34;bat\u0026#34;]] Clarifying Questions \u0026amp; Constraints All strings are lowercase English letters. Anagrams have the same characters with the same frequencies. Two strings are anagrams of each other if sorting them produces the same result. Approach Discussion Approach 1: Sort Each String, Group by Sorted Key ✅\nFor each string, sort its characters to produce a canonical form. Group strings with the same sorted form. Time: O(n × k log k) where n = number of strings, k = max string length Space: O(n × k) Approach 2: Frequency Tuple as Key (Slightly Better) ✅\nInstead of sorting, count character frequencies and use the frequency as a key. For lowercase English: a tuple of 26 counts like (1, 0, 0, ..., 1, 0) for \u0026ldquo;ae\u0026rdquo;. Time: O(n × k) - counting is O(k), no sorting needed. Space: O(n × k) Both approaches use the grouping by key pattern. The key insight is: two strings are anagrams if and only if they produce the same key (whether that key is a sorted string or a frequency tuple).\nCode (Both Approaches) from collections import defaultdict # ============================================================ # APPROACH 1: Sort as Key - O(n × k log k) # ============================================================ def groupAnagrams_sort(strs: list[str]) -\u0026gt; list[list[str]]: \u0026#34;\u0026#34;\u0026#34; Group anagrams by using the sorted string as a dictionary key. Two strings are anagrams ⟺ their sorted forms are identical. Time Complexity: O(n × k log k) - sorting each string of length k Space Complexity: O(n × k) - storing all strings in groups \u0026#34;\u0026#34;\u0026#34; groups = defaultdict(list) for s in strs: # The key is the sorted version of the string key = tuple(sorted(s)) # tuple because lists aren\u0026#39;t hashable groups[key].append(s) return list(groups.values()) # ============================================================ # APPROACH 2: Frequency Tuple as Key - O(n × k) ✅ # ============================================================ def groupAnagrams(strs: list[str]) -\u0026gt; list[list[str]]: \u0026#34;\u0026#34;\u0026#34; Group anagrams by using character frequency as a dictionary key. Instead of sorting (O(k log k)), we count characters (O(k)). The frequency tuple (count of \u0026#39;a\u0026#39;, count of \u0026#39;b\u0026#39;, ..., count of \u0026#39;z\u0026#39;) uniquely identifies an anagram group. Time Complexity: O(n × k) - counting characters in each string Space Complexity: O(n × k) - storing all strings in groups \u0026#34;\u0026#34;\u0026#34; groups = defaultdict(list) for s in strs: # Build a frequency key: 26-element tuple count = [0] * 26 for char in s: count[ord(char) - ord(\u0026#39;a\u0026#39;)] += 1 key = tuple(count) # e.g., \u0026#34;eat\u0026#34; → (1,0,0,0,1,0,...,1,0,0,0) groups[key].append(s) return list(groups.values()) Edge Cases All same strings: [\u0026quot;a\u0026quot;, \u0026quot;a\u0026quot;, \u0026quot;a\u0026quot;] → [[\u0026quot;a\u0026quot;, \u0026quot;a\u0026quot;, \u0026quot;a\u0026quot;]]. All different: [\u0026quot;a\u0026quot;, \u0026quot;b\u0026quot;, \u0026quot;c\u0026quot;] → [[\u0026quot;a\u0026quot;], [\u0026quot;b\u0026quot;], [\u0026quot;c\u0026quot;]]. Empty strings: [\u0026quot;\u0026quot;, \u0026quot;\u0026quot;] → [[\u0026quot;\u0026quot;, \u0026quot;\u0026quot;]] (both sort to \u0026ldquo;\u0026rdquo;). Single string: [\u0026quot;abc\u0026quot;] → [[\u0026quot;abc\u0026quot;]]. Dry Run Input: strs = [\u0026#34;eat\u0026#34;, \u0026#34;tea\u0026#34;, \u0026#34;tan\u0026#34;, \u0026#34;ate\u0026#34;, \u0026#34;nat\u0026#34;, \u0026#34;bat\u0026#34;] Using Approach 1 (sorted key): \u0026#34;eat\u0026#34; → sorted = \u0026#34;aet\u0026#34; → groups[\u0026#34;aet\u0026#34;] = [\u0026#34;eat\u0026#34;] \u0026#34;tea\u0026#34; → sorted = \u0026#34;aet\u0026#34; → groups[\u0026#34;aet\u0026#34;] = [\u0026#34;eat\u0026#34;, \u0026#34;tea\u0026#34;] \u0026#34;tan\u0026#34; → sorted = \u0026#34;ant\u0026#34; → groups[\u0026#34;ant\u0026#34;] = [\u0026#34;tan\u0026#34;] \u0026#34;ate\u0026#34; → sorted = \u0026#34;aet\u0026#34; → groups[\u0026#34;aet\u0026#34;] = [\u0026#34;eat\u0026#34;, \u0026#34;tea\u0026#34;, \u0026#34;ate\u0026#34;] \u0026#34;nat\u0026#34; → sorted = \u0026#34;ant\u0026#34; → groups[\u0026#34;ant\u0026#34;] = [\u0026#34;tan\u0026#34;, \u0026#34;nat\u0026#34;] \u0026#34;bat\u0026#34; → sorted = \u0026#34;abt\u0026#34; → groups[\u0026#34;abt\u0026#34;] = [\u0026#34;bat\u0026#34;] Output: [[\u0026#34;eat\u0026#34;, \u0026#34;tea\u0026#34;, \u0026#34;ate\u0026#34;], [\u0026#34;tan\u0026#34;, \u0026#34;nat\u0026#34;], [\u0026#34;bat\u0026#34;]] ✅ Problem 4: Top K Frequent Elements LeetCode Link: https://leetcode.com/problems/top-k-frequent-elements/\nProblem Statement Given an integer array nums and an integer k, return the k most frequent elements. You may return the answer in any order.\nExample:\nInput: nums = [1, 1, 1, 2, 2, 3], k = 2 Output: [1, 2] Clarifying Questions \u0026amp; Constraints k is always valid (1 ≤ k ≤ number of distinct elements). The answer is guaranteed to be unique. Follow-up: Can you solve it in better than O(n log n)? Approach Discussion Approach 1: Sort by Frequency\nCount frequencies, sort by count, take top k. Time: O(n log n), Space: O(n) Approach 2: Heap (Min-Heap of Size k)\nCount frequencies. Maintain a min-heap of size k. Push each element; if heap exceeds size k, pop the smallest. Time: O(n log k), Space: O(n) Approach 3: Bucket Sort (Optimal) ✅\nCount frequencies. Create \u0026ldquo;buckets\u0026rdquo; indexed by frequency. Bucket i contains all elements that appear i times. Walk buckets from highest to lowest, collecting elements until we have k. Time: O(n), Space: O(n) Why bucket sort works here: The maximum possible frequency is n (the array length). So we create n+1 buckets. This avoids comparison-based sorting entirely.\nCode (All Three Approaches) from collections import Counter import heapq # ============================================================ # APPROACH 1: Sort by Frequency - O(n log n) # ============================================================ def topKFrequent_sort(nums: list[int], k: int) -\u0026gt; list[int]: \u0026#34;\u0026#34;\u0026#34; Count frequencies, sort by count, return top k. Time Complexity: O(n log n) - sorting dominates Space Complexity: O(n) - for the frequency map \u0026#34;\u0026#34;\u0026#34; freq = Counter(nums) # Sort by frequency (descending) and take the first k return [num for num, count in freq.most_common(k)] # ============================================================ # APPROACH 2: Min-Heap of Size k - O(n log k) # ============================================================ def topKFrequent_heap(nums: list[int], k: int) -\u0026gt; list[int]: \u0026#34;\u0026#34;\u0026#34; Use a min-heap of size k to efficiently find top k elements. We push (frequency, element) pairs. When the heap exceeds size k, pop the smallest - this ensures only the k largest remain. Time Complexity: O(n log k) - n insertions, each O(log k) Space Complexity: O(n) for frequency map + O(k) for heap \u0026#34;\u0026#34;\u0026#34; freq = Counter(nums) # Use a min-heap of size k # heapq is a min-heap, so smallest frequency gets popped heap = [] for num, count in freq.items(): heapq.heappush(heap, (count, num)) if len(heap) \u0026gt; k: heapq.heappop(heap) # Remove the least frequent return [num for count, num in heap] # ============================================================ # APPROACH 3: Bucket Sort - O(n) ✅ # ============================================================ def topKFrequent(nums: list[int], k: int) -\u0026gt; list[int]: \u0026#34;\u0026#34;\u0026#34; Use bucket sort to find top k frequent elements in O(n). Idea: Create an array of buckets where bucket[i] holds all elements that appear exactly i times. Since max frequency is n, we need n+1 buckets. Then walk from the highest bucket downward, collecting elements. Why this is O(n): - Counting frequencies: O(n) - Filling buckets: O(n) - each element goes into exactly one bucket - Collecting results: O(n) - we scan at most n buckets Time Complexity: O(n) Space Complexity: O(n) - for the frequency map and buckets \u0026#34;\u0026#34;\u0026#34; freq = Counter(nums) # Create buckets: index = frequency, value = list of elements with that frequency # Max frequency possible is len(nums) n = len(nums) buckets = [[] for _ in range(n + 1)] for num, count in freq.items(): buckets[count].append(num) # Collect top k elements, starting from highest frequency result = [] for i in range(n, 0, -1): # From frequency n down to 1 for num in buckets[i]: result.append(num) if len(result) == k: return result return result Edge Cases k equals number of distinct elements: Return all distinct elements. All same element: [1,1,1], k=1 → [1]. All unique: [1,2,3], k=2 → any 2 of the three (all have frequency 1). Single element: [1], k=1 → [1]. Dry Run Input: nums = [1, 1, 1, 2, 2, 3], k = 2 Step 1: Count frequencies freq = {1: 3, 2: 2, 3: 1} Step 2: Fill buckets (n = 6) buckets[0] = [] buckets[1] = [3] ← 3 appears 1 time buckets[2] = [2] ← 2 appears 2 times buckets[3] = [1] ← 1 appears 3 times buckets[4] = [] buckets[5] = [] buckets[6] = [] Step 3: Collect from highest frequency i=6: empty i=5: empty i=4: empty i=3: result = [1] (need 1 more) i=2: result = [1, 2] (got k=2 elements) → return! Output: [1, 2] ✅ Problem 5: Longest Consecutive Sequence LeetCode Link: https://leetcode.com/problems/longest-consecutive-sequence/\nProblem Statement Given an unsorted array of integers nums, return the length of the longest consecutive elements sequence. You must write an algorithm that runs in O(n) time.\nExample:\nInput: nums = [100, 4, 200, 1, 3, 2] Output: 4 (sequence: [1, 2, 3, 4]) Clarifying Questions \u0026amp; Constraints Elements can be negative and can contain duplicates. \u0026ldquo;Consecutive\u0026rdquo; means the values differ by 1: [1, 2, 3, 4] not [1, 3, 5, 7]. Must be O(n) - so sorting (O(n log n)) is not allowed. Approach Discussion Approach 1: Sort\nSort the array, scan for consecutive runs. Time: O(n log n) ❌ violates the O(n) constraint. Space: O(1) (or O(n) depending on sort). Approach 2: HashSet + Smart Sequence Start Detection (Optimal) ✅\nThe key insight: we only want to start counting a sequence from its beginning - the smallest number in that sequence.\nHow do we know if a number is the start of a sequence? If num - 1 is NOT in the set. If num - 1 exists, then num is in the middle of some sequence, and we\u0026rsquo;ll count it when we process the actual start.\nPut all numbers in a set (O(n)). For each number, if it\u0026rsquo;s a sequence start (num - 1 not in set), count how long the sequence goes (num, num+1, num+2, \u0026hellip;). Track the maximum length. Time: O(n) - each number is visited at most twice (once when checking if it\u0026rsquo;s a start, once when extending a sequence). Space: O(n).\nCode (Both Approaches) # ============================================================ # APPROACH 1: Sort - O(n log n) # ============================================================ def longestConsecutive_sort(nums: list[int]) -\u0026gt; int: \u0026#34;\u0026#34;\u0026#34; Sort and scan for consecutive runs. Time Complexity: O(n log n) Space Complexity: O(1) extra \u0026#34;\u0026#34;\u0026#34; if not nums: return 0 nums.sort() max_length = 1 current_length = 1 for i in range(1, len(nums)): if nums[i] == nums[i - 1]: continue # Skip duplicates elif nums[i] == nums[i - 1] + 1: current_length += 1 else: current_length = 1 max_length = max(max_length, current_length) return max_length # ============================================================ # APPROACH 2: HashSet - O(n) ✅ # ============================================================ def longestConsecutive(nums: list[int]) -\u0026gt; int: \u0026#34;\u0026#34;\u0026#34; Find the longest consecutive sequence using a set for O(1) lookups. Key Insight: Only start counting from the BEGINNING of a sequence. A number is a sequence start if (num - 1) is NOT in the set. Why this is O(n) and not O(n²): The inner while loop seems dangerous, but consider: each number in the array is part of exactly ONE sequence. It gets visited at most once by the inner loop (when we extend the sequence from its starting point) and once by the outer loop (where we skip it because num-1 exists). So total work across all iterations is O(n). Time Complexity: O(n) - each element is processed at most twice Space Complexity: O(n) - storing elements in a set \u0026#34;\u0026#34;\u0026#34; if not nums: return 0 num_set = set(nums) # O(n) to build, handles duplicates automatically max_length = 0 for num in num_set: # Only start counting if this is the BEGINNING of a sequence # (i.e., num-1 is NOT in the set) if num - 1 not in num_set: # This is a sequence start - count how far it goes current = num length = 1 while current + 1 in num_set: current += 1 length += 1 max_length = max(max_length, length) return max_length Edge Cases Empty array: [] → 0. Single element: [7] → 1. All same: [5, 5, 5] → 1 (set removes duplicates). Negatives: [-2, -1, 0, 1] → 4. No consecutive: [10, 30, 50] → 1. Dry Run Input: nums = [100, 4, 200, 1, 3, 2] num_set = {1, 2, 3, 4, 100, 200} num=1: 1-1=0 not in set → sequence START 1→2→3→4→ (5 not in set) → length=4, max_length=4 num=2: 2-1=1 IS in set → SKIP (not a start) num=3: 3-1=2 IS in set → SKIP num=4: 4-1=3 IS in set → SKIP num=100: 100-1=99 not in set → sequence START 100→ (101 not in set) → length=1, max_length=4 num=200: 200-1=199 not in set → sequence START 200→ (201 not in set) → length=1, max_length=4 Output: 4 ✅ Problem 6: Contains Duplicate II LeetCode Link: https://leetcode.com/problems/contains-duplicate-ii/\nProblem Statement Given an integer array nums and an integer k, return True if there are two distinct indices i and j such that nums[i] == nums[j] and abs(i - j) \u0026lt;= k.\nExample:\nInput: nums = [1, 2, 3, 1], k = 3 Output: True (nums[0] == nums[3] and |0 - 3| = 3 ≤ 3) Clarifying Questions \u0026amp; Constraints We need both conditions: same value AND indices within distance k. Multiple duplicates may exist; we need any valid pair. Approach Discussion Approach 1: Brute Force\nFor each pair (i, j), check both conditions. Time: O(n × k) or O(n²), Space: O(1) Approach 2: HashMap - Track Last Index (Optimal) ✅\nStore each value\u0026rsquo;s most recent index. When we see a value again, check if the distance to the previous index is ≤ k. Time: O(n), Space: O(n) Approach 3: Sliding Window Set (Alternative) ✅\nMaintain a set of elements in the current window of size k. If a new element is already in the set → duplicate within distance k. Time: O(n), Space: O(k) Code (Approaches 2 and 3) # ============================================================ # APPROACH 2: HashMap - Track Last Index - O(n) ✅ # ============================================================ def containsNearbyDuplicate(nums: list[int], k: int) -\u0026gt; bool: \u0026#34;\u0026#34;\u0026#34; For each element, check if it appeared before within distance k. We store the most recent index of each value. When we see a value again, we check if (current_index - last_index) \u0026lt;= k. Why store the most recent index (not the first)? If the first occurrence is too far away, a later occurrence might be close enough. We want the CLOSEST previous match. Time Complexity: O(n) - single pass Space Complexity: O(n) - storing indices of all distinct values \u0026#34;\u0026#34;\u0026#34; last_index = {} # value → most recent index for i, num in enumerate(nums): if num in last_index and i - last_index[num] \u0026lt;= k: return True # Update to the most recent index last_index[num] = i return False # ============================================================ # APPROACH 3: Sliding Window Set - O(n), O(k) space ✅ # ============================================================ def containsNearbyDuplicate_set(nums: list[int], k: int) -\u0026gt; bool: \u0026#34;\u0026#34;\u0026#34; Maintain a set of elements within a window of size k. If a new element is already in the set, it must be within distance k. When the window exceeds size k, remove the oldest element. Time Complexity: O(n) - single pass, O(1) per set operation Space Complexity: O(k) - set holds at most k+1 elements \u0026#34;\u0026#34;\u0026#34; window = set() for i, num in enumerate(nums): # If num is already in the window, found a duplicate within distance k if num in window: return True # Add the new element window.add(num) # If window exceeds size k, remove the element that\u0026#39;s now too far away if len(window) \u0026gt; k: window.remove(nums[i - k]) return False Edge Cases k = 0: abs(i-j) \u0026lt;= 0 means i == j, but we need distinct indices → always False. No duplicates: [1, 2, 3], k=2 → False. Duplicate too far: [1, 2, 3, 1], k=2 → False (distance is 3 \u0026gt; 2). Adjacent duplicates: [1, 1], k=1 → True. Dry Run Input: nums = [1, 2, 3, 1], k = 3 Using Approach 2 (HashMap): last_index = {} i=0, num=1: not in map → last_index = {1: 0} i=1, num=2: not in map → last_index = {1: 0, 2: 1} i=2, num=3: not in map → last_index = {1: 0, 2: 1, 3: 2} i=3, num=1: IN map, last at index 0. |3 - 0| = 3 ≤ 3 → return True ✅ Problem 7: Intersection of Two Arrays II LeetCode Link: https://leetcode.com/problems/intersection-of-two-arrays-ii/\nProblem Statement Given two integer arrays nums1 and nums2, return an array of their intersection. Each element in the result must appear as many times as it shows in both arrays. The result can be in any order.\nExample:\nInput: nums1 = [1, 2, 2, 1], nums2 = [2, 2] Output: [2, 2] Clarifying Questions \u0026amp; Constraints Result includes duplicates (unlike set intersection). [2, 2] intersected with [2] → [2] (limited by the smaller count). Follow-up: What if nums1 is much smaller? What if nums2 is stored on disk? Approach Discussion Approach 1: Sort Both Arrays + Two Pointers\nSort both arrays, use two pointers to find common elements. Time: O(n log n + m log m), Space: O(1) extra. ✅ Good when arrays are already sorted or memory is limited. Approach 2: HashMap - Frequency Intersection (Optimal) ✅\nCount frequencies in the smaller array (saves space). Iterate through the larger array; for each match, decrement the count. Time: O(n + m), Space: O(min(n, m)). Code (Both Approaches) from collections import Counter # ============================================================ # APPROACH 1: Sort + Two Pointers - O(n log n + m log m) # ============================================================ def intersect_sort(nums1: list[int], nums2: list[int]) -\u0026gt; list[int]: \u0026#34;\u0026#34;\u0026#34; Sort both arrays and use two pointers to find matches. Time Complexity: O(n log n + m log m) for sorting Space Complexity: O(1) extra (not counting the result) \u0026#34;\u0026#34;\u0026#34; nums1.sort() nums2.sort() i, j = 0, 0 result = [] while i \u0026lt; len(nums1) and j \u0026lt; len(nums2): if nums1[i] == nums2[j]: result.append(nums1[i]) i += 1 j += 1 elif nums1[i] \u0026lt; nums2[j]: i += 1 else: j += 1 return result # ============================================================ # APPROACH 2: HashMap - O(n + m) ✅ # ============================================================ def intersect(nums1: list[int], nums2: list[int]) -\u0026gt; list[int]: \u0026#34;\u0026#34;\u0026#34; Count frequencies in the smaller array, then match against the larger. Why count the smaller array? The map size is proportional to the array we count. Using the smaller one minimizes space usage. For each element in the larger array: - If it exists in the map with count \u0026gt; 0: add to result, decrement count. - Decrementing prevents using the same occurrence twice. Time Complexity: O(n + m) - one pass each Space Complexity: O(min(n, m)) - map for the smaller array \u0026#34;\u0026#34;\u0026#34; # Always count the smaller array if len(nums1) \u0026gt; len(nums2): return intersect(nums2, nums1) freq = Counter(nums1) # Count the smaller array result = [] for num in nums2: if freq.get(num, 0) \u0026gt; 0: result.append(num) freq[num] -= 1 # \u0026#34;Use up\u0026#34; one occurrence return result Edge Cases No intersection: [1, 2], [3, 4] → []. Complete overlap: [2, 2], [2, 2] → [2, 2]. One empty: [], [1, 2] → []. Partial overlap: [4, 9, 5], [9, 4, 9, 8, 4] → [4, 9] (or [9, 4]). Dry Run Input: nums1 = [1, 2, 2, 1], nums2 = [2, 2] nums2 is smaller → freq = Counter([2, 2]) = {2: 2} Iterate through nums1: num=1: freq.get(1, 0) = 0 → skip num=2: freq[2] = 2 \u0026gt; 0 → result=[2], freq[2]=1 num=2: freq[2] = 1 \u0026gt; 0 → result=[2, 2], freq[2]=0 num=1: freq.get(1, 0) = 0 → skip Output: [2, 2] ✅ Problem 8: First Unique Character in a String LeetCode Link: https://leetcode.com/problems/first-unique-character-in-a-string/\nProblem Statement Given a string s, find the first non-repeating character and return its index. If it does not exist, return -1.\nExample:\nInput: s = \u0026#34;leetcode\u0026#34; Output: 0 (\u0026#39;l\u0026#39; appears only once and is the first such character) Clarifying Questions \u0026amp; Constraints String consists of only lowercase English letters. We need the first (leftmost) unique character. Approach Discussion Approach 1: Brute Force\nFor each character, scan the entire string to check if it appears elsewhere. Time: O(n²), Space: O(1) Approach 2: Two-Pass with Frequency Map (Optimal) ✅\nPass 1: Count the frequency of every character. Pass 2: Scan left to right, return the first character with count 1. Time: O(n), Space: O(1) - at most 26 entries. Why two passes? In one pass, when we see a character for the first time, we don\u0026rsquo;t yet know if it will repeat later. We need the full frequency picture before we can answer \u0026ldquo;which character is unique.\u0026rdquo;\nCode (Optimal Solution) from collections import Counter def firstUniqChar(s: str) -\u0026gt; int: \u0026#34;\u0026#34;\u0026#34; Find the first character that appears exactly once. Pass 1: Count all character frequencies. Pass 2: Scan left to right, return the first character with count == 1. Why can\u0026#39;t we do this in one pass? At position i, we don\u0026#39;t know if s[i] will repeat later. We need the complete frequency count before we can identify unique characters. Time Complexity: O(n) - two passes through the string Space Complexity: O(1) - at most 26 lowercase letters in the map \u0026#34;\u0026#34;\u0026#34; # Pass 1: Count frequencies freq = Counter(s) # Pass 2: Find the first character with count 1 for i, char in enumerate(s): if freq[char] == 1: return i return -1 # No unique character found Edge Cases All unique: \u0026quot;abc\u0026quot; → 0. All same: \u0026quot;aaa\u0026quot; → -1. Unique at end: \u0026quot;aabb c\u0026quot; → 4 (space at index 4 is unique if present, but constraints say lowercase letters only, so \u0026quot;aabbc\u0026quot; → 4, 'c' is first unique). Single character: \u0026quot;z\u0026quot; → 0. Empty string: \u0026quot;\u0026quot; → -1. Dry Run Input: s = \u0026#34;loveleetcode\u0026#34; Pass 1: freq = {\u0026#39;l\u0026#39;: 1, \u0026#39;o\u0026#39;: 2, \u0026#39;v\u0026#39;: 1, \u0026#39;e\u0026#39;: 4, \u0026#39;t\u0026#39;: 1, \u0026#39;c\u0026#39;: 1, \u0026#39;d\u0026#39;: 1} Pass 2: i=0, \u0026#39;l\u0026#39;: freq[\u0026#39;l\u0026#39;]=1 → not anymore, wait... Actually: let me recount. s = \u0026#34;loveleetcode\u0026#34; l:1, o:2, v:1, e:4, l:wait... s = l, o, v, e, l, e, e, t, c, o, d, e freq = {\u0026#39;l\u0026#39;:2, \u0026#39;o\u0026#39;:2, \u0026#39;v\u0026#39;:1, \u0026#39;e\u0026#39;:4, \u0026#39;t\u0026#39;:1, \u0026#39;c\u0026#39;:1, \u0026#39;d\u0026#39;:1} Pass 2: i=0, \u0026#39;l\u0026#39;: freq=2 → skip i=1, \u0026#39;o\u0026#39;: freq=2 → skip i=2, \u0026#39;v\u0026#39;: freq=1 → return 2 ✅ Output: 2 Problem 9: 4Sum II LeetCode Link: https://leetcode.com/problems/4sum-ii/\nProblem Statement Given four integer arrays nums1, nums2, nums3, and nums4, all of length n, return the number of tuples (i, j, k, l) such that nums1[i] + nums2[j] + nums3[k] + nums4[l] == 0.\nExample:\nInput: nums1 = [1, 2], nums2 = [-2, -1], nums3 = [-1, 2], nums4 = [0, 2] Output: 2 Tuples: (0,0,0,1) → 1+(-2)+(-1)+2 = 0 (1,1,0,0) → 2+(-1)+(-1)+0 = 0 Clarifying Questions \u0026amp; Constraints All four arrays have the same length n. Elements can be from different arrays (indices are independent). We count all valid tuples (not just unique values). n ≤ 200. Approach Discussion Approach 1: Brute Force - Four Nested Loops\nCheck every combination of (i, j, k, l). Time: O(n⁴) ❌ - way too slow even for n=200 (1.6 billion operations). Approach 2: Three Loops + Set Lookup\nPrecompute all values of nums4 into a set. Three nested loops for nums1, nums2, nums3, check if -(a+b+c) is in the set. Time: O(n³) - still too slow for n=200. Approach 3: Two-Map Split (Optimal) ✅\nSplit the four arrays into two groups: (nums1, nums2) and (nums3, nums4). Compute all possible sums of pairs from group 1 → store in a hashmap with counts. Compute all possible sums of pairs from group 2 → look up -(c+d) in the map. Time: O(n²), Space: O(n²) This is the \u0026ldquo;meet in the middle\u0026rdquo; technique - a powerful application of the complement lookup pattern.\nCode (Optimal Solution) from collections import Counter def fourSumCount(nums1: list[int], nums2: list[int], nums3: list[int], nums4: list[int]) -\u0026gt; int: \u0026#34;\u0026#34;\u0026#34; Count tuples (i,j,k,l) where nums1[i]+nums2[j]+nums3[k]+nums4[l] = 0. Strategy - \u0026#34;Meet in the Middle\u0026#34;: 1. Compute all possible sums (a + b) for a in nums1, b in nums2. Store each sum and how many ways it can be formed. 2. For each possible sum (c + d) where c in nums3, d in nums4, check if -(c + d) exists in the map from step 1. Why split into two halves? Four arrays with n elements each → n⁴ brute force. Splitting into pairs: n² pairs in each half → n² + n² = O(n²) total. Time Complexity: O(n²) - n² pairs per half Space Complexity: O(n²) - storing up to n² sums in the map \u0026#34;\u0026#34;\u0026#34; # Step 1: Compute all (a + b) sums and their counts ab_sums = Counter() for a in nums1: for b in nums2: ab_sums[a + b] += 1 # Step 2: For each (c + d), check if -(c+d) was a valid (a+b) sum count = 0 for c in nums3: for d in nums4: target = -(c + d) if target in ab_sums: count += ab_sums[target] return count Edge Cases All zeros: [0],[0],[0],[0] → 1. No valid tuples: All positive numbers → 0 (can\u0026rsquo;t sum to 0). Large counts: If many pairs produce the same sum, the count multiplies. Dry Run Input: nums1=[1,2], nums2=[-2,-1], nums3=[-1,2], nums4=[0,2] Step 1: All (a+b) sums: 1+(-2) = -1 1+(-1) = 0 2+(-2) = 0 2+(-1) = 1 ab_sums = {-1: 1, 0: 2, 1: 1} Step 2: For each (c+d), look up -(c+d): c=-1, d=0: target = -(-1+0) = 1 → ab_sums[1] = 1 → count += 1 c=-1, d=2: target = -(-1+2) = -1 → ab_sums[-1] = 1 → count += 1 c=2, d=0: target = -(2+0) = -2 → not in map → count += 0 c=2, d=2: target = -(2+2) = -4 → not in map → count += 0 Output: 2 ✅ Problem 10: Encode and Decode TinyURL LeetCode Link: https://leetcode.com/problems/encode-and-decode-tinyurl/\nProblem Statement Design a URL shortening service. Implement encode(longUrl) which converts a long URL to a short URL, and decode(shortUrl) which converts the short URL back to the original.\nExample:\nurl = \u0026#34;https://leetcode.com/problems/design-tinyurl\u0026#34; tiny = encode(url) # e.g., \u0026#34;http://tinyurl.com/abc123\u0026#34; original = decode(tiny) # \u0026#34;https://leetcode.com/problems/design-tinyurl\u0026#34; Clarifying Questions \u0026amp; Constraints There\u0026rsquo;s no restriction on how your encode/decode algorithm works. The short URL should map back to exactly the original long URL. This is a design problem - the focus is on demonstrating hashmap-based bidirectional mapping. Approach Discussion Approach 1: Counter-Based (Simple)\nAssign each URL an incrementing integer ID. Short URL = base URL + ID. Pros: Simple, deterministic, no collisions. Cons: Predictable (someone can guess URLs), IDs grow large. Approach 2: Random Code Generation (Better) ✅\nGenerate a random alphanumeric code for each URL. Store the mapping in both directions: code → long URL, long URL → code. Check for collisions (unlikely but possible). Pros: Unpredictable, short codes. Cons: Tiny collision risk. Approach 3: Hashing (Alternative)\nHash the long URL to produce the short code. Pros: Deterministic (same URL always gets same code). Cons: Hash collisions need handling. The core data structure for all approaches is a bidirectional hashmap - two maps that let you look up in both directions.\nCode (Approaches 1 and 2) import random import string # ============================================================ # APPROACH 1: Counter-Based - Simple # ============================================================ class Codec_Counter: \u0026#34;\u0026#34;\u0026#34; Assign each URL an incrementing integer ID. Encode: Store longUrl with the next available ID. Decode: Look up the long URL by ID. Time Complexity: O(1) for both encode and decode Space Complexity: O(n) where n = number of URLs stored \u0026#34;\u0026#34;\u0026#34; def __init__(self): self.id_to_url = {} # id → long URL self.url_to_id = {} # long URL → id (avoid encoding same URL twice) self.counter = 0 def encode(self, longUrl: str) -\u0026gt; str: # If already encoded, return the existing short URL if longUrl in self.url_to_id: return f\u0026#34;http://tinyurl.com/{self.url_to_id[longUrl]}\u0026#34; self.counter += 1 self.id_to_url[self.counter] = longUrl self.url_to_id[longUrl] = self.counter return f\u0026#34;http://tinyurl.com/{self.counter}\u0026#34; def decode(self, shortUrl: str) -\u0026gt; str: # Extract the ID from the short URL url_id = int(shortUrl.split(\u0026#39;/\u0026#39;)[-1]) return self.id_to_url[url_id] # ============================================================ # APPROACH 2: Random Code - More Realistic ✅ # ============================================================ class Codec: \u0026#34;\u0026#34;\u0026#34; Generate a random 6-character code for each URL. Two hashmaps form a bidirectional mapping: - code_to_url: short code → original long URL (for decoding) - url_to_code: long URL → short code (to avoid encoding same URL twice) With 62 characters and 6 positions: 62^6 ≈ 56.8 billion possible codes. Collision probability is negligible for reasonable usage. Time Complexity: O(1) amortized for both encode and decode Space Complexity: O(n) where n = number of URLs stored \u0026#34;\u0026#34;\u0026#34; BASE = \u0026#34;http://tinyurl.com/\u0026#34; CHARS = string.ascii_letters + string.digits # a-z, A-Z, 0-9 (62 chars) CODE_LENGTH = 6 def __init__(self): self.code_to_url = {} # code → long URL self.url_to_code = {} # long URL → code def _generate_code(self) -\u0026gt; str: \u0026#34;\u0026#34;\u0026#34;Generate a random 6-character alphanumeric code.\u0026#34;\u0026#34;\u0026#34; return \u0026#39;\u0026#39;.join(random.choices(self.CHARS, k=self.CODE_LENGTH)) def encode(self, longUrl: str) -\u0026gt; str: # If already encoded, return the existing short URL if longUrl in self.url_to_code: return self.BASE + self.url_to_code[longUrl] # Generate a unique code (handle unlikely collisions) code = self._generate_code() while code in self.code_to_url: code = self._generate_code() # Store in both directions self.code_to_url[code] = longUrl self.url_to_code[longUrl] = code return self.BASE + code def decode(self, shortUrl: str) -\u0026gt; str: # Extract the code from the short URL and look it up code = shortUrl.replace(self.BASE, \u0026#34;\u0026#34;) return self.code_to_url[code] Edge Cases Same URL encoded twice: Should return the same short URL (handled by url_to_code check). Empty URL: Works, but unusual. Very long URL: No issue - the code length is independent of the URL length. Decode an invalid short URL: Would raise a KeyError (in production, return an error). Dry Run Using Approach 2 (Random Code): encode(\u0026#34;https://leetcode.com/problems/design-tinyurl\u0026#34;): Not in url_to_code → generate code \u0026#34;aB3xY7\u0026#34; code_to_url[\u0026#34;aB3xY7\u0026#34;] = \u0026#34;https://leetcode.com/problems/design-tinyurl\u0026#34; url_to_code[\u0026#34;https://leetcode.com/problems/design-tinyurl\u0026#34;] = \u0026#34;aB3xY7\u0026#34; Return: \u0026#34;http://tinyurl.com/aB3xY7\u0026#34; decode(\u0026#34;http://tinyurl.com/aB3xY7\u0026#34;): code = \u0026#34;aB3xY7\u0026#34; code_to_url[\u0026#34;aB3xY7\u0026#34;] = \u0026#34;https://leetcode.com/problems/design-tinyurl\u0026#34; Return: \u0026#34;https://leetcode.com/problems/design-tinyurl\u0026#34; ✅ encode(\u0026#34;https://leetcode.com/problems/design-tinyurl\u0026#34;) again: Already in url_to_code → return \u0026#34;http://tinyurl.com/aB3xY7\u0026#34; (same code) ✅ Key Takeaways \u0026amp; Summary Quick Reference Table Problem HashMap Pattern Time Space Core Trick Two Sum Complement lookup O(n) O(n) Look up target - num Valid Anagram Frequency comparison O(n) O(1) Increment for s, decrement for t Group Anagrams Grouping by key O(n×k) O(n×k) Sorted string or freq tuple as key Top K Frequent Frequency + bucket sort O(n) O(n) Buckets indexed by frequency Longest Consecutive Set membership O(n) O(n) Only start from sequence beginnings Contains Duplicate II Index tracking O(n) O(n) Store last seen index, check gap Intersection of Arrays Frequency intersection O(n+m) O(min(n,m)) Count smaller, decrement on match First Unique Char Frequency + scan O(n) O(1) Two-pass: count then scan 4Sum II Meet in the middle O(n²) O(n²) Split into two halves, complement Encode/Decode TinyURL Bidirectional map O(1) O(n) Two maps: code↔URL The Five HashMap Patterns - When to Use Each What does the problem ask you to do? 1. \u0026#34;Find a PAIR/COMPLEMENT that satisfies some condition\u0026#34; └─ Complement Lookup Store seen values, check if (target - current) exists. Examples: Two Sum, 4Sum II 2. \u0026#34;Count/compare CHARACTER or ELEMENT frequencies\u0026#34; └─ Frequency Counting Use Counter or manual dict. Compare, find max, check validity. Examples: Valid Anagram, Top K Frequent, First Unique 3. \u0026#34;GROUP elements that share a property\u0026#34; └─ Grouping by Key Compute a key from each element, group into lists by key. Examples: Group Anagrams 4. \u0026#34;Check for DUPLICATES or DISTANCES between occurrences\u0026#34; └─ Index Tracking Store the index of each element. Compare indices when seen again. Examples: Contains Duplicate II, Longest Consecutive Sequence 5. \u0026#34;Map between two DOMAINS (lookup in both directions)\u0026#34; └─ Bidirectional Mapping Two maps: A→B and B→A. Examples: Encode/Decode TinyURL HashMap vs Other Approaches - Choosing Wisely Scenario HashMap Sorting Two Pointers Notes Array is unsorted, need pairs ✅ O(n) O(n log n) Needs sort first HashMap is faster Array is already sorted, need pairs Works O(n) Already done ✅ O(n), O(1) space Two pointers saves space Need to group elements ✅ Natural fit Can sort, but grouping is awkward N/A HashMap is cleaner Memory is very limited ❌ O(n) space ✅ O(1) space ✅ O(1) space Sort/two-pointers save space Need elements in order Extra work ✅ Natural ✅ Natural HashMap loses insertion order (unless ordered dict) Common Mistakes to Avoid Forgetting to handle the {0: 1} initialization in prefix sum + hashmap problems. This isn\u0026rsquo;t a hashmap-pattern mistake per se, but it overlaps heavily since those problems use hashmaps.\nUsing a list where a set would suffice. If you only need \u0026ldquo;seen or not,\u0026rdquo; a set is cleaner and communicates intent better. x in set is O(1); x in list is O(n).\nNot considering what to store as the value. Sometimes you need the index, sometimes the count, sometimes a list of items. The choice of value type determines which pattern you\u0026rsquo;re using.\nMutating the hashmap while iterating over it. In Python, this raises a RuntimeError. If you need to modify, iterate over a copy: for key in list(my_dict.keys()).\nAssuming hashmap operations are always O(1). They\u0026rsquo;re O(1) amortized. In the worst case (many hash collisions), they can degrade. For interviews, O(1) is the standard assumption.\nWhat\u0026rsquo;s Next? We\u0026rsquo;ve now covered four core array patterns: Two Pointers, Sliding Window, Prefix Sum, and HashMap. These four alone will handle the majority of array problems you\u0026rsquo;ll encounter. Next up is Pattern 5: Kadane\u0026rsquo;s Algorithm - a deceptively simple technique for maximum subarray problems that shows up constantly in interviews. Stay tuned!\n💡 Practice Tip: The hashmap is the most versatile tool in your algorithm toolkit. Almost every array problem has a hashmap-based solution. When you\u0026rsquo;re stuck, ask yourself: \u0026ldquo;What would I store in a hashmap that would let me answer the question I need in O(1)?\u0026rdquo; That single question unlocks most solutions.\n","permalink":"https://blogs.codeverra.com/blog/dsa-patterns-05-hashmap-frequency/","summary":"\u003ch1 id=\"-pattern-4-hashmap--frequency-counting\"\u003e🔰 Pattern 4: HashMap / Frequency Counting\u003c/h1\u003e\n\u003ch2 id=\"table-of-contents\"\u003eTable of Contents\u003c/h2\u003e\n\u003col\u003e\n\u003cli\u003e\u003ca href=\"#what-is-the-hashmap-pattern\"\u003eWhat is the HashMap Pattern?\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#when-to-use-hashmaps\"\u003eWhen to Use HashMaps?\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#types-of-hashmap-patterns\"\u003eTypes of HashMap Patterns\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#template-code\"\u003eTemplate Code\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#problem-set\"\u003eProblem Set\u003c/a\u003e\n\u003cul\u003e\n\u003cli\u003e\u003ca href=\"#problem-1-two-sum\"\u003eProblem 1: Two Sum\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#problem-2-valid-anagram\"\u003eProblem 2: Valid Anagram\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#problem-3-group-anagrams\"\u003eProblem 3: Group Anagrams\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#problem-4-top-k-frequent-elements\"\u003eProblem 4: Top K Frequent Elements\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#problem-5-longest-consecutive-sequence\"\u003eProblem 5: Longest Consecutive Sequence\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#problem-6-contains-duplicate-ii\"\u003eProblem 6: Contains Duplicate II\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#problem-7-intersection-of-two-arrays-ii\"\u003eProblem 7: Intersection of Two Arrays II\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#problem-8-first-unique-character-in-a-string\"\u003eProblem 8: First Unique Character in a String\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#problem-9-4sum-ii\"\u003eProblem 9: 4Sum II\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#problem-10-encode-and-decode-tinyurl\"\u003eProblem 10: Encode and Decode TinyURL\u003c/a\u003e\u003c/li\u003e\n\u003c/ul\u003e\n\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#key-takeaways--summary\"\u003eKey Takeaways \u0026amp; Summary\u003c/a\u003e\u003c/li\u003e\n\u003c/ol\u003e\n\u003chr\u003e\n\u003ch2 id=\"what-is-the-hashmap-pattern\"\u003eWhat is the HashMap Pattern?\u003c/h2\u003e\n\u003cp\u003eIn the previous patterns, we\u0026rsquo;ve already used hashmaps as a supporting tool  -  tracking character frequencies in sliding window, storing prefix sums for subarray problems. In this section, we focus on problems where \u003cstrong\u003ethe hashmap is the core of the solution\u003c/strong\u003e, not just a helper.\u003c/p\u003e","tags":["dsa","dsa-patterns"],"title":"HashMap and Frequency Counting Pattern  -  Complete Guide"},{"content":"Linked Lists Table of Contents What You Will Learn in This Blog Why Linked Lists Exist What is a Node What is a Linked List Singly Linked List Building the Node Class Building the LinkedList Class Append \u0026ndash; Insert at Tail Prepend \u0026ndash; Insert at Head Insert at Position Delete by Value Delete at Position Search Traverse and Print Length of the List Reverse a Linked List Singly Linked List \u0026ndash; Complete Implementation Time and Space Complexity Summary Doubly Linked List Building the DoublyNode Class Building the DoublyLinkedList Class Append and Prepend Insert at Position Delete by Value Traverse Forward and Backward Doubly Linked List \u0026ndash; Complete Implementation Time and Space Complexity Summary Circular Linked List What Makes it Circular Building the CircularLinkedList Class Append and Prepend Delete by Value Traverse Circular Linked List \u0026ndash; Complete Implementation Time and Space Complexity Summary Arrays vs Linked Lists \u0026ndash; When to Use Which Python and Linked Lists Patterns and Techniques Pattern 1 \u0026ndash; Fast and Slow Pointers Pattern 2 \u0026ndash; In-place Reversal Summary Key Takeaways Practice Problems Next Steps What You Will Learn in This Blog Why linked lists exist and what problem they solve compared to arrays What a node and a pointer are How to build a Singly Linked List from scratch with all operations How to build a Doubly Linked List from scratch with all operations How to build a Circular Linked List from scratch with all operations Time and space complexity of every operation Two patterns: Fast and Slow Pointers, and In-place Reversal Solved LeetCode problems with full explanation A practice problem table for further study Why Linked Lists Exist In the Arrays blog, we learned that arrays store elements in contiguous memory \u0026ndash; one right after the other. This gives us O(1) access by index, which is powerful. But contiguous memory comes with a cost:\nWhen you insert or delete an element in the middle of an array, every element after that position must shift. That is O(n). When a dynamic array runs out of capacity, it must allocate a new larger block and copy everything over. You need a contiguous block of memory large enough to hold the entire array. If your memory is fragmented, this can be a problem. A linked list solves these problems by giving up contiguous memory entirely. Elements can live anywhere in memory. Each element just needs to know where the next one is. This makes insertion and deletion at any position O(1) \u0026ndash; provided you already have a reference to the position \u0026ndash; because no shifting is required.\nThe trade-off is that you lose O(1) index access. To reach the nth element, you must walk through all n-1 elements before it. That is O(n).\nUnderstanding this trade-off is the entire reason linked lists exist.\nWhat is a Node A linked list is made up of nodes. Each node is a small object that holds two things:\nThe data \u0026ndash; the actual value being stored A pointer (also called a reference or link) \u0026ndash; the memory address of the next node in the list Think of nodes like train carriages. Each carriage holds passengers (the data) and is connected to the next carriage by a coupling (the pointer). The train can be as long as needed, and carriages do not need to be in any particular location in the train yard \u0026ndash; they just need to be connected.\nWhat is a Linked List A linked list is a chain of nodes. The list keeps track of:\nThe head \u0026ndash; a pointer to the first node The tail \u0026ndash; a pointer to the last node (optional but useful) The size \u0026ndash; the number of nodes (optional but useful) The last node\u0026rsquo;s pointer points to None, signalling the end of the list.\nhead | [10 | *] --\u0026gt; [20 | *] --\u0026gt; [30 | *] --\u0026gt; [40 | None] | tail Singly Linked List In a singly linked list, each node has one pointer \u0026ndash; pointing to the next node only. You can only traverse in one direction: forward.\nBuilding the Node Class The node is the fundamental building block. We implement it as a simple class with two attributes.\nclass Node: \u0026#34;\u0026#34;\u0026#34; A single node in a singly linked list. Attributes: data: the value stored in this node next: a reference to the next node, or None if this is the last node \u0026#34;\u0026#34;\u0026#34; def __init__(self, data): self.data = data # the value self.next = None # pointer to the next node, starts as None Let us create a few nodes manually to understand how they connect:\n# Create three nodes node1 = Node(10) node2 = Node(20) node3 = Node(30) # Connect them manually node1.next = node2 # node1 points to node2 node2.next = node3 # node2 points to node3 # node3.next is already None -- end of list # Traverse manually current = node1 while current is not None: print(current.data) # 10, then 20, then 30 current = current.next This is exactly what a linked list does internally \u0026ndash; just wrapped in a cleaner interface.\nBuilding the LinkedList Class The LinkedList class manages the chain of nodes. It keeps track of the head, tail, and size.\nclass LinkedList: \u0026#34;\u0026#34;\u0026#34; A singly linked list. Attributes: head: reference to the first node, or None if the list is empty tail: reference to the last node, or None if the list is empty size: number of nodes in the list \u0026#34;\u0026#34;\u0026#34; def __init__(self): self.head = None # empty list has no head self.tail = None # empty list has no tail self.size = 0 Append \u0026ndash; Insert at Tail Adding a new node at the end of the list.\ndef append(self, data): \u0026#34;\u0026#34;\u0026#34; Insert a new node at the tail of the list. Steps: 1. Create a new node 2. If list is empty, new node is both head and tail 3. Otherwise, point current tail\u0026#39;s next to new node, then update tail to new node Time complexity: O(1) -- we have a direct reference to the tail Space complexity: O(1) -- only one new node created \u0026#34;\u0026#34;\u0026#34; new_node = Node(data) if self.head is None: # list is empty self.head = new_node self.tail = new_node else: self.tail.next = new_node # link current tail to new node self.tail = new_node # update tail reference self.size += 1 Prepend \u0026ndash; Insert at Head Adding a new node at the beginning of the list.\ndef prepend(self, data): \u0026#34;\u0026#34;\u0026#34; Insert a new node at the head of the list. Steps: 1. Create a new node 2. Point new node\u0026#39;s next to current head 3. Update head to new node 4. If list was empty, also update tail Time complexity: O(1) -- direct head reference Space complexity: O(1) \u0026#34;\u0026#34;\u0026#34; new_node = Node(data) if self.head is None: # list is empty self.head = new_node self.tail = new_node else: new_node.next = self.head # new node points to old head self.head = new_node # head now points to new node self.size += 1 Insert at Position Inserting a node at a specific index (0-based).\ndef insert_at(self, data, position): \u0026#34;\u0026#34;\u0026#34; Insert a new node at the given position (0-based index). Steps: 1. Validate position 2. If position is 0, use prepend 3. If position is last, use append 4. Otherwise, traverse to the node just before the position, then rewire the pointers Time complexity: O(n) -- must traverse to find the position Space complexity: O(1) \u0026#34;\u0026#34;\u0026#34; if position \u0026lt; 0 or position \u0026gt; self.size: raise IndexError(\u0026#34;Position out of range\u0026#34;) if position == 0: self.prepend(data) return if position == self.size: self.append(data) return new_node = Node(data) # Traverse to the node just BEFORE the target position current = self.head for _ in range(position - 1): current = current.next # Rewire pointers: # new_node points to the node that was at position # current (node at position-1) points to new_node new_node.next = current.next current.next = new_node self.size += 1 Let us visualise what happens when we insert at position 2 in [10 -\u0026gt; 20 -\u0026gt; 30 -\u0026gt; 40]:\nBefore: [10 | *] --\u0026gt; [20 | *] --\u0026gt; [30 | *] --\u0026gt; [40 | None] ^ current (position 1, just before position 2) After inserting 99 at position 2: [10 | *] --\u0026gt; [20 | *] --\u0026gt; [99 | *] --\u0026gt; [30 | *] --\u0026gt; [40 | None] new_node Delete by Value Remove the first node that contains a given value.\ndef delete_by_value(self, data): \u0026#34;\u0026#34;\u0026#34; Remove the first node with the given value. Steps: 1. If list is empty, do nothing 2. If head contains the value, update head 3. Otherwise, traverse to find the node just before the target, then bypass the target node Time complexity: O(n) -- may need to traverse entire list Space complexity: O(1) \u0026#34;\u0026#34;\u0026#34; if self.head is None: return # empty list, nothing to delete # Special case: head is the node to delete if self.head.data == data: self.head = self.head.next if self.head is None: # list is now empty self.tail = None self.size -= 1 return # Traverse to find the node just BEFORE the target current = self.head while current.next is not None: if current.next.data == data: # Found it -- bypass the target node if current.next == self.tail: # target is the tail self.tail = current current.next = current.next.next self.size -= 1 return current = current.next # Value not found -- do nothing Visualising deletion of value 20 from [10 -\u0026gt; 20 -\u0026gt; 30 -\u0026gt; 40]:\nBefore: [10 | *] --\u0026gt; [20 | *] --\u0026gt; [30 | *] --\u0026gt; [40 | None] ^ ^ current current.next (this is the one to delete) After: [10 | *] --\u0026gt; [30 | *] --\u0026gt; [40 | None] ^ current.next now skips over 20 Delete at Position Remove the node at a specific index.\ndef delete_at(self, position): \u0026#34;\u0026#34;\u0026#34; Remove the node at the given position (0-based index). Time complexity: O(n) -- traverse to position Space complexity: O(1) \u0026#34;\u0026#34;\u0026#34; if position \u0026lt; 0 or position \u0026gt;= self.size: raise IndexError(\u0026#34;Position out of range\u0026#34;) if position == 0: # Delete head self.head = self.head.next if self.head is None: self.tail = None self.size -= 1 return # Traverse to the node just before the target position current = self.head for _ in range(position - 1): current = current.next # Bypass the node at position if current.next == self.tail: # deleting the tail self.tail = current current.next = current.next.next self.size -= 1 Search Find whether a value exists and return its position.\ndef search(self, data): \u0026#34;\u0026#34;\u0026#34; Search for a value in the list. Returns the 0-based index of the first occurrence, or -1 if not found. Time complexity: O(n) -- may scan the entire list Space complexity: O(1) \u0026#34;\u0026#34;\u0026#34; current = self.head index = 0 while current is not None: if current.data == data: return index # found at this index current = current.next index += 1 return -1 # not found Traverse and Print Walk through every node and print the values.\ndef print_list(self): \u0026#34;\u0026#34;\u0026#34; Print all values in the list, separated by arrows. Time complexity: O(n) -- visits every node Space complexity: O(1) \u0026#34;\u0026#34;\u0026#34; elements = [] current = self.head while current is not None: elements.append(str(current.data)) current = current.next print(\u0026#34; -\u0026gt; \u0026#34;.join(elements) + \u0026#34; -\u0026gt; None\u0026#34;) Length of the List def __len__(self): \u0026#34;\u0026#34;\u0026#34; Return the number of nodes. We track size as we insert/delete so this is O(1). Time complexity: O(1) Space complexity: O(1) \u0026#34;\u0026#34;\u0026#34; return self.size Reverse a Linked List Reversing a linked list in place is one of the most important linked list operations and a common interview topic. The key is to reverse the direction of every pointer.\ndef reverse(self): \u0026#34;\u0026#34;\u0026#34; Reverse the linked list in place by reversing all next pointers. We use three pointers: - prev: the node that current.next should point to after reversal - current: the node we are currently processing - next_node: saved reference to current.next before we overwrite it Walk through the list, reversing each pointer as we go. At the end, swap head and tail. Time complexity: O(n) -- visits every node exactly once Space complexity: O(1) -- only three pointer variables \u0026#34;\u0026#34;\u0026#34; prev = None current = self.head self.tail = self.head # current head will become the new tail while current is not None: next_node = current.next # save next before we overwrite current.next = prev # reverse the pointer prev = current # move prev forward current = next_node # move current forward self.head = prev # prev is now the last node we processed -- the new head Let us trace through reversing [10 -\u0026gt; 20 -\u0026gt; 30 -\u0026gt; None] step by step:\nStep prev current current.next (after reversal) Start None 10 \u0026ndash; Step 1 None 10 None (was 20) Step 2 10 20 10 (was 30) Step 3 20 30 20 (was None) End 30 None \u0026ndash; Result: [30 -\u0026gt; 20 -\u0026gt; 10 -\u0026gt; None], head = 30, tail = 10.\nSingly Linked List \u0026ndash; Complete Implementation Here is the complete implementation in one place for easy reference.\nclass Node: def __init__(self, data): self.data = data self.next = None class LinkedList: def __init__(self): self.head = None self.tail = None self.size = 0 def append(self, data): \u0026#34;\u0026#34;\u0026#34;Insert at tail -- O(1)\u0026#34;\u0026#34;\u0026#34; new_node = Node(data) if self.head is None: self.head = new_node self.tail = new_node else: self.tail.next = new_node self.tail = new_node self.size += 1 def prepend(self, data): \u0026#34;\u0026#34;\u0026#34;Insert at head -- O(1)\u0026#34;\u0026#34;\u0026#34; new_node = Node(data) if self.head is None: self.head = new_node self.tail = new_node else: new_node.next = self.head self.head = new_node self.size += 1 def insert_at(self, data, position): \u0026#34;\u0026#34;\u0026#34;Insert at position -- O(n)\u0026#34;\u0026#34;\u0026#34; if position \u0026lt; 0 or position \u0026gt; self.size: raise IndexError(\u0026#34;Position out of range\u0026#34;) if position == 0: self.prepend(data) return if position == self.size: self.append(data) return new_node = Node(data) current = self.head for _ in range(position - 1): current = current.next new_node.next = current.next current.next = new_node self.size += 1 def delete_by_value(self, data): \u0026#34;\u0026#34;\u0026#34;Delete first node with value -- O(n)\u0026#34;\u0026#34;\u0026#34; if self.head is None: return if self.head.data == data: self.head = self.head.next if self.head is None: self.tail = None self.size -= 1 return current = self.head while current.next is not None: if current.next.data == data: if current.next == self.tail: self.tail = current current.next = current.next.next self.size -= 1 return current = current.next def delete_at(self, position): \u0026#34;\u0026#34;\u0026#34;Delete node at position -- O(n)\u0026#34;\u0026#34;\u0026#34; if position \u0026lt; 0 or position \u0026gt;= self.size: raise IndexError(\u0026#34;Position out of range\u0026#34;) if position == 0: self.head = self.head.next if self.head is None: self.tail = None self.size -= 1 return current = self.head for _ in range(position - 1): current = current.next if current.next == self.tail: self.tail = current current.next = current.next.next self.size -= 1 def search(self, data): \u0026#34;\u0026#34;\u0026#34;Search for value, return index or -1 -- O(n)\u0026#34;\u0026#34;\u0026#34; current = self.head index = 0 while current is not None: if current.data == data: return index current = current.next index += 1 return -1 def reverse(self): \u0026#34;\u0026#34;\u0026#34;Reverse the list in place -- O(n)\u0026#34;\u0026#34;\u0026#34; prev = None current = self.head self.tail = self.head while current is not None: next_node = current.next current.next = prev prev = current current = next_node self.head = prev def print_list(self): \u0026#34;\u0026#34;\u0026#34;Print all values -- O(n)\u0026#34;\u0026#34;\u0026#34; elements = [] current = self.head while current is not None: elements.append(str(current.data)) current = current.next print(\u0026#34; -\u0026gt; \u0026#34;.join(elements) + \u0026#34; -\u0026gt; None\u0026#34;) def __len__(self): return self.size # --- Test the complete implementation --- ll = LinkedList() ll.append(10) ll.append(20) ll.append(30) ll.append(40) ll.print_list() # 10 -\u0026gt; 20 -\u0026gt; 30 -\u0026gt; 40 -\u0026gt; None ll.prepend(5) ll.print_list() # 5 -\u0026gt; 10 -\u0026gt; 20 -\u0026gt; 30 -\u0026gt; 40 -\u0026gt; None ll.insert_at(99, 2) ll.print_list() # 5 -\u0026gt; 10 -\u0026gt; 99 -\u0026gt; 20 -\u0026gt; 30 -\u0026gt; 40 -\u0026gt; None ll.delete_by_value(99) ll.print_list() # 5 -\u0026gt; 10 -\u0026gt; 20 -\u0026gt; 30 -\u0026gt; 40 -\u0026gt; None ll.delete_at(0) ll.print_list() # 10 -\u0026gt; 20 -\u0026gt; 30 -\u0026gt; 40 -\u0026gt; None print(ll.search(30)) # 2 print(ll.search(99)) # -1 print(len(ll)) # 4 ll.reverse() ll.print_list() # 40 -\u0026gt; 30 -\u0026gt; 20 -\u0026gt; 10 -\u0026gt; None Time and Space Complexity Summary Operation Time Complexity Space Complexity Notes Append (insert at tail) O(1) O(1) Direct tail reference Prepend (insert at head) O(1) O(1) Direct head reference Insert at position O(n) O(1) Must traverse to position Delete by value O(n) O(1) Must traverse to find value Delete at position O(n) O(1) Must traverse to position Search O(n) O(1) Must scan entire list Access by index O(n) O(1) No direct index access Reverse O(n) O(1) Visits every node once Traverse / Print O(n) O(1) Length O(1) O(1) Tracked as a counter Doubly Linked List In a doubly linked list, each node has two pointers \u0026ndash; one to the next node and one to the previous node. This allows traversal in both directions.\nNone \u0026lt;-- [10 | prev | next] \u0026lt;--\u0026gt; [20 | prev | next] \u0026lt;--\u0026gt; [30 | prev | next] --\u0026gt; None head tail The main advantages over a singly linked list:\nYou can traverse backward Deleting a node is O(1) if you have a direct reference to it (no need to find the previous node) Easier to implement some operations like delete at tail The cost is slightly more memory per node (one extra pointer) and slightly more complex pointer management.\nBuilding the DoublyNode Class class DoublyNode: \u0026#34;\u0026#34;\u0026#34; A single node in a doubly linked list. Attributes: data: the value stored next: reference to the next node prev: reference to the previous node \u0026#34;\u0026#34;\u0026#34; def __init__(self, data): self.data = data self.next = None # pointer to next node self.prev = None # pointer to previous node Building the DoublyLinkedList Class class DoublyLinkedList: \u0026#34;\u0026#34;\u0026#34; A doubly linked list where each node has both next and prev pointers. \u0026#34;\u0026#34;\u0026#34; def __init__(self): self.head = None self.tail = None self.size = 0 Append and Prepend def append(self, data): \u0026#34;\u0026#34;\u0026#34; Insert at tail. Time complexity: O(1) Space complexity: O(1) \u0026#34;\u0026#34;\u0026#34; new_node = DoublyNode(data) if self.head is None: self.head = new_node self.tail = new_node else: new_node.prev = self.tail # new node\u0026#39;s prev points to old tail self.tail.next = new_node # old tail\u0026#39;s next points to new node self.tail = new_node # update tail self.size += 1 def prepend(self, data): \u0026#34;\u0026#34;\u0026#34; Insert at head. Time complexity: O(1) Space complexity: O(1) \u0026#34;\u0026#34;\u0026#34; new_node = DoublyNode(data) if self.head is None: self.head = new_node self.tail = new_node else: new_node.next = self.head # new node\u0026#39;s next points to old head self.head.prev = new_node # old head\u0026#39;s prev points to new node self.head = new_node # update head self.size += 1 Insert at Position (Doubly) def insert_at(self, data, position): \u0026#34;\u0026#34;\u0026#34; Insert at given position (0-based). Time complexity: O(n) Space complexity: O(1) \u0026#34;\u0026#34;\u0026#34; if position \u0026lt; 0 or position \u0026gt; self.size: raise IndexError(\u0026#34;Position out of range\u0026#34;) if position == 0: self.prepend(data) return if position == self.size: self.append(data) return new_node = DoublyNode(data) # Traverse to the node currently at the target position current = self.head for _ in range(position): current = current.next # Insert new_node before current prev_node = current.prev new_node.next = current # new node points forward to current new_node.prev = prev_node # new node points back to prev prev_node.next = new_node # prev points forward to new node current.prev = new_node # current points back to new node self.size += 1 Delete by Value (Doubly) The doubly linked list makes deletion cleaner \u0026ndash; when we find the node, we can directly access its previous node through node.prev.\ndef delete_by_value(self, data): \u0026#34;\u0026#34;\u0026#34; Delete first node with given value. Because we have prev pointers, we do not need a separate \u0026#39;previous node\u0026#39; variable during traversal. Time complexity: O(n) -- traverse to find the value Space complexity: O(1) \u0026#34;\u0026#34;\u0026#34; current = self.head while current is not None: if current.data == data: # Rewire prev and next nodes to skip current if current.prev is not None: current.prev.next = current.next # skip current going forward else: self.head = current.next # deleting the head if current.next is not None: current.next.prev = current.prev # skip current going backward else: self.tail = current.prev # deleting the tail self.size -= 1 return current = current.next Traverse Forward and Backward def print_forward(self): \u0026#34;\u0026#34;\u0026#34;Traverse from head to tail -- O(n)\u0026#34;\u0026#34;\u0026#34; elements = [] current = self.head while current is not None: elements.append(str(current.data)) current = current.next print(\u0026#34;Forward: None \u0026lt;-\u0026gt; \u0026#34; + \u0026#34; \u0026lt;-\u0026gt; \u0026#34;.join(elements) + \u0026#34; \u0026lt;-\u0026gt; None\u0026#34;) def print_backward(self): \u0026#34;\u0026#34;\u0026#34;Traverse from tail to head -- O(n)\u0026#34;\u0026#34;\u0026#34; elements = [] current = self.tail while current is not None: elements.append(str(current.data)) current = current.prev print(\u0026#34;Backward: None \u0026lt;-\u0026gt; \u0026#34; + \u0026#34; \u0026lt;-\u0026gt; \u0026#34;.join(elements) + \u0026#34; \u0026lt;-\u0026gt; None\u0026#34;) Doubly Linked List \u0026ndash; Complete Implementation class DoublyNode: def __init__(self, data): self.data = data self.next = None self.prev = None class DoublyLinkedList: def __init__(self): self.head = None self.tail = None self.size = 0 def append(self, data): \u0026#34;\u0026#34;\u0026#34;Insert at tail -- O(1)\u0026#34;\u0026#34;\u0026#34; new_node = DoublyNode(data) if self.head is None: self.head = new_node self.tail = new_node else: new_node.prev = self.tail self.tail.next = new_node self.tail = new_node self.size += 1 def prepend(self, data): \u0026#34;\u0026#34;\u0026#34;Insert at head -- O(1)\u0026#34;\u0026#34;\u0026#34; new_node = DoublyNode(data) if self.head is None: self.head = new_node self.tail = new_node else: new_node.next = self.head self.head.prev = new_node self.head = new_node self.size += 1 def insert_at(self, data, position): \u0026#34;\u0026#34;\u0026#34;Insert at position -- O(n)\u0026#34;\u0026#34;\u0026#34; if position \u0026lt; 0 or position \u0026gt; self.size: raise IndexError(\u0026#34;Position out of range\u0026#34;) if position == 0: self.prepend(data) return if position == self.size: self.append(data) return new_node = DoublyNode(data) current = self.head for _ in range(position): current = current.next prev_node = current.prev new_node.next = current new_node.prev = prev_node prev_node.next = new_node current.prev = new_node self.size += 1 def delete_by_value(self, data): \u0026#34;\u0026#34;\u0026#34;Delete first node with value -- O(n)\u0026#34;\u0026#34;\u0026#34; current = self.head while current is not None: if current.data == data: if current.prev is not None: current.prev.next = current.next else: self.head = current.next if current.next is not None: current.next.prev = current.prev else: self.tail = current.prev self.size -= 1 return current = current.next def print_forward(self): \u0026#34;\u0026#34;\u0026#34;Print head to tail -- O(n)\u0026#34;\u0026#34;\u0026#34; elements = [] current = self.head while current is not None: elements.append(str(current.data)) current = current.next print(\u0026#34;Forward: \u0026#34; + \u0026#34; \u0026lt;-\u0026gt; \u0026#34;.join(elements)) def print_backward(self): \u0026#34;\u0026#34;\u0026#34;Print tail to head -- O(n)\u0026#34;\u0026#34;\u0026#34; elements = [] current = self.tail while current is not None: elements.append(str(current.data)) current = current.prev print(\u0026#34;Backward: \u0026#34; + \u0026#34; \u0026lt;-\u0026gt; \u0026#34;.join(elements)) def __len__(self): return self.size # --- Test --- dll = DoublyLinkedList() dll.append(10) dll.append(20) dll.append(30) dll.prepend(5) dll.print_forward() # Forward: 5 \u0026lt;-\u0026gt; 10 \u0026lt;-\u0026gt; 20 \u0026lt;-\u0026gt; 30 dll.print_backward() # Backward: 30 \u0026lt;-\u0026gt; 20 \u0026lt;-\u0026gt; 10 \u0026lt;-\u0026gt; 5 dll.insert_at(99, 2) dll.print_forward() # Forward: 5 \u0026lt;-\u0026gt; 10 \u0026lt;-\u0026gt; 99 \u0026lt;-\u0026gt; 20 \u0026lt;-\u0026gt; 30 dll.delete_by_value(99) dll.print_forward() # Forward: 5 \u0026lt;-\u0026gt; 10 \u0026lt;-\u0026gt; 20 \u0026lt;-\u0026gt; 30 Doubly Linked List Complexity Operation Time Complexity Space Complexity Notes Append O(1) O(1) Direct tail reference Prepend O(1) O(1) Direct head reference Insert at position O(n) O(1) Traverse to position Delete by value O(n) O(1) Traverse to find value Delete with direct reference O(1) O(1) No traversal needed \u0026ndash; key advantage Traverse forward O(n) O(1) Traverse backward O(n) O(1) Not possible with singly Search O(n) O(1) Circular Linked List In a circular linked list, the last node does not point to None. Instead, it points back to the first node, forming a circle.\nhead | [10 | *] --\u0026gt; [20 | *] --\u0026gt; [30 | *] --\u0026gt; [40 | *] ^ | |________________________________________| Where is this useful?\nRound-robin scheduling (CPU processes, game turns) Music or video playlists that loop Any problem where you need to cycle through a collection repeatedly without checking for the end The key challenge with circular lists is avoiding infinite loops during traversal. You must track when you have come back to the head.\nWhat Makes it Circular The only structural difference from a singly linked list: the tail\u0026rsquo;s next pointer points to head instead of None.\n# In a regular singly list: tail.next = None # In a circular list: tail.next = head # connects back to the beginning Building the CircularLinkedList Class We reuse the same Node class. The circular list class tracks head and tail.\nclass CircularLinkedList: \u0026#34;\u0026#34;\u0026#34; A circular singly linked list. The tail\u0026#39;s next pointer points back to the head. \u0026#34;\u0026#34;\u0026#34; def __init__(self): self.head = None self.tail = None self.size = 0 Circular Append and Prepend def append(self, data): \u0026#34;\u0026#34;\u0026#34; Insert at tail and maintain the circular connection. Time complexity: O(1) Space complexity: O(1) \u0026#34;\u0026#34;\u0026#34; new_node = Node(data) if self.head is None: # first node -- points to itself self.head = new_node self.tail = new_node new_node.next = self.head # circular: points to itself else: new_node.next = self.head # new node points back to head (circular) self.tail.next = new_node # old tail points to new node self.tail = new_node # update tail self.size += 1 def prepend(self, data): \u0026#34;\u0026#34;\u0026#34; Insert at head and maintain the circular connection. Time complexity: O(1) Space complexity: O(1) \u0026#34;\u0026#34;\u0026#34; new_node = Node(data) if self.head is None: self.head = new_node self.tail = new_node new_node.next = self.head else: new_node.next = self.head # new node points to old head self.head = new_node # update head self.tail.next = self.head # tail must now point to new head (circular) self.size += 1 Circular Delete by Value def delete_by_value(self, data): \u0026#34;\u0026#34;\u0026#34; Delete first node with the given value. Must handle the circular connection carefully: - Deleting the head requires updating tail.next - Deleting the tail requires updating tail reference and its prev\u0026#39;s next Time complexity: O(n) Space complexity: O(1) \u0026#34;\u0026#34;\u0026#34; if self.head is None: return # Only one node in the list if self.head == self.tail and self.head.data == data: self.head = None self.tail = None self.size -= 1 return # Deleting the head if self.head.data == data: self.head = self.head.next self.tail.next = self.head # maintain circular connection self.size -= 1 return # Traverse to find the node just before the target current = self.head while current.next != self.head: # stop before we loop back if current.next.data == data: if current.next == self.tail: # deleting the tail self.tail = current self.tail.next = self.head else: current.next = current.next.next self.size -= 1 return current = current.next Circular Traverse Traversal must stop when we reach the head again \u0026ndash; otherwise we loop forever.\ndef print_list(self): \u0026#34;\u0026#34;\u0026#34; Traverse and print all values. Must stop when we return to head to avoid infinite loop. Time complexity: O(n) Space complexity: O(1) \u0026#34;\u0026#34;\u0026#34; if self.head is None: print(\u0026#34;Empty list\u0026#34;) return elements = [] current = self.head while True: elements.append(str(current.data)) current = current.next if current == self.head: # we have looped back to start -- stop break print(\u0026#34; -\u0026gt; \u0026#34;.join(elements) + \u0026#34; -\u0026gt; (back to head)\u0026#34;) Circular Linked List \u0026ndash; Complete Implementation class CircularLinkedList: def __init__(self): self.head = None self.tail = None self.size = 0 def append(self, data): \u0026#34;\u0026#34;\u0026#34;Insert at tail -- O(1)\u0026#34;\u0026#34;\u0026#34; new_node = Node(data) if self.head is None: self.head = new_node self.tail = new_node new_node.next = self.head else: new_node.next = self.head self.tail.next = new_node self.tail = new_node self.size += 1 def prepend(self, data): \u0026#34;\u0026#34;\u0026#34;Insert at head -- O(1)\u0026#34;\u0026#34;\u0026#34; new_node = Node(data) if self.head is None: self.head = new_node self.tail = new_node new_node.next = self.head else: new_node.next = self.head self.head = new_node self.tail.next = self.head self.size += 1 def delete_by_value(self, data): \u0026#34;\u0026#34;\u0026#34;Delete first node with value -- O(n)\u0026#34;\u0026#34;\u0026#34; if self.head is None: return if self.head == self.tail and self.head.data == data: self.head = None self.tail = None self.size -= 1 return if self.head.data == data: self.head = self.head.next self.tail.next = self.head self.size -= 1 return current = self.head while current.next != self.head: if current.next.data == data: if current.next == self.tail: self.tail = current self.tail.next = self.head else: current.next = current.next.next self.size -= 1 return current = current.next def print_list(self): \u0026#34;\u0026#34;\u0026#34;Print all values -- O(n)\u0026#34;\u0026#34;\u0026#34; if self.head is None: print(\u0026#34;Empty list\u0026#34;) return elements = [] current = self.head while True: elements.append(str(current.data)) current = current.next if current == self.head: break print(\u0026#34; -\u0026gt; \u0026#34;.join(elements) + \u0026#34; -\u0026gt; (back to head)\u0026#34;) def __len__(self): return self.size # --- Test --- cll = CircularLinkedList() cll.append(10) cll.append(20) cll.append(30) cll.prepend(5) cll.print_list() # 5 -\u0026gt; 10 -\u0026gt; 20 -\u0026gt; 30 -\u0026gt; (back to head) cll.delete_by_value(20) cll.print_list() # 5 -\u0026gt; 10 -\u0026gt; 30 -\u0026gt; (back to head) print(len(cll)) # 3 Circular Linked List Complexity Operation Time Complexity Space Complexity Notes Append O(1) O(1) Direct tail reference Prepend O(1) O(1) Direct head reference, tail.next update Delete by value O(n) O(1) Traverse to find value Traverse O(n) O(1) Stop condition is returning to head Search O(n) O(1) Arrays vs Linked Lists \u0026ndash; When to Use Which Factor Array Linked List Access by index O(1) O(n) Search O(n) O(n) Insert at head O(n) \u0026ndash; shift required O(1) Insert at tail O(1) amortised O(1) with tail pointer Insert in middle O(n) \u0026ndash; shift required O(n) \u0026ndash; traverse + O(1) rewire Delete at head O(n) \u0026ndash; shift required O(1) Delete in middle O(n) \u0026ndash; shift required O(n) traverse + O(1) rewire Memory layout Contiguous Scattered Memory overhead Low (values only) Higher (values + pointers) Cache performance Better (contiguous) Worse (scattered) Dynamic size Yes (amortised) Yes (naturally) Use an array when:\nYou need frequent access by index You know the size in advance or it does not change much Memory locality and cache performance matter Use a linked list when:\nYou need frequent insertions or deletions at the head You are implementing a stack, queue, or deque You do not need random access by index Python and Linked Lists Python does not have a built-in linked list. However, collections.deque is implemented as a doubly linked list internally. This is why deque gives O(1) append and pop from both ends, while a list gives O(n) for operations at the front.\nfrom collections import deque dq = deque([1, 2, 3]) dq.appendleft(0) # O(1) -- uses the doubly linked list structure dq.popleft() # O(1) -- same reason In DSA problems you will usually build your own linked list from scratch using the Node class, because problems test your ability to manipulate pointers directly.\nPatterns and Techniques Pattern 1 \u0026ndash; Fast and Slow Pointers The core idea: Use two pointers that move through the list at different speeds \u0026ndash; one moves one step at a time (slow), the other moves two steps at a time (fast). This technique detects cycles, finds midpoints, and solves several other problems without extra space.\nAlso called Floyd\u0026rsquo;s Tortoise and Hare algorithm.\nThe template:\nslow = head fast = head while fast is not None and fast.next is not None: slow = slow.next # moves one step fast = fast.next.next # moves two steps When fast reaches the end, slow is at the middle. If fast ever meets slow again (after moving past the start), there is a cycle.\nSolved Problem \u0026ndash; Middle of the Linked List LeetCode 876 \u0026ndash; Middle of the Linked List\nProblem: Given the head of a singly linked list, return the middle node. If there are two middle nodes, return the second one.\nInput: [1, 2, 3, 4, 5] Output: Node with value 3 Input: [1, 2, 3, 4, 5, 6] Output: Node with value 4 (second middle) Thinking through it:\nThe brute force would be to count the nodes, then traverse to n//2. That is O(n) time with O(1) space but requires two passes. Fast and slow pointers solve it in one pass: when fast reaches the end, slow is at the middle.\ndef middle_node(head): \u0026#34;\u0026#34;\u0026#34; Fast pointer moves twice as fast as slow. When fast reaches the end, slow is at the middle. For even length: fast ends at None (past last node) slow ends at second middle node Time complexity: O(n) -- single pass Space complexity: O(1) -- two pointer variables only \u0026#34;\u0026#34;\u0026#34; slow = head fast = head while fast is not None and fast.next is not None: slow = slow.next # one step fast = fast.next.next # two steps return slow # slow is now at the middle # Trace for [1, 2, 3, 4, 5]: # Start: slow=1, fast=1 # Step 1: slow=2, fast=3 # Step 2: slow=3, fast=5 # Step 3: fast.next is None -- stop. slow=3 (middle) Solved Problem \u0026ndash; Linked List Cycle LeetCode 141 \u0026ndash; Linked List Cycle\nProblem: Given the head of a linked list, determine if the list has a cycle. A cycle exists if some node can be reached again by following next pointers.\nInput: [3, 2, 0, -4] with tail connecting back to node at index 1 Output: True Input: [1, 2] with no cycle Output: False Thinking through it:\nIf there is no cycle, the fast pointer will reach None. If there is a cycle, the fast pointer will eventually lap the slow pointer and they will meet. Think of two runners on a circular track \u0026ndash; the faster one always catches the slower one.\ndef has_cycle(head): \u0026#34;\u0026#34;\u0026#34; Fast and slow pointers. If they ever meet, there is a cycle. If fast reaches None, there is no cycle. Time complexity: O(n) -- in a cycle, fast catches slow in at most n steps Space complexity: O(1) -- two pointer variables only \u0026#34;\u0026#34;\u0026#34; slow = head fast = head while fast is not None and fast.next is not None: slow = slow.next fast = fast.next.next if slow == fast: # they met -- cycle detected return True return False # fast reached end -- no cycle Pattern 2 \u0026ndash; In-place Reversal The core idea: Reverse part or all of a linked list by rewiring pointers as you traverse, without allocating a new list. This is the same reversal technique shown in the implementation section, but now applied to specific subranges.\nWhen to use it:\nReverse the entire list Reverse a sublist between two positions Reverse in groups of k Solved Problem \u0026ndash; Reverse Linked List LeetCode 206 \u0026ndash; Reverse Linked List\nProblem: Given the head of a singly linked list, reverse the list and return the new head.\nInput: [1, 2, 3, 4, 5] Output: [5, 4, 3, 2, 1] Two approaches \u0026ndash; iterative and recursive:\ndef reverse_list_iterative(head): \u0026#34;\u0026#34;\u0026#34; Three-pointer approach. At each step, reverse one pointer. Time complexity: O(n) Space complexity: O(1) \u0026#34;\u0026#34;\u0026#34; prev = None current = head while current is not None: next_node = current.next # save next current.next = prev # reverse the pointer prev = current # move prev forward current = next_node # move current forward return prev # prev is the new head def reverse_list_recursive(head): \u0026#34;\u0026#34;\u0026#34; Recursive approach: reverse the rest of the list, then fix the current node\u0026#39;s pointer. Base case: empty list or single node -- already reversed. Recursive case: reverse everything after head, then make head.next point back to head. Time complexity: O(n) Space complexity: O(n) -- call stack depth \u0026#34;\u0026#34;\u0026#34; # Base case if head is None or head.next is None: return head # Reverse the rest of the list new_head = reverse_list_recursive(head.next) # Make the next node point back to current head head.next.next = head head.next = None # current head is now the tail return new_head Solved Problem \u0026ndash; Merge Two Sorted Lists LeetCode 21 \u0026ndash; Merge Two Sorted Lists\nProblem: You are given the heads of two sorted linked lists. Merge them into one sorted linked list and return its head.\nInput: list1 = [1, 2, 4], list2 = [1, 3, 4] Output: [1, 1, 2, 3, 4, 4] Thinking through it:\nUse a dummy head node to simplify edge cases. Compare the current nodes of both lists, attach the smaller one to the result, and advance that pointer. When one list is exhausted, attach the remainder of the other.\ndef merge_two_lists(list1, list2): \u0026#34;\u0026#34;\u0026#34; Use a dummy head to avoid special-casing the first node. At each step, attach the smaller of the two current nodes. Time complexity: O(n + m) -- n and m are lengths of the two lists Space complexity: O(1) -- rewiring existing nodes, no new nodes created \u0026#34;\u0026#34;\u0026#34; # Dummy node -- its next will be the head of the merged list dummy = Node(0) current = dummy while list1 is not None and list2 is not None: if list1.data \u0026lt;= list2.data: current.next = list1 # attach list1\u0026#39;s node list1 = list1.next # advance list1 else: current.next = list2 # attach list2\u0026#39;s node list2 = list2.next # advance list2 current = current.next # advance result pointer # Attach the remaining nodes from whichever list is not exhausted if list1 is not None: current.next = list1 else: current.next = list2 return dummy.next # skip the dummy node, return actual head Summary What We Covered Topic Key Point Why linked lists exist Solve the O(n) insert/delete cost of arrays by giving up contiguous memory Nodes and pointers A node holds data and a reference to the next node Singly linked list One direction traversal, O(1) insert/delete at head and tail Doubly linked list Two direction traversal, O(1) delete with direct reference Circular linked list Tail points back to head, used for cyclic problems and round-robin Arrays vs linked lists Arrays for index access, linked lists for frequent head insertions Python deque Internally a doubly linked list, O(1) both ends Operation Complexity at a Glance Operation Singly Doubly Circular Insert at head O(1) O(1) O(1) Insert at tail O(1) O(1) O(1) Insert at position O(n) O(n) O(n) Delete at head O(1) O(1) O(1) Delete by value O(n) O(n) O(n) Delete with direct reference O(n) O(1) O(n) Search O(n) O(n) O(n) Access by index O(n) O(n) O(n) Reverse O(n) O(n) O(n) Patterns Introduced Pattern Core Idea Use When Fast and Slow Pointers Two pointers at different speeds Cycle detection, finding midpoint, nth from end In-place Reversal Rewire pointers as you traverse Reverse full list or sublist without extra space Key Takeaways Linked lists trade O(1) index access (which arrays have) for O(1) insert and delete at the head (which arrays do not have). Neither is strictly better \u0026ndash; the right choice depends on what operations you need most. Every pointer reassignment in a linked list must be done carefully and in the right order. Losing a reference means losing access to the rest of the list permanently. The dummy head node technique simplifies many linked list problems by removing special cases for operating on the head. Fast and slow pointers solve cycle detection and midpoint finding in a single O(n) pass with O(1) space. Recursive reversal is elegant but uses O(n) stack space. Iterative reversal is O(1) space and is preferred in practice. Always check for None before accessing .next. A null pointer error is the most common linked list bug. Practice Problems Problem Link Difficulty Pattern Reverse Linked List LC 206 Easy In-place Reversal Merge Two Sorted Lists LC 21 Easy Pointer manipulation Linked List Cycle LC 141 Easy Fast and Slow Pointers Middle of the Linked List LC 876 Easy Fast and Slow Pointers Remove Duplicates from Sorted List LC 83 Easy Pointer manipulation Remove Nth Node From End LC 19 Medium Fast and Slow Pointers Linked List Cycle II LC 142 Medium Fast and Slow Pointers Reverse Linked List II LC 92 Medium In-place Reversal Reorder List LC 143 Medium Fast/Slow + Reversal Copy List with Random Pointer LC 138 Medium Hash Map Merge K Sorted Lists LC 23 Hard Heap / Divide and Conquer Reverse Nodes in K-Group LC 25 Hard In-place Reversal Next Steps Next blog: [Stacks] \u0026ndash; a data structure built on top of linked lists or arrays, with a strict Last In First Out access pattern Reference: Codeverra DSA Sheet ","permalink":"https://blogs.codeverra.com/blog/dsa-06-linked-lists/","summary":"\u003ch1 id=\"linked-lists\"\u003eLinked Lists\u003c/h1\u003e\n\u003chr\u003e\n\u003ch2 id=\"table-of-contents\"\u003eTable of Contents\u003c/h2\u003e\n\u003cul\u003e\n\u003cli\u003e\u003ca href=\"#what-you-will-learn-in-this-blog\"\u003eWhat You Will Learn in This Blog\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#why-linked-lists-exist\"\u003eWhy Linked Lists Exist\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#what-is-a-node\"\u003eWhat is a Node\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#what-is-a-linked-list\"\u003eWhat is a Linked List\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#singly-linked-list\"\u003eSingly Linked List\u003c/a\u003e\n\u003cul\u003e\n\u003cli\u003e\u003ca href=\"#building-the-node-class\"\u003eBuilding the Node Class\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#building-the-linkedlist-class\"\u003eBuilding the LinkedList Class\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#append----insert-at-tail\"\u003eAppend \u0026ndash; Insert at Tail\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#prepend----insert-at-head\"\u003ePrepend \u0026ndash; Insert at Head\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#insert-at-position\"\u003eInsert at Position\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#delete-by-value\"\u003eDelete by Value\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#delete-at-position\"\u003eDelete at Position\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#search\"\u003eSearch\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#traverse-and-print\"\u003eTraverse and Print\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#length-of-the-list\"\u003eLength of the List\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#reverse-a-linked-list\"\u003eReverse a Linked List\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#singly-linked-list----complete-implementation\"\u003eSingly Linked List \u0026ndash; Complete Implementation\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#time-and-space-complexity-summary\"\u003eTime and Space Complexity Summary\u003c/a\u003e\u003c/li\u003e\n\u003c/ul\u003e\n\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#doubly-linked-list\"\u003eDoubly Linked List\u003c/a\u003e\n\u003cul\u003e\n\u003cli\u003e\u003ca href=\"#building-the-doublynode-class\"\u003eBuilding the DoublyNode Class\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#building-the-doublylinkedlist-class\"\u003eBuilding the DoublyLinkedList Class\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#append-and-prepend\"\u003eAppend and Prepend\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#insert-at-position-doubly\"\u003eInsert at Position\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#delete-by-value-doubly\"\u003eDelete by Value\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#traverse-forward-and-backward\"\u003eTraverse Forward and Backward\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#doubly-linked-list----complete-implementation\"\u003eDoubly Linked List \u0026ndash; Complete Implementation\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#doubly-linked-list-complexity\"\u003eTime and Space Complexity Summary\u003c/a\u003e\u003c/li\u003e\n\u003c/ul\u003e\n\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#circular-linked-list\"\u003eCircular Linked List\u003c/a\u003e\n\u003cul\u003e\n\u003cli\u003e\u003ca href=\"#what-makes-it-circular\"\u003eWhat Makes it Circular\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#building-the-circularlinkedlist-class\"\u003eBuilding the CircularLinkedList Class\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#circular-append-and-prepend\"\u003eAppend and Prepend\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#circular-delete-by-value\"\u003eDelete by Value\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#circular-traverse\"\u003eTraverse\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#circular-linked-list----complete-implementation\"\u003eCircular Linked List \u0026ndash; Complete Implementation\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#circular-linked-list-complexity\"\u003eTime and Space Complexity Summary\u003c/a\u003e\u003c/li\u003e\n\u003c/ul\u003e\n\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#arrays-vs-linked-lists----when-to-use-which\"\u003eArrays vs Linked Lists \u0026ndash; When to Use Which\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#python-and-linked-lists\"\u003ePython and Linked Lists\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#patterns-and-techniques\"\u003ePatterns and Techniques\u003c/a\u003e\n\u003cul\u003e\n\u003cli\u003e\u003ca href=\"#pattern-1----fast-and-slow-pointers\"\u003ePattern 1 \u0026ndash; Fast and Slow Pointers\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#pattern-2----in-place-reversal\"\u003ePattern 2 \u0026ndash; In-place Reversal\u003c/a\u003e\u003c/li\u003e\n\u003c/ul\u003e\n\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#summary\"\u003eSummary\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#key-takeaways\"\u003eKey Takeaways\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#practice-problems\"\u003ePractice Problems\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#next-steps\"\u003eNext Steps\u003c/a\u003e\u003c/li\u003e\n\u003c/ul\u003e\n\u003chr\u003e\n\u003ch2 id=\"what-you-will-learn-in-this-blog\"\u003eWhat You Will Learn in This Blog\u003c/h2\u003e\n\u003cul\u003e\n\u003cli\u003eWhy linked lists exist and what problem they solve compared to arrays\u003c/li\u003e\n\u003cli\u003eWhat a node and a pointer are\u003c/li\u003e\n\u003cli\u003eHow to build a Singly Linked List from scratch with all operations\u003c/li\u003e\n\u003cli\u003eHow to build a Doubly Linked List from scratch with all operations\u003c/li\u003e\n\u003cli\u003eHow to build a Circular Linked List from scratch with all operations\u003c/li\u003e\n\u003cli\u003eTime and space complexity of every operation\u003c/li\u003e\n\u003cli\u003eTwo patterns: Fast and Slow Pointers, and In-place Reversal\u003c/li\u003e\n\u003cli\u003eSolved LeetCode problems with full explanation\u003c/li\u003e\n\u003cli\u003eA practice problem table for further study\u003c/li\u003e\n\u003c/ul\u003e\n\u003chr\u003e\n\u003ch2 id=\"why-linked-lists-exist\"\u003eWhy Linked Lists Exist\u003c/h2\u003e\n\u003cp\u003eIn the Arrays blog, we learned that arrays store elements in \u003cstrong\u003econtiguous memory\u003c/strong\u003e \u0026ndash; one right after the other. This gives us O(1) access by index, which is powerful. But contiguous memory comes with a cost:\u003c/p\u003e","tags":["dsa"],"title":"Linked Lists in Python  -  Complete Guide"},{"content":"Logging and System Interaction in Python Professional Observability with the logging Module and the os Module Before We Begin \u0026ndash; Why This Matters More Than It Looks Here is a situation every developer eventually faces.\nYou write a script that processes data overnight. It runs on a server while you sleep. In the morning you check the output and something is wrong. Half the records are missing. You have no idea what happened because your script used print() for all its messages and those prints went nowhere \u0026ndash; there was nobody watching the terminal.\nOr this. A script runs fine on your laptop but crashes in production. You look at the server logs and find nothing useful because your code never wrote any logs. You have to add print statements, redeploy, and wait for the problem to happen again.\nBoth situations have the same root cause. print() is not a logging system. It has no levels, no timestamps, no file output, no way to turn it on or off without editing code, and no way to know which part of a large codebase generated a particular message.\nThe logging module solves all of this. It is what every professional Python project uses. Learning it properly is the difference between code that is observable and code that is a black box.\nThe os module solves a different but equally practical problem: your Python code needs to interact with the operating system. Read environment variables. Walk directory trees. Check if paths exist. Run shell commands. os is how Python talks to the machine it runs on.\nTable of Contents The Problem with print() The Five Log Levels Basic Logging with basicConfig The Right Way \u0026ndash; Named Loggers Handlers \u0026ndash; Where Logs Go Formatters \u0026ndash; What Logs Look Like Logging to Multiple Destinations Rotating Log Files Logging Across Multiple Modules A Production-Ready Logging Setup The os Module \u0026ndash; System Interaction Environment Variables \u0026ndash; The Right Way Walking Directory Trees with os.walk Running Shell Commands \u0026ndash; os.system vs subprocess Putting It Together \u0026ndash; A Data Pipeline Summary and Key Takeaways 1. The Problem with print() # Typical beginner approach -- print everything def process_orders(orders): print(\u0026#34;Starting order processing\u0026#34;) for order in orders: print(f\u0026#34;Processing order {order[\u0026#39;id\u0026#39;]}\u0026#34;) if order[\u0026#39;amount\u0026#39;] \u0026gt; 10000: print(f\u0026#34;Large order detected: {order[\u0026#39;id\u0026#39;]}\u0026#34;) result = calculate_total(order) print(f\u0026#34;Order {order[\u0026#39;id\u0026#39;]} done. Total: {result}\u0026#34;) print(\u0026#34;All orders processed\u0026#34;) This looks fine when you are watching a terminal. In production:\nNo timestamps. You cannot tell when something happened. No severity. Is \u0026ldquo;Large order detected\u0026rdquo; information or a warning? No persistence. Output disappears when the terminal closes. Cannot be filtered. You cannot turn off the noise without editing code. Cannot be routed. You cannot send errors to one place and info to another. No source information. In a large codebase, which file generated this? The logging module solves every one of these problems.\n# The professional approach import logging logger = logging.getLogger(__name__) def process_orders(orders): logger.info(\u0026#34;Starting order processing. Count: %d\u0026#34;, len(orders)) for order in orders: logger.debug(\u0026#34;Processing order %s\u0026#34;, order[\u0026#39;id\u0026#39;]) if order[\u0026#39;amount\u0026#39;] \u0026gt; 10000: logger.warning(\u0026#34;Large order: %s (Rs.%s)\u0026#34;, order[\u0026#39;id\u0026#39;], order[\u0026#39;amount\u0026#39;]) result = calculate_total(order) logger.debug(\u0026#34;Order %s complete. Total: Rs.%.2f\u0026#34;, order[\u0026#39;id\u0026#39;], result) logger.info(\u0026#34;All orders processed successfully\u0026#34;) Now every message has a level, can be filtered, can be written to a file, and carries the module name automatically.\n2. The Five Log Levels Log levels let you categorise the severity of each message. You set a minimum level and anything below it is silently ignored.\nLevel Value When to use it ──────────────────────────────────────────────────────────────────── DEBUG 10 Detailed diagnostic information. The most verbose level. Used during development to trace exactly what the program is doing step by step. Example: \u0026#34;Entering function calculate_total with args (order_id=42)\u0026#34; INFO 20 Confirmation that things are working as expected. High-level progress markers. Example: \u0026#34;Successfully loaded 500 records from CSV\u0026#34; WARNING 30 Something unexpected happened but the program can continue. Or a potential problem in the near future. Example: \u0026#34;Disk space below 10%. Consider cleaning up.\u0026#34; ERROR 40 A serious problem. The program could not do something it was supposed to do. Example: \u0026#34;Failed to connect to database after 3 retries\u0026#34; CRITICAL 50 A very serious error that may prevent the program from continuing at all. Example: \u0026#34;Could not write to output directory. Stopping.\u0026#34; import logging # Setting level to INFO means: show INFO, WARNING, ERROR, CRITICAL # but ignore DEBUG messages logging.basicConfig(level=logging.INFO) logging.debug(\u0026#34;This will NOT appear -- below INFO level\u0026#34;) logging.info(\u0026#34;This WILL appear\u0026#34;) logging.warning(\u0026#34;This WILL appear\u0026#34;) logging.error(\u0026#34;This WILL appear\u0026#34;) logging.critical(\u0026#34;This WILL appear\u0026#34;) The rule of thumb:\nDEBUG -- you want to see when actively debugging a problem INFO -- normal operation, always safe to show WARNING -- something to investigate, does not require immediate action ERROR -- something failed, requires attention CRITICAL -- the application cannot continue, requires immediate action 3. Basic Logging with basicConfig basicConfig is the quickest way to get logging working. It configures the root logger \u0026ndash; the parent of all loggers in your application.\nimport logging # Most basic setup -- logs to console at WARNING level by default logging.basicConfig() # Set level and format logging.basicConfig( level=logging.DEBUG, format=\u0026#34;%(asctime)s -- %(levelname)s -- %(message)s\u0026#34; ) # Log to a file instead of console logging.basicConfig( level=logging.INFO, filename=\u0026#34;app.log\u0026#34;, filemode=\u0026#34;a\u0026#34;, # \u0026#34;a\u0026#34; appends, \u0026#34;w\u0026#34; overwrites each run encoding=\u0026#34;utf-8\u0026#34;, format=\u0026#34;%(asctime)s -- %(levelname)s -- %(name)s -- %(message)s\u0026#34;, datefmt=\u0026#34;%Y-%m-%d %H:%M:%S\u0026#34; ) Important limitation of basicConfig: It only works once. If it has already been called (even by an imported library), subsequent calls do nothing. For anything beyond a quick script, use named loggers with explicit handlers instead.\n4. The Right Way \u0026ndash; Named Loggers The professional way to use logging is with named loggers, not the root logger.\nimport logging # Create a logger for this specific module # __name__ resolves to the module\u0026#39;s full dotted name # e.g., \u0026#34;mypackage.data_loader\u0026#34; or \u0026#34;__main__\u0026#34; in a script logger = logging.getLogger(__name__) This one line at the top of every module is the standard pattern. Here is why it matters:\n# file: data_loader.py import logging logger = logging.getLogger(__name__) # logger.name == \u0026#34;data_loader\u0026#34; # file: analytics.py import logging logger = logging.getLogger(__name__) # logger.name == \u0026#34;analytics\u0026#34; # file: payments/billing.py import logging logger = logging.getLogger(__name__) # logger.name == \u0026#34;payments.billing\u0026#34; Every log message now carries its origin automatically. When you see:\n2024-03-15 14:30:22 -- ERROR -- payments.billing -- Payment failed for order 1042 You know immediately which file to open.\nThe Logger Hierarchy Loggers form a tree based on their names, with the root logger at the top.\nroot | +-- data_loader | +-- analytics | +-- payments | +-- payments.billing | +-- payments.invoice A logger propagates messages up to its parent by default. This means if you configure the root logger, all child loggers inherit that configuration unless you override it.\nimport logging # Configure the root logger -- affects all loggers in the application logging.basicConfig(level=logging.DEBUG) # Now all loggers at any level will output their messages logger = logging.getLogger(\u0026#34;payments.billing\u0026#34;) logger.info(\u0026#34;This reaches the root logger\u0026#39;s handler\u0026#34;) 5. Handlers \u0026ndash; Where Logs Go A Handler decides where log messages are sent. You can attach multiple handlers to a single logger to send messages to multiple destinations.\nimport logging logger = logging.getLogger(\u0026#34;myapp\u0026#34;) logger.setLevel(logging.DEBUG) # set on the logger itself # StreamHandler -- sends to console (stdout or stderr) console_handler = logging.StreamHandler() console_handler.setLevel(logging.DEBUG) # FileHandler -- writes to a file file_handler = logging.FileHandler(\u0026#34;app.log\u0026#34;, encoding=\u0026#34;utf-8\u0026#34;) file_handler.setLevel(logging.WARNING) # only warnings and above to file # Attach both handlers to the logger logger.addHandler(console_handler) logger.addHandler(file_handler) Common handlers:\nimport logging import logging.handlers # Console logging.StreamHandler() # stdout/stderr # File logging.FileHandler(\u0026#34;app.log\u0026#34;) # plain file # Rotating by size logging.handlers.RotatingFileHandler( \u0026#34;app.log\u0026#34;, maxBytes=5 * 1024 * 1024, # 5 MB backupCount=3 # keep 3 old files ) # Rotating by time logging.handlers.TimedRotatingFileHandler( \u0026#34;app.log\u0026#34;, when=\u0026#34;midnight\u0026#34;, # rotate at midnight interval=1, backupCount=7 # keep 7 days of logs ) # Memory (stores in a list -- useful for testing) logging.handlers.MemoryHandler(capacity=1000) 6. Formatters \u0026ndash; What Logs Look Like A Formatter controls the structure of each log line.\nimport logging # Create a formatter formatter = logging.Formatter( fmt=\u0026#34;%(asctime)s | %(levelname)-8s | %(name)s | %(message)s\u0026#34;, datefmt=\u0026#34;%Y-%m-%d %H:%M:%S\u0026#34; ) # Apply to a handler handler = logging.StreamHandler() handler.setFormatter(formatter) All available format attributes:\n%(asctime)s Timestamp (formatted by datefmt) %(created)f Time as Unix timestamp float %(filename)s Source file name (e.g., main.py) %(funcName)s Function name where log was called %(levelname)s Level name: DEBUG, INFO, WARNING, ERROR, CRITICAL %(levelno)d Level number: 10, 20, 30, 40, 50 %(lineno)d Line number in source file %(message)s The actual log message %(module)s Module name (filename without .py) %(name)s Logger name %(pathname)s Full path to source file %(process)d Process ID %(processName)s Process name %(thread)d Thread ID %(threadName)s Thread name Practical format examples:\n# Minimal -- for development \u0026#34;%(levelname)s: %(message)s\u0026#34; # Standard -- for most applications \u0026#34;%(asctime)s | %(levelname)-8s | %(name)s | %(message)s\u0026#34; # Detailed -- for debugging production issues \u0026#34;%(asctime)s | %(levelname)-8s | %(name)s:%(lineno)d | %(funcName)s | %(message)s\u0026#34; # JSON format (useful for log aggregation tools) \u0026#39;{\u0026#34;time\u0026#34;: \u0026#34;%(asctime)s\u0026#34;, \u0026#34;level\u0026#34;: \u0026#34;%(levelname)s\u0026#34;, \u0026#34;module\u0026#34;: \u0026#34;%(name)s\u0026#34;, \u0026#34;msg\u0026#34;: \u0026#34;%(message)s\u0026#34;}\u0026#39; The -8s in %(levelname)-8s left-aligns the level name and pads to 8 characters so all messages line up:\n2024-03-15 14:30:22 | DEBUG | data_loader | Loading file 2024-03-15 14:30:22 | INFO | data_loader | 500 records loaded 2024-03-15 14:30:22 | WARNING | analytics | Missing values in column price 2024-03-15 14:30:22 | ERROR | payments | Payment gateway timeout 7. Logging to Multiple Destinations A very common real-world setup: DEBUG and above to the console during development, WARNING and above to a persistent log file.\nimport logging def setup_logger(name, log_file=None, level=logging.DEBUG): \u0026#34;\u0026#34;\u0026#34; Create a logger that outputs to console and optionally to a file. Args: name (str): Logger name (use __name__ from calling module) log_file (str, optional): Path to log file. None means console only. level (int): Minimum level for this logger. Returns: logging.Logger: Configured logger instance. \u0026#34;\u0026#34;\u0026#34; logger = logging.getLogger(name) logger.setLevel(level) # Avoid adding duplicate handlers if this function is called multiple times if logger.handlers: return logger # Console handler -- all messages at DEBUG and above console_fmt = logging.Formatter(\u0026#34;%(levelname)-8s | %(name)s | %(message)s\u0026#34;) console_hdlr = logging.StreamHandler() console_hdlr.setLevel(logging.DEBUG) console_hdlr.setFormatter(console_fmt) logger.addHandler(console_hdlr) # File handler -- WARNING and above only if log_file: file_fmt = logging.Formatter( \u0026#34;%(asctime)s | %(levelname)-8s | %(name)s:%(lineno)d | %(message)s\u0026#34;, datefmt=\u0026#34;%Y-%m-%d %H:%M:%S\u0026#34; ) file_hdlr = logging.FileHandler(log_file, encoding=\u0026#34;utf-8\u0026#34;) file_hdlr.setLevel(logging.WARNING) file_hdlr.setFormatter(file_fmt) logger.addHandler(file_hdlr) return logger # Usage logger = setup_logger(__name__, log_file=\u0026#34;app.log\u0026#34;) logger.debug(\u0026#34;Loading configuration\u0026#34;) # console only logger.info(\u0026#34;Application started\u0026#34;) # console only logger.warning(\u0026#34;Config file missing\u0026#34;) # console AND file logger.error(\u0026#34;Database connection failed\u0026#34;) # console AND file 8. Rotating Log Files In production, log files grow indefinitely. Rotating log files prevents them from consuming all available disk space.\nRotate by size (RotatingFileHandler) import logging import logging.handlers logger = logging.getLogger(\u0026#34;myapp\u0026#34;) logger.setLevel(logging.DEBUG) # Creates: app.log, app.log.1, app.log.2, app.log.3 # When app.log reaches 5MB it is renamed to app.log.1 # The old app.log.1 becomes app.log.2, etc. # app.log.3 (the oldest) is deleted when a new rotation happens handler = logging.handlers.RotatingFileHandler( filename=\u0026#34;app.log\u0026#34;, maxBytes=5 * 1024 * 1024, # 5 MB backupCount=3, # keep 3 backup files encoding=\u0026#34;utf-8\u0026#34; ) formatter = logging.Formatter( \u0026#34;%(asctime)s | %(levelname)-8s | %(name)s | %(message)s\u0026#34;, datefmt=\u0026#34;%Y-%m-%d %H:%M:%S\u0026#34; ) handler.setFormatter(formatter) logger.addHandler(handler) Rotate by time (TimedRotatingFileHandler) import logging.handlers # Creates: app.log, app.log.2024-03-14, app.log.2024-03-13 ... # At midnight, today\u0026#39;s log is archived and a fresh file starts handler = logging.handlers.TimedRotatingFileHandler( filename=\u0026#34;app.log\u0026#34;, when=\u0026#34;midnight\u0026#34;, # rotate at midnight interval=1, # every 1 day backupCount=30, # keep 30 days of logs encoding=\u0026#34;utf-8\u0026#34; ) # Other \u0026#39;when\u0026#39; options: # \u0026#34;S\u0026#34; -- seconds # \u0026#34;M\u0026#34; -- minutes # \u0026#34;H\u0026#34; -- hours # \u0026#34;D\u0026#34; -- days # \u0026#34;W0\u0026#34; -- every Monday (W0=Monday, W6=Sunday) # \u0026#34;midnight\u0026#34; 9. Logging Across Multiple Modules This is the pattern used in every real Python project.\nThe key insight: configure logging once at the application entry point (usually main.py). Every other module just creates its own named logger with logging.getLogger(__name__) and uses it. The configuration propagates automatically through the hierarchy.\n# config/logging_config.py # Centralised logging configuration import logging import logging.handlers from pathlib import Path def configure_logging(log_level=logging.INFO, log_dir=\u0026#34;logs\u0026#34;): \u0026#34;\u0026#34;\u0026#34; Set up logging for the entire application. Call this ONCE at the start of main.py. \u0026#34;\u0026#34;\u0026#34; Path(log_dir).mkdir(exist_ok=True) # Configure the ROOT logger -- all child loggers inherit this root_logger = logging.getLogger() root_logger.setLevel(logging.DEBUG) # capture everything at root # Remove any existing handlers (in case this is called multiple times) root_logger.handlers.clear() # Console handler console_handler = logging.StreamHandler() console_handler.setLevel(log_level) console_handler.setFormatter(logging.Formatter( \u0026#34;%(asctime)s | %(levelname)-8s | %(name)s | %(message)s\u0026#34;, datefmt=\u0026#34;%H:%M:%S\u0026#34; )) # File handler -- all DEBUG messages to detailed log debug_handler = logging.handlers.RotatingFileHandler( f\u0026#34;{log_dir}/debug.log\u0026#34;, maxBytes=10 * 1024 * 1024, backupCount=5, encoding=\u0026#34;utf-8\u0026#34; ) debug_handler.setLevel(logging.DEBUG) debug_handler.setFormatter(logging.Formatter( \u0026#34;%(asctime)s | %(levelname)-8s | %(name)s:%(lineno)d | %(message)s\u0026#34;, datefmt=\u0026#34;%Y-%m-%d %H:%M:%S\u0026#34; )) # Error file -- only errors and criticals error_handler = logging.handlers.RotatingFileHandler( f\u0026#34;{log_dir}/errors.log\u0026#34;, maxBytes=10 * 1024 * 1024, backupCount=10, encoding=\u0026#34;utf-8\u0026#34; ) error_handler.setLevel(logging.ERROR) error_handler.setFormatter(logging.Formatter( \u0026#34;%(asctime)s | %(levelname)-8s | %(name)s:%(lineno)d | %(funcName)s | %(message)s\u0026#34;, datefmt=\u0026#34;%Y-%m-%d %H:%M:%S\u0026#34; )) root_logger.addHandler(console_handler) root_logger.addHandler(debug_handler) root_logger.addHandler(error_handler) logging.getLogger(__name__).info(\u0026#34;Logging configured. Log dir: %s\u0026#34;, log_dir) # data_loader.py import logging # One line at the top of every module -- that is all logger = logging.getLogger(__name__) def load_csv(filepath): logger.debug(\u0026#34;Attempting to load: %s\u0026#34;, filepath) try: import pandas as pd df = pd.read_csv(filepath) logger.info(\u0026#34;Loaded %d rows from %s\u0026#34;, len(df), filepath) return df except FileNotFoundError: logger.error(\u0026#34;File not found: %s\u0026#34;, filepath) return None except Exception as e: logger.exception(\u0026#34;Unexpected error loading %s\u0026#34;, filepath) # logger.exception logs the full traceback automatically return None # analytics.py import logging logger = logging.getLogger(__name__) def calculate_summary(df): logger.debug(\u0026#34;Calculating summary for DataFrame with shape %s\u0026#34;, df.shape) if df.empty: logger.warning(\u0026#34;DataFrame is empty -- summary will be trivial\u0026#34;) summary = df.describe() logger.info(\u0026#34;Summary calculated successfully\u0026#34;) return summary # main.py from config.logging_config import configure_logging import logging # Configure once at the entry point configure_logging(log_level=logging.INFO) # Now import and use other modules -- they all use the configured logging from data_loader import load_csv from analytics import calculate_summary logger = logging.getLogger(__name__) def main(): logger.info(\u0026#34;Application starting\u0026#34;) df = load_csv(\u0026#34;data/sales.csv\u0026#34;) if df is None: logger.critical(\u0026#34;Could not load data. Exiting.\u0026#34;) return summary = calculate_summary(df) logger.info(\u0026#34;Pipeline completed successfully\u0026#34;) if __name__ == \u0026#34;__main__\u0026#34;: main() Logging exceptions properly import logging logger = logging.getLogger(__name__) # WRONG -- loses the traceback try: result = 1 / 0 except ZeroDivisionError as e: logger.error(\u0026#34;Division failed: %s\u0026#34;, e) # no traceback in the log # RIGHT -- logger.exception includes the full traceback try: result = 1 / 0 except ZeroDivisionError: logger.exception(\u0026#34;Division failed\u0026#34;) # logs message + full traceback # Or manually include traceback with logger.error import traceback try: result = 1 / 0 except ZeroDivisionError as e: logger.error(\u0026#34;Division failed: %s\\n%s\u0026#34;, e, traceback.format_exc()) 10. A Production-Ready Logging Setup Here is a complete, drop-in logging configuration that you can use in any project. Save it as logger.py in your project root.\n# logger.py # Drop this into any project. Import and call configure_logging() from main.py. import logging import logging.handlers import sys from pathlib import Path def configure_logging( app_name: str = \u0026#34;app\u0026#34;, log_level: int = logging.INFO, log_dir: str = \u0026#34;logs\u0026#34;, console: bool = True, log_to_file: bool = True, max_bytes: int = 10 * 1024 * 1024, # 10 MB backup_count: int = 5, ): \u0026#34;\u0026#34;\u0026#34; Configure application-wide logging. Call this once at the start of main.py before importing other modules. Args: app_name : Used as prefix for log filenames log_level : Minimum level for console output log_dir : Directory where log files are stored console : Whether to output to console log_to_file : Whether to write logs to files max_bytes : Max size per log file before rotation backup_count: Number of backup files to keep \u0026#34;\u0026#34;\u0026#34; # Create log directory if log_to_file: Path(log_dir).mkdir(parents=True, exist_ok=True) # Root logger captures everything root = logging.getLogger() root.setLevel(logging.DEBUG) root.handlers.clear() # Formats simple_fmt = logging.Formatter( \u0026#34;%(asctime)s | %(levelname)-8s | %(name)s | %(message)s\u0026#34;, datefmt=\u0026#34;%H:%M:%S\u0026#34; ) detailed_fmt = logging.Formatter( \u0026#34;%(asctime)s | %(levelname)-8s | %(name)s:%(lineno)d | %(funcName)s() | %(message)s\u0026#34;, datefmt=\u0026#34;%Y-%m-%d %H:%M:%S\u0026#34; ) # Console handler if console: ch = logging.StreamHandler(sys.stdout) ch.setLevel(log_level) ch.setFormatter(simple_fmt) root.addHandler(ch) if log_to_file: # All messages (DEBUG+) to a rolling debug log fh_debug = logging.handlers.RotatingFileHandler( Path(log_dir) / f\u0026#34;{app_name}_debug.log\u0026#34;, maxBytes=max_bytes, backupCount=backup_count, encoding=\u0026#34;utf-8\u0026#34; ) fh_debug.setLevel(logging.DEBUG) fh_debug.setFormatter(detailed_fmt) root.addHandler(fh_debug) # Errors only -- a clean, small file for monitoring fh_error = logging.handlers.RotatingFileHandler( Path(log_dir) / f\u0026#34;{app_name}_errors.log\u0026#34;, maxBytes=max_bytes, backupCount=backup_count * 2, # keep more error history encoding=\u0026#34;utf-8\u0026#34; ) fh_error.setLevel(logging.ERROR) fh_error.setFormatter(detailed_fmt) root.addHandler(fh_error) # Silence overly verbose third-party loggers logging.getLogger(\u0026#34;urllib3\u0026#34;).setLevel(logging.WARNING) logging.getLogger(\u0026#34;requests\u0026#34;).setLevel(logging.WARNING) logging.getLogger(\u0026#34;matplotlib\u0026#34;).setLevel(logging.WARNING) logging.getLogger(__name__).debug( \u0026#34;Logging configured. app=%s level=%s dir=%s\u0026#34;, app_name, logging.getLevelName(log_level), log_dir ) Usage in any project:\n# main.py from logger import configure_logging import logging configure_logging(app_name=\u0026#34;cricket_analysis\u0026#34;, log_level=logging.INFO) # Now every module that does logging.getLogger(__name__) works correctly 11. The os Module \u0026ndash; System Interaction The os module is Python\u0026rsquo;s interface to the operating system. It lets your code interact with the file system, processes, and the environment your program runs in.\nWe covered pathlib in the file handling guide \u0026ndash; pathlib handles most path operations better in modern Python. What os does that pathlib does not is covered here.\nWhat os is good for import os # Current working directory print(os.getcwd()) # /home/aarav/projects/cricket_analysis # Change directory os.chdir(\u0026#34;/home/aarav/data\u0026#34;) # Path operations (prefer pathlib, but os.path still works) os.path.exists(\u0026#34;data.csv\u0026#34;) # True or False os.path.isfile(\u0026#34;data.csv\u0026#34;) # is it a file? os.path.isdir(\u0026#34;results/\u0026#34;) # is it a directory? os.path.abspath(\u0026#34;data.csv\u0026#34;) # full absolute path os.path.basename(\u0026#34;/a/b/data.csv\u0026#34;) # \u0026#34;data.csv\u0026#34; os.path.dirname(\u0026#34;/a/b/data.csv\u0026#34;) # \u0026#34;/a/b\u0026#34; os.path.splitext(\u0026#34;data.csv\u0026#34;) # (\u0026#34;data\u0026#34;, \u0026#34;.csv\u0026#34;) os.path.join(\u0026#34;data\u0026#34;, \u0026#34;raw\u0026#34;, \u0026#34;x.csv\u0026#34;)# \u0026#34;data/raw/x.csv\u0026#34; # File metadata os.path.getsize(\u0026#34;data.csv\u0026#34;) # size in bytes os.path.getmtime(\u0026#34;data.csv\u0026#34;) # last modified time (Unix timestamp) # Directory contents os.listdir(\u0026#34;.\u0026#34;) # list files and folders in current dir os.listdir(\u0026#34;/home/aarav/data\u0026#34;) # list a specific directory # Create and remove directories os.mkdir(\u0026#34;output\u0026#34;) # create one directory (fails if exists) os.makedirs(\u0026#34;output/charts/2024\u0026#34;, # create full path exist_ok=True) # no error if already exists os.rmdir(\u0026#34;output\u0026#34;) # remove empty directory only os.remove(\u0026#34;old_file.csv\u0026#34;) # delete a file # Rename and move os.rename(\u0026#34;old_name.csv\u0026#34;, \u0026#34;new_name.csv\u0026#34;) # Get file permissions (Unix only) os.stat(\u0026#34;data.csv\u0026#34;).st_mode os vs pathlib \u0026ndash; when to use which from pathlib import Path import os # pathlib is cleaner for most path operations p = Path(\u0026#34;data\u0026#34;) / \u0026#34;raw\u0026#34; / \u0026#34;sales.csv\u0026#34; if p.exists(): content = p.read_text(encoding=\u0026#34;utf-8\u0026#34;) # Use os for: # 1. Environment variables -- pathlib does not handle these api_key = os.environ.get(\u0026#34;API_KEY\u0026#34;) # 2. os.walk for recursive directory traversal -- pathlib\u0026#39;s rglob is simpler # for simple cases but os.walk gives more control (covered in section 13) # 3. Process-level operations os.getpid() # current process ID os.getppid() # parent process ID os.cpu_count() # number of CPUs # 4. os.path when working with legacy code that uses strings # pathlib is preferred for new code 12. Environment Variables \u0026ndash; The Right Way Environment variables are key-value pairs set in the operating system (or in a .env file) that your program can read at runtime. They are the standard way to pass configuration and secrets to Python programs without hardcoding them.\nimport os # Read an environment variable api_key = os.environ.get(\u0026#34;OPENAI_API_KEY\u0026#34;) print(api_key) # None if not set # Read with a default value debug_mode = os.environ.get(\u0026#34;DEBUG\u0026#34;, \u0026#34;false\u0026#34;).lower() == \u0026#34;true\u0026#34; port = int(os.environ.get(\u0026#34;PORT\u0026#34;, \u0026#34;8000\u0026#34;)) db_host = os.environ.get(\u0026#34;DB_HOST\u0026#34;, \u0026#34;localhost\u0026#34;) # Read a required variable -- raise if missing def get_required_env(key): \u0026#34;\u0026#34;\u0026#34;Get an environment variable or raise if not set.\u0026#34;\u0026#34;\u0026#34; value = os.environ.get(key) if value is None: raise EnvironmentError( f\u0026#34;Required environment variable \u0026#39;{key}\u0026#39; is not set. \u0026#34; f\u0026#34;Set it before running this script.\u0026#34; ) return value api_key = get_required_env(\u0026#34;RAZORPAY_API_KEY\u0026#34;) db_url = get_required_env(\u0026#34;DATABASE_URL\u0026#34;) # List all environment variables for key, value in os.environ.items(): print(f\u0026#34;{key} = {value[:20]}...\u0026#34;) # show first 20 chars for safety Setting environment variables in the terminal # Mac/Linux -- for the current terminal session export API_KEY=\u0026#34;your_key_here\u0026#34; export DB_HOST=\u0026#34;localhost\u0026#34; export DEBUG=\u0026#34;true\u0026#34; # Run your script python main.py # Unset a variable unset API_KEY # Set for a single command only API_KEY=\u0026#34;test_key\u0026#34; python main.py # Windows PowerShell $env:API_KEY = \u0026#34;your_key_here\u0026#34; python main.py Using .env files with python-dotenv Manually exporting variables before every run is tedious. python-dotenv reads a .env file and loads all variables automatically.\npip install python-dotenv # .env (in your project root -- NEVER commit this to git) API_KEY=your_key_here DB_HOST=localhost DB_PORT=5432 DB_NAME=cricket_db DEBUG=false LOG_LEVEL=INFO # main.py from dotenv import load_dotenv import os # Load .env file -- must be called before reading os.environ load_dotenv() api_key = os.environ.get(\u0026#34;API_KEY\u0026#34;) db_host = os.environ.get(\u0026#34;DB_HOST\u0026#34;, \u0026#34;localhost\u0026#34;) debug = os.environ.get(\u0026#34;DEBUG\u0026#34;, \u0026#34;false\u0026#34;).lower() == \u0026#34;true\u0026#34; print(f\u0026#34;DB Host: {db_host}\u0026#34;) print(f\u0026#34;Debug: {debug}\u0026#34;) # For more control over which .env file to load from dotenv import load_dotenv from pathlib import Path # Load from a specific path load_dotenv(Path(__file__).parent / \u0026#34;.env.production\u0026#34;) # Override existing environment variables (by default, dotenv does NOT # override variables already set in the environment) load_dotenv(override=True) Building a config module with environment variables # config.py # Centralised configuration reading from environment variables import os from dataclasses import dataclass from dotenv import load_dotenv load_dotenv() @dataclass class DatabaseConfig: host: str port: int name: str user: str password: str @property def url(self): return f\u0026#34;postgresql://{self.user}:{self.password}@{self.host}:{self.port}/{self.name}\u0026#34; @dataclass class AppConfig: debug: bool log_level: str api_key: str db: DatabaseConfig def load_config() -\u0026gt; AppConfig: \u0026#34;\u0026#34;\u0026#34;Load and validate all configuration from environment variables.\u0026#34;\u0026#34;\u0026#34; def required(key: str) -\u0026gt; str: val = os.environ.get(key) if not val: raise EnvironmentError(f\u0026#34;Required config \u0026#39;{key}\u0026#39; not set.\u0026#34;) return val return AppConfig( debug = os.environ.get(\u0026#34;DEBUG\u0026#34;, \u0026#34;false\u0026#34;).lower() == \u0026#34;true\u0026#34;, log_level = os.environ.get(\u0026#34;LOG_LEVEL\u0026#34;, \u0026#34;INFO\u0026#34;).upper(), api_key = required(\u0026#34;API_KEY\u0026#34;), db=DatabaseConfig( host = os.environ.get(\u0026#34;DB_HOST\u0026#34;, \u0026#34;localhost\u0026#34;), port = int(os.environ.get(\u0026#34;DB_PORT\u0026#34;, \u0026#34;5432\u0026#34;)), name = required(\u0026#34;DB_NAME\u0026#34;), user = required(\u0026#34;DB_USER\u0026#34;), password = required(\u0026#34;DB_PASSWORD\u0026#34;), ) ) # Usage config = load_config() print(config.db.url) print(f\u0026#34;Debug mode: {config.debug}\u0026#34;) 13. Walking Directory Trees with os.walk os.walk recursively walks a directory tree, yielding the contents of each subdirectory. It is the right tool when you need to process files across a nested folder structure.\nimport os # os.walk yields (dirpath, dirnames, filenames) for each directory for dirpath, dirnames, filenames in os.walk(\u0026#34;my_project\u0026#34;): print(f\u0026#34;Directory: {dirpath}\u0026#34;) for filename in filenames: print(f\u0026#34; File: {filename}\u0026#34;) full_path = os.path.join(dirpath, filename) print(f\u0026#34; Full path: {full_path}\u0026#34;) Practical examples import os # Find all Python files in a project (recursively) def find_python_files(root_dir): \u0026#34;\u0026#34;\u0026#34;Return a list of all .py files under root_dir.\u0026#34;\u0026#34;\u0026#34; py_files = [] for dirpath, dirnames, filenames in os.walk(root_dir): # Exclude hidden directories and __pycache__ dirnames[:] = [ d for d in dirnames if not d.startswith(\u0026#34;.\u0026#34;) and d != \u0026#34;__pycache__\u0026#34; ] for filename in filenames: if filename.endswith(\u0026#34;.py\u0026#34;): py_files.append(os.path.join(dirpath, filename)) return py_files files = find_python_files(\u0026#34;.\u0026#34;) print(f\u0026#34;Found {len(files)} Python files\u0026#34;) for f in files[:5]: print(f\u0026#34; {f}\u0026#34;) import os # Calculate total size of a directory tree def directory_size(path): \u0026#34;\u0026#34;\u0026#34;Return total size of all files in a directory tree (bytes).\u0026#34;\u0026#34;\u0026#34; total = 0 for dirpath, dirnames, filenames in os.walk(path): for filename in filenames: filepath = os.path.join(dirpath, filename) try: total += os.path.getsize(filepath) except OSError: pass # skip files we cannot access return total size = directory_size(\u0026#34;my_project\u0026#34;) print(f\u0026#34;Total size: {size / 1024 / 1024:.1f} MB\u0026#34;) import os import shutil # Find and process all CSV files in a data directory def process_all_csvs(data_dir, output_dir): \u0026#34;\u0026#34;\u0026#34; Walk a directory tree and process every CSV file found. \u0026#34;\u0026#34;\u0026#34; os.makedirs(output_dir, exist_ok=True) processed = 0 errors = 0 for dirpath, dirnames, filenames in os.walk(data_dir): # Skip hidden directories dirnames[:] = [d for d in dirnames if not d.startswith(\u0026#34;.\u0026#34;)] for filename in filenames: if not filename.lower().endswith(\u0026#34;.csv\u0026#34;): continue input_path = os.path.join(dirpath, filename) output_path = os.path.join(output_dir, filename) try: # Preserve relative directory structure in output relative = os.path.relpath(dirpath, data_dir) out_dir = os.path.join(output_dir, relative) os.makedirs(out_dir, exist_ok=True) out_path = os.path.join(out_dir, filename) print(f\u0026#34;Processing: {input_path}\u0026#34;) # ... your processing logic here ... shutil.copy2(input_path, out_path) # example: just copy processed += 1 except Exception as e: print(f\u0026#34;Error processing {input_path}: {e}\u0026#34;) errors += 1 print(f\u0026#34;\\nDone. Processed: {processed}, Errors: {errors}\u0026#34;) For simple cases, pathlib.Path.rglob(\u0026quot;*.csv\u0026quot;) is shorter and more readable than os.walk. Use os.walk when you need to modify the directory traversal (like skipping certain folders with dirnames[:]) or need more control over the walk order.\n14. Running Shell Commands \u0026ndash; os.system vs subprocess Sometimes Python needs to run a shell command \u0026ndash; call a CLI tool, run a script, or execute a system command.\nos.system \u0026ndash; simple but limited import os # Runs a command in a shell os.system(\u0026#34;ls -la\u0026#34;) # Mac/Linux os.system(\u0026#34;dir\u0026#34;) # Windows # Returns the exit code (0 = success, non-zero = error) exit_code = os.system(\u0026#34;python --version\u0026#34;) print(exit_code) # 0 # Problem: you cannot capture the output # Problem: security risk if command includes user input (shell injection) subprocess \u0026ndash; the right way The subprocess module gives you full control: capture output, check exit codes, handle errors, set timeouts, and avoid shell injection.\nimport subprocess # Run a command and wait for it to finish result = subprocess.run( [\u0026#34;python\u0026#34;, \u0026#34;--version\u0026#34;], capture_output=True, # capture stdout and stderr text=True, # decode output as string (not bytes) timeout=30 # raise TimeoutExpired after 30 seconds ) print(result.stdout) # \u0026#34;Python 3.12.0\\n\u0026#34; print(result.returncode) # 0 (success) # Check if command succeeded if result.returncode != 0: print(f\u0026#34;Command failed: {result.stderr}\u0026#34;) import subprocess # check=True raises CalledProcessError if command exits with non-zero code try: result = subprocess.run( [\u0026#34;pip\u0026#34;, \u0026#34;install\u0026#34;, \u0026#34;pandas\u0026#34;], capture_output=True, text=True, check=True ) print(\u0026#34;Installed successfully\u0026#34;) print(result.stdout) except subprocess.CalledProcessError as e: print(f\u0026#34;Installation failed (exit code {e.returncode})\u0026#34;) print(e.stderr) except subprocess.TimeoutExpired: print(\u0026#34;Command took too long\u0026#34;) except FileNotFoundError: print(\u0026#34;Command not found -- is pip installed?\u0026#34;) import subprocess # Pass arguments safely -- no shell injection risk # WRONG (shell injection risk): filename = \u0026#34;file with spaces.csv; rm -rf /\u0026#34; # malicious input os.system(f\u0026#34;wc -l {filename}\u0026#34;) # dangerous! # RIGHT (subprocess with list -- shell=False by default): result = subprocess.run( [\u0026#34;wc\u0026#34;, \u0026#34;-l\u0026#34;, filename], # filename is treated as a single argument capture_output=True, text=True ) import subprocess # Running a Python script from Python result = subprocess.run( [\u0026#34;python\u0026#34;, \u0026#34;process_data.py\u0026#34;, \u0026#34;--input\u0026#34;, \u0026#34;data.csv\u0026#34;, \u0026#34;--output\u0026#34;, \u0026#34;results/\u0026#34;], capture_output=True, text=True, check=True ) # Run a command in a specific directory result = subprocess.run( [\u0026#34;git\u0026#34;, \u0026#34;status\u0026#34;], capture_output=True, text=True, cwd=\u0026#34;/home/aarav/projects/cricket_analysis\u0026#34; # working directory ) print(result.stdout) Rule: Use subprocess.run() for everything. Only use os.system() for throwaway scripts where you do not need the output.\n15. Putting It Together \u0026ndash; A Data Pipeline This example shows both modules working together in a realistic scenario: a daily data pipeline that reads sales files, processes them, and logs every step. Configuration comes from environment variables. Every important event is logged at the appropriate level.\n\u0026#34;\u0026#34;\u0026#34; pipeline.py Daily sales data pipeline. Reads CSV files from an input directory, validates them, summarises each file, and writes results to an output directory. Environment variables required: INPUT_DIR -- directory containing raw CSV files OUTPUT_DIR -- directory to write processed results LOG_LEVEL -- logging level (default: INFO) MIN_RECORDS -- minimum records per file to be considered valid (default: 10) \u0026#34;\u0026#34;\u0026#34; import os import csv import logging import logging.handlers from pathlib import Path from datetime import datetime # ─── Logging setup ──────────────────────────────────────────────────────── def setup_pipeline_logging(log_dir=\u0026#34;logs\u0026#34;, level_name=\u0026#34;INFO\u0026#34;): \u0026#34;\u0026#34;\u0026#34;Configure logging for the pipeline.\u0026#34;\u0026#34;\u0026#34; log_level = getattr(logging, level_name.upper(), logging.INFO) Path(log_dir).mkdir(exist_ok=True) root = logging.getLogger() root.setLevel(logging.DEBUG) root.handlers.clear() # Console -- INFO and above console = logging.StreamHandler() console.setLevel(log_level) console.setFormatter(logging.Formatter( \u0026#34;%(asctime)s | %(levelname)-8s | %(message)s\u0026#34;, datefmt=\u0026#34;%H:%M:%S\u0026#34; )) root.addHandler(console) # File -- everything (DEBUG+), rotated daily today = datetime.now().strftime(\u0026#34;%Y%m%d\u0026#34;) file_hdlr = logging.handlers.TimedRotatingFileHandler( f\u0026#34;{log_dir}/pipeline.log\u0026#34;, when=\u0026#34;midnight\u0026#34;, backupCount=14, encoding=\u0026#34;utf-8\u0026#34; ) file_hdlr.setLevel(logging.DEBUG) file_hdlr.setFormatter(logging.Formatter( \u0026#34;%(asctime)s | %(levelname)-8s | %(name)s:%(lineno)d | %(message)s\u0026#34;, datefmt=\u0026#34;%Y-%m-%d %H:%M:%S\u0026#34; )) root.addHandler(file_hdlr) logger = logging.getLogger(__name__) # ─── Configuration ──────────────────────────────────────────────────────── def load_config(): \u0026#34;\u0026#34;\u0026#34;Load and validate pipeline configuration from environment variables.\u0026#34;\u0026#34;\u0026#34; logger.debug(\u0026#34;Loading configuration from environment variables\u0026#34;) required_vars = [\u0026#34;INPUT_DIR\u0026#34;, \u0026#34;OUTPUT_DIR\u0026#34;] missing = [v for v in required_vars if not os.environ.get(v)] if missing: raise EnvironmentError( f\u0026#34;Missing required environment variables: {missing}\u0026#34; ) config = { \u0026#34;input_dir\u0026#34;: os.environ[\u0026#34;INPUT_DIR\u0026#34;], \u0026#34;output_dir\u0026#34;: os.environ[\u0026#34;OUTPUT_DIR\u0026#34;], \u0026#34;log_level\u0026#34;: os.environ.get(\u0026#34;LOG_LEVEL\u0026#34;, \u0026#34;INFO\u0026#34;), \u0026#34;min_records\u0026#34;: int(os.environ.get(\u0026#34;MIN_RECORDS\u0026#34;, \u0026#34;10\u0026#34;)), } logger.info(\u0026#34;Configuration loaded: input=%s output=%s min_records=%d\u0026#34;, config[\u0026#34;input_dir\u0026#34;], config[\u0026#34;output_dir\u0026#34;], config[\u0026#34;min_records\u0026#34;]) return config # ─── File discovery ─────────────────────────────────────────────────────── def find_csv_files(directory): \u0026#34;\u0026#34;\u0026#34; Recursively find all CSV files in a directory. Returns a list of absolute paths. \u0026#34;\u0026#34;\u0026#34; logger.debug(\u0026#34;Scanning for CSV files in: %s\u0026#34;, directory) if not os.path.isdir(directory): logger.error(\u0026#34;Input directory does not exist: %s\u0026#34;, directory) return [] csv_files = [] for dirpath, dirnames, filenames in os.walk(directory): # Skip hidden directories dirnames[:] = [d for d in dirnames if not d.startswith(\u0026#34;.\u0026#34;)] for filename in filenames: if filename.lower().endswith(\u0026#34;.csv\u0026#34;): full_path = os.path.join(dirpath, filename) csv_files.append(full_path) logger.debug(\u0026#34;Found: %s\u0026#34;, full_path) logger.info(\u0026#34;Found %d CSV file(s) in %s\u0026#34;, len(csv_files), directory) return csv_files # ─── File processing ────────────────────────────────────────────────────── def process_csv(filepath, min_records): \u0026#34;\u0026#34;\u0026#34; Read a CSV file and return a summary dict. Returns None if the file is invalid. \u0026#34;\u0026#34;\u0026#34; logger.debug(\u0026#34;Processing: %s\u0026#34;, filepath) try: with open(filepath, \u0026#34;r\u0026#34;, encoding=\u0026#34;utf-8\u0026#34;, newline=\u0026#34;\u0026#34;) as f: reader = csv.DictReader(f) rows = list(reader) headers = reader.fieldnames or [] if len(rows) \u0026lt; min_records: logger.warning( \u0026#34;File has only %d records (minimum: %d). Skipping: %s\u0026#34;, len(rows), min_records, filepath ) return None # Build summary summary = { \u0026#34;filename\u0026#34;: os.path.basename(filepath), \u0026#34;filepath\u0026#34;: filepath, \u0026#34;rows\u0026#34;: len(rows), \u0026#34;columns\u0026#34;: len(headers), \u0026#34;headers\u0026#34;: headers, } # Try to sum any numeric columns numeric_sums = {} for col in headers: try: total = sum(float(row[col]) for row in rows if row.get(col)) numeric_sums[col] = round(total, 2) except (ValueError, TypeError): pass # not a numeric column -- skip summary[\u0026#34;numeric_sums\u0026#34;] = numeric_sums logger.info(\u0026#34;Processed %s: %d rows, %d columns\u0026#34;, os.path.basename(filepath), len(rows), len(headers)) return summary except PermissionError: logger.error(\u0026#34;Permission denied reading: %s\u0026#34;, filepath) return None except UnicodeDecodeError: logger.error(\u0026#34;Encoding error in: %s. Try UTF-8-SIG.\u0026#34;, filepath) return None except Exception: logger.exception(\u0026#34;Unexpected error processing: %s\u0026#34;, filepath) return None # ─── Output writing ─────────────────────────────────────────────────────── def write_summary(summaries, output_dir): \u0026#34;\u0026#34;\u0026#34;Write a summary report of all processed files.\u0026#34;\u0026#34;\u0026#34; os.makedirs(output_dir, exist_ok=True) today = datetime.now().strftime(\u0026#34;%Y%m%d_%H%M%S\u0026#34;) report_path = os.path.join(output_dir, f\u0026#34;pipeline_report_{today}.txt\u0026#34;) logger.debug(\u0026#34;Writing summary report to: %s\u0026#34;, report_path) try: with open(report_path, \u0026#34;w\u0026#34;, encoding=\u0026#34;utf-8\u0026#34;) as f: f.write(f\u0026#34;Pipeline Report -- {datetime.now().strftime(\u0026#39;%Y-%m-%d %H:%M:%S\u0026#39;)}\\n\u0026#34;) f.write(\u0026#34;=\u0026#34; * 60 + \u0026#34;\\n\\n\u0026#34;) for s in summaries: f.write(f\u0026#34;File : {s[\u0026#39;filename\u0026#39;]}\\n\u0026#34;) f.write(f\u0026#34;Rows : {s[\u0026#39;rows\u0026#39;]}\\n\u0026#34;) f.write(f\u0026#34;Columns : {s[\u0026#39;columns\u0026#39;]}\\n\u0026#34;) f.write(f\u0026#34;Headers : {\u0026#39;, \u0026#39;.join(s[\u0026#39;headers\u0026#39;])}\\n\u0026#34;) if s[\u0026#39;numeric_sums\u0026#39;]: f.write(\u0026#34;Totals :\\n\u0026#34;) for col, total in s[\u0026#39;numeric_sums\u0026#39;].items(): f.write(f\u0026#34; {col}: {total:,.2f}\\n\u0026#34;) f.write(\u0026#34;\\n\u0026#34;) f.write(f\u0026#34;Total files processed: {len(summaries)}\\n\u0026#34;) logger.info(\u0026#34;Report written to: %s\u0026#34;, report_path) return report_path except IOError as e: logger.error(\u0026#34;Could not write report: %s\u0026#34;, e) return None # ─── Main pipeline ──────────────────────────────────────────────────────── def run_pipeline(): \u0026#34;\u0026#34;\u0026#34;Main pipeline entry point.\u0026#34;\u0026#34;\u0026#34; start_time = datetime.now() logger.info(\u0026#34;Pipeline starting at %s\u0026#34;, start_time.strftime(\u0026#34;%H:%M:%S\u0026#34;)) # Load config try: config = load_config() except EnvironmentError as e: logger.critical(\u0026#34;Configuration error: %s\u0026#34;, e) return False # Discover files csv_files = find_csv_files(config[\u0026#34;input_dir\u0026#34;]) if not csv_files: logger.warning(\u0026#34;No CSV files found. Nothing to process.\u0026#34;) return True # Process each file summaries = [] failed = 0 for filepath in csv_files: summary = process_csv(filepath, config[\u0026#34;min_records\u0026#34;]) if summary: summaries.append(summary) else: failed += 1 # Write output if summaries: report_path = write_summary(summaries, config[\u0026#34;output_dir\u0026#34;]) if not report_path: logger.error(\u0026#34;Failed to write summary report\u0026#34;) # Final summary elapsed = (datetime.now() - start_time).total_seconds() logger.info( \u0026#34;Pipeline complete. Processed: %d, Skipped/failed: %d, Time: %.1fs\u0026#34;, len(summaries), failed, elapsed ) if failed \u0026gt; 0: logger.warning(\u0026#34;%d file(s) could not be processed.\u0026#34;, failed) return failed == 0 if __name__ == \u0026#34;__main__\u0026#34;: # Load .env if present try: from dotenv import load_dotenv load_dotenv() except ImportError: pass # dotenv not installed -- rely on actual env vars # Set up logging (using LOG_LEVEL from environment or default INFO) setup_pipeline_logging( log_dir=\u0026#34;logs\u0026#34;, level_name=os.environ.get(\u0026#34;LOG_LEVEL\u0026#34;, \u0026#34;INFO\u0026#34;) ) success = run_pipeline() exit(0 if success else 1) 16. Summary and Key Takeaways The logging module Use named loggers everywhere. One line at the top of every module: logger = logging.getLogger(__name__). Never use print() for anything that matters in production.\nConfigure once at the entry point. Call your logging setup function in main.py before importing anything else. All child loggers inherit the configuration automatically.\nUse the right level for the right message. DEBUG for diagnostic detail. INFO for normal progress. WARNING for something unexpected that is not a failure. ERROR for a failure. CRITICAL for something that prevents the program from continuing.\nUse logger.exception() inside except blocks. It automatically includes the full traceback. logger.error() alone does not.\nUse logger.info(\u0026quot;Value: %s\u0026quot;, value) not logger.info(f\u0026quot;Value: {value}\u0026quot;). The %s style defers string formatting until the message is actually logged. If the message is below the minimum level, no formatting happens at all. This matters for performance in tight loops.\nRotate log files in production. RotatingFileHandler and TimedRotatingFileHandler prevent log files from filling your disk. Always set backupCount and maxBytes.\nThe os module Environment variables are how configuration and secrets get into your program. Never hardcode API keys or database credentials. Read them from environment variables. Use python-dotenv for local development. Use your platform\u0026rsquo;s secret management for production.\nPrefer pathlib for file paths, os for everything else. pathlib.Path is cleaner for creating, reading, and manipulating paths. Use os for environment variables, process information, os.walk, and running shell commands.\nUse subprocess.run() not os.system(). subprocess.run() lets you capture output, check exit codes, set timeouts, and pass arguments safely as a list (no shell injection risk).\nos.walk gives you recursive directory traversal with control. Use it when you need to skip specific directories or process nested structures. For simple cases, pathlib.Path.rglob(\u0026quot;*.csv\u0026quot;) is shorter.\nCommon mistakes to avoid # 1. Using print() instead of logging in anything that runs unattended print(\u0026#34;Processing started\u0026#34;) # disappears when terminal closes logger.info(\u0026#34;Processing started\u0026#34;) # persists, timestamped, filterable # 2. Logging at the wrong level logger.debug(\u0026#34;Payment failed\u0026#34;) # debug -- will be invisible in production logger.error(\u0026#34;Payment failed\u0026#34;) # correct # 3. Using the root logger directly in library/module code logging.info(\u0026#34;Message\u0026#34;) # pollutes any application that imports you logger = logging.getLogger(__name__) # correct -- namespaced to your module logger.info(\u0026#34;Message\u0026#34;) # 4. Hardcoding secrets API_KEY = \u0026#34;sk-abc123xyz\u0026#34; # ends up in git history API_KEY = os.environ.get(\u0026#34;API_KEY\u0026#34;) # correct # 5. Using os.system() when you need the output os.system(\u0026#34;git status\u0026#34;) # output goes to terminal, you cannot capture it result = subprocess.run([\u0026#34;git\u0026#34;, \u0026#34;status\u0026#34;], capture_output=True, text=True) # correct # 6. Not checking that required environment variables are set db_host = os.environ.get(\u0026#34;DB_HOST\u0026#34;) # silently returns None db_host = os.environ[\u0026#34;DB_HOST\u0026#34;] # raises KeyError if not set -- fails fast # Better: check explicitly with a helpful error message The ten things to remember 1. logger = logging.getLogger(__name__) -- one line, every module 2. Configure logging once in main.py before importing other modules 3. DEBUG for dev, INFO for progress, WARNING for anomalies, ERROR for failures 4. logger.exception() inside except blocks -- includes traceback automatically 5. Use %s formatting, not f-strings, in log calls 6. RotatingFileHandler prevents log files from growing forever 7. os.environ.get(\u0026#34;KEY\u0026#34;, \u0026#34;default\u0026#34;) for optional config 8. os.environ[\u0026#34;KEY\u0026#34;] or raise EnvironmentError for required config 9. subprocess.run([\u0026#34;cmd\u0026#34;, \u0026#34;arg\u0026#34;], capture_output=True, text=True, check=True) 10. Never commit .env files -- always add to .gitignore Made with care for Codeverra learners | codeverra.com\n","permalink":"https://blogs.codeverra.com/blog/python-16-logging-and-os/","summary":"\u003ch1 id=\"logging-and-system-interaction-in-python\"\u003eLogging and System Interaction in Python\u003c/h1\u003e\n\u003ch3 id=\"professional-observability-with-the-logging-module-and-the-os-module\"\u003eProfessional Observability with the logging Module and the os Module\u003c/h3\u003e\n\u003chr\u003e\n\u003ch2 id=\"before-we-begin--why-this-matters-more-than-it-looks\"\u003eBefore We Begin \u0026ndash; Why This Matters More Than It Looks\u003c/h2\u003e\n\u003cp\u003eHere is a situation every developer eventually faces.\u003c/p\u003e\n\u003cp\u003eYou write a script that processes data overnight. It runs on a server\nwhile you sleep. In the morning you check the output and something is\nwrong. Half the records are missing. You have no idea what happened\nbecause your script used \u003ccode\u003eprint()\u003c/code\u003e for all its messages and those prints\nwent nowhere \u0026ndash; there was nobody watching the terminal.\u003c/p\u003e","tags":["python"],"title":"Logging and System Interaction in Python"},{"content":"Modules, Packages, and the Python Ecosystem Understanding Imports, the Standard Library, pip, and Virtual Environments Before We Begin \u0026ndash; Why This Topic Is More Important Than It Looks Here is something that happens to almost every beginner.\nYou are working through a tutorial. It says import pandas as pd and moves on. You copy it. It works. You move on too.\nSix months later you are working on two projects at the same time. One needs pandas 1.5. The other needs pandas 2.1. You install one and the other breaks. You have no idea why. You cannot share your project with a teammate because it works on your machine and crashes on theirs.\nOr you write a 500-line script and realise you have copy-pasted the same helper function into three different files. When you fix a bug in one, you forget to fix the other two.\nBoth problems have the same root cause \u0026ndash; you never learned how Python\u0026rsquo;s import system actually works, or how to structure code across files.\nThis guide fixes that. By the end you will understand:\nWhat actually happens when you write import something How to split your own code across multiple files so it stays maintainable What the standard library is and which parts of it you will use every day How pip works and what requirements.txt is for Why virtual environments are not optional and how to use them properly This is not exciting material on the surface. But it is the infrastructure that every Python project you will ever build depends on.\nTable of Contents What is a Module? Importing Modules \u0026ndash; All the Ways How Python Finds What You Import The __name__ Variable and if __name__ == \u0026quot;__main__\u0026quot; Writing Your Own Modules What is a Package? Building Your Own Package Relative and Absolute Imports The Standard Library \u0026ndash; A Practical Tour pip \u0026ndash; Installing Third-Party Packages requirements.txt \u0026ndash; Sharing Dependencies Virtual Environments \u0026ndash; Why They Are Non-Negotiable venv \u0026ndash; The Built-In Solution conda \u0026ndash; The Data Science Standard Other Tools \u0026ndash; virtualenv, uv Which Tool Should You Use? Practice Questions Solutions What Comes Next 1. What is a Module? A module is the simplest possible concept in Python packaging.\nA module is just a .py file.\nThat is it. If you create a file called maths_utils.py, that file is a module. Any other Python file can import it and use everything defined inside it.\nThis matters because as your programs grow, keeping everything in one file becomes unmanageable. A 2000-line script where functions for database access, business logic, and user interface are all mixed together is hard to read, hard to test, and hard to maintain.\nModules let you break that 2000-line script into focused, reusable pieces:\nmy_project/ database.py -- everything related to database access analytics.py -- calculation and analysis functions reports.py -- report generation main.py -- ties everything together Each file is a module. Each module has a clear responsibility. When a bug appears in the database layer, you know exactly which file to open.\nWhat can a module contain? Anything you can write in a Python file:\n# maths_utils.py -- a simple module # Variables PI = 3.14159265358979 E = 2.71828182845905 GOLDEN_RATIO = 1.61803398874989 # Functions def circle_area(radius): \u0026#34;\u0026#34;\u0026#34;Calculate the area of a circle.\u0026#34;\u0026#34;\u0026#34; return PI * radius ** 2 def is_prime(n): \u0026#34;\u0026#34;\u0026#34;Return True if n is a prime number.\u0026#34;\u0026#34;\u0026#34; if n \u0026lt; 2: return False for i in range(2, int(n**0.5) + 1): if n % i == 0: return False return True def factorial(n): \u0026#34;\u0026#34;\u0026#34;Return n! using recursion.\u0026#34;\u0026#34;\u0026#34; if n \u0026lt;= 1: return 1 return n * factorial(n - 1) # Classes class Vector2D: def __init__(self, x, y): self.x = x self.y = y def magnitude(self): return (self.x**2 + self.y**2) ** 0.5 def __repr__(self): return f\u0026#34;Vector2D({self.x}, {self.y})\u0026#34; 2. Importing Modules \u0026ndash; All the Ways import module_name The most basic form. Imports the entire module and gives you access to everything in it via dot notation.\nimport maths_utils # Access everything with the module name as a prefix print(maths_utils.PI) # 3.14159... print(maths_utils.circle_area(7)) # 153.938... print(maths_utils.is_prime(17)) # True v = maths_utils.Vector2D(3, 4) print(v.magnitude()) # 5.0 The module name acts as a namespace \u0026ndash; it prevents name collisions. If you have your own variable called PI, it does not conflict with maths_utils.PI because they live in different namespaces.\nfrom module import name Import specific names from a module directly into your current namespace. No prefix needed when using them.\nfrom maths_utils import PI, circle_area, is_prime # Use directly -- no prefix print(PI) # 3.14159... print(circle_area(7)) # 153.938... print(is_prime(17)) # True # But Vector2D is NOT imported -- you only got what you asked for v = Vector2D(3, 4) # NameError: name \u0026#39;Vector2D\u0026#39; is not defined from module import * Imports everything from the module into the current namespace. Avoid this in production code. It pollutes your namespace and makes it impossible to tell where a name came from.\nfrom maths_utils import * # imports PI, E, GOLDEN_RATIO, circle_area, etc. # Now everything is available, but this is dangerous: PI = 3.14 # which PI are you using now? yours or maths_utils.PI? The only acceptable use of import * is in a package\u0026rsquo;s __init__.py to selectively expose its public API, and even then it requires a carefully defined __all__ list.\nimport module as alias Give the module a shorter name. This is the standard pattern for well-known libraries that have conventional aliases.\nimport numpy as np import pandas as pd import matplotlib.pyplot as plt import seaborn as sns # These aliases are universal conventions -- use them arr = np.array([1, 2, 3]) df = pd.DataFrame({\u0026#34;a\u0026#34;: [1, 2, 3]}) You can alias your own modules too:\nimport maths_utils as mu print(mu.circle_area(5)) from module import name as alias Combine both \u0026ndash; import a specific name and give it an alias.\nfrom maths_utils import circle_area as area from maths_utils import is_prime as prime_check print(area(7)) # 153.938... print(prime_check(17)) # True Importing from the standard library vs your own files # Standard library -- Python finds these automatically import os import sys import math import json import datetime # Your own modules -- Python looks in the current directory first import maths_utils # looks for maths_utils.py in the same folder # Third-party libraries (installed via pip) import numpy # Python looks in site-packages import pandas 3. How Python Finds What You Import When you write import something, Python searches for it in a specific order. Understanding this order explains why some imports work and others fail.\nimport sys # sys.path is the list of directories Python searches, in order print(sys.path) The search order is:\n1. sys.modules cache -- Already-imported modules are cached here. -- Python checks here first to avoid re-importing. 2. Built-in modules -- Modules compiled into the Python interpreter itself. -- Examples: sys, builtins, _io -- Check with: import sys; print(sys.builtin_module_names) 3. The current directory (or the script\u0026#39;s directory) -- If you run python main.py from /home/aarav/project/, Python looks in /home/aarav/project/ first. 4. PYTHONPATH environment variable directories -- A colon-separated list of directories you can set. -- Rarely needed in everyday work. 5. Standard library directories -- Where Python\u0026#39;s built-in modules live on your system. 6. Site-packages -- Where pip installs third-party libraries. -- Typically: .../lib/python3.x/site-packages/ import sys # See exactly where Python is looking for path in sys.path: print(path) # Find where a specific module is located import numpy print(numpy.__file__) # /path/to/site-packages/numpy/__init__.py import os print(os.__file__) # /path/to/lib/python3.x/os.py The most common import error explained # You are in /home/aarav/projects/analytics/ # Your file structure is: # # analytics/ # main.py # utils/ # helper.py # In main.py: import helper # ModuleNotFoundError -- helper.py is in utils/, not here # Correct: from utils import helper # or: import utils.helper This error almost always means either:\nThe file does not exist where Python is looking You are running the script from the wrong directory The package structure is not set up correctly (missing __init__.py) 4. The __name__ Variable and if __name__ == \u0026quot;__main__\u0026quot; Every Python module has a special variable called __name__.\nWhen a file is run directly (python my_file.py), __name__ is set to \u0026quot;__main__\u0026quot; When a file is imported by another file, __name__ is set to the module\u0026rsquo;s name (e.g., \u0026quot;my_file\u0026quot;) This distinction is critical. Here is why:\n# greetings.py def greet(name): return f\u0026#34;Namaste, {name}!\u0026#34; def farewell(name): return f\u0026#34;Alvida, {name}!\u0026#34; # This block runs ONLY when you execute: python greetings.py # It does NOT run when another file does: import greetings if __name__ == \u0026#34;__main__\u0026#34;: print(greet(\u0026#34;Aarav\u0026#34;)) print(farewell(\u0026#34;Priya\u0026#34;)) Without the if __name__ == \u0026quot;__main__\u0026quot;: guard:\n# greetings_bad.py def greet(name): return f\u0026#34;Namaste, {name}!\u0026#34; # This runs every time the module is imported -- WRONG print(greet(\u0026#34;Aarav\u0026#34;)) # executes when imported -- pollutes any importer # main.py import greetings_bad # this triggers the print() -- not what you want Why this pattern matters in practice # calculator.py def add(a, b): return a + b def subtract(a, b): return a - b def multiply(a, b): return a * b def divide(a, b): if b == 0: raise ValueError(\u0026#34;Cannot divide by zero\u0026#34;) return a / b # Tests and demos -- only run when executed directly if __name__ == \u0026#34;__main__\u0026#34;: print(\u0026#34;Running calculator tests...\u0026#34;) print(f\u0026#34;add(3, 4) = {add(3, 4)}\u0026#34;) print(f\u0026#34;subtract(10, 3) = {subtract(10, 3)}\u0026#34;) print(f\u0026#34;multiply(5, 6) = {multiply(5, 6)}\u0026#34;) print(f\u0026#34;divide(15, 3) = {divide(15, 3)}\u0026#34;) print(\u0026#34;All tests passed.\u0026#34;) Now calculator.py works as both a standalone script (run directly to test) and as a reusable module (imported by other files without side effects).\n5. Writing Your Own Modules Let us build a real, practical example. Suppose you are building a simple data processing project for cricket match statistics.\n# cricket_utils.py # A reusable module for cricket statistics calculations \u0026#34;\u0026#34;\u0026#34; cricket_utils -- Utility functions for cricket statistics. Functions: batting_average(runs, dismissals) strike_rate(runs, balls) economy_rate(runs, overs) bowling_average(runs, wickets) classify_performance(average) \u0026#34;\u0026#34;\u0026#34; def batting_average(runs, dismissals): \u0026#34;\u0026#34;\u0026#34; Calculate batting average. Args: runs (int): Total runs scored dismissals (int): Number of times dismissed Returns: float: Batting average, or None if never dismissed \u0026#34;\u0026#34;\u0026#34; if dismissals == 0: return None # Not out -- average is undefined, not zero return round(runs / dismissals, 2) def strike_rate(runs, balls): \u0026#34;\u0026#34;\u0026#34;Calculate batting strike rate (runs per 100 balls).\u0026#34;\u0026#34;\u0026#34; if balls == 0: return 0.0 return round((runs / balls) * 100, 2) def economy_rate(runs_conceded, overs_bowled): \u0026#34;\u0026#34;\u0026#34;Calculate bowling economy rate (runs per over).\u0026#34;\u0026#34;\u0026#34; if overs_bowled == 0: return 0.0 return round(runs_conceded / overs_bowled, 2) def bowling_average(runs_conceded, wickets): \u0026#34;\u0026#34;\u0026#34;Calculate bowling average (runs per wicket).\u0026#34;\u0026#34;\u0026#34; if wickets == 0: return None # No wickets taken -- average undefined return round(runs_conceded / wickets, 2) def classify_performance(batting_avg): \u0026#34;\u0026#34;\u0026#34; Classify a batsman\u0026#39;s performance based on average. Returns: str: \u0026#34;World Class\u0026#34;, \u0026#34;Excellent\u0026#34;, \u0026#34;Good\u0026#34;, \u0026#34;Average\u0026#34;, or \u0026#34;Developing\u0026#34; \u0026#34;\u0026#34;\u0026#34; if batting_avg is None: return \u0026#34;Insufficient data\u0026#34; if batting_avg \u0026gt;= 50: return \u0026#34;World Class\u0026#34; if batting_avg \u0026gt;= 40: return \u0026#34;Excellent\u0026#34; if batting_avg \u0026gt;= 30: return \u0026#34;Good\u0026#34; if batting_avg \u0026gt;= 20: return \u0026#34;Average\u0026#34; return \u0026#34;Developing\u0026#34; # Only runs when this file is executed directly if __name__ == \u0026#34;__main__\u0026#34;: # Quick demo print(\u0026#34;=== Cricket Stats Demo ===\u0026#34;) players = [ (\u0026#34;Virat Kohli\u0026#34;, 12898, 241, 8944, 6432), (\u0026#34;Rohit Sharma\u0026#34;, 9825, 243, 8714, 7145), (\u0026#34;MS Dhoni\u0026#34;, 10773, 350, 10000, 9350), ] for name, runs, dismissals, balls_faced, _ in players: avg = batting_average(runs, dismissals) sr = strike_rate(runs, balls_faced) cls = classify_performance(avg) print(f\u0026#34;{name:\u0026lt;20} Avg: {str(avg):\u0026lt;7} SR: {sr:\u0026lt;8} [{cls}]\u0026#34;) Now use it from another file:\n# main.py import cricket_utils # Using the module kohli_avg = cricket_utils.batting_average(12898, 241) kohli_sr = cricket_utils.strike_rate(12898, 6432) print(f\u0026#34;Kohli Average: {kohli_avg}\u0026#34;) print(f\u0026#34;Kohli SR: {kohli_sr}\u0026#34;) print(f\u0026#34;Performance: {cricket_utils.classify_performance(kohli_avg)}\u0026#34;) # Or import specific functions from cricket_utils import batting_average, strike_rate, classify_performance avg = batting_average(9825, 243) print(classify_performance(avg)) 6. What is a Package? A package is a directory that contains Python modules, with a special file called __init__.py that tells Python \u0026ldquo;this directory is a package, not just a folder.\u0026rdquo;\nWithout __init__.py: With __init__.py: analytics/ analytics/ \u0026lt;-- package data_loader.py __init__.py calculator.py data_loader.py \u0026lt;-- module visualiser.py calculator.py \u0026lt;-- module visualiser.py \u0026lt;-- module # Python treats this as # Python can import from this # just a directory -- you # as: import analytics # cannot import from it # or: from analytics import calculator Why packages? As a project grows, you end up with many modules. Packages let you organise related modules into a hierarchy:\ncodeverra_app/ __init__.py students/ __init__.py profile.py grades.py enrollment.py courses/ __init__.py curriculum.py schedule.py payments/ __init__.py billing.py invoice.py utils/ __init__.py validators.py formatters.py Now you can import very specifically:\nfrom codeverra_app.students import grades from codeverra_app.payments.billing import calculate_fee import codeverra_app.utils.validators as validators 7. Building Your Own Package Let us build a small but real package step by step.\nStep 1 \u0026ndash; Create the directory structure cricket_stats/ __init__.py batting.py bowling.py fielding.py utils.py Step 2 \u0026ndash; Write the modules # cricket_stats/batting.py def average(runs, dismissals): \u0026#34;\u0026#34;\u0026#34;Calculate batting average.\u0026#34;\u0026#34;\u0026#34; if dismissals == 0: return None return round(runs / dismissals, 2) def strike_rate(runs, balls): \u0026#34;\u0026#34;\u0026#34;Calculate batting strike rate.\u0026#34;\u0026#34;\u0026#34; if balls == 0: return 0.0 return round((runs / balls) * 100, 2) def centuries_count(scores): \u0026#34;\u0026#34;\u0026#34;Count number of centuries (100+) in a list of scores.\u0026#34;\u0026#34;\u0026#34; return sum(1 for s in scores if s \u0026gt;= 100) def highest_score(scores): \u0026#34;\u0026#34;\u0026#34;Return the highest score from a list.\u0026#34;\u0026#34;\u0026#34; return max(scores) if scores else 0 # cricket_stats/bowling.py def economy_rate(runs, overs): \u0026#34;\u0026#34;\u0026#34;Calculate bowling economy rate.\u0026#34;\u0026#34;\u0026#34; if overs == 0: return 0.0 return round(runs / overs, 2) def bowling_average(runs, wickets): \u0026#34;\u0026#34;\u0026#34;Calculate bowling average.\u0026#34;\u0026#34;\u0026#34; if wickets == 0: return None return round(runs / wickets, 2) def bowling_strike_rate(balls, wickets): \u0026#34;\u0026#34;\u0026#34;Calculate bowling strike rate (balls per wicket).\u0026#34;\u0026#34;\u0026#34; if wickets == 0: return None return round(balls / wickets, 2) # cricket_stats/utils.py def classify_batsman(average): \u0026#34;\u0026#34;\u0026#34;Classify a batsman based on batting average.\u0026#34;\u0026#34;\u0026#34; if average is None: return \u0026#34;Insufficient data\u0026#34; if average \u0026gt;= 50: return \u0026#34;World Class\u0026#34; if average \u0026gt;= 40: return \u0026#34;Excellent\u0026#34; if average \u0026gt;= 30: return \u0026#34;Good\u0026#34; if average \u0026gt;= 20: return \u0026#34;Average\u0026#34; return \u0026#34;Developing\u0026#34; def format_player_report(name, stats_dict): \u0026#34;\u0026#34;\u0026#34;Format a player report as a readable string.\u0026#34;\u0026#34;\u0026#34; lines = [f\u0026#34;\\n=== {name} ===\u0026#34;] for key, value in stats_dict.items(): lines.append(f\u0026#34; {key:\u0026lt;20}: {value}\u0026#34;) return \u0026#34;\\n\u0026#34;.join(lines) Step 3 \u0026ndash; Write __init__.py The __init__.py file controls what gets exposed when someone does import cricket_stats. You can leave it empty, or use it to:\nImport commonly used items so they are accessible directly from the package Define __all__ to control what from package import * exports Run package-level initialisation code # cricket_stats/__init__.py \u0026#34;\u0026#34;\u0026#34; cricket_stats -- A Python package for cricket statistics. Usage: import cricket_stats cricket_stats.batting.average(runs, dismissals) # Or import directly from the package (because of this __init__.py): from cricket_stats import batting_average \u0026#34;\u0026#34;\u0026#34; # Package metadata __version__ = \u0026#34;1.0.0\u0026#34; __author__ = \u0026#34;Codeverra\u0026#34; # Import commonly used functions directly into the package namespace # This lets users do: from cricket_stats import batting_average # instead of: from cricket_stats.batting import average from cricket_stats.batting import average as batting_average from cricket_stats.batting import strike_rate as batting_sr from cricket_stats.bowling import economy_rate from cricket_stats.bowling import bowling_average from cricket_stats.utils import classify_batsman, format_player_report # Control what \u0026#39;from cricket_stats import *\u0026#39; exports __all__ = [ \u0026#34;batting_average\u0026#34;, \u0026#34;batting_sr\u0026#34;, \u0026#34;economy_rate\u0026#34;, \u0026#34;bowling_average\u0026#34;, \u0026#34;classify_batsman\u0026#34;, \u0026#34;format_player_report\u0026#34;, ] Step 4 \u0026ndash; Use your package # main.py (in the same directory as the cricket_stats/ folder) # Option 1: import the package and use submodules import cricket_stats.batting as batting import cricket_stats.bowling as bowling from cricket_stats.utils import classify_batsman avg = batting.average(12898, 241) sr = batting.strike_rate(12898, 6432) eco = bowling.economy_rate(850, 120) print(f\u0026#34;Batting Average : {avg}\u0026#34;) print(f\u0026#34;Strike Rate : {sr}\u0026#34;) print(f\u0026#34;Economy Rate : {eco}\u0026#34;) print(f\u0026#34;Classification : {classify_batsman(avg)}\u0026#34;) # Option 2: use the shortcuts defined in __init__.py from cricket_stats import batting_average, batting_sr, classify_batsman avg = batting_average(9825, 243) print(f\u0026#34;Rohit Average: {avg} [{classify_batsman(avg)}]\u0026#34;) # Option 3: check package metadata import cricket_stats print(cricket_stats.__version__) # 1.0.0 8. Relative and Absolute Imports When code inside a package needs to import from another module in the same package, you have two choices.\nAbsolute imports \u0026ndash; always use the full path from the project root # cricket_stats/bowling.py # Absolute import -- works from anywhere from cricket_stats.utils import classify_batsman from cricket_stats.batting import average Relative imports \u0026ndash; use dots to mean \u0026ldquo;relative to this file\u0026rdquo; # cricket_stats/bowling.py # Relative import -- . means \u0026#34;same package\u0026#34; from .utils import classify_batsman # . = cricket_stats/ from .batting import average # . = cricket_stats/ # .. means \u0026#34;parent package\u0026#34; from ..database import connection # .. = one level up Which to use?\nPEP 8 (Python\u0026rsquo;s style guide) recommends absolute imports for clarity. They are always unambiguous \u0026ndash; you know exactly where something comes from. Relative imports can be confusing in deep package hierarchies.\nUse relative imports only within a package when you want the package to be relocatable (i.e., if you rename the top-level package, internal imports still work).\n9. The Standard Library \u0026ndash; A Practical Tour Python ships with a vast standard library \u0026ndash; hundreds of modules covering everything from file handling to cryptography to HTTP servers. You do not need to install any of it. It is already there.\nThe full library is documented at: https://docs.python.org/3/library/\nHere are the modules you will reach for most often in everyday work:\nos \u0026ndash; Operating System Interface import os # Current working directory print(os.getcwd()) # /home/aarav/projects # Change directory os.chdir(\u0026#34;/home/aarav/data\u0026#34;) # List files in a directory print(os.listdir(\u0026#34;.\u0026#34;)) # [\u0026#39;data.csv\u0026#39;, \u0026#39;notes.txt\u0026#39;, ...] # Check if a path exists print(os.path.exists(\u0026#34;data.csv\u0026#34;)) # True or False # Join paths safely (works on Windows and Mac/Linux) path = os.path.join(\u0026#34;data\u0026#34;, \u0026#34;raw\u0026#34;, \u0026#34;sales.csv\u0026#34;) print(path) # data/raw/sales.csv (or data\\raw\\sales.csv on Windows) # Create a directory os.makedirs(\u0026#34;output/charts\u0026#34;, exist_ok=True) # exist_ok=True -- no error if already exists # Environment variables print(os.environ.get(\u0026#34;HOME\u0026#34;)) # /home/aarav api_key = os.environ.get(\u0026#34;OPENAI_API_KEY\u0026#34;, \u0026#34;not set\u0026#34;) # Run a shell command os.system(\u0026#34;ls -la\u0026#34;) # avoid this -- use subprocess instead # Get file size print(os.path.getsize(\u0026#34;data.csv\u0026#34;)) # size in bytes Note: For most path operations, prefer pathlib.Path over os.path. pathlib is more modern, more readable, and object-oriented. os is still useful for environment variables, process management, and things pathlib does not cover.\nsys \u0026ndash; System-Specific Parameters import sys # Python version print(sys.version) # 3.12.0 (...) print(sys.version_info) # sys.version_info(major=3, minor=12, ...) # Command line arguments (when running: python script.py arg1 arg2) print(sys.argv) # [\u0026#39;script.py\u0026#39;, \u0026#39;arg1\u0026#39;, \u0026#39;arg2\u0026#39;] script_name = sys.argv[0] arguments = sys.argv[1:] # Exit the program # sys.exit(0) -- clean exit (0 = success) # sys.exit(1) -- exit with error code # The module search path (as discussed in Section 3) print(sys.path) # Platform print(sys.platform) # \u0026#39;linux\u0026#39;, \u0026#39;darwin\u0026#39; (macOS), \u0026#39;win32\u0026#39; # Standard output and error streams print(\u0026#34;Normal output\u0026#34;, file=sys.stdout) print(\u0026#34;Error output\u0026#34;, file=sys.stderr) datetime \u0026ndash; Working with Dates and Times from datetime import datetime, date, time, timedelta # Current date and time now = datetime.now() today = date.today() print(now) # 2024-03-15 14:30:22.123456 print(today) # 2024-03-15 # Create specific dates independence_day = date(1947, 8, 15) ipl_start = datetime(2024, 3, 22, 19, 30, 0) # 7:30 PM # Formatting -- strftime (datetime TO string) print(now.strftime(\u0026#34;%d %B %Y\u0026#34;)) # 15 March 2024 print(now.strftime(\u0026#34;%d/%m/%Y %H:%M\u0026#34;)) # 15/03/2024 14:30 print(now.strftime(\u0026#34;%A, %d %B %Y\u0026#34;)) # Friday, 15 March 2024 # Parsing -- strptime (string TO datetime) date_str = \u0026#34;15-03-2024\u0026#34; parsed = datetime.strptime(date_str, \u0026#34;%d-%m-%Y\u0026#34;) print(parsed) # 2024-03-15 00:00:00 api_timestamp = \u0026#34;2024-03-15T14:30:22Z\u0026#34; parsed_api = datetime.strptime(api_timestamp, \u0026#34;%Y-%m-%dT%H:%M:%SZ\u0026#34;) # Arithmetic with timedelta deadline = date.today() + timedelta(days=30) last_week = datetime.now() - timedelta(weeks=1) diff = date.today() - independence_day print(f\u0026#34;India has been independent for {diff.days} days\u0026#34;) # Comparing dates if date.today() \u0026gt; independence_day: print(\u0026#34;We are past independence day\u0026#34;) # Timestamps (Unix epoch -- seconds since Jan 1, 1970) import time timestamp = time.time() # current timestamp as float dt = datetime.fromtimestamp(timestamp) print(dt) collections \u0026ndash; Specialised Container Types from collections import Counter, defaultdict, OrderedDict, namedtuple, deque # Counter -- count occurrences of elements ipl_winners = [\u0026#34;CSK\u0026#34;, \u0026#34;MI\u0026#34;, \u0026#34;CSK\u0026#34;, \u0026#34;RCB\u0026#34;, \u0026#34;MI\u0026#34;, \u0026#34;CSK\u0026#34;, \u0026#34;KKR\u0026#34;, \u0026#34;MI\u0026#34;, \u0026#34;MI\u0026#34;, \u0026#34;CSK\u0026#34;] counts = Counter(ipl_winners) print(counts) # Counter({\u0026#39;CSK\u0026#39;: 4, \u0026#39;MI\u0026#39;: 4, \u0026#39;KKR\u0026#39;: 1, \u0026#39;RCB\u0026#39;: 1}) print(counts.most_common(2)) # [(\u0026#39;CSK\u0026#39;, 4), (\u0026#39;MI\u0026#39;, 4)] # defaultdict -- dict that creates a default value for missing keys # No more \u0026#34;if key not in dict: dict[key] = []\u0026#34; from collections import defaultdict dept_employees = defaultdict(list) # default value is an empty list employees = [(\u0026#34;Aarav\u0026#34;, \u0026#34;Engineering\u0026#34;), (\u0026#34;Priya\u0026#34;, \u0026#34;HR\u0026#34;), (\u0026#34;Rohan\u0026#34;, \u0026#34;Engineering\u0026#34;), (\u0026#34;Sneha\u0026#34;, \u0026#34;HR\u0026#34;), (\u0026#34;Karan\u0026#34;, \u0026#34;Sales\u0026#34;)] for name, dept in employees: dept_employees[dept].append(name) # no KeyError even on first access print(dict(dept_employees)) # {\u0026#39;Engineering\u0026#39;: [\u0026#39;Aarav\u0026#39;, \u0026#39;Rohan\u0026#39;], \u0026#39;HR\u0026#39;: [\u0026#39;Priya\u0026#39;, \u0026#39;Sneha\u0026#39;], \u0026#39;Sales\u0026#39;: [\u0026#39;Karan\u0026#39;]} # namedtuple -- tuple with named fields (lightweight class) Student = namedtuple(\u0026#34;Student\u0026#34;, [\u0026#34;name\u0026#34;, \u0026#34;roll\u0026#34;, \u0026#34;cgpa\u0026#34;]) s = Student(\u0026#34;Aarav Sharma\u0026#34;, 101, 8.9) print(s.name) # Aarav Sharma print(s.cgpa) # 8.9 print(s[0]) # Aarav Sharma (still indexable like a tuple) # deque -- double-ended queue (fast appends and pops from both ends) from collections import deque queue = deque([\u0026#34;Rohit\u0026#34;, \u0026#34;Virat\u0026#34;, \u0026#34;Dhoni\u0026#34;]) queue.appendleft(\u0026#34;Shubman\u0026#34;) # add to front -- O(1) queue.append(\u0026#34;Jadeja\u0026#34;) # add to back -- O(1) print(queue.popleft()) # \u0026#34;Shubman\u0026#34; -- remove from front math \u0026ndash; Mathematical Functions import math print(math.sqrt(144)) # 12.0 print(math.ceil(4.1)) # 5 print(math.floor(4.9)) # 4 print(math.log(100, 10)) # 2.0 print(math.log2(1024)) # 10.0 print(math.factorial(10)) # 3628800 print(math.gcd(48, 18)) # 6 print(math.pi) # 3.14159... print(math.e) # 2.71828... print(math.inf) # inf print(math.isnan(float(\u0026#34;nan\u0026#34;))) # True print(math.isinf(math.inf)) # True random \u0026ndash; Random Number Generation import random random.seed(42) # set seed for reproducibility # Random float between 0 and 1 print(random.random()) # 0.6394... # Random integer in range (inclusive) print(random.randint(1, 6)) # dice roll # Random float in range print(random.uniform(10.5, 20.5)) # 14.32... # Random choice from a sequence players = [\u0026#34;Rohit\u0026#34;, \u0026#34;Virat\u0026#34;, \u0026#34;Dhoni\u0026#34;, \u0026#34;Jadeja\u0026#34;, \u0026#34;Bumrah\u0026#34;] print(random.choice(players)) # one random player # Multiple choices (with replacement) print(random.choices(players, k=3)) # [\u0026#39;Virat\u0026#39;, \u0026#39;Rohit\u0026#39;, \u0026#39;Virat\u0026#39;] # Sample without replacement print(random.sample(players, k=3)) # [\u0026#39;Dhoni\u0026#39;, \u0026#39;Bumrah\u0026#39;, \u0026#39;Jadeja\u0026#39;] # Shuffle a list in place random.shuffle(players) print(players) json \u0026ndash; JSON Encoding and Decoding import json # Python dict to JSON string data = { \u0026#34;name\u0026#34;: \u0026#34;Aarav Sharma\u0026#34;, \u0026#34;city\u0026#34;: \u0026#34;Bangalore\u0026#34;, \u0026#34;scores\u0026#34;: [88, 92, 75], \u0026#34;enrolled\u0026#34;: True, \u0026#34;grade\u0026#34;: None } json_string = json.dumps(data, indent=2) # indent for pretty printing print(json_string) # JSON string to Python dict parsed = json.loads(json_string) print(parsed[\u0026#34;name\u0026#34;]) # Aarav Sharma print(type(parsed)) # \u0026lt;class \u0026#39;dict\u0026#39;\u0026gt; # Read JSON from file with open(\u0026#34;data.json\u0026#34;, \u0026#34;r\u0026#34;, encoding=\u0026#34;utf-8\u0026#34;) as f: data = json.load(f) # Write JSON to file with open(\u0026#34;output.json\u0026#34;, \u0026#34;w\u0026#34;, encoding=\u0026#34;utf-8\u0026#34;) as f: json.dump(data, f, indent=2, ensure_ascii=False) # ensure_ascii=False preserves non-ASCII chars like Indian language text time \u0026ndash; Time Measurement import time # Current time as Unix timestamp print(time.time()) # 1710504622.123 # Sleep (pause execution) time.sleep(1.5) # sleep for 1.5 seconds # Measure execution time start = time.time() result = sum(range(10_000_000)) elapsed = time.time() - start print(f\u0026#34;Took {elapsed:.4f} seconds\u0026#34;) # More precise timing start = time.perf_counter() # ... code to time ... elapsed = time.perf_counter() - start copy \u0026ndash; Shallow and Deep Copy import copy original = {\u0026#34;name\u0026#34;: \u0026#34;Aarav\u0026#34;, \u0026#34;scores\u0026#34;: [88, 92, 75]} # Shallow copy -- top-level dict is new, but nested list is shared shallow = copy.copy(original) shallow[\u0026#34;scores\u0026#34;].append(100) print(original[\u0026#34;scores\u0026#34;]) # [88, 92, 75, 100] -- original affected! # Deep copy -- completely independent at all levels original = {\u0026#34;name\u0026#34;: \u0026#34;Aarav\u0026#34;, \u0026#34;scores\u0026#34;: [88, 92, 75]} deep = copy.deepcopy(original) deep[\u0026#34;scores\u0026#34;].append(100) print(original[\u0026#34;scores\u0026#34;]) # [88, 92, 75] -- original unaffected typing \u0026ndash; Type Hints from typing import List, Dict, Tuple, Optional, Union, Callable, Any def process_students( students: List[Dict[str, Any]], min_cgpa: float = 6.0 ) -\u0026gt; List[str]: \u0026#34;\u0026#34;\u0026#34;Return names of students above the minimum CGPA.\u0026#34;\u0026#34;\u0026#34; return [s[\u0026#34;name\u0026#34;] for s in students if s[\u0026#34;cgpa\u0026#34;] \u0026gt;= min_cgpa] def safe_divide(a: float, b: float) -\u0026gt; Optional[float]: \u0026#34;\u0026#34;\u0026#34;Return a/b or None if b is zero.\u0026#34;\u0026#34;\u0026#34; if b == 0: return None return a / b # Python does NOT enforce these at runtime # They are hints for developers and tools like mypy 10. pip \u0026ndash; Installing Third-Party Packages pip is Python\u0026rsquo;s package installer. It downloads packages from PyPI (the Python Package Index at https://pypi.org) and installs them into your Python environment.\nBasic pip commands # Install a package pip install requests # Install a specific version pip install requests==2.31.0 # Install the latest version that is compatible with a constraint pip install \u0026#34;requests\u0026gt;=2.28,\u0026lt;3.0\u0026#34; # Upgrade a package to the latest version pip install --upgrade pandas # Uninstall a package pip uninstall numpy # List all installed packages pip list # Show details about a specific package pip show pandas # Search for packages (deprecated on PyPI, use the website instead) # pip search \u0026#34;data visualisation\u0026#34; # Check for outdated packages pip list --outdated Installing from a requirements file pip install -r requirements.txt Where pip installs packages By default, pip installs into the global Python environment. This is the source of many problems (covered in the virtual environments section). The right workflow is always: create a virtual environment first, then pip install.\n# WRONG -- installs globally, causes version conflicts pip install pandas # RIGHT -- install into an active virtual environment python -m venv venv source venv/bin/activate # Mac/Linux pip install pandas # now installs into venv only pip vs pip3 On systems with both Python 2 and Python 3 installed, pip might refer to Python 2\u0026rsquo;s pip and pip3 to Python 3\u0026rsquo;s. On modern systems with only Python 3, they are the same. The safest form is always:\npython -m pip install requests # This explicitly uses the pip associated with whichever python you are using 11. requirements.txt \u0026ndash; Sharing Dependencies A requirements.txt file lists all the packages your project depends on. It is how you tell teammates (and servers) exactly what needs to be installed to run your code.\nCreating requirements.txt # Manually -- write out what your project actually needs # requirements.txt: pandas==2.1.4 numpy==1.26.2 matplotlib==3.8.2 seaborn==0.13.0 requests==2.31.0 # Automatically -- capture everything currently installed pip freeze \u0026gt; requirements.txt # Warning: pip freeze captures ALL packages, including indirect dependencies. # This creates a very long file. Fine for exact reproducibility, # but consider manually listing only your direct dependencies for clarity. requirements.txt version specifiers # Exact version -- maximum reproducibility pandas==2.1.4 # Minimum version -- \u0026#34;at least this\u0026#34; requests\u0026gt;=2.28.0 # Compatible release -- \u0026#34;same major.minor, any patch\u0026#34; numpy~=1.26.0 # means \u0026gt;=1.26.0, \u0026lt;1.27.0 # Range -- useful for avoiding known-broken versions scipy\u0026gt;=1.11,\u0026lt;2.0 # No version constraint -- just needs to be installed matplotlib Multiple requirements files Large projects often have separate requirements files for different purposes:\nrequirements.txt -- production dependencies only requirements-dev.txt -- adds testing and development tools requirements-test.txt -- testing tools only # requirements-dev.txt -r requirements.txt # include everything from production pytest==7.4.3 black==23.11.0 mypy==1.7.1 jupyter==1.0.0 12. Virtual Environments \u0026ndash; Why They Are Non-Negotiable This is the concept that most beginners skip and regret later.\nThe problem When you run pip install pandas, it installs pandas into your global Python environment \u0026ndash; the one Python installation on your machine.\nNow imagine:\nProject A (built 2022): pandas 1.3.5 numpy 1.21.0 scikit-learn 1.0.2 Project B (built 2024): pandas 2.1.4 numpy 1.26.2 scikit-learn 1.3.2 These two projects need different versions of the same libraries. If you install pandas 2.1.4 globally, Project A might break because it uses APIs that were removed in pandas 2.x.\nThis is the dependency conflict problem. It affects every Python developer who works on more than one project.\nThe solution \u0026ndash; virtual environments A virtual environment is an isolated Python installation specific to one project. It has its own copy of pip and its own site-packages directory.\nmy_machine/ Python 3.12 (global) \u0026lt;-- base installation, keep it clean | project_a/ venv/ \u0026lt;-- isolated environment for project_a lib/python3.12/site-packages/ pandas-1.3.5/ numpy-1.21.0/ | project_b/ venv/ \u0026lt;-- isolated environment for project_b lib/python3.12/site-packages/ pandas-2.1.4/ numpy-1.26.2/ Project A and Project B have completely separate dependencies. They never conflict. You can work on both simultaneously.\nThe golden rule Every project gets its own virtual environment. No exceptions.\n13. venv \u0026ndash; The Built-In Solution venv is Python\u0026rsquo;s built-in virtual environment tool. No installation required.\nCreating and using a virtual environment # Step 1: Create a virtual environment # Convention: name it \u0026#39;venv\u0026#39; or \u0026#39;.venv\u0026#39; inside your project directory python -m venv venv # What this creates: # venv/ # bin/ (Mac/Linux) or Scripts/ (Windows) # python # pip # lib/ # python3.12/ # site-packages/ \u0026lt;-- packages install here # pyvenv.cfg # Step 2: Activate the environment # Mac / Linux: source venv/bin/activate # Windows (Command Prompt): venv\\Scripts\\activate.bat # Windows (PowerShell): venv\\Scripts\\Activate.ps1 # You will see (venv) prefix in your terminal: # (venv) aarav@laptop:~/projects/my_project$ # Step 3: Install packages -- they go into the venv, not globally pip install pandas numpy matplotlib # Step 4: Work on your project python main.py jupyter notebook # Step 5: Deactivate when done deactivate # (venv) prefix disappears -- you are back to global environment The .gitignore rule Always add your venv directory to .gitignore. You do not commit the virtual environment to version control \u0026ndash; it is too large and machine-specific. Instead, commit requirements.txt so anyone can recreate it.\n# .gitignore venv/ .venv/ __pycache__/ *.pyc .env Recreating an environment from requirements.txt # Another developer (or you, on a new machine) does: git clone https://github.com/your/project cd project python -m venv venv source venv/bin/activate pip install -r requirements.txt # Exact same environment, every time 14. conda \u0026ndash; The Data Science Standard conda is a package and environment manager from Anaconda. Unlike pip + venv, conda manages both Python packages AND non-Python dependencies (like C libraries that NumPy and SciPy depend on).\nThis makes it particularly well-suited for data science, machine learning, and scientific computing where many packages have complex native dependencies.\nAnaconda vs Miniconda Anaconda: Full distribution. Installs Python, conda, and 250+ data science packages out of the box (pandas, numpy, matplotlib, scikit-learn, jupyter, etc.). Large download (~3GB). Good for beginners who want everything ready.\nMiniconda: Minimal distribution. Just Python and conda. You install only what you need. Smaller, faster, recommended for experienced users.\nDownload links:\nAnaconda: https://www.anaconda.com/download Miniconda: https://docs.anaconda.com/miniconda/ Core conda commands # Check conda version conda --version # Update conda itself conda update conda # Create a new environment conda create --name my_project python=3.11 # Create with specific packages conda create --name ds_project python=3.11 pandas numpy matplotlib jupyter # Activate an environment conda activate my_project # Deactivate conda deactivate # List all environments conda env list # or: conda info --envs # Install a package into active environment conda install pandas conda install -c conda-forge some_package # install from conda-forge channel # You can still use pip inside a conda environment pip install some_package_not_on_conda # List packages in current environment conda list # Remove an environment conda env remove --name my_project # Export environment to a file conda env export \u0026gt; environment.yml # Recreate environment from file conda env create -f environment.yml environment.yml \u0026ndash; conda\u0026rsquo;s requirements file # environment.yml name: codeverra_ds channels: - conda-forge - defaults dependencies: - python=3.11 - pandas=2.1.4 - numpy=1.26.2 - matplotlib=3.8.2 - seaborn=0.13.0 - scikit-learn=1.3.2 - jupyter=1.0.0 - pip - pip: - some-package-only-on-pypi==1.2.3 # Create environment from yml conda env create -f environment.yml # Update environment when yml changes conda env update -f environment.yml --prune conda channels Channels are repositories where conda looks for packages. The two most important ones:\ndefaults: Anaconda\u0026rsquo;s own channel. Most stable, slightly outdated. conda-forge: Community-maintained channel. More packages, more up-to-date. # Install from conda-forge conda install -c conda-forge plotly # Set conda-forge as default channel conda config --add channels conda-forge conda config --set channel_priority strict Official conda documentation: https://docs.conda.io/en/latest/\n15. Other Tools \u0026ndash; virtualenv, uv virtualenv virtualenv is the original virtual environment tool that inspired Python\u0026rsquo;s built-in venv. It is slightly faster than venv, supports older Python versions, and has a few extra features that venv lacks. For most modern Python 3 development, venv is sufficient. virtualenv is worth knowing because many older tutorials and projects use it.\npip install virtualenv virtualenv my_env source my_env/bin/activate Documentation: https://virtualenv.pypa.io/en/latest/\nuv \u0026ndash; The Modern High-Performance Alternative uv is a very new (2024) Rust-based Python package manager and environment tool built by Astral (the team behind the ruff linter). It is designed to be a drop-in replacement for pip and venv, but dramatically faster.\nKey claim: uv installs packages 10x to 100x faster than pip. It also resolves dependencies more reliably.\n# Install uv pip install uv # or on Mac/Linux: curl -LsSf https://astral.sh/uv/install.sh | sh # Create a virtual environment uv venv # Install packages (replaces pip install) uv pip install pandas numpy matplotlib # Install from requirements.txt uv pip install -r requirements.txt # Sync environment to exact requirements uv pip sync requirements.txt # Run a script in an isolated environment (no manual activation needed) uv run script.py uv is gaining adoption quickly in the Python community. It is worth watching even if you start with venv + pip.\nDocumentation: https://docs.astral.sh/uv/\npipenv Another tool that combines pip and virtualenv. Was popular around 2018-2020 but has largely been superseded by venv (for simplicity) and uv (for speed). Mentioned here because you will encounter it in older projects.\nDocumentation: https://pipenv.pypa.io/en/latest/\n16. Which Tool Should You Use? The tool landscape can be confusing. Here is a practical guide:\nAre you doing data science / ML / scientific computing? YES --\u0026gt; Use conda (Miniconda recommended) conda manages non-Python dependencies much better Handles numpy, scipy, tensorflow, pytorch cleanly NO --\u0026gt; Continue below Are you on a modern project (2024+) and value speed? YES --\u0026gt; Consider uv Dramatically faster installs Compatible with pip and requirements.txt Growing community adoption NO --\u0026gt; Continue below Are you a beginner or on a simple project? YES --\u0026gt; Use venv + pip Built into Python, no extra installation Works everywhere, well-documented Fine for 90% of projects Summary table:\nTool Best for Install Speed Non-Python deps venv + pip General Python, beginners Built-in Moderate No conda Data science, ML, scientific Separate download Moderate Yes uv Modern projects, speed matters pip install uv Very fast No virtualenv Compatibility with older Python pip install Fast No The one rule that applies to all of them:\nAlways use an isolated environment. Never install project dependencies globally.\n17. Practice Questions Try each question before reading the solution in Section 18. Questions are ordered: Easy (1-5), Medium (6-10), Hard (11-15).\nEasy Q1 \u0026ndash; Import Styles Given a module geometry.py with functions circle_area(r), rectangle_area(l, w), and a constant PI = 3.14159, write four different ways to import and use circle_area. Explain when you would use each style.\nQ2 \u0026ndash; name Guard The file below has a bug \u0026ndash; it prints output every time it is imported. Fix it so the print statements only run when the file is executed directly.\n# temperature.py def celsius_to_fahrenheit(c): return (c * 9/5) + 32 def fahrenheit_to_celsius(f): return (f - 32) * 5/9 print(\u0026#34;Testing conversions:\u0026#34;) print(celsius_to_fahrenheit(100)) print(fahrenheit_to_celsius(32)) Q3 \u0026ndash; Standard Library Using only the Python standard library (no pip installs), write a script that:\nGets the current date and time Formats it as \u0026ldquo;Day DD Month YYYY, HH:MM\u0026rdquo; (e.g., \u0026ldquo;Friday 15 March 2024, 14:30\u0026rdquo;) Calculates how many days until the next New Year\u0026rsquo;s Day Prints both Q4 \u0026ndash; collections.Counter Given a list of 50 random dice rolls (use random.randint(1, 6)), use Counter to:\nCount how many times each number appeared Find the most common roll Find the least common roll Print a simple frequency bar chart using only string multiplication Q5 \u0026ndash; defaultdict Without using any if statements, rewrite the following code using defaultdict:\n# Original -- uses if to handle missing keys word_positions = {} words = \u0026#34;the cat sat on the mat the cat wore a hat\u0026#34;.split() for i, word in enumerate(words): if word not in word_positions: word_positions[word] = [] word_positions[word].append(i) print(word_positions) Medium Q6 \u0026ndash; Build a Module Create a module called indian_holidays.py that contains:\nA dictionary of major Indian national holidays with their dates (at least 6) A function get_holiday(date_str) that returns the holiday name for a given \u0026ldquo;DD-MM\u0026rdquo; string, or \u0026ldquo;No holiday\u0026rdquo; if none A function holidays_in_month(month_num) that returns all holidays in that month A function days_until_next_holiday() that returns the name and number of days until the next upcoming holiday A proper if __name__ == \u0026quot;__main__\u0026quot;: section that demos all three functions Q7 \u0026ndash; Package Structure Design the folder structure for a package called school_manager that handles:\nStudent records (create, update, delete, search) Course management (add courses, assign to students) Grade tracking (record grades, calculate averages, generate reports) Utilities (validators, formatters) Write out the full directory tree with all file names. Then write the __init__.py that exposes the 5 most useful functions directly from the top-level package.\nQ8 \u0026ndash; sys.argv Write a command-line script csv_stats.py that:\nTakes a CSV filename as a command-line argument Validates the argument exists and the file exists Reads the CSV using only the standard library Prints: number of rows, number of columns, column names, and any rows where any field is empty Shows usage instructions if called with wrong arguments Usage: python csv_stats.py data.csv\nQ9 \u0026ndash; requirements.txt and Environments You are given the following scenario. Write out the exact sequence of terminal commands you would run for each step:\nCreate a new project directory called cricket_analytics Create a virtual environment inside it Activate the environment Install pandas 2.1.4, matplotlib 3.8.2, and seaborn 0.13.0 Create a requirements.txt Deactivate the environment Then write the .gitignore entries for this project.\nQ10 \u0026ndash; Module Search Path Explain what happens step by step when Python executes this line:\nimport pandas Include: where Python looks, what file it finds, what sys.modules does, and what you can inspect on the pandas module object after import. Write this as a numbered explanation, not code.\nHard Q11 \u0026ndash; Custom Namespace Package Build a package data_toolkit with the following submodules:\ndata_toolkit.io \u0026ndash; read_csv(path), write_csv(data, path), read_json(path) data_toolkit.clean \u0026ndash; remove_duplicates(data), fill_missing(data, value), trim_strings(data) data_toolkit.stats \u0026ndash; mean(values), median(values), std_dev(values), percentile(values, p) Write a complete __init__.py that:\nExposes the most commonly used functions at the top level Sets __version__, __author__, __all__ Prints a deprecation warning if Python version is below 3.9 Q12 \u0026ndash; Environment Investigation Write a Python script env_doctor.py that diagnoses the current Python environment:\nPython version (and whether it is at least 3.9) Whether running inside a virtual environment or not Location of the Python executable Number of installed packages Location of site-packages Top 5 largest installed packages by directory size Whether common data science packages are installed (pandas, numpy, matplotlib, sklearn) and their versions if installed Hint: use sys, os, importlib, and pkg_resources or importlib.metadata.\nQ13 \u0026ndash; Lazy Import Pattern Some imports are expensive (take significant time or memory). A common pattern is to delay importing a heavy module until it is actually needed.\nWrite a class LazyImporter that:\nTakes a module name in __init__ Does NOT import the module immediately Imports it on first attribute access Caches the module after first import Works transparently \u0026ndash; lazy.some_function() should work exactly like module.some_function() # Usage: np = LazyImporter(\u0026#34;numpy\u0026#34;) pd = LazyImporter(\u0026#34;pandas\u0026#34;) # No import happens yet -- just two LazyImporter objects # Import happens HERE, on first use arr = np.array([1, 2, 3]) # numpy imported now df = pd.DataFrame({\u0026#34;a\u0026#34;: [1,2,3]}) # pandas imported now # Subsequent uses use the cached module arr2 = np.array([4, 5, 6]) # no re-import Q14 \u0026ndash; Plugin System Build a simple plugin architecture using Python\u0026rsquo;s import machinery. The system should:\nHave a plugins/ directory where new functionality can be dropped in Each plugin is a Python file with a register() function and a PLUGIN_NAME constant A PluginManager class that scans the plugins directory, imports all valid plugins, and registers them Support calling a plugin by name: manager.run(\u0026quot;csv_exporter\u0026quot;, data) Write two sample plugins: csv_exporter.py and json_exporter.py.\nQ15 \u0026ndash; Full Project Setup You are starting a new data analysis project for IPL statistics. Create the complete project scaffold:\nFull directory structure with all files (including empty ones) A setup.py or pyproject.toml with project metadata requirements.txt and requirements-dev.txt .gitignore A README.md template A config.py module that reads settings from environment variables with sensible defaults A logger.py module that sets up a standard logger for the project An __init__.py for the main package that exposes version info The project should follow professional Python project structure conventions.\n18. Solutions Q1 \u0026ndash; Import Styles # geometry.py (the module we are importing from) PI = 3.14159 def circle_area(r): return PI * r ** 2 def rectangle_area(l, w): return l * w # Way 1: import the whole module # Use when: you want explicit namespace, avoid name collisions import geometry area = geometry.circle_area(7) # Way 2: from ... import specific name # Use when: you use the function frequently and want shorter syntax from geometry import circle_area area = circle_area(7) # Way 3: import with alias # Use when: the module name is long or conflicts with something import geometry as geo area = geo.circle_area(7) # Way 4: from ... import with alias # Use when: the function name conflicts with something in current scope from geometry import circle_area as c_area area = c_area(7) print(area) # 153.938... Q2 \u0026ndash; name Guard # temperature.py -- fixed def celsius_to_fahrenheit(c): return (c * 9/5) + 32 def fahrenheit_to_celsius(f): return (f - 32) * 5/9 # Only runs when this file is executed directly, not when imported if __name__ == \u0026#34;__main__\u0026#34;: print(\u0026#34;Testing conversions:\u0026#34;) print(celsius_to_fahrenheit(100)) # 212.0 print(fahrenheit_to_celsius(32)) # 0.0 Q3 \u0026ndash; Standard Library Date Work from datetime import datetime, date now = datetime.now() today = date.today() # Format: \u0026#34;Friday 15 March 2024, 14:30\u0026#34; formatted = now.strftime(\u0026#34;%A %d %B %Y, %H:%M\u0026#34;) print(f\u0026#34;Current date and time: {formatted}\u0026#34;) # Days until next New Year\u0026#39;s Day current_year = today.year next_new_year = date(current_year + 1, 1, 1) days_until = (next_new_year - today).days print(f\u0026#34;Days until New Year ({next_new_year}): {days_until}\u0026#34;) Q4 \u0026ndash; Counter Dice Rolls import random from collections import Counter random.seed(42) rolls = [random.randint(1, 6) for _ in range(50)] counts = Counter(rolls) most_common = counts.most_common(1)[0] least_common = counts.most_common()[-1] print(f\u0026#34;Most common roll : {most_common[0]} (appeared {most_common[1]} times)\u0026#34;) print(f\u0026#34;Least common roll : {least_common[0]} (appeared {least_common[1]} times)\u0026#34;) print(\u0026#34;\\nFrequency chart:\u0026#34;) for face in range(1, 7): bar = \u0026#34;#\u0026#34; * counts[face] print(f\u0026#34; {face}: {bar:\u0026lt;20} ({counts[face]})\u0026#34;) Q5 \u0026ndash; defaultdict from collections import defaultdict word_positions = defaultdict(list) # default value is an empty list words = \u0026#34;the cat sat on the mat the cat wore a hat\u0026#34;.split() for i, word in enumerate(words): word_positions[word].append(i) # no KeyError, no if statement needed print(dict(word_positions)) # {\u0026#39;the\u0026#39;: [0, 4, 6], \u0026#39;cat\u0026#39;: [1, 7], \u0026#39;sat\u0026#39;: [2], ...} Q6 \u0026ndash; Indian Holidays Module # indian_holidays.py \u0026#34;\u0026#34;\u0026#34; indian_holidays -- Indian national and major public holidays. Functions: get_holiday(date_str) -- look up by \u0026#34;DD-MM\u0026#34; holidays_in_month(month_num) -- all holidays in a month days_until_next_holiday() -- countdown to next holiday \u0026#34;\u0026#34;\u0026#34; from datetime import date, datetime, timedelta # Major Indian national and public holidays # Format: \u0026#34;DD-MM\u0026#34; -\u0026gt; \u0026#34;Holiday Name\u0026#34; HOLIDAYS = { \u0026#34;26-01\u0026#34;: \u0026#34;Republic Day\u0026#34;, \u0026#34;14-04\u0026#34;: \u0026#34;Dr. Ambedkar Jayanti\u0026#34;, \u0026#34;01-05\u0026#34;: \u0026#34;Labour Day / Maharashtra Day\u0026#34;, \u0026#34;15-08\u0026#34;: \u0026#34;Independence Day\u0026#34;, \u0026#34;02-10\u0026#34;: \u0026#34;Gandhi Jayanti\u0026#34;, \u0026#34;25-12\u0026#34;: \u0026#34;Christmas Day\u0026#34;, \u0026#34;01-01\u0026#34;: \u0026#34;New Year\u0026#39;s Day\u0026#34;, \u0026#34;14-01\u0026#34;: \u0026#34;Makar Sankranti / Pongal\u0026#34;, } def get_holiday(date_str): \u0026#34;\u0026#34;\u0026#34; Return holiday name for a given date string \u0026#34;DD-MM\u0026#34;. Args: date_str (str): Date in \u0026#34;DD-MM\u0026#34; format (e.g., \u0026#34;15-08\u0026#34;) Returns: str: Holiday name or \u0026#34;No holiday on this date\u0026#34; \u0026#34;\u0026#34;\u0026#34; return HOLIDAYS.get(date_str, \u0026#34;No holiday on this date\u0026#34;) def holidays_in_month(month_num): \u0026#34;\u0026#34;\u0026#34; Return all holidays in the given month. Args: month_num (int): Month number 1-12 Returns: dict: {date_str: holiday_name} for all holidays in the month \u0026#34;\u0026#34;\u0026#34; return { date_str: name for date_str, name in HOLIDAYS.items() if int(date_str.split(\u0026#34;-\u0026#34;)[1]) == month_num } def days_until_next_holiday(): \u0026#34;\u0026#34;\u0026#34; Find the next upcoming Indian holiday and how many days away it is. Returns: tuple: (holiday_name, days_remaining, holiday_date) \u0026#34;\u0026#34;\u0026#34; today = date.today() current_year = today.year closest_name = None closest_days = float(\u0026#34;inf\u0026#34;) closest_date = None for date_str, name in HOLIDAYS.items(): day, month = map(int, date_str.split(\u0026#34;-\u0026#34;)) # Try this year first holiday_date = date(current_year, month, day) if holiday_date \u0026lt; today: # Holiday has passed -- try next year holiday_date = date(current_year + 1, month, day) days_away = (holiday_date - today).days if days_away \u0026lt; closest_days: closest_days = days_away closest_name = name closest_date = holiday_date return closest_name, closest_days, closest_date if __name__ == \u0026#34;__main__\u0026#34;: print(\u0026#34;=== Indian Holidays Demo ===\\n\u0026#34;) # get_holiday print(\u0026#34;15-08:\u0026#34;, get_holiday(\u0026#34;15-08\u0026#34;)) # Independence Day print(\u0026#34;10-07:\u0026#34;, get_holiday(\u0026#34;10-07\u0026#34;)) # No holiday # holidays_in_month print(\u0026#34;\\nHolidays in January:\u0026#34;) for date_str, name in holidays_in_month(1).items(): print(f\u0026#34; {date_str}: {name}\u0026#34;) # days_until_next_holiday name, days, hdate = days_until_next_holiday() print(f\u0026#34;\\nNext holiday: {name} on {hdate} ({days} days away)\u0026#34;) Q7 \u0026ndash; Package Structure school_manager/ __init__.py students/ __init__.py records.py -- create_student, update_student, delete_student, search_students enrollment.py -- enroll_student, unenroll_student, get_enrolled_courses courses/ __init__.py management.py -- add_course, remove_course, update_course, list_courses assignment.py -- assign_course_to_student, get_course_students grades/ __init__.py tracking.py -- record_grade, update_grade, get_student_grades reports.py -- generate_report, calculate_average, get_class_topper utils/ __init__.py validators.py -- validate_roll_number, validate_grade, validate_email formatters.py -- format_student_report, format_grade_card # school_manager/__init__.py \u0026#34;\u0026#34;\u0026#34; school_manager -- A complete school management system. \u0026#34;\u0026#34;\u0026#34; __version__ = \u0026#34;1.0.0\u0026#34; __author__ = \u0026#34;Codeverra\u0026#34; # Expose the 5 most useful functions at the top level from school_manager.students.records import create_student, search_students from school_manager.grades.tracking import record_grade from school_manager.grades.reports import generate_report, get_class_topper __all__ = [ \u0026#34;create_student\u0026#34;, \u0026#34;search_students\u0026#34;, \u0026#34;record_grade\u0026#34;, \u0026#34;generate_report\u0026#34;, \u0026#34;get_class_topper\u0026#34;, ] Q8 \u0026ndash; sys.argv CSV Stats # csv_stats.py import sys import csv import os def print_usage(): print(\u0026#34;Usage: python csv_stats.py \u0026lt;filename.csv\u0026gt;\u0026#34;) print(\u0026#34;Example: python csv_stats.py students.csv\u0026#34;) def analyse_csv(filepath): \u0026#34;\u0026#34;\u0026#34;Read and print statistics about a CSV file.\u0026#34;\u0026#34;\u0026#34; rows = [] empty_rows = [] with open(filepath, \u0026#34;r\u0026#34;, encoding=\u0026#34;utf-8\u0026#34;, newline=\u0026#34;\u0026#34;) as f: reader = csv.DictReader(f) headers = reader.fieldnames for i, row in enumerate(reader, start=2): # start=2 because row 1 is header rows.append(row) empty_fields = [col for col, val in row.items() if not val.strip()] if empty_fields: empty_rows.append((i, empty_fields)) print(f\u0026#34;\\nFile : {filepath}\u0026#34;) print(f\u0026#34;Rows : {len(rows)}\u0026#34;) print(f\u0026#34;Columns : {len(headers)}\u0026#34;) print(f\u0026#34;Column names: {\u0026#39;, \u0026#39;.join(headers)}\u0026#34;) if empty_rows: print(f\u0026#34;\\nRows with empty fields ({len(empty_rows)} found):\u0026#34;) for row_num, fields in empty_rows[:10]: # show first 10 only print(f\u0026#34; Row {row_num}: empty fields = {fields}\u0026#34;) else: print(\u0026#34;\\nNo empty fields found.\u0026#34;) def main(): # Validate arguments if len(sys.argv) != 2: print(\u0026#34;Error: exactly one argument required.\u0026#34;) print_usage() sys.exit(1) filepath = sys.argv[1] if not os.path.exists(filepath): print(f\u0026#34;Error: file not found: {filepath}\u0026#34;) sys.exit(1) if not filepath.endswith(\u0026#34;.csv\u0026#34;): print(f\u0026#34;Warning: file does not have .csv extension: {filepath}\u0026#34;) try: analyse_csv(filepath) except Exception as e: print(f\u0026#34;Error reading file: {e}\u0026#34;) sys.exit(1) if __name__ == \u0026#34;__main__\u0026#34;: main() Q9 \u0026ndash; Virtual Environment Commands # 1. Create project directory mkdir cricket_analytics cd cricket_analytics # 2. Create virtual environment python -m venv venv # 3. Activate source venv/bin/activate # Mac/Linux # venv\\Scripts\\activate.bat # Windows # 4. Install packages pip install pandas==2.1.4 matplotlib==3.8.2 seaborn==0.13.0 # 5. Create requirements.txt pip freeze \u0026gt; requirements.txt # 6. Deactivate deactivate # .gitignore venv/ .venv/ __pycache__/ *.pyc *.pyo .env .env.local *.egg-info/ dist/ build/ .ipynb_checkpoints/ .DS_Store Q10 \u0026ndash; What Happens When You Import pandas 1. Python checks sys.modules first. If \u0026#34;pandas\u0026#34; is already in sys.modules (imported earlier in this session), Python returns the cached module object immediately -- no re-import. 2. If not cached, Python searches sys.path in order: a. The current directory / script directory b. PYTHONPATH directories (if set) c. Standard library directories d. site-packages directory (where pip installs packages) 3. Python finds pandas in site-packages, typically at: .../lib/python3.12/site-packages/pandas/__init__.py (pandas is a package, so Python enters the pandas/ directory) 4. Python executes pandas/__init__.py, which: - Imports pandas submodules (core, io, plotting, etc.) - Sets up the public API (__all__, version info, etc.) - Runs any package-level initialisation 5. The resulting module object is stored in sys.modules[\u0026#34;pandas\u0026#34;] so future imports of pandas skip steps 2-4. 6. The name \u0026#39;pandas\u0026#39; is bound in the current namespace, pointing to the module object in sys.modules. After import, you can inspect: pandas.__file__ -- path to pandas/__init__.py pandas.__version__ -- version string pandas.__path__ -- package directory pandas.__name__ -- \u0026#34;pandas\u0026#34; dir(pandas) -- all exported names Q11 \u0026ndash; Q15 Questions 11 through 15 are extended project-level exercises. Full reference solutions are available on learn.codeverra.com.\nGuidance:\nQ11: Write each submodule independently, test it, then write __init__.py last Q12: sys.prefix != sys.base_prefix is how you detect a virtual environment Q13: The key is __getattr__ on the LazyImporter class Q14: Use importlib.import_module() to dynamically import plugins by name Q15: See the Python project structure guide at https://packaging.python.org 19. What Comes Next What This Guide Covered Section Topics Modules .py files as modules, all import styles, namespaces Module discovery sys.path, search order, common import errors name The if __name__ == \u0026quot;__main__\u0026quot; guard and why it exists Your own modules Writing reusable modules with docstrings and clean APIs Packages Directory structure, init.py, building a real package Relative imports Dot notation, when to use absolute vs relative Standard library os, sys, datetime, collections, math, random, json, time, copy, typing pip Install, upgrade, uninstall, pip freeze, python -m pip requirements.txt Version specifiers, multiple files, dev dependencies Virtual environments The dependency conflict problem and why venv is essential venv Create, activate, deactivate, .gitignore rules conda Anaconda vs Miniconda, commands, environment.yml, channels Other tools virtualenv, uv (with documentation links) Choosing a tool Decision guide for which environment tool to use Next in the Curriculum With modules and packages understood, everything else becomes clearer:\nWhen you write import requests in the next module (Working with the Real World), you now know exactly what Python is doing to find and load it When you create a data science project and conda creates an environment.yml, you know what that file is for When a teammate says \u0026ldquo;just pip install from requirements.txt\u0026rdquo;, you know exactly what to do The next blog in this series is:\nWorking with the Real World \u0026ndash; requests, datetime, and JSON APIs, HTTP requests, parsing responses, working with timestamps, and building a complete project that calls a real API and processes the data.\nMade with care for Codeverra learners | codeverra.com\n","permalink":"https://blogs.codeverra.com/blog/python-10-modules-packages/","summary":"\u003ch1 id=\"modules-packages-and-the-python-ecosystem\"\u003eModules, Packages, and the Python Ecosystem\u003c/h1\u003e\n\u003ch3 id=\"understanding-imports-the-standard-library-pip-and-virtual-environments\"\u003eUnderstanding Imports, the Standard Library, pip, and Virtual Environments\u003c/h3\u003e\n\u003chr\u003e\n\u003ch2 id=\"before-we-begin--why-this-topic-is-more-important-than-it-looks\"\u003eBefore We Begin \u0026ndash; Why This Topic Is More Important Than It Looks\u003c/h2\u003e\n\u003cp\u003eHere is something that happens to almost every beginner.\u003c/p\u003e\n\u003cp\u003eYou are working through a tutorial. It says \u003ccode\u003eimport pandas as pd\u003c/code\u003e and moves on.\nYou copy it. It works. You move on too.\u003c/p\u003e\n\u003cp\u003eSix months later you are working on two projects at the same time. One needs\npandas 1.5. The other needs pandas 2.1. You install one and the other breaks.\nYou have no idea why. You cannot share your project with a teammate because\nit works on your machine and crashes on theirs.\u003c/p\u003e","tags":["python"],"title":"Modules, Packages, and the Python Ecosystem"},{"content":"Object-Oriented Programming in Python A Structured Guide from First Principles to Advanced Patterns Before We Begin - How This Guide is Organised Most OOP tutorials throw syntax at you immediately. You copy the code, it works, and you think you understand it. Two weeks later you cannot write a class from scratch.\nThis guide takes a different approach. We build understanding in three layers, each one preparing you for the next.\nPart 1 \u0026ndash; Building Intuition (Beginner) The goal of Part 1 is not to teach you syntax. It is to make OOP feel obvious and natural \u0026ndash; something you already think about in real life, just without the vocabulary.\nSection What you will learn 1 Why OOP exists \u0026ndash; the problem it actually solves 2 Classes and Objects \u0026ndash; the blueprint and the building 3 __init__ \u0026ndash; setting up a fresh object 4 Instance methods \u0026ndash; what an object can do 5 Class variables vs instance variables 6 Class methods and static methods 7 Custom exceptions using classes Part 2 \u0026ndash; The Four Pillars (Intermediate) Once you can build basic classes, Part 2 teaches the four principles that make OOP genuinely powerful. Every serious Python codebase uses all four.\nSection What you will learn 8 Encapsulation \u0026ndash; hiding internal details, _ and __ naming 9 Properties \u0026ndash; controlled access with @property 10 Inheritance \u0026ndash; building on existing classes, super() 11 Polymorphism \u0026ndash; same interface, different behaviour 12 Abstraction \u0026ndash; ABC and @abstractmethod 13 Magic (Dunder) Methods \u0026ndash; making your objects feel built-in Part 3 \u0026ndash; Advanced OOP Part 3 is for students going into library development, ML engineering, or framework work. These are the patterns used inside Pandas, Scikit-learn, Django, and FastAPI.\nSection What you will learn 14 Multiple Inheritance and the MRO 15 Mixins \u0026ndash; composable, reusable behaviour 16 __slots__ \u0026ndash; memory optimisation 17 Metaclasses \u0026ndash; classes that create classes 18 Dataclasses \u0026ndash; modern, clean class definitions 19 OOP in the Wild \u0026ndash; a peek at Pandas and Scikit-learn Practice and Solutions Section What you will find 20 Practice Questions (20 questions, Easy to Hard, grouped together) 21 Solutions (all 20, complete working code) A note on pace: If you are new to OOP, read Part 1 slowly and do the practice questions for each section before moving on. Part 2 will make much more sense once Part 1 is genuinely comfortable, not just familiar.\nTable of Contents Why OOP Exists Classes and Objects The init Method and Instance Attributes Instance Methods Class Variables vs Instance Variables Class Methods and Static Methods Custom Exceptions Encapsulation Properties Inheritance Polymorphism Abstraction Magic (Dunder) Methods Multiple Inheritance and MRO Mixins slots Metaclasses Dataclasses OOP in the Wild Practice Questions Solutions What Comes Next Part 1 \u0026ndash; Building Intuition 1. Why OOP Exists Before writing a single line of OOP code, let us understand the problem it solves.\nThe problem with procedural code at scale Imagine you are building a system to manage students at a college in Pune. With what you know so far, you might write this:\n# Procedural approach -- works fine for 1 student student_name = \u0026#34;Aarav Sharma\u0026#34; student_roll = 101 student_marks = [88, 92, 75, 90, 85] student_course = \u0026#34;Computer Science\u0026#34; def calculate_average(marks): return sum(marks) / len(marks) def get_grade(average): if average \u0026gt;= 85: return \u0026#34;Distinction\u0026#34; elif average \u0026gt;= 60: return \u0026#34;First Class\u0026#34; else: return \u0026#34;Pass\u0026#34; avg = calculate_average(student_marks) grade = get_grade(avg) print(f\u0026#34;{student_name} -- Average: {avg:.1f}, Grade: {grade}\u0026#34;) This works perfectly for one student. Now add a second student:\n# Things start getting messy fast student1_name = \u0026#34;Aarav Sharma\u0026#34; student1_roll = 101 student1_marks = [88, 92, 75, 90, 85] student1_course = \u0026#34;Computer Science\u0026#34; student2_name = \u0026#34;Priya Patel\u0026#34; student2_roll = 102 student2_marks = [95, 88, 91, 87, 93] student2_course = \u0026#34;Electronics\u0026#34; # For 500 students, this approach completely falls apart # You end up with 2000 variables and no structure What you really want is a way to say: \u0026ldquo;A student is a thing that has a name, roll number, marks, and a course. And a student can calculate their own average and grade.\u0026rdquo;\nThat is exactly what OOP gives you.\nThe OOP solution - bundle data and behaviour together class Student: def __init__(self, name, roll, marks, course): self.name = name self.roll = roll self.marks = marks self.course = course def average(self): return sum(self.marks) / len(self.marks) def grade(self): avg = self.average() if avg \u0026gt;= 85: return \u0026#34;Distinction\u0026#34; elif avg \u0026gt;= 60: return \u0026#34;First Class\u0026#34; else: return \u0026#34;Pass\u0026#34; # Create as many students as you need -- all organised, all consistent s1 = Student(\u0026#34;Aarav Sharma\u0026#34;, 101, [88, 92, 75, 90, 85], \u0026#34;Computer Science\u0026#34;) s2 = Student(\u0026#34;Priya Patel\u0026#34;, 102, [95, 88, 91, 87, 93], \u0026#34;Electronics\u0026#34;) s3 = Student(\u0026#34;Rohan Verma\u0026#34;, 103, [65, 70, 68, 72, 60], \u0026#34;Mechanical\u0026#34;) for student in [s1, s2, s3]: print(f\u0026#34;{student.name} -- Avg: {student.average():.1f}, Grade: {student.grade()}\u0026#34;) Aarav Sharma -- Avg: 86.0, Grade: Distinction Priya Patel -- Avg: 90.8, Grade: Distinction Rohan Verma -- Avg: 67.0, Grade: First Class The data (name, roll, marks) and the behaviour (calculate average, get grade) are now bundled together in one place. That bundle is called a class.\nThe three core benefits 1. Organisation: Related data and functions live together, not scattered across the file.\n2. Reusability: Define the Student class once, create a thousand Student objects.\n3. Maintainability: If the grading logic changes, you update one method in one class. Not twenty functions spread across ten files.\n2. Classes and Objects The blueprint and the building A class is a blueprint. It defines what something looks like and what it can do. An object is a specific instance created from that blueprint.\nThink of it like this:\nClass = Blueprint for a house (specifies: 3 bedrooms, 2 bathrooms, one kitchen) Object = An actual house built from that blueprint House in Bandra -- 3 bed, 2 bath, one kitchen, painted blue House in Koramangala -- 3 bed, 2 bath, one kitchen, painted white House in Salt Lake -- 3 bed, 2 bath, one kitchen, painted yellow Each house is unique (different colour, different address, different residents) but all follow the same blueprint. Defining a class # The \u0026#39;class\u0026#39; keyword defines a new blueprint # Class names use PascalCase (capital first letter of each word) class BankAccount: pass # \u0026#39;pass\u0026#39; means empty body -- valid Python, does nothing yet Creating objects (instances) # Creating an object is called \u0026#34;instantiation\u0026#34; # You call the class like a function account1 = BankAccount() account2 = BankAccount() # These are two separate objects -- completely independent print(type(account1)) # \u0026lt;class \u0026#39;__main__.BankAccount\u0026#39;\u0026gt; print(account1 is account2) # False -- different objects in memory Every object knows its class print(isinstance(account1, BankAccount)) # True print(type(account1).__name__) # BankAccount 3. The init Method and Instance Attributes Right now our BankAccount class creates empty objects. That is not very useful. We need a way to set up each object with its own data when it is created.\nThat is what __init__ is for.\nWhat init does __init__ is called automatically every time you create a new object. It is your chance to set up the object\u0026rsquo;s initial state.\nclass BankAccount: def __init__(self, account_holder, account_number, balance=0): \u0026#34;\u0026#34;\u0026#34; Called automatically when a new BankAccount is created. Parameters: account_holder (str): Name of the account holder account_number (str): Unique account number balance (float): Starting balance (default 0) \u0026#34;\u0026#34;\u0026#34; # These are instance attributes -- each object gets its own copy self.account_holder = account_holder self.account_number = account_number self.balance = balance # Creating objects now initialises their data acc1 = BankAccount(\u0026#34;Aarav Sharma\u0026#34;, \u0026#34;SBI001234\u0026#34;, 15000) acc2 = BankAccount(\u0026#34;Priya Patel\u0026#34;, \u0026#34;SBI005678\u0026#34;, 50000) acc3 = BankAccount(\u0026#34;Rohan Verma\u0026#34;, \u0026#34;SBI009012\u0026#34;) # balance defaults to 0 # Each object has its own independent data print(acc1.account_holder) # Aarav Sharma print(acc2.balance) # 50000 print(acc3.balance) # 0 What is self? self is the first parameter of every instance method, including __init__. It refers to the specific object being worked on right now.\nWhen you write:\nacc1 = BankAccount(\u0026#34;Aarav Sharma\u0026#34;, \u0026#34;SBI001234\u0026#34;, 15000) Python internally does:\nBankAccount.__init__(acc1, \u0026#34;Aarav Sharma\u0026#34;, \u0026#34;SBI001234\u0026#34;, 15000) self is acc1. The dot notation self.balance = balance means \u0026ldquo;set the balance attribute on THIS specific object\u0026rdquo;.\n# You can name it anything, but \u0026#39;self\u0026#39; is universal convention # Never deviate from this convention in real code class Demo: def __init__(this_object, value): # \u0026#39;this_object\u0026#39; works but never do this this_object.value = value Instance attributes \u0026ndash; each object has its own Instance attributes are variables that belong to a specific object. They are created by assigning to self.something inside __init__.\nclass Student: def __init__(self, name, roll, course): # Each student object gets its own name, roll, and course self.name = name self.roll = roll self.course = course self.marks = [] # starts empty for every new student self.active = True # all students start active # Two students -- completely independent attribute sets s1 = Student(\u0026#34;Sneha Iyer\u0026#34;, 201, \u0026#34;Data Science\u0026#34;) s2 = Student(\u0026#34;Karan Singh\u0026#34;, 202, \u0026#34;Cybersecurity\u0026#34;) s1.marks.append(88) # only s1\u0026#39;s marks list is affected print(s1.marks) # [88] print(s2.marks) # [] -- s2 is unaffected Adding attributes outside init You can add attributes to an object after creation, but this is generally bad practice \u0026ndash; it makes the object\u0026rsquo;s structure unpredictable.\ns1 = Student(\u0026#34;Sneha Iyer\u0026#34;, 201, \u0026#34;Data Science\u0026#34;) # Valid but not recommended -- other code won\u0026#39;t know this attribute exists s1.hometown = \u0026#34;Chennai\u0026#34; # Preferred: define all attributes in __init__, even if they start as None class Student: def __init__(self, name, roll, course): self.name = name self.roll = roll self.course = course self.hometown = None # known attribute, just not set yet 4. Instance Methods Instance methods are functions defined inside a class that operate on a specific object. They always take self as the first parameter.\nclass BankAccount: def __init__(self, account_holder, account_number, balance=0): self.account_holder = account_holder self.account_number = account_number self.balance = balance self.transactions = [] # list to track all transactions def deposit(self, amount): \u0026#34;\u0026#34;\u0026#34;Add money to the account.\u0026#34;\u0026#34;\u0026#34; if amount \u0026lt;= 0: raise ValueError(\u0026#34;Deposit amount must be positive.\u0026#34;) self.balance += amount self.transactions.append((\u0026#34;deposit\u0026#34;, amount)) print(f\u0026#34;Rs.{amount:,} deposited. New balance: Rs.{self.balance:,}\u0026#34;) def withdraw(self, amount): \u0026#34;\u0026#34;\u0026#34;Remove money from the account if sufficient balance exists.\u0026#34;\u0026#34;\u0026#34; if amount \u0026lt;= 0: raise ValueError(\u0026#34;Withdrawal amount must be positive.\u0026#34;) if amount \u0026gt; self.balance: print(f\u0026#34;Insufficient balance. Available: Rs.{self.balance:,}\u0026#34;) return False self.balance -= amount self.transactions.append((\u0026#34;withdrawal\u0026#34;, amount)) print(f\u0026#34;Rs.{amount:,} withdrawn. New balance: Rs.{self.balance:,}\u0026#34;) return True def get_statement(self): \u0026#34;\u0026#34;\u0026#34;Print a summary of all transactions.\u0026#34;\u0026#34;\u0026#34; print(f\u0026#34;\\n--- Statement for {self.account_holder} ---\u0026#34;) print(f\u0026#34;Account: {self.account_number}\u0026#34;) for txn_type, amount in self.transactions: symbol = \u0026#34;+\u0026#34; if txn_type == \u0026#34;deposit\u0026#34; else \u0026#34;-\u0026#34; print(f\u0026#34; {symbol} Rs.{amount:,} ({txn_type})\u0026#34;) print(f\u0026#34;Current Balance: Rs.{self.balance:,}\u0026#34;) print(\u0026#34;-\u0026#34; * 40) def __str__(self): \u0026#34;\u0026#34;\u0026#34;String representation -- what print(account) shows.\u0026#34;\u0026#34;\u0026#34; return (f\u0026#34;BankAccount({self.account_holder}, \u0026#34; f\u0026#34;Acc: {self.account_number}, \u0026#34; f\u0026#34;Balance: Rs.{self.balance:,})\u0026#34;) # Using the methods acc = BankAccount(\u0026#34;Meera Nair\u0026#34;, \u0026#34;HDFC00456\u0026#34;, 10000) acc.deposit(5000) acc.deposit(2500) acc.withdraw(3000) acc.withdraw(20000) # insufficient balance acc.get_statement() print(acc) # calls __str__ Rs.5,000 deposited. New balance: Rs.15,000 Rs.2,500 deposited. New balance: Rs.17,500 Rs.3,000 withdrawn. New balance: Rs.14,500 Insufficient balance. Available: Rs.14,500 --- Statement for Meera Nair --- Account: HDFC00456 + Rs.5,000 (deposit) + Rs.2,500 (deposit) - Rs.3,000 (withdrawal) Current Balance: Rs.14,500 ---------------------------------------- BankAccount(Meera Nair, Acc: HDFC00456, Balance: Rs.14,500) Methods can call other methods class Student: def __init__(self, name, marks): self.name = name self.marks = marks # list of marks across 5 subjects def average(self): \u0026#34;\u0026#34;\u0026#34;Calculate average marks.\u0026#34;\u0026#34;\u0026#34; return sum(self.marks) / len(self.marks) def grade(self): \u0026#34;\u0026#34;\u0026#34;Determine grade based on average.\u0026#34;\u0026#34;\u0026#34; avg = self.average() # calling another method using self if avg \u0026gt;= 85: return \u0026#34;Distinction\u0026#34; elif avg \u0026gt;= 60: return \u0026#34;First Class\u0026#34; elif avg \u0026gt;= 40: return \u0026#34;Pass\u0026#34; else: return \u0026#34;Fail\u0026#34; def report(self): \u0026#34;\u0026#34;\u0026#34;Print full report -- calls both average() and grade().\u0026#34;\u0026#34;\u0026#34; print(f\u0026#34;Student : {self.name}\u0026#34;) print(f\u0026#34;Marks : {self.marks}\u0026#34;) print(f\u0026#34;Average : {self.average():.1f}\u0026#34;) print(f\u0026#34;Grade : {self.grade()}\u0026#34;) s = Student(\u0026#34;Arjun Kumar\u0026#34;, [88, 92, 75, 90, 85]) s.report() 5. Class Variables vs Instance Variables This is one of the most important and most misunderstood concepts in OOP. Understanding the difference prevents some very subtle bugs.\nInstance variables \u0026ndash; belong to one object Instance variables are set with self.something and are unique to each object.\nclass Employee: def __init__(self, name, salary): self.name = name # instance variable -- unique to each Employee self.salary = salary # instance variable -- unique to each Employee e1 = Employee(\u0026#34;Vikram Nair\u0026#34;, 85000) e2 = Employee(\u0026#34;Ananya Sharma\u0026#34;, 72000) # Each employee has their own name and salary print(e1.name) # Vikram Nair print(e2.salary) # 72000 Class variables \u0026ndash; shared across all objects Class variables are defined directly inside the class body, outside any method. All objects of that class share the same class variable.\nclass Employee: # Class variables -- shared by ALL Employee objects company_name = \u0026#34;Codeverra Technologies\u0026#34; employee_count = 0 ANNUAL_BONUS_RATE = 0.10 # 10% bonus, same for all def __init__(self, name, salary, department): # Instance variables -- unique to each Employee self.name = name self.salary = salary self.department = department # Modify the class variable to track how many employees exist Employee.employee_count += 1 # use class name, not self, for clarity def annual_bonus(self): \u0026#34;\u0026#34;\u0026#34;Calculate bonus using the shared class variable.\u0026#34;\u0026#34;\u0026#34; return self.salary * Employee.ANNUAL_BONUS_RATE def company_info(self): \u0026#34;\u0026#34;\u0026#34;Access class variable through self -- also works.\u0026#34;\u0026#34;\u0026#34; return f\u0026#34;{self.name} works at {self.company_name}\u0026#34; # self.company_name works but Employee.company_name is clearer # Creating employees e1 = Employee(\u0026#34;Vikram Nair\u0026#34;, 85000, \u0026#34;Engineering\u0026#34;) e2 = Employee(\u0026#34;Ananya Sharma\u0026#34;, 72000, \u0026#34;Product\u0026#34;) e3 = Employee(\u0026#34;Suresh Kumar\u0026#34;, 68000, \u0026#34;Sales\u0026#34;) print(Employee.employee_count) # 3 -- class variable updated by all three print(e1.employee_count) # 3 -- accessible via instance too print(e2.employee_count) # 3 -- same value for everyone print(e1.company_info()) # Vikram Nair works at Codeverra Technologies print(e2.company_info()) # Ananya Sharma works at Codeverra Technologies print(f\u0026#34;Vikram\u0026#39;s bonus: Rs.{e1.annual_bonus():,}\u0026#34;) # Rs.8,500 print(f\u0026#34;Ananya\u0026#39;s bonus: Rs.{e2.annual_bonus():,}\u0026#34;) # Rs.7,200 The critical danger \u0026ndash; modifying class variables via self class Employee: company_name = \u0026#34;Codeverra Technologies\u0026#34; employee_count = 0 def __init__(self, name): self.name = name Employee.employee_count += 1 e1 = Employee(\u0026#34;Vikram\u0026#34;) e2 = Employee(\u0026#34;Ananya\u0026#34;) # DANGEROUS -- this does NOT modify the class variable # It creates a NEW instance variable on e1 that shadows the class variable e1.company_name = \u0026#34;New Company\u0026#34; print(e1.company_name) # \u0026#34;New Company\u0026#34; -- e1\u0026#39;s own instance variable print(e2.company_name) # \u0026#34;Codeverra Technologies\u0026#34; -- still the class variable print(Employee.company_name) # \u0026#34;Codeverra Technologies\u0026#34; -- class variable unchanged # CORRECT way to update a class variable (affects all objects) Employee.company_name = \u0026#34;Codeverra Pvt Ltd\u0026#34; print(e1.company_name) # still \u0026#34;New Company\u0026#34; -- e1 has its own instance variable now print(e2.company_name) # \u0026#34;Codeverra Pvt Ltd\u0026#34; -- uses the updated class variable Rule: Always modify class variables using the class name (Employee.count += 1), never via self (self.count += 1). The latter creates a shadowing instance variable.\n6. Class Methods and Static Methods Python gives you three types of methods. Knowing which to use is a mark of clean design.\nType 1 \u0026ndash; Instance methods (what you have seen so far) First parameter is self \u0026ndash; the specific object Can access and modify instance attributes via self Most methods you write will be instance methods class Student: def __init__(self, name, marks): self.name = name self.marks = marks def average(self): # instance method return sum(self.marks) / len(self.marks) Type 2 \u0026ndash; Class methods (@classmethod) Decorated with @classmethod First parameter is cls \u0026ndash; the class itself, not any specific object Can access and modify class variables Common use: alternative constructors (different ways to create an object) class Student: # Class variable institution = \u0026#34;Codeverra Institute, Pune\u0026#34; def __init__(self, name, roll, marks): self.name = name self.roll = roll self.marks = marks @classmethod def from_csv_string(cls, csv_string): \u0026#34;\u0026#34;\u0026#34; Alternative constructor -- create a Student from a CSV line. Usage: Student.from_csv_string(\u0026#34;Aarav,101,88,92,75,90,85\u0026#34;) \u0026#34;\u0026#34;\u0026#34; parts = csv_string.split(\u0026#34;,\u0026#34;) name = parts[0].strip() roll = int(parts[1].strip()) marks = [int(m.strip()) for m in parts[2:]] return cls(name, roll, marks) # cls(...) is same as Student(...) @classmethod def from_dict(cls, data_dict): \u0026#34;\u0026#34;\u0026#34;Alternative constructor -- create from a dictionary.\u0026#34;\u0026#34;\u0026#34; return cls(data_dict[\u0026#34;name\u0026#34;], data_dict[\u0026#34;roll\u0026#34;], data_dict[\u0026#34;marks\u0026#34;]) @classmethod def set_institution(cls, new_name): \u0026#34;\u0026#34;\u0026#34;Modify the class variable for all students.\u0026#34;\u0026#34;\u0026#34; cls.institution = new_name def __str__(self): avg = sum(self.marks) / len(self.marks) return f\u0026#34;Student({self.name}, Roll: {self.roll}, Avg: {avg:.1f})\u0026#34; # Using regular constructor s1 = Student(\u0026#34;Aarav Sharma\u0026#34;, 101, [88, 92, 75, 90, 85]) # Using class method constructors -- much cleaner for real data pipelines s2 = Student.from_csv_string(\u0026#34;Priya Patel, 102, 95, 88, 91, 87, 93\u0026#34;) s3 = Student.from_dict({\u0026#34;name\u0026#34;: \u0026#34;Rohan Verma\u0026#34;, \u0026#34;roll\u0026#34;: 103, \u0026#34;marks\u0026#34;: [72, 68, 75, 70, 65]}) print(s1) # Student(Aarav Sharma, Roll: 101, Avg: 86.0) print(s2) # Student(Priya Patel, Roll: 102, Avg: 90.8) print(s3) # Student(Rohan Verma, Roll: 103, Avg: 70.0) print(Student.institution) # Codeverra Institute, Pune Student.set_institution(\u0026#34;Codeverra Academy, Mumbai\u0026#34;) print(Student.institution) # Codeverra Academy, Mumbai print(s1.institution) # Codeverra Academy, Mumbai (class var updated) Type 3 \u0026ndash; Static methods (@staticmethod) Decorated with @staticmethod No self or cls parameter Cannot access instance or class attributes \u0026ndash; completely independent Used for utility functions that logically belong to the class but do not need any object or class data class TemperatureConverter: \u0026#34;\u0026#34;\u0026#34;A utility class for temperature conversions.\u0026#34;\u0026#34;\u0026#34; @staticmethod def celsius_to_fahrenheit(c): \u0026#34;\u0026#34;\u0026#34;Convert Celsius to Fahrenheit.\u0026#34;\u0026#34;\u0026#34; return (c * 9/5) + 32 @staticmethod def fahrenheit_to_celsius(f): \u0026#34;\u0026#34;\u0026#34;Convert Fahrenheit to Celsius.\u0026#34;\u0026#34;\u0026#34; return (f - 32) * 5/9 @staticmethod def celsius_to_kelvin(c): \u0026#34;\u0026#34;\u0026#34;Convert Celsius to Kelvin.\u0026#34;\u0026#34;\u0026#34; return c + 273.15 # Static methods can be called on the class directly -- no object needed print(TemperatureConverter.celsius_to_fahrenheit(37)) # 98.6 (body temp) print(TemperatureConverter.celsius_to_fahrenheit(0)) # 32.0 print(TemperatureConverter.celsius_to_kelvin(100)) # 373.15 # Also callable via an instance, though the class version is more common tc = TemperatureConverter() print(tc.fahrenheit_to_celsius(98.6)) # 37.0 class Student: institution = \u0026#34;Codeverra Institute\u0026#34; def __init__(self, name, marks): self.name = name self.marks = marks def average(self): return sum(self.marks) / len(self.marks) @staticmethod def is_valid_mark(mark): \u0026#34;\u0026#34;\u0026#34; Check if a mark is in the valid range (0-100). Does not need any student data -- pure utility. \u0026#34;\u0026#34;\u0026#34; return 0 \u0026lt;= mark \u0026lt;= 100 @staticmethod def letter_grade(average): \u0026#34;\u0026#34;\u0026#34; Convert a numeric average to a letter grade. Useful as a standalone utility -- does not need self or cls. \u0026#34;\u0026#34;\u0026#34; if average \u0026gt;= 85: return \u0026#34;A\u0026#34; if average \u0026gt;= 70: return \u0026#34;B\u0026#34; if average \u0026gt;= 55: return \u0026#34;C\u0026#34; if average \u0026gt;= 40: return \u0026#34;D\u0026#34; return \u0026#34;F\u0026#34; # Validate marks before creating a student raw_marks = [88, 105, 72, -5, 90] # two invalid marks valid_marks = [m for m in raw_marks if Student.is_valid_mark(m)] print(valid_marks) # [88, 72, 90] s = Student(\u0026#34;Divya Menon\u0026#34;, valid_marks) print(Student.letter_grade(s.average())) # B Summary \u0026ndash; which type to use? Instance method --\u0026gt; needs to access or modify object data (use self) Class method --\u0026gt; needs to access or modify class data, OR is an alternative constructor (use cls) Static method --\u0026gt; is a utility function related to the class but does not need any data from it 7. Custom Exceptions Before moving to Part 2, here is one very practical early use of classes: building your own exception types.\nWhy custom exceptions? Python has built-in exceptions: ValueError, TypeError, FileNotFoundError. But when you build an application, you want errors that describe YOUR domain.\nA banking app should raise InsufficientFundsError, not a generic ValueError. A course platform should raise EnrollmentLimitError, not a generic Exception. Custom exceptions make your code self-documenting and easier to handle precisely.\nCreating a custom exception All you need to do is inherit from Exception (or any built-in exception class).\n# Custom exceptions are just classes that inherit from Exception class InsufficientFundsError(Exception): \u0026#34;\u0026#34;\u0026#34;Raised when a withdrawal exceeds the available balance.\u0026#34;\u0026#34;\u0026#34; pass class InvalidAmountError(Exception): \u0026#34;\u0026#34;\u0026#34;Raised when a transaction amount is zero or negative.\u0026#34;\u0026#34;\u0026#34; pass class AccountFrozenError(Exception): \u0026#34;\u0026#34;\u0026#34;Raised when a transaction is attempted on a frozen account.\u0026#34;\u0026#34;\u0026#34; pass Adding useful information to custom exceptions class InsufficientFundsError(Exception): \u0026#34;\u0026#34;\u0026#34;Raised when a withdrawal exceeds the available balance.\u0026#34;\u0026#34;\u0026#34; def __init__(self, requested, available): self.requested = requested self.available = available # Always call super().__init__() with a helpful message super().__init__( f\u0026#34;Cannot withdraw Rs.{requested:,}. \u0026#34; f\u0026#34;Available balance: Rs.{available:,}. \u0026#34; f\u0026#34;Shortfall: Rs.{requested - available:,}.\u0026#34; ) class InvalidAmountError(Exception): \u0026#34;\u0026#34;\u0026#34;Raised when a transaction amount is zero or negative.\u0026#34;\u0026#34;\u0026#34; def __init__(self, amount, operation): self.amount = amount self.operation = operation super().__init__( f\u0026#34;Invalid {operation} amount: Rs.{amount}. \u0026#34; f\u0026#34;Amount must be greater than zero.\u0026#34; ) class AccountFrozenError(Exception): \u0026#34;\u0026#34;\u0026#34;Raised when a transaction is attempted on a frozen account.\u0026#34;\u0026#34;\u0026#34; def __init__(self, account_number): self.account_number = account_number super().__init__( f\u0026#34;Account {account_number} is frozen. \u0026#34; f\u0026#34;Contact your branch to resolve this.\u0026#34; ) Using custom exceptions in a class class BankAccount: def __init__(self, holder, account_number, balance=0): self.holder = holder self.account_number = account_number self.balance = balance self.is_frozen = False def freeze(self): self.is_frozen = True print(f\u0026#34;Account {self.account_number} has been frozen.\u0026#34;) def deposit(self, amount): if self.is_frozen: raise AccountFrozenError(self.account_number) if amount \u0026lt;= 0: raise InvalidAmountError(amount, \u0026#34;deposit\u0026#34;) self.balance += amount return self.balance def withdraw(self, amount): if self.is_frozen: raise AccountFrozenError(self.account_number) if amount \u0026lt;= 0: raise InvalidAmountError(amount, \u0026#34;withdrawal\u0026#34;) if amount \u0026gt; self.balance: raise InsufficientFundsError(amount, self.balance) self.balance -= amount return self.balance # Using the account with proper exception handling acc = BankAccount(\u0026#34;Aarav Sharma\u0026#34;, \u0026#34;SBI001234\u0026#34;, 10000) # Test InsufficientFundsError try: acc.withdraw(25000) except InsufficientFundsError as e: print(f\u0026#34;Transaction failed: {e}\u0026#34;) print(f\u0026#34;Requested: Rs.{e.requested:,}\u0026#34;) print(f\u0026#34;Available: Rs.{e.available:,}\u0026#34;) # Test AccountFrozenError acc.freeze() try: acc.deposit(5000) except AccountFrozenError as e: print(f\u0026#34;Transaction failed: {e}\u0026#34;) # Test InvalidAmountError acc2 = BankAccount(\u0026#34;Priya Patel\u0026#34;, \u0026#34;HDFC005678\u0026#34;, 50000) try: acc2.deposit(-1000) except InvalidAmountError as e: print(f\u0026#34;Transaction failed: {e}\u0026#34;) Transaction failed: Cannot withdraw Rs.25,000. Available balance: Rs.10,000. Shortfall: Rs.15,000. Requested: Rs.25,000 Available: Rs.10,000 Account SBI001234 has been frozen. Transaction failed: Account SBI001234 is frozen. Contact your branch to resolve this. Transaction failed: Invalid deposit amount: Rs.-1000. Amount must be greater than zero. Exception hierarchy \u0026ndash; building related exceptions # Create a base exception for your entire application class CodevenaError(Exception): \u0026#34;\u0026#34;\u0026#34;Base exception for all Codeverra application errors.\u0026#34;\u0026#34;\u0026#34; pass # All domain-specific exceptions inherit from your base class StudentError(CodevenaError): \u0026#34;\u0026#34;\u0026#34;Base for all student-related errors.\u0026#34;\u0026#34;\u0026#34; pass class EnrollmentError(StudentError): \u0026#34;\u0026#34;\u0026#34;Raised when enrollment fails.\u0026#34;\u0026#34;\u0026#34; pass class EnrollmentLimitError(EnrollmentError): \u0026#34;\u0026#34;\u0026#34;Raised when a course is full.\u0026#34;\u0026#34;\u0026#34; def __init__(self, course_name, limit): super().__init__( f\u0026#34;Course \u0026#39;{course_name}\u0026#39; is full. Maximum enrollment: {limit}.\u0026#34; ) class PrerequisiteError(EnrollmentError): \u0026#34;\u0026#34;\u0026#34;Raised when prerequisite courses are not completed.\u0026#34;\u0026#34;\u0026#34; def __init__(self, course_name, missing): super().__init__( f\u0026#34;Cannot enroll in \u0026#39;{course_name}\u0026#39;. \u0026#34; f\u0026#34;Missing prerequisites: {\u0026#39;, \u0026#39;.join(missing)}.\u0026#34; ) # Now you can catch at different levels of specificity try: raise PrerequisiteError(\u0026#34;Machine Learning\u0026#34;, [\u0026#34;Python Basics\u0026#34;, \u0026#34;Statistics\u0026#34;]) except EnrollmentError as e: print(f\u0026#34;Enrollment failed: {e}\u0026#34;) # catches PrerequisiteError too except StudentError as e: print(f\u0026#34;Student error: {e}\u0026#34;) except CodevenaError as e: print(f\u0026#34;Application error: {e}\u0026#34;) Part 2 \u0026ndash; The Four Pillars 8. Encapsulation Encapsulation means bundling data and the methods that operate on it together, and controlling what the outside world can access.\nThink of a TV remote. You press buttons and the TV changes channels. You do not need to know about the infrared signals, the circuit board, or how the microcontroller encodes the command. The internals are hidden. You get a clean, simple interface.\nThat is encapsulation.\nWhy it matters Without encapsulation, any part of your code can reach into any object and mess with its data directly:\n# Without encapsulation -- dangerous acc = BankAccount(\u0026#34;Aarav\u0026#34;, \u0026#34;SBI001\u0026#34;, 10000) acc.balance = -999999 # nothing stops this -- corrupted state acc.balance = \u0026#34;hello\u0026#34; # also valid Python -- completely wrong type With encapsulation, you control how data is accessed and modified.\nPython\u0026rsquo;s naming conventions for access control Python does not have true private variables like Java or C++. Instead, it uses naming conventions that signal intent:\nNo underscore: public -- anyone can access and modify Single underscore: _name -- \u0026#34;internal use\u0026#34; -- technically accessible but convention says \u0026#34;do not touch from outside\u0026#34; Double underscore: __name -- name-mangled -- harder to access from outside, genuine discourage from external access class CricketPlayer: def __init__(self, name, team, runs, is_contracted): self.name = name # public -- access freely self.team = team # public self._runs = runs # protected -- internal use self.__is_contracted = is_contracted # private -- name-mangled def get_runs(self): \u0026#34;\u0026#34;\u0026#34;Public method to access protected attribute.\u0026#34;\u0026#34;\u0026#34; return self._runs def is_available(self): \u0026#34;\u0026#34;\u0026#34;Public method to check private attribute.\u0026#34;\u0026#34;\u0026#34; return self.__is_contracted player = CricketPlayer(\u0026#34;Rohit Sharma\u0026#34;, \u0026#34;Mumbai Indians\u0026#34;, 9825, True) # Public -- fine print(player.name) # Rohit Sharma print(player.team) # Mumbai Indians # Protected -- accessible but signals \u0026#34;please do not\u0026#34; print(player._runs) # 9825 -- works but is bad practice # Private -- name-mangled, harder to reach # print(player.__is_contracted) # AttributeError! # Python mangles the name to _ClassName__attribute print(player._CricketPlayer__is_contracted) # True -- but never do this # Use the public methods instead print(player.get_runs()) # 9825 print(player.is_available()) # True Name mangling in detail class BankAccount: def __init__(self, holder, balance): self.holder = holder # public self._log = [] # protected -- internal transaction log self.__balance = balance # private -- name mangled to _BankAccount__balance def deposit(self, amount): if amount \u0026gt; 0: self.__balance += amount # accessed freely inside the class self._log.append(f\u0026#34;+{amount}\u0026#34;) else: raise ValueError(\u0026#34;Amount must be positive\u0026#34;) def get_balance(self): \u0026#34;\u0026#34;\u0026#34;Public interface to read the private balance.\u0026#34;\u0026#34;\u0026#34; return self.__balance acc = BankAccount(\u0026#34;Priya Patel\u0026#34;, 50000) acc.deposit(10000) print(acc.get_balance()) # 60000 # acc.__balance --\u0026gt; AttributeError # acc._BankAccount__balance --\u0026gt; 60000 (works but is a violation of intent) 9. Properties The @property decorator is the Pythonic way to implement encapsulation. It lets you control attribute access with getter, setter, and deleter methods while keeping the clean object.attribute syntax from the caller\u0026rsquo;s perspective.\nThe problem @property solves class Circle: def __init__(self, radius): self.radius = radius # public -- anyone can set radius = -5 c = Circle(7) c.radius = -10 # nothing stops this -- physically meaningless You want to add validation but you do not want to break every line that currently reads c.radius or writes c.radius = 5.\n@property lets you add that validation without changing the interface.\nimport math class Circle: def __init__(self, radius): # Note: calls the setter below (not direct assignment) self.radius = radius @property def radius(self): \u0026#34;\u0026#34;\u0026#34;Getter -- called when you READ circle.radius\u0026#34;\u0026#34;\u0026#34; return self.__radius @radius.setter def radius(self, value): \u0026#34;\u0026#34;\u0026#34;Setter -- called when you WRITE circle.radius = value\u0026#34;\u0026#34;\u0026#34; if value \u0026lt;= 0: raise ValueError(f\u0026#34;Radius must be positive. Got: {value}\u0026#34;) self.__radius = value # store in private attribute @property def diameter(self): \u0026#34;\u0026#34;\u0026#34;Computed property -- no setter because it is derived.\u0026#34;\u0026#34;\u0026#34; return self.__radius * 2 @property def area(self): \u0026#34;\u0026#34;\u0026#34;Computed property.\u0026#34;\u0026#34;\u0026#34; return round(math.pi * self.__radius ** 2, 4) @property def circumference(self): \u0026#34;\u0026#34;\u0026#34;Computed property.\u0026#34;\u0026#34;\u0026#34; return round(2 * math.pi * self.__radius, 4) c = Circle(7) print(c.radius) # 7 -- calls the getter print(c.diameter) # 14 -- computed property print(c.area) # 153.938 print(c.circumference) # 43.9823 c.radius = 10 # calls the setter -- validated print(c.area) # 314.1593 c.radius = -5 # ValueError: Radius must be positive. Got: -5 # diameter has no setter -- it is read-only c.diameter = 20 # AttributeError: can\u0026#39;t set attribute A more complete property example class Employee: # Class variable: valid department names VALID_DEPARTMENTS = {\u0026#34;Engineering\u0026#34;, \u0026#34;Product\u0026#34;, \u0026#34;Sales\u0026#34;, \u0026#34;HR\u0026#34;, \u0026#34;Marketing\u0026#34;, \u0026#34;Finance\u0026#34;} def __init__(self, name, salary, department): # These all go through their setters for validation self.name = name self.salary = salary self.department = department @property def name(self): return self.__name @name.setter def name(self, value): if not isinstance(value, str) or not value.strip(): raise ValueError(\u0026#34;Name must be a non-empty string.\u0026#34;) self.__name = value.strip().title() # clean and normalise @property def salary(self): return self.__salary @salary.setter def salary(self, value): if not isinstance(value, (int, float)): raise TypeError(\u0026#34;Salary must be a number.\u0026#34;) if value \u0026lt; 15000: raise ValueError(f\u0026#34;Salary cannot be below Rs.15,000. Got: Rs.{value:,}\u0026#34;) self.__salary = float(value) @salary.deleter def salary(self): \u0026#34;\u0026#34;\u0026#34;Called when you do: del employee.salary\u0026#34;\u0026#34;\u0026#34; print(f\u0026#34;Salary record for {self.__name} has been cleared.\u0026#34;) self.__salary = None @property def department(self): return self.__department @department.setter def department(self, value): if value not in Employee.VALID_DEPARTMENTS: raise ValueError( f\u0026#34;\u0026#39;{value}\u0026#39; is not a valid department. \u0026#34; f\u0026#34;Choose from: {sorted(Employee.VALID_DEPARTMENTS)}\u0026#34; ) self.__department = value @property def annual_salary(self): \u0026#34;\u0026#34;\u0026#34;Read-only computed property.\u0026#34;\u0026#34;\u0026#34; if self.__salary is None: return None return self.__salary * 12 def __str__(self): return (f\u0026#34;Employee({self.__name}, \u0026#34; f\u0026#34;Dept: {self.__department}, \u0026#34; f\u0026#34;Salary: Rs.{self.__salary:,.0f}/month)\u0026#34;) # All validation happens automatically through setters emp = Employee(\u0026#34;vikram nair\u0026#34;, 85000, \u0026#34;Engineering\u0026#34;) print(emp) # Employee(Vikram Nair, Dept: Engineering, Salary: Rs.85,000/month) print(emp.annual_salary) # 1020000.0 emp.salary = 95000 # valid -- goes through setter print(emp.salary) # 95000.0 try: emp.salary = 5000 # invalid except ValueError as e: print(e) # Salary cannot be below Rs.15,000. Got: Rs.5,000 try: emp.department = \u0026#34;Accounts\u0026#34; # invalid except ValueError as e: print(e) del emp.salary # calls the deleter print(emp.annual_salary) # None 10. Inheritance Inheritance lets one class acquire the attributes and methods of another class. It models the \u0026ldquo;is-a\u0026rdquo; relationship: a SavingsAccount IS-A BankAccount. A Dog IS-AN Animal.\nThis eliminates duplication. Define common behaviour once in a parent class. Let child classes inherit it and add only what is unique to them.\nBasic inheritance class Animal: \u0026#34;\u0026#34;\u0026#34;Parent class -- defines common behaviour for all animals.\u0026#34;\u0026#34;\u0026#34; def __init__(self, name, species, age): self.name = name self.species = species self.age = age def breathe(self): print(f\u0026#34;{self.name} is breathing.\u0026#34;) def eat(self, food): print(f\u0026#34;{self.name} is eating {food}.\u0026#34;) def describe(self): print(f\u0026#34;{self.name} is a {self.age}-year-old {self.species}.\u0026#34;) class Dog(Animal): \u0026#34;\u0026#34;\u0026#34;Child class -- inherits everything from Animal, adds dog-specific behaviour.\u0026#34;\u0026#34;\u0026#34; def __init__(self, name, age, breed): # Call the parent __init__ to set up the inherited attributes super().__init__(name, species=\u0026#34;Dog\u0026#34;, age=age) self.breed = breed # dog-specific attribute def bark(self): print(f\u0026#34;{self.name} says: Woof!\u0026#34;) def fetch(self, item): print(f\u0026#34;{self.name} fetched the {item}!\u0026#34;) class Cat(Animal): \u0026#34;\u0026#34;\u0026#34;Another child class of Animal.\u0026#34;\u0026#34;\u0026#34; def __init__(self, name, age, is_indoor): super().__init__(name, species=\u0026#34;Cat\u0026#34;, age=age) self.is_indoor = is_indoor def meow(self): print(f\u0026#34;{self.name} says: Meow!\u0026#34;) def purr(self): print(f\u0026#34;{self.name} is purring...\u0026#34;) # Creating objects tommy = Dog(\u0026#34;Tommy\u0026#34;, 3, \u0026#34;Labrador\u0026#34;) whiskers = Cat(\u0026#34;Whiskers\u0026#34;, 5, is_indoor=True) # Inherited methods -- available without redefining tommy.describe() # Tommy is a 3-year-old Dog. tommy.breathe() # Tommy is breathing. tommy.eat(\u0026#34;chicken\u0026#34;) # Tommy is eating chicken. # Dog-specific methods tommy.bark() # Tommy says: Woof! tommy.fetch(\u0026#34;ball\u0026#34;) # Tommy fetched the ball! # Cat-specific methods whiskers.meow() whiskers.purr() whiskers.describe() # Whiskers is a 5-year-old Cat. # isinstance() checks inheritance hierarchy print(isinstance(tommy, Dog)) # True print(isinstance(tommy, Animal)) # True -- Dog IS-AN Animal print(isinstance(tommy, Cat)) # False super() \u0026ndash; calling the parent class super() gives you access to the parent class. Most commonly used in __init__ to initialise inherited attributes.\nclass Vehicle: \u0026#34;\u0026#34;\u0026#34;Base class for all vehicles.\u0026#34;\u0026#34;\u0026#34; def __init__(self, make, model, year, fuel_type): self.make = make self.model = model self.year = year self.fuel_type = fuel_type self.speed = 0 # all vehicles start at rest def accelerate(self, amount): self.speed += amount print(f\u0026#34;{self.make} {self.model} accelerating to {self.speed} km/h\u0026#34;) def brake(self, amount): self.speed = max(0, self.speed - amount) print(f\u0026#34;{self.make} {self.model} slowing to {self.speed} km/h\u0026#34;) def __str__(self): return f\u0026#34;{self.year} {self.make} {self.model} ({self.fuel_type})\u0026#34; class ElectricVehicle(Vehicle): \u0026#34;\u0026#34;\u0026#34;Electric vehicles -- inherits from Vehicle, adds battery management.\u0026#34;\u0026#34;\u0026#34; def __init__(self, make, model, year, battery_kwh, range_km): # Call Vehicle.__init__ for the shared attributes super().__init__(make, model, year, fuel_type=\u0026#34;Electric\u0026#34;) # Add EV-specific attributes self.battery_kwh = battery_kwh self.range_km = range_km self.charge_percent = 100 # starts fully charged def charge(self, percent): self.charge_percent = min(100, self.charge_percent + percent) print(f\u0026#34;{self.make} {self.model} charged to {self.charge_percent}%\u0026#34;) def estimated_range(self): return self.range_km * (self.charge_percent / 100) def __str__(self): # Extend the parent\u0026#39;s __str__ with EV-specific info base = super().__str__() return f\u0026#34;{base} | Battery: {self.battery_kwh}kWh | Range: {self.range_km}km\u0026#34; class Tata_Nexon_EV(ElectricVehicle): \u0026#34;\u0026#34;\u0026#34;Specific model -- inherits from ElectricVehicle.\u0026#34;\u0026#34;\u0026#34; # Tata Nexon EV Long Range specs DEFAULT_BATTERY = 40.5 # kWh DEFAULT_RANGE = 465 # km (claimed) def __init__(self, year, color): super().__init__( make=\u0026#34;Tata\u0026#34;, model=\u0026#34;Nexon EV\u0026#34;, year=year, battery_kwh=self.DEFAULT_BATTERY, range_km=self.DEFAULT_RANGE ) self.color = color def __str__(self): return f\u0026#34;{super().__str__()} | Color: {self.color}\u0026#34; # Using the hierarchy nexon = Tata_Nexon_EV(2024, \u0026#34;Pristine White\u0026#34;) print(nexon) nexon.accelerate(60) nexon.accelerate(40) nexon.brake(30) print(f\u0026#34;Estimated remaining range: {nexon.estimated_range():.0f}km\u0026#34;) # All three are True -- full hierarchy print(isinstance(nexon, Tata_Nexon_EV)) # True print(isinstance(nexon, ElectricVehicle)) # True print(isinstance(nexon, Vehicle)) # True Method overriding A child class can redefine a method inherited from the parent. When you call that method on a child object, Python uses the child\u0026rsquo;s version.\nclass Shape: \u0026#34;\u0026#34;\u0026#34;Base class for geometric shapes.\u0026#34;\u0026#34;\u0026#34; def __init__(self, colour=\u0026#34;white\u0026#34;): self.colour = colour def area(self): \u0026#34;\u0026#34;\u0026#34;To be overridden by each specific shape.\u0026#34;\u0026#34;\u0026#34; raise NotImplementedError(\u0026#34;Subclasses must implement area()\u0026#34;) def describe(self): print(f\u0026#34;I am a {self.__class__.__name__} with area {self.area():.2f} sq units.\u0026#34;) class Rectangle(Shape): def __init__(self, length, width, colour=\u0026#34;white\u0026#34;): super().__init__(colour) self.length = length self.width = width def area(self): \u0026#34;\u0026#34;\u0026#34;Override parent\u0026#39;s area() with rectangle-specific formula.\u0026#34;\u0026#34;\u0026#34; return self.length * self.width def perimeter(self): return 2 * (self.length + self.width) class Circle(Shape): def __init__(self, radius, colour=\u0026#34;white\u0026#34;): import math super().__init__(colour) self.radius = radius self._pi = math.pi def area(self): \u0026#34;\u0026#34;\u0026#34;Override parent\u0026#39;s area() with circle-specific formula.\u0026#34;\u0026#34;\u0026#34; return self._pi * self.radius ** 2 class Triangle(Shape): def __init__(self, base, height, colour=\u0026#34;white\u0026#34;): super().__init__(colour) self.base = base self.height = height def area(self): \u0026#34;\u0026#34;\u0026#34;Override parent\u0026#39;s area() with triangle-specific formula.\u0026#34;\u0026#34;\u0026#34; return 0.5 * self.base * self.height # Each shape uses its own area() method r = Rectangle(8, 5) c = Circle(7) t = Triangle(6, 4) r.describe() # I am a Rectangle with area 40.00 sq units. c.describe() # I am a Circle with area 153.94 sq units. t.describe() # I am a Triangle with area 12.00 sq units. 11. Polymorphism Polymorphism means one interface, many implementations. The same method name works differently depending on which object you call it on.\nYou already saw this with area() above. The word \u0026ldquo;polymorphism\u0026rdquo; just gives this pattern a name and makes it explicit that you are designing for it.\nPolymorphism through method overriding class PaymentGateway: \u0026#34;\u0026#34;\u0026#34;Base class for all payment methods.\u0026#34;\u0026#34;\u0026#34; def __init__(self, name): self.name = name def process_payment(self, amount): \u0026#34;\u0026#34;\u0026#34;Each subclass implements this differently.\u0026#34;\u0026#34;\u0026#34; raise NotImplementedError def refund(self, amount, reason): \u0026#34;\u0026#34;\u0026#34;Each subclass implements this differently.\u0026#34;\u0026#34;\u0026#34; raise NotImplementedError class UPIPayment(PaymentGateway): def __init__(self, upi_id): super().__init__(\u0026#34;UPI\u0026#34;) self.upi_id = upi_id def process_payment(self, amount): print(f\u0026#34;Processing Rs.{amount:,} via UPI ({self.upi_id})\u0026#34;) print(f\u0026#34; Sending payment request to {self.upi_id}...\u0026#34;) print(f\u0026#34; Payment of Rs.{amount:,} successful via UPI.\u0026#34;) return True def refund(self, amount, reason): print(f\u0026#34;Initiating UPI refund of Rs.{amount:,} to {self.upi_id}\u0026#34;) print(f\u0026#34; Reason: {reason}\u0026#34;) print(f\u0026#34; Refund will reflect in 1-3 business days.\u0026#34;) class NetBankingPayment(PaymentGateway): def __init__(self, bank_name, account_last4): super().__init__(\u0026#34;Net Banking\u0026#34;) self.bank_name = bank_name self.account_last4 = account_last4 def process_payment(self, amount): print(f\u0026#34;Processing Rs.{amount:,} via {self.bank_name} Net Banking\u0026#34;) print(f\u0026#34; Account ending: ****{self.account_last4}\u0026#34;) print(f\u0026#34; Payment of Rs.{amount:,} successful via Net Banking.\u0026#34;) return True def refund(self, amount, reason): print(f\u0026#34;Initiating Net Banking refund of Rs.{amount:,}\u0026#34;) print(f\u0026#34; To: {self.bank_name} account ****{self.account_last4}\u0026#34;) print(f\u0026#34; Reason: {reason}\u0026#34;) print(f\u0026#34; Refund will reflect in 5-7 business days.\u0026#34;) class WalletPayment(PaymentGateway): def __init__(self, wallet_name, wallet_balance): super().__init__(wallet_name) self.wallet_balance = wallet_balance def process_payment(self, amount): if amount \u0026gt; self.wallet_balance: print(f\u0026#34;Insufficient wallet balance. Available: Rs.{self.wallet_balance:,}\u0026#34;) return False self.wallet_balance -= amount print(f\u0026#34;Processing Rs.{amount:,} via {self.name} Wallet\u0026#34;) print(f\u0026#34; Remaining wallet balance: Rs.{self.wallet_balance:,}\u0026#34;) return True def refund(self, amount, reason): self.wallet_balance += amount print(f\u0026#34;Rs.{amount:,} refunded to {self.name} Wallet\u0026#34;) print(f\u0026#34; New balance: Rs.{self.wallet_balance:,}\u0026#34;) # THE POWER OF POLYMORPHISM: # This function works with ANY payment gateway -- present or future def checkout(order_total, payment_method): \u0026#34;\u0026#34;\u0026#34; Process a payment -- does not care which gateway is used. All gateways have process_payment() -- polymorphism handles the rest. \u0026#34;\u0026#34;\u0026#34; print(f\u0026#34;\\n--- Checkout: Rs.{order_total:,} ---\u0026#34;) success = payment_method.process_payment(order_total) if success: print(\u0026#34;Order confirmed!\u0026#34;) else: print(\u0026#34;Payment failed. Please try another method.\u0026#34;) # Create different payment methods upi = UPIPayment(\u0026#34;aarav@okaxis\u0026#34;) netbank = NetBankingPayment(\u0026#34;HDFC Bank\u0026#34;, \u0026#34;4521\u0026#34;) paytm = WalletPayment(\u0026#34;Paytm\u0026#34;, 2000) # Same function call -- completely different behaviour checkout(1299, upi) checkout(5499, netbank) checkout(1299, paytm) checkout(3500, paytm) # insufficient balance # Iterate over a mixed list of payment objects print(\u0026#34;\\n--- All payment methods ---\u0026#34;) all_methods = [upi, netbank, paytm] for method in all_methods: # process_payment is called on each -- Python picks the right version method.process_payment(100) Duck typing \u0026ndash; Python\u0026rsquo;s informal polymorphism Python does not require inheritance for polymorphism. If an object has the method you need, it works. Period. \u0026ldquo;If it walks like a duck and quacks like a duck, it is a duck.\u0026rdquo;\nclass Printer: def print_document(self, text): print(f\u0026#34;[Printer]: {text}\u0026#34;) class EmailSender: def print_document(self, text): print(f\u0026#34;[Email]: Sending \u0026#39;{text}\u0026#39; to inbox...\u0026#34;) class SMSService: def print_document(self, text): print(f\u0026#34;[SMS]: Sending \u0026#39;{text}\u0026#39; as SMS...\u0026#34;) # None of these inherit from a common base class # But they all have print_document() -- duck typing makes them interchangeable def distribute_report(report_text, *output_channels): for channel in output_channels: channel.print_document(report_text) printer = Printer() email = EmailSender() sms = SMSService() distribute_report(\u0026#34;Q3 Sales Report\u0026#34;, printer, email, sms) 12. Abstraction Abstraction means hiding the complexity and showing only what is necessary.\nYou have already seen informal abstraction \u0026ndash; raise NotImplementedError in a base class method. Python also has a formal mechanism: Abstract Base Classes (ABCs).\nAn abstract class:\nCannot be instantiated directly Forces subclasses to implement specified methods Acts as a contract: \u0026ldquo;anything that calls itself a PaymentGateway MUST have process_payment()\u0026rdquo; from abc import ABC, abstractmethod class PaymentGateway(ABC): \u0026#34;\u0026#34;\u0026#34; Abstract base class for payment gateways. Any concrete payment gateway MUST implement: - process_payment(amount) - refund(amount, reason) - get_transaction_id() \u0026#34;\u0026#34;\u0026#34; def __init__(self, gateway_name): self.gateway_name = gateway_name self._transaction_history = [] @abstractmethod def process_payment(self, amount): \u0026#34;\u0026#34;\u0026#34;Process a payment of the given amount.\u0026#34;\u0026#34;\u0026#34; pass # subclasses must implement this @abstractmethod def refund(self, amount, reason): \u0026#34;\u0026#34;\u0026#34;Refund the given amount.\u0026#34;\u0026#34;\u0026#34; pass # subclasses must implement this @abstractmethod def get_transaction_id(self): \u0026#34;\u0026#34;\u0026#34;Return the last transaction ID.\u0026#34;\u0026#34;\u0026#34; pass # Concrete method -- shared by all gateways, no need to override def get_history(self): \u0026#34;\u0026#34;\u0026#34;Return list of all transactions -- already implemented.\u0026#34;\u0026#34;\u0026#34; return self._transaction_history def add_to_history(self, record): self._transaction_history.append(record) # Trying to instantiate the abstract class directly raises TypeError try: gateway = PaymentGateway(\u0026#34;Test\u0026#34;) except TypeError as e: print(f\u0026#34;Cannot instantiate: {e}\u0026#34;) # TypeError: Can\u0026#39;t instantiate abstract class PaymentGateway # with abstract methods get_transaction_id, process_payment, refund class RazorpayGateway(PaymentGateway): \u0026#34;\u0026#34;\u0026#34;Concrete implementation using Razorpay.\u0026#34;\u0026#34;\u0026#34; def __init__(self, api_key): super().__init__(\u0026#34;Razorpay\u0026#34;) self.api_key = api_key self._last_txn_id = None self._txn_counter = 1000 def process_payment(self, amount): self._txn_counter += 1 self._last_txn_id = f\u0026#34;pay_RZP{self._txn_counter}\u0026#34; record = {\u0026#34;txn_id\u0026#34;: self._last_txn_id, \u0026#34;type\u0026#34;: \u0026#34;payment\u0026#34;, \u0026#34;amount\u0026#34;: amount} self.add_to_history(record) print(f\u0026#34;Razorpay: Payment of Rs.{amount:,} processed. TXN: {self._last_txn_id}\u0026#34;) return True def refund(self, amount, reason): self._txn_counter += 1 refund_id = f\u0026#34;rfnd_RZP{self._txn_counter}\u0026#34; record = {\u0026#34;txn_id\u0026#34;: refund_id, \u0026#34;type\u0026#34;: \u0026#34;refund\u0026#34;, \u0026#34;amount\u0026#34;: amount} self.add_to_history(record) print(f\u0026#34;Razorpay: Refund of Rs.{amount:,} initiated. Reason: {reason}\u0026#34;) def get_transaction_id(self): return self._last_txn_id class PhonePeGateway(PaymentGateway): \u0026#34;\u0026#34;\u0026#34;Concrete implementation using PhonePe.\u0026#34;\u0026#34;\u0026#34; def __init__(self, merchant_id): super().__init__(\u0026#34;PhonePe\u0026#34;) self.merchant_id = merchant_id self._last_txn_id = None self._txn_counter = 2000 def process_payment(self, amount): self._txn_counter += 1 self._last_txn_id = f\u0026#34;PP{self._txn_counter}_{self.merchant_id}\u0026#34; record = {\u0026#34;txn_id\u0026#34;: self._last_txn_id, \u0026#34;type\u0026#34;: \u0026#34;payment\u0026#34;, \u0026#34;amount\u0026#34;: amount} self.add_to_history(record) print(f\u0026#34;PhonePe: Payment of Rs.{amount:,} processed. TXN: {self._last_txn_id}\u0026#34;) return True def refund(self, amount, reason): self._txn_counter += 1 refund_id = f\u0026#34;PP_R{self._txn_counter}\u0026#34; self.add_to_history({\u0026#34;txn_id\u0026#34;: refund_id, \u0026#34;type\u0026#34;: \u0026#34;refund\u0026#34;, \u0026#34;amount\u0026#34;: amount}) print(f\u0026#34;PhonePe: Refund of Rs.{amount:,} initiated. TXN: {refund_id}\u0026#34;) def get_transaction_id(self): return self._last_txn_id # Both work through the same interface -- abstraction in action rzp = RazorpayGateway(api_key=\u0026#34;rzp_live_abc123\u0026#34;) php = PhonePeGateway(merchant_id=\u0026#34;CODEVERRA\u0026#34;) rzp.process_payment(2499) rzp.process_payment(1299) rzp.refund(1299, \u0026#34;Customer returned item\u0026#34;) php.process_payment(5000) print(\u0026#34;\\nRazorpay history:\u0026#34;, rzp.get_history()) print(\u0026#34;Last TXN:\u0026#34;, rzp.get_transaction_id()) # issubclass and ABC print(issubclass(RazorpayGateway, PaymentGateway)) # True print(issubclass(PhonePeGateway, PaymentGateway)) # True 13. Magic (Dunder) Methods Magic methods (also called dunder methods \u0026ndash; double underscore) let your custom objects work with Python\u0026rsquo;s built-in syntax and functions.\nThey are what makes len(my_object) work, or obj1 + obj2, or print(my_object) give a meaningful output.\nstr and repr class Product: def __init__(self, name, price, category): self.name = name self.price = price self.category = category def __str__(self): \u0026#34;\u0026#34;\u0026#34; Called by print() and str() -- meant for end users. Should be readable and informative. \u0026#34;\u0026#34;\u0026#34; return f\u0026#34;{self.name} (Rs.{self.price:,}) -- {self.category}\u0026#34; def __repr__(self): \u0026#34;\u0026#34;\u0026#34; Called in the REPL and for debugging -- meant for developers. Should ideally be valid Python that recreates the object. \u0026#34;\u0026#34;\u0026#34; return (f\u0026#34;Product(name={self.name!r}, \u0026#34; f\u0026#34;price={self.price}, \u0026#34; f\u0026#34;category={self.category!r})\u0026#34;) p = Product(\u0026#34;iPhone 15\u0026#34;, 79900, \u0026#34;Electronics\u0026#34;) print(p) # calls __str__: iPhone 15 (Rs.79,900) -- Electronics repr(p) # calls __repr__: Product(name=\u0026#39;iPhone 15\u0026#39;, price=79900, ...) # In a list, Python uses __repr__ products = [Product(\u0026#34;Book\u0026#34;, 299, \u0026#34;Books\u0026#34;), Product(\u0026#34;Pen\u0026#34;, 15, \u0026#34;Stationery\u0026#34;)] print(products) # uses __repr__ for each item len, contains, getitem class ShoppingCart: def __init__(self, customer_name): self.customer = customer_name self._items = [] # list of (product, quantity) tuples def add(self, product, quantity=1): self._items.append((product, quantity)) def __len__(self): \u0026#34;\u0026#34;\u0026#34;Called by len(cart) -- returns number of items.\u0026#34;\u0026#34;\u0026#34; return len(self._items) def __contains__(self, product_name): \u0026#34;\u0026#34;\u0026#34;Called by \u0026#39;x in cart\u0026#39; -- check if product is in cart.\u0026#34;\u0026#34;\u0026#34; return any(item[0].name == product_name for item in self._items) def __getitem__(self, index): \u0026#34;\u0026#34;\u0026#34;Called by cart[0], cart[1] -- index access.\u0026#34;\u0026#34;\u0026#34; return self._items[index] def __iter__(self): \u0026#34;\u0026#34;\u0026#34;Called by \u0026#39;for item in cart\u0026#39; -- makes cart iterable.\u0026#34;\u0026#34;\u0026#34; return iter(self._items) def total(self): return sum(p.price * qty for p, qty in self._items) def __str__(self): lines = [f\u0026#34;Cart for {self.customer}:\u0026#34;] for product, qty in self._items: lines.append(f\u0026#34; {product.name} x{qty} = Rs.{product.price * qty:,}\u0026#34;) lines.append(f\u0026#34; Total: Rs.{self.total():,}\u0026#34;) return \u0026#34;\\n\u0026#34;.join(lines) p1 = Product(\u0026#34;Laptop\u0026#34;, 65000, \u0026#34;Electronics\u0026#34;) p2 = Product(\u0026#34;Mouse\u0026#34;, 850, \u0026#34;Electronics\u0026#34;) p3 = Product(\u0026#34;Python Book\u0026#34;, 599, \u0026#34;Books\u0026#34;) cart = ShoppingCart(\u0026#34;Aarav Sharma\u0026#34;) cart.add(p1, 1) cart.add(p2, 2) cart.add(p3, 3) print(len(cart)) # 3 -- calls __len__ print(\u0026#34;Laptop\u0026#34; in cart) # True -- calls __contains__ print(\u0026#34;TV\u0026#34; in cart) # False print(cart[0]) # first item -- calls __getitem__ for product, qty in cart: # calls __iter__ print(f\u0026#34; {product.name}: {qty} unit(s)\u0026#34;) print(cart) # calls __str__ Comparison dunder methods from functools import total_ordering @total_ordering # automatically generates all comparison methods from __eq__ and __lt__ class Student: def __init__(self, name, cgpa): self.name = name self.cgpa = cgpa def __eq__(self, other): \u0026#34;\u0026#34;\u0026#34;Called by == and !=\u0026#34;\u0026#34;\u0026#34; if not isinstance(other, Student): return NotImplemented return self.cgpa == other.cgpa def __lt__(self, other): \u0026#34;\u0026#34;\u0026#34;Called by \u0026lt;. @total_ordering derives \u0026lt;=, \u0026gt;, \u0026gt;= from this.\u0026#34;\u0026#34;\u0026#34; if not isinstance(other, Student): return NotImplemented return self.cgpa \u0026lt; other.cgpa def __str__(self): return f\u0026#34;{self.name} (CGPA: {self.cgpa})\u0026#34; students = [ Student(\u0026#34;Aarav\u0026#34;, 8.9), Student(\u0026#34;Priya\u0026#34;, 9.4), Student(\u0026#34;Rohan\u0026#34;, 7.8), Student(\u0026#34;Sneha\u0026#34;, 9.4), Student(\u0026#34;Karan\u0026#34;, 8.2), ] # These all work because of our dunder methods print(max(students)) # Priya (CGPA: 9.4) print(min(students)) # Rohan (CGPA: 7.8) sorted_students = sorted(students) for s in sorted_students: print(s) print(students[0] == students[1]) # False (8.9 != 9.4) print(students[1] == students[3]) # True (9.4 == 9.4) print(students[0] \u0026lt; students[1]) # True (8.9 \u0026lt; 9.4) print(students[0] \u0026gt; students[4]) # True (8.9 \u0026gt; 8.2) Arithmetic dunder methods class Vector: \u0026#34;\u0026#34;\u0026#34;2D vector for physics calculations.\u0026#34;\u0026#34;\u0026#34; def __init__(self, x, y): self.x = x self.y = y def __add__(self, other): \u0026#34;\u0026#34;\u0026#34;Called by v1 + v2\u0026#34;\u0026#34;\u0026#34; return Vector(self.x + other.x, self.y + other.y) def __sub__(self, other): \u0026#34;\u0026#34;\u0026#34;Called by v1 - v2\u0026#34;\u0026#34;\u0026#34; return Vector(self.x - other.x, self.y - other.y) def __mul__(self, scalar): \u0026#34;\u0026#34;\u0026#34;Called by v * 3 (scalar multiplication)\u0026#34;\u0026#34;\u0026#34; return Vector(self.x * scalar, self.y * scalar) def __rmul__(self, scalar): \u0026#34;\u0026#34;\u0026#34;Called by 3 * v (right multiply -- Python tries this if left fails)\u0026#34;\u0026#34;\u0026#34; return self.__mul__(scalar) def __abs__(self): \u0026#34;\u0026#34;\u0026#34;Called by abs(v) -- magnitude of the vector\u0026#34;\u0026#34;\u0026#34; import math return math.sqrt(self.x**2 + self.y**2) def __neg__(self): \u0026#34;\u0026#34;\u0026#34;Called by -v -- negate the vector\u0026#34;\u0026#34;\u0026#34; return Vector(-self.x, -self.y) def __bool__(self): \u0026#34;\u0026#34;\u0026#34;Called in boolean context -- zero vector is falsy\u0026#34;\u0026#34;\u0026#34; return self.x != 0 or self.y != 0 def __str__(self): return f\u0026#34;Vector({self.x}, {self.y})\u0026#34; def __repr__(self): return f\u0026#34;Vector(x={self.x}, y={self.y})\u0026#34; v1 = Vector(3, 4) v2 = Vector(1, 2) print(v1 + v2) # Vector(4, 6) print(v1 - v2) # Vector(2, 2) print(v1 * 3) # Vector(9, 12) print(3 * v1) # Vector(9, 12) -- uses __rmul__ print(abs(v1)) # 5.0 print(-v1) # Vector(-3, -4) print(bool(Vector(0, 0))) # False print(bool(v1)) # True Context manager dunders \u0026ndash; enter and exit class DatabaseConnection: \u0026#34;\u0026#34;\u0026#34; A database connection that automatically closes itself. Used with the \u0026#39;with\u0026#39; statement. \u0026#34;\u0026#34;\u0026#34; def __init__(self, host, database): self.host = host self.database = database self.connection = None def __enter__(self): \u0026#34;\u0026#34;\u0026#34;Called when entering the \u0026#39;with\u0026#39; block -- set up the resource.\u0026#34;\u0026#34;\u0026#34; print(f\u0026#34;Connecting to {self.database} on {self.host}...\u0026#34;) self.connection = f\u0026#34;CONN_{self.host}_{self.database}\u0026#34; # simulated print(\u0026#34;Connection established.\u0026#34;) return self # the \u0026#39;as\u0026#39; variable in \u0026#39;with ... as conn:\u0026#39; gets this def __exit__(self, exc_type, exc_val, exc_tb): \u0026#34;\u0026#34;\u0026#34; Called when leaving the \u0026#39;with\u0026#39; block -- clean up the resource. Even called if an exception was raised inside the block. exc_type, exc_val, exc_tb: exception info (None if no exception) Return True to suppress the exception, False to let it propagate. \u0026#34;\u0026#34;\u0026#34; print(f\u0026#34;Closing connection to {self.database}...\u0026#34;) self.connection = None if exc_type: print(f\u0026#34;Exception occurred: {exc_val}\u0026#34;) return False # let the exception propagate print(\u0026#34;Connection closed cleanly.\u0026#34;) return False def query(self, sql): if not self.connection: raise RuntimeError(\u0026#34;No active connection.\u0026#34;) print(f\u0026#34;Executing: {sql}\u0026#34;) return [{\u0026#34;id\u0026#34;: 1, \u0026#34;name\u0026#34;: \u0026#34;Aarav\u0026#34;}, {\u0026#34;id\u0026#34;: 2, \u0026#34;name\u0026#34;: \u0026#34;Priya\u0026#34;}] # simulated # The with statement calls __enter__ and __exit__ automatically with DatabaseConnection(\u0026#34;db.codeverra.com\u0026#34;, \u0026#34;students_db\u0026#34;) as conn: results = conn.query(\u0026#34;SELECT * FROM students WHERE active = 1\u0026#34;) for row in results: print(f\u0026#34; {row}\u0026#34;) # __exit__ is called here automatically -- even if an exception occurred Part 3 \u0026ndash; Advanced OOP 14. Multiple Inheritance and MRO Python allows a class to inherit from more than one parent class. This is more powerful than single inheritance but requires understanding how Python decides which method to use when multiple parents define it.\nBasic multiple inheritance class Flyable: \u0026#34;\u0026#34;\u0026#34;Mixin-style class: gives the ability to fly.\u0026#34;\u0026#34;\u0026#34; def fly(self): print(f\u0026#34;{self.__class__.__name__} is flying!\u0026#34;) def land(self): print(f\u0026#34;{self.__class__.__name__} has landed.\u0026#34;) class Swimmable: \u0026#34;\u0026#34;\u0026#34;Mixin-style class: gives the ability to swim.\u0026#34;\u0026#34;\u0026#34; def swim(self): print(f\u0026#34;{self.__class__.__name__} is swimming!\u0026#34;) def dive(self): print(f\u0026#34;{self.__class__.__name__} has dived.\u0026#34;) class Animal: def __init__(self, name): self.name = name def breathe(self): print(f\u0026#34;{self.name} is breathing.\u0026#34;) class Duck(Animal, Flyable, Swimmable): \u0026#34;\u0026#34;\u0026#34;Duck inherits from Animal, Flyable, and Swimmable.\u0026#34;\u0026#34;\u0026#34; def quack(self): print(f\u0026#34;{self.name} says: Quack!\u0026#34;) donald = Duck(\u0026#34;Donald\u0026#34;) donald.breathe() # from Animal donald.fly() # from Flyable donald.swim() # from Swimmable donald.quack() # Duck\u0026#39;s own method print(Duck.__mro__) # shows the Method Resolution Order The Method Resolution Order (MRO) When multiple parent classes define the same method, Python needs a rule to decide which one to use. This rule is the MRO \u0026ndash; the order in which Python searches through the class hierarchy.\nPython uses the C3 Linearisation algorithm to compute the MRO. For everyday use, remember this: left to right, depth first, no class listed twice.\nclass A: def hello(self): print(\u0026#34;Hello from A\u0026#34;) class B(A): def hello(self): print(\u0026#34;Hello from B\u0026#34;) class C(A): def hello(self): print(\u0026#34;Hello from C\u0026#34;) class D(B, C): pass d = D() d.hello() # \u0026#34;Hello from B\u0026#34; -- B comes before C in D\u0026#39;s definition print(D.__mro__) # (\u0026lt;class \u0026#39;D\u0026#39;\u0026gt;, \u0026lt;class \u0026#39;B\u0026#39;\u0026gt;, \u0026lt;class \u0026#39;C\u0026#39;\u0026gt;, \u0026lt;class \u0026#39;A\u0026#39;\u0026gt;, \u0026lt;class \u0026#39;object\u0026#39;\u0026gt;) # MRO: D --\u0026gt; B --\u0026gt; C --\u0026gt; A --\u0026gt; object # Python searches in this exact order for any method Visualising the diamond: A / \\ B C \\ / D MRO: D, B, C, A, object super() in multiple inheritance super() follows the MRO, not just \u0026ldquo;the parent class\u0026rdquo;.\nclass A: def __init__(self): print(\u0026#34;A.__init__\u0026#34;) super().__init__() # follows MRO -- calls next in line class B(A): def __init__(self): print(\u0026#34;B.__init__\u0026#34;) super().__init__() # follows MRO -- calls C next, not A directly class C(A): def __init__(self): print(\u0026#34;C.__init__\u0026#34;) super().__init__() # follows MRO -- calls A next class D(B, C): def __init__(self): print(\u0026#34;D.__init__\u0026#34;) super().__init__() # starts the MRO chain d = D() # Output: # D.__init__ # B.__init__ # C.__init__ # A.__init__ # Every class in the MRO gets called exactly once -- cooperative multiple inheritance 15. Mixins A Mixin is a class designed to be mixed into other classes to add a specific piece of functionality. Mixins:\nAre not meant to be instantiated on their own Do not define __init__ (usually) Do one thing and do it well Are the Pythonic alternative to multiple inheritance soup Think of them as plug-in features you can add to any class by including them in the inheritance list.\nclass LoggingMixin: \u0026#34;\u0026#34;\u0026#34; Mixin that adds logging capability to any class. Mix this into any class that needs to log its method calls. \u0026#34;\u0026#34;\u0026#34; def log(self, message, level=\u0026#34;INFO\u0026#34;): import datetime timestamp = datetime.datetime.now().strftime(\u0026#34;%Y-%m-%d %H:%M:%S\u0026#34;) print(f\u0026#34;[{level}] {timestamp} [{self.__class__.__name__}] {message}\u0026#34;) class ValidationMixin: \u0026#34;\u0026#34;\u0026#34; Mixin that adds input validation helpers. \u0026#34;\u0026#34;\u0026#34; def validate_positive(self, value, field_name): if value \u0026lt;= 0: raise ValueError(f\u0026#34;{field_name} must be positive. Got: {value}\u0026#34;) return value def validate_string(self, value, field_name, min_length=1): if not isinstance(value, str) or len(value.strip()) \u0026lt; min_length: raise ValueError(f\u0026#34;{field_name} must be a non-empty string.\u0026#34;) return value.strip() def validate_range(self, value, field_name, min_val, max_val): if not (min_val \u0026lt;= value \u0026lt;= max_val): raise ValueError( f\u0026#34;{field_name} must be between {min_val} and {max_val}. Got: {value}\u0026#34; ) return value class SerializationMixin: \u0026#34;\u0026#34;\u0026#34; Mixin that adds JSON serialization capability. Assumes all attributes are stored in __dict__. \u0026#34;\u0026#34;\u0026#34; def to_dict(self): \u0026#34;\u0026#34;\u0026#34;Convert object to dictionary.\u0026#34;\u0026#34;\u0026#34; return { k: v for k, v in self.__dict__.items() if not k.startswith(\u0026#34;_\u0026#34;) # skip private attributes } def to_json(self): \u0026#34;\u0026#34;\u0026#34;Convert object to JSON string.\u0026#34;\u0026#34;\u0026#34; import json return json.dumps(self.to_dict(), indent=2, default=str) # Now mix these into real classes -- pick and choose what you need class BankAccount(LoggingMixin, ValidationMixin, SerializationMixin): \u0026#34;\u0026#34;\u0026#34;A full-featured bank account using three mixins.\u0026#34;\u0026#34;\u0026#34; def __init__(self, holder, account_number, balance=0): self.holder = self.validate_string(holder, \u0026#34;Account holder\u0026#34;) self.account_number = account_number self.balance = self.validate_positive(balance + 1, \u0026#34;Balance\u0026#34;) - 1 # balance can be 0, so we add 1 before validating positive, then subtract self.balance = balance self.log(f\u0026#34;Account created for {self.holder} with balance Rs.{balance:,}\u0026#34;) def deposit(self, amount): self.validate_positive(amount, \u0026#34;Deposit amount\u0026#34;) self.balance += amount self.log(f\u0026#34;Deposited Rs.{amount:,}. New balance: Rs.{self.balance:,}\u0026#34;) return self.balance def withdraw(self, amount): self.validate_positive(amount, \u0026#34;Withdrawal amount\u0026#34;) if amount \u0026gt; self.balance: self.log(f\u0026#34;Withdrawal of Rs.{amount:,} failed -- insufficient funds.\u0026#34;, \u0026#34;WARN\u0026#34;) raise ValueError(f\u0026#34;Insufficient funds. Balance: Rs.{self.balance:,}\u0026#34;) self.balance -= amount self.log(f\u0026#34;Withdrew Rs.{amount:,}. New balance: Rs.{self.balance:,}\u0026#34;) return self.balance class Student(LoggingMixin, ValidationMixin, SerializationMixin): \u0026#34;\u0026#34;\u0026#34;A student that also uses the three mixins.\u0026#34;\u0026#34;\u0026#34; def __init__(self, name, roll, cgpa): self.name = self.validate_string(name, \u0026#34;Student name\u0026#34;) self.roll = roll self.cgpa = self.validate_range(cgpa, \u0026#34;CGPA\u0026#34;, 0.0, 10.0) self.log(f\u0026#34;Student record created: {self.name} (Roll: {self.roll})\u0026#34;) def update_cgpa(self, new_cgpa): self.cgpa = self.validate_range(new_cgpa, \u0026#34;CGPA\u0026#34;, 0.0, 10.0) self.log(f\u0026#34;CGPA updated to {self.cgpa}\u0026#34;) # Using BankAccount with all three mixins acc = BankAccount(\u0026#34;Aarav Sharma\u0026#34;, \u0026#34;SBI001234\u0026#34;, 10000) acc.deposit(5000) acc.withdraw(3000) print(acc.to_json()) # SerializationMixin gives us this # Using Student with all three mixins s = Student(\u0026#34;Priya Patel\u0026#34;, 102, 9.2) s.update_cgpa(9.4) print(s.to_dict()) 16. slots By default, Python stores instance attributes in a dictionary called __dict__. This is flexible but uses more memory than necessary when you have thousands of objects.\n__slots__ replaces the __dict__ with a fixed set of attribute slots, reducing memory and slightly improving attribute access speed.\nimport sys # Without __slots__ class RegularPoint: def __init__(self, x, y): self.x = x self.y = y # With __slots__ class SlottedPoint: __slots__ = (\u0026#34;x\u0026#34;, \u0026#34;y\u0026#34;) # only these attributes are allowed def __init__(self, x, y): self.x = x self.y = y p1 = RegularPoint(3, 4) p2 = SlottedPoint(3, 4) print(sys.getsizeof(p1.__dict__)) # ~232 bytes (the dict overhead) print(hasattr(p2, \u0026#34;__dict__\u0026#34;)) # False -- no __dict__ for slotted objects # Memory difference becomes significant at scale regular_points = [RegularPoint(i, i) for i in range(100000)] slotted_points = [SlottedPoint(i, i) for i in range(100000)] # Slotted points use significantly less memory # Restriction: cannot add attributes not in __slots__ try: p2.z = 5 # AttributeError: \u0026#39;SlottedPoint\u0026#39; object has no attribute \u0026#39;z\u0026#39; except AttributeError as e: print(e) # Practical use: a GPS coordinate system tracking millions of points class GPSCoordinate: \u0026#34;\u0026#34;\u0026#34; Tracks lat/long for a large number of delivery locations. __slots__ is important when you have 100,000+ objects. \u0026#34;\u0026#34;\u0026#34; __slots__ = (\u0026#34;latitude\u0026#34;, \u0026#34;longitude\u0026#34;, \u0026#34;timestamp\u0026#34;, \u0026#34;label\u0026#34;) def __init__(self, lat, lon, label=\u0026#34;\u0026#34;): import datetime self.latitude = lat self.longitude = lon self.timestamp = datetime.datetime.now() self.label = label def distance_to(self, other): \u0026#34;\u0026#34;\u0026#34;Approximate distance in km using Haversine formula.\u0026#34;\u0026#34;\u0026#34; import math R = 6371 dlat = math.radians(other.latitude - self.latitude) dlon = math.radians(other.longitude - self.longitude) a = (math.sin(dlat/2)**2 + math.cos(math.radians(self.latitude)) * math.cos(math.radians(other.latitude)) * math.sin(dlon/2)**2) return R * 2 * math.atan2(math.sqrt(a), math.sqrt(1-a)) def __repr__(self): return f\u0026#34;GPS({self.latitude:.4f}, {self.longitude:.4f})\u0026#34; mumbai = GPSCoordinate(19.0760, 72.8777, \u0026#34;Mumbai\u0026#34;) bangalore = GPSCoordinate(12.9716, 77.5946, \u0026#34;Bangalore\u0026#34;) print(mumbai) print(f\u0026#34;Distance Mumbai to Bangalore: {mumbai.distance_to(bangalore):.0f}km\u0026#34;) 17. Metaclasses A metaclass is a class whose instances are themselves classes. In Python, type is the default metaclass \u0026ndash; it is what creates all classes.\nThis is advanced Python. You rarely need to write metaclasses. But understanding them demystifies things like Django models, SQLAlchemy, and the ABC mechanism.\n# type() with three arguments creates a new class dynamically # type(name, bases, dict) DynamicClass = type(\u0026#34;DynamicClass\u0026#34;, (object,), {\u0026#34;x\u0026#34;: 42, \u0026#34;greet\u0026#34;: lambda self: \u0026#34;Hello\u0026#34;}) obj = DynamicClass() print(obj.x) # 42 print(obj.greet()) # Hello # Every class is an instance of type print(type(int)) # \u0026lt;class \u0026#39;type\u0026#39;\u0026gt; print(type(str)) # \u0026lt;class \u0026#39;type\u0026#39;\u0026gt; print(type(DynamicClass)) # \u0026lt;class \u0026#39;type\u0026#39;\u0026gt; Writing a custom metaclass class SingletonMeta(type): \u0026#34;\u0026#34;\u0026#34; A metaclass that makes any class using it a Singleton. A Singleton is a class that can only ever have ONE instance. This is how the pattern is used: class DatabasePool(metaclass=SingletonMeta): ... pool1 = DatabasePool() pool2 = DatabasePool() pool1 is pool2 --\u0026gt; True (same object, not two separate ones) \u0026#34;\u0026#34;\u0026#34; _instances = {} # stores one instance per class def __call__(cls, *args, **kwargs): \u0026#34;\u0026#34;\u0026#34; __call__ on a metaclass is called when you do ClassName(). We intercept it to return the existing instance if one exists. \u0026#34;\u0026#34;\u0026#34; if cls not in cls._instances: # First time -- create and store the instance cls._instances[cls] = super().__call__(*args, **kwargs) return cls._instances[cls] class DatabasePool(metaclass=SingletonMeta): \u0026#34;\u0026#34;\u0026#34;Only one database pool should exist in the entire application.\u0026#34;\u0026#34;\u0026#34; def __init__(self, host, max_connections=10): self.host = host self.max_connections = max_connections self.active_connections = 0 print(f\u0026#34;DatabasePool created: {self.host}\u0026#34;) def get_connection(self): if self.active_connections \u0026lt; self.max_connections: self.active_connections += 1 return f\u0026#34;Connection #{self.active_connections}\u0026#34; raise RuntimeError(\u0026#34;Connection pool exhausted\u0026#34;) pool1 = DatabasePool(\u0026#34;db.codeverra.com\u0026#34;, max_connections=5) pool2 = DatabasePool(\u0026#34;different-host.com\u0026#34;) # this argument is IGNORED print(pool1 is pool2) # True -- same object print(pool1.host) # db.codeverra.com print(pool2.host) # db.codeverra.com (pool2 IS pool1) conn = pool1.get_connection() print(conn) # Connection #1 A validation metaclass class ValidatedMeta(type): \u0026#34;\u0026#34;\u0026#34; Metaclass that checks all method names follow snake_case convention. Useful for enforcing coding standards across a codebase. \u0026#34;\u0026#34;\u0026#34; def __new__(mcs, name, bases, namespace): for method_name in namespace: if method_name.startswith(\u0026#34;_\u0026#34;): continue # skip dunder and private methods if not method_name == method_name.lower(): raise TypeError( f\u0026#34;Method \u0026#39;{method_name}\u0026#39; in class \u0026#39;{name}\u0026#39; must be snake_case.\u0026#34; ) return super().__new__(mcs, name, bases, namespace) class GoodClass(metaclass=ValidatedMeta): def calculate_total(self): # valid snake_case pass def get_name(self): # valid pass try: class BadClass(metaclass=ValidatedMeta): def CalculateTotal(self): # invalid -- raises TypeError pass except TypeError as e: print(e) # Method \u0026#39;CalculateTotal\u0026#39; in class \u0026#39;BadClass\u0026#39; must be snake_case. 18. Dataclasses Dataclasses (introduced in Python 3.7) let you define classes that primarily hold data with far less boilerplate. They automatically generate __init__, __repr__, __eq__, and more.\nfrom dataclasses import dataclass, field, asdict, astuple from typing import List # Before dataclasses -- lots of boilerplate class OldStudent: def __init__(self, name, roll, marks): self.name = name self.roll = roll self.marks = marks def __repr__(self): return f\u0026#34;OldStudent(name={self.name!r}, roll={self.roll}, marks={self.marks})\u0026#34; def __eq__(self, other): return (self.name, self.roll, self.marks) == (other.name, other.roll, other.marks) # With dataclasses -- clean and concise @dataclass class Student: name: str roll: int marks: List[int] = field(default_factory=list) # mutable default -- use field() # You can still add methods def average(self): return sum(self.marks) / len(self.marks) if self.marks else 0 def grade(self): avg = self.average() if avg \u0026gt;= 85: return \u0026#34;Distinction\u0026#34; if avg \u0026gt;= 60: return \u0026#34;First Class\u0026#34; return \u0026#34;Pass\u0026#34; # __init__, __repr__, __eq__ all generated automatically s1 = Student(\u0026#34;Aarav Sharma\u0026#34;, 101, [88, 92, 75, 90, 85]) s2 = Student(\u0026#34;Priya Patel\u0026#34;, 102, [95, 88, 91, 87, 93]) s3 = Student(\u0026#34;Aarav Sharma\u0026#34;, 101, [88, 92, 75, 90, 85]) print(s1) # Student(name=\u0026#39;Aarav Sharma\u0026#39;, roll=101, marks=[88, 92, 75, 90, 85]) print(s1 == s3) # True -- __eq__ compares all fields print(s1 == s2) # False print(s1.average()) # 86.0 print(s1.grade()) # Distinction # Convert to dict or tuple print(asdict(s1)) # {\u0026#39;name\u0026#39;: \u0026#39;Aarav Sharma\u0026#39;, \u0026#39;roll\u0026#39;: 101, \u0026#39;marks\u0026#39;: [...]} print(astuple(s1)) # (\u0026#39;Aarav Sharma\u0026#39;, 101, [88, 92, 75, 90, 85]) Dataclass options from dataclasses import dataclass, field import datetime @dataclass( order=True, # generate __lt__, __le__, __gt__, __ge__ based on field order frozen=True, # make immutable -- generates __hash__, prevents attribute changes ) class GeoPoint: \u0026#34;\u0026#34;\u0026#34;Immutable, comparable GPS coordinate.\u0026#34;\u0026#34;\u0026#34; latitude: float longitude: float label: str = \u0026#34;\u0026#34; def __post_init__(self): \u0026#34;\u0026#34;\u0026#34;Called after __init__ -- for validation.\u0026#34;\u0026#34;\u0026#34; if not (-90 \u0026lt;= self.latitude \u0026lt;= 90): raise ValueError(f\u0026#34;Invalid latitude: {self.latitude}\u0026#34;) if not (-180 \u0026lt;= self.longitude \u0026lt;= 180): raise ValueError(f\u0026#34;Invalid longitude: {self.longitude}\u0026#34;) mumbai = GeoPoint(19.0760, 72.8777, \u0026#34;Mumbai\u0026#34;) bangalore = GeoPoint(12.9716, 77.5946, \u0026#34;Bangalore\u0026#34;) delhi = GeoPoint(28.6139, 77.2090, \u0026#34;Delhi\u0026#34;) # Sortable (by latitude first, then longitude, then label) cities = [mumbai, bangalore, delhi] print(sorted(cities)) # Hashable (because frozen=True) city_set = {mumbai, bangalore, delhi} # Immutable -- raises FrozenInstanceError try: mumbai.latitude = 20.0 except Exception as e: print(f\u0026#34;Cannot modify frozen dataclass: {e}\u0026#34;) # Dataclass with computed fields and post-init @dataclass class Order: product_name: str quantity: int unit_price: float discount_pct: float = 0.0 order_id: str = field(default=\u0026#34;\u0026#34;, init=False) # not in __init__ total: float = field(default=0.0, init=False) # computed def __post_init__(self): import random # Generate order ID after regular init self.order_id = f\u0026#34;ORD{random.randint(10000, 99999)}\u0026#34; # Compute total subtotal = self.quantity * self.unit_price discount = subtotal * (self.discount_pct / 100) self.total = round(subtotal - discount, 2) def __str__(self): return (f\u0026#34;Order {self.order_id}: {self.product_name} x{self.quantity} \u0026#34; f\u0026#34;@ Rs.{self.unit_price:,} \u0026#34; f\u0026#34;(Discount: {self.discount_pct}%) \u0026#34; f\u0026#34;= Rs.{self.total:,}\u0026#34;) o1 = Order(\u0026#34;MacBook Pro\u0026#34;, 1, 129900, discount_pct=5) o2 = Order(\u0026#34;Python Book\u0026#34;, 3, 599, discount_pct=10) print(o1) print(o2) 19. OOP in the Wild You have been using OOP without knowing it this entire time. Every object you use in Python is an instance of some class.\nimport pandas as pd import numpy as np # A DataFrame IS an object -- an instance of pd.DataFrame df = pd.DataFrame({\u0026#34;city\u0026#34;: [\u0026#34;Delhi\u0026#34;, \u0026#34;Mumbai\u0026#34;], \u0026#34;population\u0026#34;: [33, 20]}) print(type(df)) # \u0026lt;class \u0026#39;pandas.core.frame.DataFrame\u0026#39;\u0026gt; print(isinstance(df, pd.DataFrame)) # True # .groupby() returns a GroupBy object grp = df.groupby(\u0026#34;city\u0026#34;) print(type(grp)) # \u0026lt;class \u0026#39;pandas.core.groupby...DataFrameGroupBy\u0026#39;\u0026gt; # .sum() is a method on that GroupBy object result = grp.sum() print(type(result)) # \u0026lt;class \u0026#39;pandas.core.frame.DataFrame\u0026#39;\u0026gt; # NumPy arrays are ndarray objects arr = np.array([1, 2, 3]) print(type(arr)) # \u0026lt;class \u0026#39;numpy.ndarray\u0026#39;\u0026gt; print(arr.shape) # (3,) -- a property on the ndarray class print(arr.dtype) # dtype(\u0026#39;int64\u0026#39;) -- another property How Scikit-learn uses OOP \u0026ndash; a teaser Scikit-learn\u0026rsquo;s entire API is built on two classes from the base module: BaseEstimator and mixins like TransformerMixin, ClassifierMixin.\n# Every sklearn model follows this pattern: # 1. fit(X, y) -- learn from data # 2. predict(X) -- use what was learned # 3. score(X, y) -- evaluate from sklearn.linear_model import LogisticRegression model = LogisticRegression() print(type(model)) # \u0026lt;class \u0026#39;sklearn.linear_model._logistic.LogisticRegression\u0026#39;\u0026gt; # LogisticRegression inherits from: # LogisticRegression --\u0026gt; LinearClassifierMixin --\u0026gt; ClassifierMixin # --\u0026gt; BaseEstimator # # ClassifierMixin provides: score() # BaseEstimator provides: get_params(), set_params() # LogisticRegression provides: fit(), predict(), predict_proba() # You can build your own sklearn-compatible transformer using the same pattern. # That is covered in the Advanced OOP Applications document. The key insight When you learned functions, you learned to bundle behaviour. When you learn OOP, you learn to bundle data + behaviour + state together.\nEvery library you use \u0026ndash; Pandas, NumPy, Matplotlib, Scikit-learn, FastAPI, Django \u0026ndash; is built from classes following exactly the patterns in this guide.\nUnderstanding OOP does not just help you write better code. It helps you read and understand the tools you depend on.\n20. Practice Questions All questions are self-contained \u0026ndash; each one has the starter code or context you need. Questions are ordered: Easy (1-7), Medium (8-14), Hard (15-20).\nEasy Q1 \u0026ndash; Your First Class Define a class Rectangle with:\n__init__ that takes length and width area() method that returns length x width perimeter() method that returns 2 x (length + width) is_square() method that returns True if length equals width __str__ that returns something like \u0026quot;Rectangle(8 x 5)\u0026quot; Create three rectangles and print their area, perimeter, and whether they are squares.\nr1 = Rectangle(8, 5) r2 = Rectangle(6, 6) r3 = Rectangle(10, 3) Q2 \u0026ndash; Class Variables and Instance Variables Build a CricketTeam class that:\nTracks the total number of teams created (class variable) Each team has a name, home city, and a list of players (instance variables) Has an add_player(name) method Has a remove_player(name) method Has a show_squad() method that prints all players # Create two teams, add players, and demonstrate the team counter Q3 \u0026ndash; Class Methods and Static Methods Build a Temperature class with:\nAn instance variable celsius A @property that gives the temperature in Fahrenheit A @classmethod called from_fahrenheit(f) that creates a Temperature from a Fahrenheit value A @staticmethod called is_fever(celsius) that returns True if temp \u0026gt; 37.5 Test with: body temp (37.0 C), fever (39.2 C), boiling water (100 C), and create one from 98.6 F.\nQ4 \u0026ndash; Custom Exceptions Build an ATM class with a custom exception hierarchy:\nATMError(Exception) \u0026ndash; base InsufficientFundsError(ATMError) InvalidPINError(ATMError) DailyLimitExceededError(ATMError) The ATM should have a withdraw(pin, amount) method that raises the appropriate exception for wrong PIN (correct PIN is 1234), insufficient balance, or exceeding the daily limit of Rs.20,000.\nQ5 \u0026ndash; Properties with Validation Build a StudentProfile class where:\nname must be a non-empty string (setter validates and title-cases it) age must be between 15 and 30 (setter validates) cgpa must be between 0.0 and 10.0 (setter validates) email must contain \u0026ldquo;@\u0026rdquo; and end with a valid domain (setter validates) All are readable properties Q6 \u0026ndash; Dunder Methods Build a Fraction class (like Python\u0026rsquo;s built-in fractions.Fraction) that:\nStores numerator and denominator Reduces itself to lowest terms on creation (use GCD) Implements +, -, *, / between two Fraction objects Implements ==, \u0026lt;, \u0026gt; for comparison Has a clean __str__ (e.g. \u0026quot;3/4\u0026quot;) and __repr__ f1 = Fraction(1, 2) f2 = Fraction(1, 3) print(f1 + f2) # 5/6 print(f1 * f2) # 1/6 print(f1 \u0026gt; f2) # True Q7 \u0026ndash; Context Manager Build a Timer class that works as a context manager and measures the time taken to execute the code inside the with block.\nimport time with Timer(\u0026#34;Sorting 1 million numbers\u0026#34;) as t: data = list(range(1000000, 0, -1)) data.sort() # Expected output: # Starting: Sorting 1 million numbers # Sorting 1 million numbers completed in 0.0842 seconds Medium Q8 \u0026ndash; Inheritance Chain Build a class hierarchy for an Indian bank:\nBankAccount |-- SavingsAccount (interest rate: 3.5% per year) |-- CurrentAccount (no interest, overdraft limit of Rs.50,000) |-- FixedDeposit (lock-in period in months, higher interest rate: 6.5%) Each should inherit deposit() and withdraw() from BankAccount but override behaviour where needed (e.g., CurrentAccount.withdraw() allows going negative up to the overdraft limit, FixedDeposit.withdraw() raises an error if the FD has not matured).\nAdd a calculate_interest() method to each.\nQ9 \u0026ndash; Polymorphism Build a reporting system for a school. Define:\nReport (abstract base class) with an @abstractmethod generate() ProgressReport(Report) \u0026ndash; generates a student\u0026rsquo;s subject-wise marks AttendanceReport(Report) \u0026ndash; generates attendance percentage per month FeeReport(Report) \u0026ndash; generates fee payment status Write a function generate_all_reports(reports) that takes a list of any Report objects and calls generate() on each. Demonstrate that it works with a mixed list.\nQ10 \u0026ndash; Mixin Composition Build three mixins:\nTimestampMixin \u0026ndash; adds created_at and updated_at attributes, and an update() method that refreshes updated_at SoftDeleteMixin \u0026ndash; adds is_deleted flag, a delete() method (sets flag, does not remove), and restore() method AuditMixin \u0026ndash; adds a change_log list that records every attribute change with timestamp Then build a Product class that uses all three mixins, and demonstrate each mixin\u0026rsquo;s behaviour independently.\nQ11 \u0026ndash; Complete dunder Suite Build a Vector3D class (3-dimensional vector) with:\n__add__, __sub__, __mul__ (scalar), __truediv__ (scalar) __abs__ (magnitude), __neg__, __bool__ __eq__, __iter__ (iterate over x, y, z), __len__ (always 3) dot_product(other), cross_product(other), normalize() __str__ and __repr__ v1 = Vector3D(1, 2, 3) v2 = Vector3D(4, 5, 6) print(v1 + v2) # Vector3D(5, 7, 9) print(v1.dot_product(v2)) # 32 print(list(v1)) # [1, 2, 3] print(abs(v1)) # 3.7417... Q12 \u0026ndash; Abstract Base Class Design Design an abstract class DataExporter for a reporting tool that can export data in multiple formats. It should enforce that all exporters implement:\nconnect() \u0026ndash; establish connection to output write(data) \u0026ndash; write data to output close() \u0026ndash; close the connection But provide concrete shared methods:\nexport(data) \u0026ndash; orchestrates connect -\u0026gt; write -\u0026gt; close, with error handling validate(data) \u0026ndash; checks data is not empty Implement CSVExporter, JSONExporter, and ExcelExporter.\nQ13 \u0026ndash; Dataclass with Full Features Model an IPL match result using a dataclass:\nTeam dataclass: name, home_city, captain MatchResult dataclass: team1, team2, venue, date, winner, margin (runs or wickets), man_of_match MatchResult should be orderable by date Add a @property called summary that returns a one-line description Add a __post_init__ that validates the winner is one of the two teams Demonstrate creating 3 match results, sorting them by date, and printing summaries.\nQ14 \u0026ndash; MRO and Multiple Inheritance Build a system for Indian food delivery platform roles:\nPerson |-- Employee(Person) -- has employee_id, department |-- Customer(Person) -- has customer_id, address |-- DeliveryPartner(Employee) -- has vehicle, rating |-- AdminUser(Employee) -- has permissions list |-- PowerUser(Customer, Employee) -- is both customer and employee Write out the MRO for PowerUser before running the code. Verify with PowerUser.__mro__. Each class should have a describe() method and the child should call super().describe() to print the full chain.\nHard Q15 \u0026ndash; Complete Banking System Build a complete banking system using everything from this guide:\nClasses required:\nBankError, InsufficientFundsError, AccountFrozenError, TransactionLimitError (custom exceptions) Transaction (dataclass): transaction_id, type, amount, timestamp, balance_after Account (abstract base class): deposit(), withdraw(), get_statement() as abstract SavingsAccount(Account): interest, minimum balance enforcement CurrentAccount(Account): overdraft facility AccountMixin (LoggingMixin + SerializationMixin combined) Bank: manages multiple accounts, can transfer between them The Bank class should:\nOpen accounts and generate account numbers Transfer funds between accounts atomically (either both sides update or neither does) Print a full bank statement for any account Show total deposits across all accounts Q16 \u0026ndash; Singleton Metaclass + Configuration Manager Build a ConfigManager using the Singleton metaclass pattern that:\nCan only ever have one instance (config should be consistent across the app) Loads settings from a dictionary on first creation Supports nested key access: config.get(\u0026quot;database.host\u0026quot;) Supports dot-notation setting: config.set(\u0026quot;database.port\u0026quot;, 5433) Tracks which settings were changed since load (change log) Has a reset() class method that destroys the singleton and allows recreating config = ConfigManager({ \u0026#34;database\u0026#34;: {\u0026#34;host\u0026#34;: \u0026#34;localhost\u0026#34;, \u0026#34;port\u0026#34;: 5432, \u0026#34;name\u0026#34;: \u0026#34;codeverra\u0026#34;}, \u0026#34;app\u0026#34;: {\u0026#34;debug\u0026#34;: True, \u0026#34;max_students\u0026#34;: 1000}, }) config2 = ConfigManager({\u0026#34;other\u0026#34;: \u0026#34;data\u0026#34;}) # returns same instance, ignores args print(config is config2) # True print(config.get(\u0026#34;database.host\u0026#34;)) # localhost config.set(\u0026#34;database.port\u0026#34;, 5433) Q17 \u0026ndash; Descriptor Protocol A descriptor is an object that defines how attribute access works through __get__, __set__, and __delete__. Properties are built on descriptors.\nBuild a Validated descriptor class that:\nAccepts a validator function and an optional type_check Raises TypeError if the value is the wrong type Raises ValueError if the validator returns False Can be used to define validated class attributes class Student: name = Validated(str, lambda v: len(v) \u0026gt;= 2, \u0026#34;Name must be at least 2 chars\u0026#34;) cgpa = Validated(float, lambda v: 0.0 \u0026lt;= v \u0026lt;= 10.0, \u0026#34;CGPA must be 0-10\u0026#34;) age = Validated(int, lambda v: 15 \u0026lt;= v \u0026lt;= 30, \u0026#34;Age must be 15-30\u0026#34;) s = Student() s.name = \u0026#34;Aarav\u0026#34; # valid s.cgpa = 9.2 # valid s.age = 21 # valid s.cgpa = 11.0 # ValueError: CGPA must be 0-10 Q18 \u0026ndash; Observer Pattern The Observer pattern is a design pattern where objects (observers) subscribe to events from another object (the subject) and are notified automatically when the event occurs.\nBuild an event system for an e-commerce platform:\nEventEmitter \u0026ndash; base class for anything that emits events Order(EventEmitter) \u0026ndash; emits order_placed, order_shipped, order_delivered NotificationService \u0026ndash; observer that sends \u0026ldquo;notifications\u0026rdquo; InventoryService \u0026ndash; observer that updates stock AnalyticsService \u0026ndash; observer that logs events When an order changes state, all subscribed services are notified automatically.\nQ19 \u0026ndash; Custom Collection Class Build a StudentRoster class that behaves like a collection:\nSupports len(roster) Supports for student in roster: Supports roster[0], roster[\u0026quot;Aarav Sharma\u0026quot;] (by name or index) Supports \u0026quot;Aarav\u0026quot; in roster (partial name match) Supports roster + other_roster (merge two rosters, no duplicates by roll) Supports roster.sort(by=\u0026quot;cgpa\u0026quot;) or roster.sort(by=\u0026quot;name\u0026quot;) Has top_n(n) returning top n students by CGPA Has filter(department=...) returning a new StudentRoster Q20 \u0026ndash; Full System Design Design and implement a Library Management System using all OOP concepts:\nHierarchy: LibraryItem (abstract) |-- Book |-- Journal |-- DVD Person (abstract) |-- Member |-- Librarian(Member) -- has extra permissions Library |-- uses Singleton pattern (one library system) |-- manages items and members Mixins: SearchableMixin -- adds search() to Library ReportableMixin -- adds generate_report() to Library Custom Exceptions: LibraryError |-- ItemNotAvailableError |-- MembershipExpiredError |-- MaxBooksExceededError (members can borrow max 3 items) Dataclasses: BorrowRecord: member_id, item_id, borrowed_date, due_date, returned_date Features: - Members can borrow and return items - Librarians can add/remove items and members - Late fee calculation (Rs.5 per day after due date) - Search by title, author, or category - Report showing all currently borrowed items with due dates 21. Solutions Q1 \u0026ndash; Rectangle class Rectangle: def __init__(self, length, width): self.length = length self.width = width def area(self): return self.length * self.width def perimeter(self): return 2 * (self.length + self.width) def is_square(self): return self.length == self.width def __str__(self): return f\u0026#34;Rectangle({self.length} x {self.width})\u0026#34; r1 = Rectangle(8, 5) r2 = Rectangle(6, 6) r3 = Rectangle(10, 3) for r in [r1, r2, r3]: print(f\u0026#34;{r}\u0026#34;) print(f\u0026#34; Area : {r.area()}\u0026#34;) print(f\u0026#34; Perimeter : {r.perimeter()}\u0026#34;) print(f\u0026#34; Is square : {r.is_square()}\u0026#34;) print() Q2 \u0026ndash; CricketTeam class CricketTeam: total_teams = 0 # class variable -- shared def __init__(self, name, home_city): self.name = name self.home_city = home_city self.players = [] CricketTeam.total_teams += 1 def add_player(self, name): if name not in self.players: self.players.append(name) print(f\u0026#34;{name} added to {self.name}.\u0026#34;) else: print(f\u0026#34;{name} is already in {self.name}.\u0026#34;) def remove_player(self, name): if name in self.players: self.players.remove(name) print(f\u0026#34;{name} removed from {self.name}.\u0026#34;) else: print(f\u0026#34;{name} not found in {self.name}.\u0026#34;) def show_squad(self): print(f\u0026#34;\\n{self.name} Squad ({self.home_city}):\u0026#34;) for i, player in enumerate(self.players, 1): print(f\u0026#34; {i}. {player}\u0026#34;) def __str__(self): return f\u0026#34;{self.name} ({self.home_city}) -- {len(self.players)} players\u0026#34; mi = CricketTeam(\u0026#34;Mumbai Indians\u0026#34;, \u0026#34;Mumbai\u0026#34;) csk = CricketTeam(\u0026#34;Chennai Super Kings\u0026#34;, \u0026#34;Chennai\u0026#34;) mi.add_player(\u0026#34;Rohit Sharma\u0026#34;) mi.add_player(\u0026#34;Jasprit Bumrah\u0026#34;) mi.add_player(\u0026#34;Suryakumar Yadav\u0026#34;) csk.add_player(\u0026#34;MS Dhoni\u0026#34;) csk.add_player(\u0026#34;Ravindra Jadeja\u0026#34;) csk.add_player(\u0026#34;Deepak Chahar\u0026#34;) mi.show_squad() csk.show_squad() print(f\u0026#34;\\nTotal teams created: {CricketTeam.total_teams}\u0026#34;) Q3 \u0026ndash; Temperature class Temperature: def __init__(self, celsius): self._celsius = celsius @property def celsius(self): return self._celsius @property def fahrenheit(self): return (self._celsius * 9/5) + 32 @classmethod def from_fahrenheit(cls, f): celsius = (f - 32) * 5/9 return cls(celsius) @staticmethod def is_fever(celsius): return celsius \u0026gt; 37.5 def __str__(self): return f\u0026#34;{self._celsius:.1f}C / {self.fahrenheit:.1f}F\u0026#34; temps = [ Temperature(37.0), Temperature(39.2), Temperature(100.0), Temperature.from_fahrenheit(98.6), ] for t in temps: fever_status = \u0026#34;FEVER\u0026#34; if Temperature.is_fever(t.celsius) else \u0026#34;Normal\u0026#34; print(f\u0026#34;{t} -- {fever_status}\u0026#34;) Q4 \u0026ndash; ATM with Custom Exceptions class ATMError(Exception): pass class InsufficientFundsError(ATMError): def __init__(self, requested, available): super().__init__( f\u0026#34;Insufficient funds. Requested: Rs.{requested:,}, Available: Rs.{available:,}\u0026#34; ) class InvalidPINError(ATMError): def __init__(self, attempts_left): super().__init__(f\u0026#34;Wrong PIN. {attempts_left} attempt(s) remaining.\u0026#34;) class DailyLimitExceededError(ATMError): def __init__(self, limit, already_withdrawn): super().__init__( f\u0026#34;Daily limit of Rs.{limit:,} exceeded. \u0026#34; f\u0026#34;Already withdrawn today: Rs.{already_withdrawn:,}.\u0026#34; ) class ATM: CORRECT_PIN = 1234 DAILY_LIMIT = 20000 def __init__(self, account_holder, balance): self.account_holder = account_holder self.balance = balance self._withdrawn_today = 0 self._pin_attempts = 3 def withdraw(self, pin, amount): # Validate PIN if pin != self.CORRECT_PIN: self._pin_attempts -= 1 raise InvalidPINError(self._pin_attempts) # Reset PIN attempts on success self._pin_attempts = 3 # Validate daily limit if self._withdrawn_today + amount \u0026gt; self.DAILY_LIMIT: raise DailyLimitExceededError(self.DAILY_LIMIT, self._withdrawn_today) # Validate balance if amount \u0026gt; self.balance: raise InsufficientFundsError(amount, self.balance) self.balance -= amount self._withdrawn_today += amount print(f\u0026#34;Rs.{amount:,} dispensed. Remaining balance: Rs.{self.balance:,}\u0026#34;) atm = ATM(\u0026#34;Aarav Sharma\u0026#34;, 15000) try: atm.withdraw(9999, 5000) # Wrong PIN except InvalidPINError as e: print(e) try: atm.withdraw(1234, 5000) # Valid except ATMError as e: print(e) try: atm.withdraw(1234, 18000) # Daily limit except DailyLimitExceededError as e: print(e) try: atm.withdraw(1234, 12000) # Insufficient balance except InsufficientFundsError as e: print(e) Q5 \u0026ndash; StudentProfile with Properties class StudentProfile: def __init__(self, name, age, cgpa, email): self.name = name self.age = age self.cgpa = cgpa self.email = email @property def name(self): return self.__name @name.setter def name(self, value): if not isinstance(value, str) or not value.strip(): raise ValueError(\u0026#34;Name must be a non-empty string.\u0026#34;) self.__name = value.strip().title() @property def age(self): return self.__age @age.setter def age(self, value): if not isinstance(value, int) or not (15 \u0026lt;= value \u0026lt;= 30): raise ValueError(\u0026#34;Age must be an integer between 15 and 30.\u0026#34;) self.__age = value @property def cgpa(self): return self.__cgpa @cgpa.setter def cgpa(self, value): if not isinstance(value, (int, float)) or not (0.0 \u0026lt;= value \u0026lt;= 10.0): raise ValueError(\u0026#34;CGPA must be a number between 0.0 and 10.0.\u0026#34;) self.__cgpa = float(value) @property def email(self): return self.__email @email.setter def email(self, value): if not isinstance(value, str) or \u0026#34;@\u0026#34; not in value: raise ValueError(\u0026#34;Email must contain \u0026#39;@\u0026#39;.\u0026#34;) domain = value.split(\u0026#34;@\u0026#34;)[-1] if \u0026#34;.\u0026#34; not in domain: raise ValueError(\u0026#34;Email domain must contain \u0026#39;.\u0026#39;.\u0026#34;) self.__email = value.lower() def __str__(self): return (f\u0026#34;Student: {self.__name} | Age: {self.__age} | \u0026#34; f\u0026#34;CGPA: {self.__cgpa} | Email: {self.__email}\u0026#34;) s = StudentProfile(\u0026#34;aarav sharma\u0026#34;, 21, 8.9, \u0026#34;Aarav@Gmail.com\u0026#34;) print(s) try: s.cgpa = 11.0 except ValueError as e: print(e) try: s.email = \u0026#34;invalid-email\u0026#34; except ValueError as e: print(e) Q6 \u0026ndash; Fraction import math class Fraction: def __init__(self, numerator, denominator): if denominator == 0: raise ZeroDivisionError(\u0026#34;Denominator cannot be zero.\u0026#34;) # Reduce to lowest terms gcd = math.gcd(abs(numerator), abs(denominator)) sign = -1 if (numerator \u0026lt; 0) != (denominator \u0026lt; 0) else 1 self.numerator = sign * abs(numerator) // gcd self.denominator = abs(denominator) // gcd def __add__(self, other): n = self.numerator * other.denominator + other.numerator * self.denominator d = self.denominator * other.denominator return Fraction(n, d) def __sub__(self, other): n = self.numerator * other.denominator - other.numerator * self.denominator d = self.denominator * other.denominator return Fraction(n, d) def __mul__(self, other): return Fraction(self.numerator * other.numerator, self.denominator * other.denominator) def __truediv__(self, other): return Fraction(self.numerator * other.denominator, self.denominator * other.numerator) def __eq__(self, other): return self.numerator == other.numerator and self.denominator == other.denominator def __lt__(self, other): return self.numerator * other.denominator \u0026lt; other.numerator * self.denominator def __gt__(self, other): return other \u0026lt; self def __str__(self): if self.denominator == 1: return str(self.numerator) return f\u0026#34;{self.numerator}/{self.denominator}\u0026#34; def __repr__(self): return f\u0026#34;Fraction({self.numerator}, {self.denominator})\u0026#34; f1 = Fraction(1, 2) f2 = Fraction(1, 3) print(f1 + f2) # 5/6 print(f1 - f2) # 1/6 print(f1 * f2) # 1/6 print(f1 / f2) # 3/2 print(f1 \u0026gt; f2) # True print(f1 == Fraction(2, 4)) # True (reduced) Q7 \u0026ndash; Timer Context Manager import time class Timer: def __init__(self, label=\u0026#34;Task\u0026#34;): self.label = label self.start = None self.elapsed = None def __enter__(self): print(f\u0026#34;Starting: {self.label}\u0026#34;) self.start = time.time() return self def __exit__(self, exc_type, exc_val, exc_tb): self.elapsed = time.time() - self.start print(f\u0026#34;{self.label} completed in {self.elapsed:.4f} seconds\u0026#34;) return False # do not suppress exceptions with Timer(\u0026#34;Sorting 1 million numbers\u0026#34;) as t: data = list(range(1000000, 0, -1)) data.sort() print(f\u0026#34;Elapsed time stored: {t.elapsed:.4f}s\u0026#34;) Q8 \u0026ndash; Banking Hierarchy from abc import ABC, abstractmethod import datetime class BankAccount(ABC): def __init__(self, holder, account_number, balance=0): self.holder = holder self.account_number = account_number self._balance = balance self._transactions = [] @property def balance(self): return self._balance def deposit(self, amount): if amount \u0026lt;= 0: raise ValueError(\u0026#34;Deposit must be positive.\u0026#34;) self._balance += amount self._transactions.append((\u0026#34;deposit\u0026#34;, amount, self._balance)) print(f\u0026#34;Deposited Rs.{amount:,}. Balance: Rs.{self._balance:,}\u0026#34;) @abstractmethod def withdraw(self, amount): pass @abstractmethod def calculate_interest(self): pass def get_statement(self): print(f\u0026#34;\\n--- {self.__class__.__name__} Statement: {self.holder} ---\u0026#34;) for txn_type, amount, bal in self._transactions: print(f\u0026#34; {txn_type.capitalize():12} Rs.{amount:\u0026gt;10,} Balance: Rs.{bal:,}\u0026#34;) print(f\u0026#34;Current Balance: Rs.{self._balance:,}\u0026#34;) class SavingsAccount(BankAccount): INTEREST_RATE = 0.035 # 3.5% per annum MINIMUM_BALANCE = 1000 def withdraw(self, amount): if amount \u0026lt;= 0: raise ValueError(\u0026#34;Withdrawal must be positive.\u0026#34;) if self._balance - amount \u0026lt; self.MINIMUM_BALANCE: raise ValueError( f\u0026#34;Cannot withdraw Rs.{amount:,}. \u0026#34; f\u0026#34;Minimum balance of Rs.{self.MINIMUM_BALANCE:,} must be maintained.\u0026#34; ) self._balance -= amount self._transactions.append((\u0026#34;withdrawal\u0026#34;, amount, self._balance)) print(f\u0026#34;Withdrawn Rs.{amount:,}. Balance: Rs.{self._balance:,}\u0026#34;) def calculate_interest(self): interest = self._balance * self.INTEREST_RATE print(f\u0026#34;Annual interest at {self.INTEREST_RATE*100}%: Rs.{interest:,.2f}\u0026#34;) return interest class CurrentAccount(BankAccount): OVERDRAFT_LIMIT = 50000 def withdraw(self, amount): if amount \u0026lt;= 0: raise ValueError(\u0026#34;Withdrawal must be positive.\u0026#34;) if self._balance - amount \u0026lt; -self.OVERDRAFT_LIMIT: raise ValueError( f\u0026#34;Exceeds overdraft limit of Rs.{self.OVERDRAFT_LIMIT:,}.\u0026#34; ) self._balance -= amount self._transactions.append((\u0026#34;withdrawal\u0026#34;, amount, self._balance)) status = \u0026#34;(overdraft)\u0026#34; if self._balance \u0026lt; 0 else \u0026#34;\u0026#34; print(f\u0026#34;Withdrawn Rs.{amount:,}. Balance: Rs.{self._balance:,} {status}\u0026#34;) def calculate_interest(self): print(\u0026#34;Current accounts do not earn interest.\u0026#34;) return 0 class FixedDeposit(BankAccount): INTEREST_RATE = 0.065 # 6.5% per annum def __init__(self, holder, account_number, principal, months): super().__init__(holder, account_number, principal) self.months = months self.maturity = datetime.date.today() + datetime.timedelta(days=30*months) print(f\u0026#34;FD created. Maturity date: {self.maturity}\u0026#34;) def withdraw(self, amount): if datetime.date.today() \u0026lt; self.maturity: raise ValueError( f\u0026#34;FD has not matured. Maturity date: {self.maturity}. \u0026#34; f\u0026#34;Premature withdrawal not allowed.\u0026#34; ) # After maturity, full withdrawal only print(f\u0026#34;FD matured. Withdrawing full amount: Rs.{self._balance:,}\u0026#34;) withdrawn = self._balance self._balance = 0 return withdrawn def calculate_interest(self): interest = self._balance * self.INTEREST_RATE * (self.months / 12) print(f\u0026#34;FD interest ({self.months} months at {self.INTEREST_RATE*100}%): Rs.{interest:,.2f}\u0026#34;) return interest # Demonstrate sa = SavingsAccount(\u0026#34;Aarav Sharma\u0026#34;, \u0026#34;SBI001\u0026#34;, 10000) sa.deposit(5000) sa.withdraw(3000) sa.calculate_interest() sa.get_statement() ca = CurrentAccount(\u0026#34;Priya Patel\u0026#34;, \u0026#34;HDFC001\u0026#34;, 5000) ca.deposit(2000) ca.withdraw(10000) # goes into overdraft ca.get_statement() fd = FixedDeposit(\u0026#34;Rohan Verma\u0026#34;, \u0026#34;SBI-FD-001\u0026#34;, 100000, months=12) fd.calculate_interest() try: fd.withdraw(100000) # not matured yet except ValueError as e: print(e) Q9 \u0026ndash; Polymorphic Reporting System from abc import ABC, abstractmethod class Report(ABC): @abstractmethod def generate(self): pass def header(self, title): print(f\u0026#34;\\n{\u0026#39;=\u0026#39;*40}\u0026#34;) print(f\u0026#34; {title}\u0026#34;) print(f\u0026#34;{\u0026#39;=\u0026#39;*40}\u0026#34;) class ProgressReport(Report): def __init__(self, student_name, subjects_marks): self.student_name = student_name self.subjects_marks = subjects_marks # dict: subject -\u0026gt; marks def generate(self): self.header(f\u0026#34;Progress Report: {self.student_name}\u0026#34;) total = 0 for subject, marks in self.subjects_marks.items(): status = \u0026#34;Pass\u0026#34; if marks \u0026gt;= 40 else \u0026#34;Fail\u0026#34; print(f\u0026#34; {subject:\u0026lt;15}: {marks:\u0026gt;3}/100 [{status}]\u0026#34;) total += marks avg = total / len(self.subjects_marks) print(f\u0026#34;\\n Average: {avg:.1f}%\u0026#34;) class AttendanceReport(Report): def __init__(self, student_name, monthly_attendance): self.student_name = student_name self.monthly_attendance = monthly_attendance # dict: month -\u0026gt; (attended, total) def generate(self): self.header(f\u0026#34;Attendance Report: {self.student_name}\u0026#34;) total_attended = total_classes = 0 for month, (attended, total) in self.monthly_attendance.items(): pct = attended / total * 100 total_attended += attended total_classes += total print(f\u0026#34; {month:\u0026lt;10}: {attended}/{total} ({pct:.1f}%)\u0026#34;) overall = total_attended / total_classes * 100 print(f\u0026#34;\\n Overall: {total_attended}/{total_classes} ({overall:.1f}%)\u0026#34;) class FeeReport(Report): def __init__(self, student_name, fee_records): self.student_name = student_name self.fee_records = fee_records # list of (month, amount, status) def generate(self): self.header(f\u0026#34;Fee Report: {self.student_name}\u0026#34;) for month, amount, status in self.fee_records: icon = \u0026#34;Paid\u0026#34; if status == \u0026#34;paid\u0026#34; else \u0026#34;PENDING\u0026#34; print(f\u0026#34; {month:\u0026lt;10}: Rs.{amount:\u0026gt;8,} [{icon}]\u0026#34;) paid = sum(a for _, a, s in self.fee_records if s == \u0026#34;paid\u0026#34;) pending = sum(a for _, a, s in self.fee_records if s != \u0026#34;paid\u0026#34;) print(f\u0026#34;\\n Paid: Rs.{paid:,} | Pending: Rs.{pending:,}\u0026#34;) def generate_all_reports(reports): \u0026#34;\u0026#34;\u0026#34;Works with any Report subclass -- polymorphism in action.\u0026#34;\u0026#34;\u0026#34; for report in reports: report.generate() reports = [ ProgressReport(\u0026#34;Aarav Sharma\u0026#34;, { \u0026#34;Maths\u0026#34;: 88, \u0026#34;Science\u0026#34;: 92, \u0026#34;English\u0026#34;: 75, \u0026#34;History\u0026#34;: 80, \u0026#34;CS\u0026#34;: 95 }), AttendanceReport(\u0026#34;Aarav Sharma\u0026#34;, { \u0026#34;January\u0026#34;: (22, 24), \u0026#34;February\u0026#34;: (18, 20), \u0026#34;March\u0026#34;: (25, 26) }), FeeReport(\u0026#34;Aarav Sharma\u0026#34;, [ (\u0026#34;January\u0026#34;, 15000, \u0026#34;paid\u0026#34;), (\u0026#34;February\u0026#34;, 15000, \u0026#34;paid\u0026#34;), (\u0026#34;March\u0026#34;, 15000, \u0026#34;pending\u0026#34;), ]), ] generate_all_reports(reports) Q10 \u0026ndash; Mixin Composition import datetime class TimestampMixin: def __init__(self, *args, **kwargs): super().__init__(*args, **kwargs) self.created_at = datetime.datetime.now() self.updated_at = datetime.datetime.now() def update(self): self.updated_at = datetime.datetime.now() print(f\u0026#34;Updated at {self.updated_at.strftime(\u0026#39;%H:%M:%S\u0026#39;)}\u0026#34;) class SoftDeleteMixin: def __init__(self, *args, **kwargs): super().__init__(*args, **kwargs) self.is_deleted = False def delete(self): self.is_deleted = True print(f\u0026#34;{self.__class__.__name__} soft-deleted.\u0026#34;) def restore(self): self.is_deleted = False print(f\u0026#34;{self.__class__.__name__} restored.\u0026#34;) class AuditMixin: def __init__(self, *args, **kwargs): super().__init__(*args, **kwargs) self.change_log = [] def __setattr__(self, name, value): if name not in (\u0026#34;change_log\u0026#34;,) and hasattr(self, \u0026#34;change_log\u0026#34;): entry = { \u0026#34;field\u0026#34;: name, \u0026#34;new_value\u0026#34;: value, \u0026#34;timestamp\u0026#34;: datetime.datetime.now().strftime(\u0026#34;%H:%M:%S\u0026#34;) } self.change_log.append(entry) super().__setattr__(name, value) class Product(TimestampMixin, SoftDeleteMixin, AuditMixin): def __init__(self, name, price, category): super().__init__() self.name = name self.price = price self.category = category def __str__(self): status = \u0026#34;DELETED\u0026#34; if self.is_deleted else \u0026#34;Active\u0026#34; return f\u0026#34;Product({self.name}, Rs.{self.price:,}, {self.category}) [{status}]\u0026#34; p = Product(\u0026#34;Laptop\u0026#34;, 75000, \u0026#34;Electronics\u0026#34;) print(p) p.price = 72000 # logged by AuditMixin p.update() # updates timestamp p.delete() print(p) p.restore() print(p) print(\u0026#34;\\nChange log:\u0026#34;) for entry in p.change_log: print(f\u0026#34; [{entry[\u0026#39;timestamp\u0026#39;]}] {entry[\u0026#39;field\u0026#39;]} = {entry[\u0026#39;new_value\u0026#39;]}\u0026#34;) Q11 \u0026ndash; Vector3D import math class Vector3D: def __init__(self, x, y, z): self.x = x self.y = y self.z = z def __add__(self, other): return Vector3D(self.x+other.x, self.y+other.y, self.z+other.z) def __sub__(self, other): return Vector3D(self.x-other.x, self.y-other.y, self.z-other.z) def __mul__(self, scalar): return Vector3D(self.x*scalar, self.y*scalar, self.z*scalar) def __rmul__(self, scalar): return self.__mul__(scalar) def __truediv__(self, scalar): return Vector3D(self.x/scalar, self.y/scalar, self.z/scalar) def __abs__(self): return math.sqrt(self.x**2 + self.y**2 + self.z**2) def __neg__(self): return Vector3D(-self.x, -self.y, -self.z) def __bool__(self): return self.x != 0 or self.y != 0 or self.z != 0 def __eq__(self, other): return (self.x, self.y, self.z) == (other.x, other.y, other.z) def __iter__(self): yield self.x yield self.y yield self.z def __len__(self): return 3 def dot_product(self, other): return self.x*other.x + self.y*other.y + self.z*other.z def cross_product(self, other): return Vector3D( self.y*other.z - self.z*other.y, self.z*other.x - self.x*other.z, self.x*other.y - self.y*other.x ) def normalize(self): magnitude = abs(self) if magnitude == 0: raise ValueError(\u0026#34;Cannot normalize zero vector.\u0026#34;) return self / magnitude def __str__(self): return f\u0026#34;Vector3D({self.x}, {self.y}, {self.z})\u0026#34; def __repr__(self): return f\u0026#34;Vector3D(x={self.x}, y={self.y}, z={self.z})\u0026#34; v1 = Vector3D(1, 2, 3) v2 = Vector3D(4, 5, 6) print(v1 + v2) # Vector3D(5, 7, 9) print(v1 - v2) # Vector3D(-3, -3, -3) print(v1 * 3) # Vector3D(3, 6, 9) print(3 * v1) # Vector3D(3, 6, 9) print(abs(v1)) # 3.7417... print(v1.dot_product(v2)) # 32 print(v1.cross_product(v2)) # Vector3D(-3, 6, -3) print(list(v1)) # [1, 2, 3] print(len(v1)) # 3 n = v1.normalize() print(f\u0026#34;{abs(n):.6f}\u0026#34;) # 1.000000 (unit vector) Q12 through Q20 Questions 12 through 20 are design and implementation exercises. The solutions involve 50 to 200+ lines each and are best attempted as project-level work. Full reference solutions are available in the Codeverra Advanced OOP Applications document.\nGuidance for Q12-Q20:\nQ12: Start with the abstract DataExporter class, then implement each exporter independently Q13: Use @dataclass(order=True) and field() for computed fields Q14: Write out the MRO on paper first \u0026ndash; draw the diamond and trace the C3 algorithm Q15: Build bottom-up: exceptions first, then dataclasses, then account types, then Bank Q16: The key is __call__ on the metaclass intercepting instance creation Q17: A descriptor is an object with __get__ and __set__ \u0026ndash; it lives as a class attribute Q18: Use a list of observer objects stored in the subject, iterate and call on event Q19: Implement dunder methods one at a time, testing each before adding the next Q20: This is a full project \u0026ndash; plan the class hierarchy on paper, then code bottom-up 22. What Comes Next What This Guide Covered Part Concepts Part 1 (Beginner) Classes, objects, __init__, instance attributes, instance methods, class vs instance variables, class methods, static methods, custom exceptions Part 2 (Intermediate) Encapsulation (_ and __), properties (@property), inheritance, super(), method overriding, polymorphism, duck typing, abstraction (ABC, @abstractmethod), magic methods Part 3 (Advanced) Multiple inheritance, MRO, mixins, __slots__, metaclasses, dataclasses, OOP in the wild Recommended Next Step \u0026ndash; Advanced OOP Applications A separate document covers OOP patterns used inside real Python libraries:\nBuilding your own Scikit-learn-compatible transformer using BaseEstimator and TransformerMixin Custom Pandas DataFrame accessors with @pd.api.extensions.register_dataframe_accessor Subclassing Matplotlib Artist for custom plot elements Django model metaclass \u0026ndash; how class Meta inside a model works FastAPI dependency injection using classes The Repository pattern for database access Other topics that build on OOP Design Patterns \u0026ndash; Singleton, Factory, Observer, Strategy, Decorator pattern (not the same as the Python decorator) SOLID Principles \u0026ndash; the five principles of good OOP design Type checking with mypy \u0026ndash; using type annotations on class attributes and methods Testing OOP code \u0026ndash; unittest, pytest, mocking objects with unittest.mock Final thought: OOP is not a set of syntax rules to memorise. It is a way of thinking about problems \u0026ndash; breaking them into things (objects) that have properties (attributes) and behaviour (methods), and defining how those things relate to each other.\nThe best way to get comfortable is to model something real. Pick anything \u0026ndash; a cricket tournament, a college system, a food delivery app \u0026ndash; and try to model it entirely with classes before looking at any reference. That exercise will teach you more than any tutorial.\nMade with care for Codeverra learners | codeverra.com\n","permalink":"https://blogs.codeverra.com/blog/python-11-oop-masterclass/","summary":"\u003ch1 id=\"object-oriented-programming-in-python\"\u003eObject-Oriented Programming in Python\u003c/h1\u003e\n\u003ch3 id=\"a-structured-guide-from-first-principles-to-advanced-patterns\"\u003eA Structured Guide from First Principles to Advanced Patterns\u003c/h3\u003e\n\u003chr\u003e\n\u003ch2 id=\"before-we-begin---how-this-guide-is-organised\"\u003eBefore We Begin - How This Guide is Organised\u003c/h2\u003e\n\u003cp\u003eMost OOP tutorials throw syntax at you immediately. You copy the code, it works,\nand you think you understand it. Two weeks later you cannot write a class from scratch.\u003c/p\u003e\n\u003cp\u003eThis guide takes a different approach. We build understanding in three layers,\neach one preparing you for the next.\u003c/p\u003e","tags":["python"],"title":"Object-Oriented Programming in Python  -  Masterclass"},{"content":"Object-Oriented Programming \u0026ndash; Assignment Four Real-World Problems to Solve Using OOP This assignment covers OOP Basics and Intermediate concepts. For each problem, read the description carefully, understand what the system needs to do, and design your classes before writing any code. Planning on paper first is not optional \u0026ndash; it is part of the exercise.\nHow to Approach Each Problem Before writing a single line of code for any problem:\nRead the entire problem description Identify the classes you need List the attributes each class should have List the methods each class should have Identify relationships between classes (inheritance, composition) Only then start coding The OOP concepts tested in each problem are clearly labelled so you know what to focus on.\nProblem 1 \u0026ndash; Library Management System Difficulty: Beginner to Intermediate Concepts tested: Classes, instance methods, class variables, __str__, __repr__, custom exceptions, properties Background You are building a digital system for a public library in Pune. The library has a collection of books. Members can borrow books and return them. The system needs to track which books are available, which are borrowed, and by whom.\nWhat You Need to Build Class: Book Each book in the library has the following information:\nTitle Author ISBN (a unique identifier \u0026ndash; no two books share an ISBN) Total copies in the library Copies currently available (starts equal to total copies) The Book class must:\nRaise a ValueError if total copies is less than 1 Have a property is_available that returns True if at least one copy is available Have a method borrow() that reduces available copies by 1. Raise a custom exception BookNotAvailableError if no copies are available Have a method return_book() that increases available copies by 1. Raise a ValueError if available copies would exceed total copies Have a class variable total_books_in_library that tracks the total number of Book objects created __str__ should return something readable for a member: e.g. \u0026quot;The White Tiger by Aravind Adiga (2 copies available)\u0026quot; __repr__ should return something useful for a developer Class: Member A library member has:\nName Member ID (auto-generated \u0026ndash; use a class variable counter starting from M001) A list of books currently borrowed (starts empty) Maximum books allowed at once (default: 3) The Member class must:\nHave a method borrow_book(book) that calls book.borrow(), adds the book to the member\u0026rsquo;s borrowed list, and prints a confirmation. Raise a BorrowLimitError if the member already has the maximum number of books. Have a method return_book(book) that calls book.return_book(), removes it from the borrowed list, and prints a confirmation. Raise a ValueError if the member does not have that book. Have a method borrowed_titles() that returns a list of titles currently borrowed __str__ should show the member\u0026rsquo;s name, ID, and how many books they have borrowed Class: Library The library itself:\nHas a name and city Maintains a collection of books (a dict mapping ISBN to Book objects) Maintains a collection of members (a dict mapping member ID to Member objects) The Library class must:\nHave a method add_book(book) that adds a book to the collection Have a method register_member(member) that registers a new member Have a method search_by_title(title) that returns all books whose title contains the search string (case-insensitive) Have a method search_by_author(author) that returns all books by that author Have a method available_books() that returns all books with at least one copy available Have a method generate_report() that prints a summary of total books, total members, and total books currently borrowed Custom Exceptions Create these exception classes:\nLibraryError(Exception) \u0026ndash; base exception for the library system BookNotAvailableError(LibraryError) \u0026ndash; raised when a book has no copies left BorrowLimitError(LibraryError) \u0026ndash; raised when a member tries to borrow beyond their limit Expected Behaviour # Sample usage (your code should produce similar output) library = Library(\u0026#34;Pune Central Library\u0026#34;, \u0026#34;Pune\u0026#34;) b1 = Book(\u0026#34;The White Tiger\u0026#34;, \u0026#34;Aravind Adiga\u0026#34;, \u0026#34;ISBN001\u0026#34;, total_copies=2) b2 = Book(\u0026#34;A Suitable Boy\u0026#34;, \u0026#34;Vikram Seth\u0026#34;, \u0026#34;ISBN002\u0026#34;, total_copies=1) b3 = Book(\u0026#34;The God of Small Things\u0026#34;, \u0026#34;Arundhati Roy\u0026#34;, \u0026#34;ISBN003\u0026#34;, total_copies=3) library.add_book(b1) library.add_book(b2) library.add_book(b3) m1 = Member(\u0026#34;Aarav Sharma\u0026#34;) m2 = Member(\u0026#34;Priya Patel\u0026#34;) library.register_member(m1) library.register_member(m2) m1.borrow_book(b1) # \u0026#34;Aarav Sharma borrowed: The White Tiger\u0026#34; m1.borrow_book(b2) # \u0026#34;Aarav Sharma borrowed: A Suitable Boy\u0026#34; m2.borrow_book(b1) # \u0026#34;Priya Patel borrowed: The White Tiger\u0026#34; print(b1) # The White Tiger by Aravind Adiga (0 copies available) print(b1.is_available) # False m2.borrow_book(b1) # BookNotAvailableError m1.return_book(b2) # \u0026#34;Aarav Sharma returned: A Suitable Boy\u0026#34; library.generate_report() # Pune Central Library -- Report # Total books : 3 titles # Total members : 2 # Books borrowed : 2 What to Highlight in Your Solution Mark the following in your code comments:\nWhere encapsulation is used (private data, controlled access) Where properties are used Where custom exceptions are used Where class variables are used vs instance variables Problem 2 \u0026ndash; Employee Payroll System Difficulty: Intermediate Concepts tested: Inheritance, method overriding, super(), polymorphism, abstract base class, @classmethod, @staticmethod Background A Bangalore tech company has three types of employees:\nFull-time employees (fixed monthly salary) Contract employees (paid per hour worked) Intern employees (fixed stipend, capped at 6 months) All employees share some common attributes and behaviour, but the way their pay is calculated is different for each type. The payroll system should be able to process any employee through the same interface regardless of type.\nWhat You Need to Build Abstract Class: Employee Use Python\u0026rsquo;s ABC module. This class should:\nHave __init__ accepting: name, employee_id, department Store name, employee_id, department, and joining_date (set to today automatically) Have an @abstractmethod called calculate_pay() that returns a float Have an @abstractmethod called employment_type() that returns a string like \u0026ldquo;Full-Time\u0026rdquo;, \u0026ldquo;Contract\u0026rdquo;, or \u0026ldquo;Intern\u0026rdquo; Have a concrete method generate_payslip() that prints a formatted payslip. It calls calculate_pay() and employment_type() internally \u0026ndash; it works for ALL employee types without any modification Have a @classmethod called from_dict(data) that creates an employee from a dictionary (you will implement this on each subclass) Have a @staticmethod called is_valid_department(dept) that returns True if the department is one of: Engineering, Product, Sales, HR, Marketing, Finance Have __str__ return: \u0026quot;[EmployeeType] Name (ID) -- Department\u0026quot; Class: FullTimeEmployee(Employee) Additional attributes:\nmonthly_salary (float) performance_rating (float, 1.0 to 5.0, default 3.0) calculate_pay() must:\nReturn monthly salary Add a 10% bonus if performance rating is 4.5 or above Add a 5% bonus if performance rating is between 4.0 and 4.49 employment_type() returns \u0026quot;Full-Time\u0026quot;\nClass: ContractEmployee(Employee) Additional attributes:\nhourly_rate (float) hours_worked (float, this month) calculate_pay() returns hourly_rate * hours_worked\nemployment_type() returns \u0026quot;Contract\u0026quot;\nClass: Intern(Employee) Additional attributes:\nmonthly_stipend (float, must not exceed 25000) duration_months (int, must not exceed 6) calculate_pay() returns monthly_stipend\nemployment_type() returns \u0026quot;Intern\u0026quot;\nRaise a ValueError in __init__ if stipend exceeds 25000 or duration exceeds 6.\nFunction: run_payroll(employees) Write a standalone function (not inside a class) that:\nAccepts a list of any mix of Employee subclass objects Calls generate_payslip() on each one Prints a summary at the end: total payroll amount, number of each employee type This function demonstrates polymorphism \u0026ndash; it treats every employee the same way regardless of type.\nExpected Behaviour employees = [ FullTimeEmployee(\u0026#34;Vikram Nair\u0026#34;, \u0026#34;E001\u0026#34;, \u0026#34;Engineering\u0026#34;, monthly_salary=95000, performance_rating=4.7), FullTimeEmployee(\u0026#34;Ananya Shah\u0026#34;, \u0026#34;E002\u0026#34;, \u0026#34;Product\u0026#34;, monthly_salary=85000, performance_rating=3.8), ContractEmployee(\u0026#34;Suresh Kumar\u0026#34;, \u0026#34;C001\u0026#34;, \u0026#34;Sales\u0026#34;, hourly_rate=800, hours_worked=160), Intern(\u0026#34;Divya Menon\u0026#34;, \u0026#34;I001\u0026#34;, \u0026#34;Engineering\u0026#34;, monthly_stipend=20000, duration_months=3), ] run_payroll(employees) # Expected payslip output for Vikram (salary + 10% bonus): # ========================================== # PAYSLIP # Name : Vikram Nair # ID : E001 # Department : Engineering # Type : Full-Time # Joining Date : 2024-03-15 # Gross Pay : Rs.1,04,500.00 # ========================================== # Summary: # Full-Time : 2 employees | Total: Rs.1,97,750.00 # Contract : 1 employee | Total: Rs.1,28,000.00 # Intern : 1 employee | Total: Rs.20,000.00 # Grand Total: Rs.3,45,750.00 What to Highlight in Your Solution Mark the following in your code comments:\nWhere inheritance is used and how super() is called Where method overriding happens (each subclass overriding calculate_pay) Where polymorphism happens (the run_payroll function) Where abstraction is enforced (the ABC preventing direct instantiation) Problem 3 \u0026ndash; Online Food Ordering System Difficulty: Intermediate Concepts tested: Encapsulation with properties, composition, dunder methods, class methods, the __enter__/__exit__ context manager, operator overloading Background You are building the core order management system for a Zomato-like food delivery app. The system needs to handle menu items, customer carts, and orders.\nWhat You Need to Build Class: MenuItem Represents a single item on a restaurant\u0026rsquo;s menu:\nname (str) price (float, must be positive \u0026ndash; use a property with validation) category (str: \u0026ldquo;Starter\u0026rdquo;, \u0026ldquo;Main\u0026rdquo;, \u0026ldquo;Dessert\u0026rdquo;, \u0026ldquo;Beverage\u0026rdquo;) is_available (bool, default True) Must implement:\n__str__: \u0026quot;Masala Dosa -- Rs.80 (Main)\u0026quot; __repr__: developer-friendly representation __eq__: two MenuItems are equal if they have the same name and price __lt__: items are ordered by price (enables sorting a menu by price) Class: CartItem A MenuItem in a cart with a quantity:\nitem (a MenuItem object \u0026ndash; composition) quantity (int, must be \u0026gt;= 1 \u0026ndash; use a property with validation) Must implement:\nsubtotal property: returns item.price * quantity __str__: \u0026quot;Masala Dosa x2 = Rs.160\u0026quot; Class: Cart A customer\u0026rsquo;s shopping cart:\ncustomer_name (str) _items (list of CartItem objects, private) restaurant_name (str) Must implement:\nadd_item(menu_item, quantity=1): adds a CartItem. If the item is already in the cart, increase the quantity instead of adding a duplicate remove_item(menu_item): removes the CartItem for that MenuItem update_quantity(menu_item, quantity): updates quantity for an existing item total property: sum of all CartItem subtotals item_count property: total number of individual items (sum of quantities) is_empty property: True if cart has no items clear(): empties the cart __len__: returns number of distinct items in cart __str__: prints a formatted cart summary __add__: combine two carts (merge their items) and return a new Cart (useful for group orders) Works as a context manager: entering the context returns the cart, exiting it prints a summary and clears it if empty Class: Order Created from a Cart when the customer checks out:\norder_id (auto-generated, format: \u0026ldquo;ORD-XXXX\u0026rdquo; where XXXX is a 4-digit number) customer_name restaurant_name items (list of CartItem \u0026ndash; copied from the Cart) order_time (set to now automatically) status (starts as \u0026ldquo;Placed\u0026rdquo;) delivery_address (str) subtotal property delivery_fee property: Rs.0 if subtotal \u0026gt;= 300, else Rs.40 gst property: 5% of subtotal total property: subtotal + delivery_fee + gst (rounded to 2 decimal places) Must implement:\nupdate_status(new_status): updates status. Valid statuses are: \u0026ldquo;Placed\u0026rdquo;, \u0026ldquo;Confirmed\u0026rdquo;, \u0026ldquo;Preparing\u0026rdquo;, \u0026ldquo;Out for Delivery\u0026rdquo;, \u0026ldquo;Delivered\u0026rdquo;, \u0026ldquo;Cancelled\u0026rdquo; Raise a ValueError for invalid statuses. @classmethod from_cart(cart, delivery_address): creates an Order from a Cart __str__: a complete formatted order receipt Expected Behaviour # Building a menu dosa = MenuItem(\u0026#34;Masala Dosa\u0026#34;, 80, \u0026#34;Main\u0026#34;) coffee = MenuItem(\u0026#34;Filter Coffee\u0026#34;, 40, \u0026#34;Beverage\u0026#34;) idli = MenuItem(\u0026#34;Idli Sambar\u0026#34;, 60, \u0026#34;Main\u0026#34;) gulab = MenuItem(\u0026#34;Gulab Jamun\u0026#34;, 50, \u0026#34;Dessert\u0026#34;) vada = MenuItem(\u0026#34;Medu Vada\u0026#34;, 45, \u0026#34;Starter\u0026#34;) # Using the Cart as a context manager with Cart(\u0026#34;Aarav Sharma\u0026#34;, \u0026#34;Udupi Palace\u0026#34;) as cart: cart.add_item(dosa, 2) cart.add_item(coffee, 1) cart.add_item(idli, 1) cart.add_item(dosa, 1) # should increase dosa quantity to 3, not add duplicate print(len(cart)) # 3 (three distinct items) print(cart.total) # 80*3 + 40 + 60 = 340 order = Order.from_cart(cart, \u0026#34;42, Koregaon Park, Pune\u0026#34;) print(order) # Order receipt should show: # ========================================= # ORDER RECEIPT # Order ID : ORD-0001 # Customer : Aarav Sharma # Restaurant : Udupi Palace # Time : 15 Mar 2024, 14:30 # ----------------------------------------- # Masala Dosa x3 Rs.240 # Filter Coffee x1 Rs.40 # Idli Sambar x1 Rs.60 # ----------------------------------------- # Subtotal : Rs.340.00 # Delivery Fee : Rs.0.00 # GST (5%) : Rs.17.00 # Total : Rs.357.00 # ----------------------------------------- # Status: Placed # ========================================= # Sorting menu items by price (uses __lt__) menu = [dosa, coffee, idli, gulab, vada] print(sorted(menu)) # Combining two carts (uses __add__) cart2 = Cart(\u0026#34;Priya Patel\u0026#34;, \u0026#34;Udupi Palace\u0026#34;) cart2.add_item(gulab, 2) cart2.add_item(vada, 1) combined = cart + cart2 print(combined.total) What to Highlight in Your Solution Mark the following in your code comments:\nWhere encapsulation is used (private _items, properties with validation) Where composition is used (CartItem has a MenuItem, Cart has CartItems) Where dunder methods make your objects feel native to Python Where the context manager protocol is implemented Problem 4 \u0026ndash; Student Grade Management System (Full System) Difficulty: Intermediate to Advanced Concepts tested: All four pillars together, mixins, dataclasses, multiple inheritance, __slots__, comprehensive dunder methods Background Build a complete grade management system for a college. The system handles students, courses, grade records, and generates reports. This problem intentionally requires you to think about the full design before writing code.\nWhat You Need to Build Mixin: SerializableMixin Adds the ability to convert any object to a dictionary and to a JSON string.\nto_dict(): returns a dict of all non-private attributes to_json(): returns a JSON string (use the json module) Mixin: TimestampMixin Adds creation and update timestamps to any class.\ncreated_at: set to now in __init__ updated_at: set to now in __init__ touch(): updates updated_at to now Dataclass: GradeRecord Use @dataclass to represent a single grade entry:\nsubject (str) marks_obtained (float) max_marks (float, default 100) exam_type (str: \u0026ldquo;Internal\u0026rdquo;, \u0026ldquo;Midterm\u0026rdquo;, \u0026ldquo;Final\u0026rdquo;, default \u0026ldquo;Final\u0026rdquo;) percentage property (computed: marks_obtained / max_marks * 100) grade_letter property (A+: \u0026gt;=90, A: \u0026gt;=80, B: \u0026gt;=70, C: \u0026gt;=60, D: \u0026gt;=50, F: below 50) The dataclass should be orderable (sort by percentage) Class: Student(SerializableMixin, TimestampMixin) Uses both mixins. Has __slots__ for memory efficiency since the college has thousands of students.\nSlots: name, student_id, department, year, _grade_records\nMust:\nAuto-generate student_id in format \u0026ldquo;STU-YYYY-NNNN\u0026rdquo; where YYYY is current year and NNNN is a zero-padded counter (STU-2024-0001, STU-2024-0002, etc.) _grade_records is a private list starting empty add_grade(grade_record): adds a GradeRecord. Raise TypeError if not a GradeRecord grades property: returns a copy of the grade records list cgpa property: average percentage across all grade records, on a 10-point scale (percentage / 10). Returns 0.0 if no grades yet top_subject property: returns the GradeRecord with the highest percentage failed_subjects property: returns list of GradeRecords where grade_letter is \u0026ldquo;F\u0026rdquo; generate_transcript(): prints a formatted transcript __str__: name, ID, department, CGPA __repr__: developer-friendly __eq__: two students are equal if they have the same student_id __lt__: students are ordered by CGPA (for ranking) Class: Course course_code (str, e.g. \u0026ldquo;CS101\u0026rdquo;) course_name (str) credits (int) instructor (str) enrolled_students (list of Student objects, starts empty) enroll(student): adds student if not already enrolled unenroll(student): removes student average_cgpa property: mean CGPA of all enrolled students top_student property: Student with highest CGPA __len__: number of enrolled students __contains__: supports student in course Class: Department name (str) head (str, name of department head) courses (dict of course_code: Course) students (dict of student_id: Student) add_course(course) add_student(student) generate_report(): prints department-wide report including: Total students Total courses Department average CGPA Topper (highest CGPA student) Number of students with CGPA below 5.0 (at risk) Distribution of grade letters across all grade records Expected Behaviour # Create department cs_dept = Department(\u0026#34;Computer Science\u0026#34;, \u0026#34;Dr. Ramesh Kumar\u0026#34;) # Create courses python_course = Course(\u0026#34;CS101\u0026#34;, \u0026#34;Python Programming\u0026#34;, credits=4, instructor=\u0026#34;Prof. Priya Nair\u0026#34;) ml_course = Course(\u0026#34;CS301\u0026#34;, \u0026#34;Machine Learning\u0026#34;, credits=4, instructor=\u0026#34;Prof. Vikram Sharma\u0026#34;) cs_dept.add_course(python_course) cs_dept.add_course(ml_course) # Create students s1 = Student(\u0026#34;Aarav Sharma\u0026#34;, \u0026#34;Computer Science\u0026#34;, year=2) s2 = Student(\u0026#34;Priya Patel\u0026#34;, \u0026#34;Computer Science\u0026#34;, year=2) s3 = Student(\u0026#34;Rohan Verma\u0026#34;, \u0026#34;Computer Science\u0026#34;, year=3) # Add grades using GradeRecord dataclass s1.add_grade(GradeRecord(\u0026#34;Python Programming\u0026#34;, 88, exam_type=\u0026#34;Final\u0026#34;)) s1.add_grade(GradeRecord(\u0026#34;Machine Learning\u0026#34;, 91, exam_type=\u0026#34;Final\u0026#34;)) s1.add_grade(GradeRecord(\u0026#34;Data Structures\u0026#34;, 79, exam_type=\u0026#34;Midterm\u0026#34;)) s2.add_grade(GradeRecord(\u0026#34;Python Programming\u0026#34;, 95)) s2.add_grade(GradeRecord(\u0026#34;Machine Learning\u0026#34;, 88)) s2.add_grade(GradeRecord(\u0026#34;Data Structures\u0026#34;, 72)) s3.add_grade(GradeRecord(\u0026#34;Python Programming\u0026#34;, 45)) # failed s3.add_grade(GradeRecord(\u0026#34;Data Structures\u0026#34;, 58)) # Enroll in courses python_course.enroll(s1) python_course.enroll(s2) python_course.enroll(s3) ml_course.enroll(s1) ml_course.enroll(s2) cs_dept.add_student(s1) cs_dept.add_student(s2) cs_dept.add_student(s3) # Generate transcript s1.generate_transcript() # Output: # ============================================ # ACADEMIC TRANSCRIPT # Name : Aarav Sharma # ID : STU-2024-0001 # Department : Computer Science # Year : 2 # CGPA : 8.60 / 10 # -------------------------------------------- # Subject Marks Grade # -------------------------------------------- # Python Programming 88/100 A (Final) # Machine Learning 91/100 A+ (Final) # Data Structures 79/100 C (Midterm) # -------------------------------------------- # Top Subject : Machine Learning (91%) # ============================================ # Serialisation (from SerializableMixin) import json print(s1.to_json()) # Ranking students (uses __lt__) all_students = [s1, s2, s3] ranked = sorted(all_students, reverse=True) # Department report cs_dept.generate_report() # Check membership (uses __contains__) print(s1 in python_course) # True print(s3 in ml_course) # False What to Highlight in Your Solution Mark the following in your code comments:\nAll four pillars: Encapsulation, Inheritance (through mixins), Polymorphism (to_dict/to_json works on any class using the mixin), Abstraction (clean interfaces hiding implementation details) Where mixins are used and why they are better than a deep inheritance chain Where dataclasses reduce boilerplate Where __slots__ is used and why Where composition is used (Student has GradeRecords, Department has Courses and Students) Submission Checklist Before submitting your solutions, verify:\nProblem 1 -- Library System [ ] Book class with all methods and properties [ ] Member class with borrow/return logic [ ] Library class with search and report [ ] Three custom exceptions in a hierarchy [ ] __str__ and __repr__ on all classes [ ] Class variable tracking total books Problem 2 -- Payroll System [ ] Abstract Employee class using ABC [ ] Three concrete subclasses with calculate_pay() overridden [ ] generate_payslip() works without modification for all types [ ] run_payroll() demonstrates polymorphism [ ] @classmethod and @staticmethod present Problem 3 -- Food Ordering System [ ] MenuItem with price validation and comparison operators [ ] CartItem with quantity validation [ ] Cart with __add__, __len__, context manager [ ] Order auto-generates ID and calculates all fees [ ] No duplicate items in cart Problem 4 -- Grade Management System [ ] SerializableMixin and TimestampMixin [ ] GradeRecord as a proper dataclass [ ] Student uses __slots__ [ ] Auto-generated student IDs [ ] Department report with full statistics [ ] All dunder methods implemented and tested Codeverra OOP Assignment | learn.codeverra.com\nObject-Oriented Programming \u0026ndash; Assignment Solutions Full Solutions with Concept Annotations Each solution includes inline comments marking exactly where each OOP concept is being applied. Read the comments as carefully as the code.\nSolution 1 \u0026ndash; Library Management System # ─── Custom Exceptions ───────────────────────────────────────────────────── # CONCEPT: Custom exception hierarchy (OOP basics) # We inherit from a base LibraryError so callers can catch all library # errors with one except clause, or be specific when needed. class LibraryError(Exception): \u0026#34;\u0026#34;\u0026#34;Base exception for all library system errors.\u0026#34;\u0026#34;\u0026#34; pass class BookNotAvailableError(LibraryError): \u0026#34;\u0026#34;\u0026#34;Raised when a book has no copies available to borrow.\u0026#34;\u0026#34;\u0026#34; def __init__(self, title): self.title = title super().__init__(f\u0026#34;\u0026#39;{title}\u0026#39; has no copies available.\u0026#34;) class BorrowLimitError(LibraryError): \u0026#34;\u0026#34;\u0026#34;Raised when a member tries to borrow beyond their allowed limit.\u0026#34;\u0026#34;\u0026#34; def __init__(self, member_name, limit): super().__init__( f\u0026#34;{member_name} has already borrowed the maximum of {limit} books.\u0026#34; ) # ─── Book ────────────────────────────────────────────────────────────────── class Book: # CONCEPT: Class variable # Shared across all Book instances -- tracks how many Book objects exist total_books_in_library = 0 def __init__(self, title, author, isbn, total_copies): if total_copies \u0026lt; 1: raise ValueError(\u0026#34;A book must have at least 1 copy.\u0026#34;) # CONCEPT: Instance variables -- unique to each Book object self.title = title self.author = author self.isbn = isbn self._total_copies = total_copies # private -- controlled via property self._available = total_copies # private -- starts equal to total # CONCEPT: Modifying class variable at instance creation Book.total_books_in_library += 1 # CONCEPT: Property -- controlled read access to private attribute @property def total_copies(self): return self._total_copies # CONCEPT: Computed property -- no setter, derived from state @property def is_available(self): return self._available \u0026gt; 0 @property def available_copies(self): return self._available def borrow(self): \u0026#34;\u0026#34;\u0026#34;Reduce available copies by 1. Raise if none left.\u0026#34;\u0026#34;\u0026#34; # CONCEPT: Encapsulation -- the borrow logic is owned by the Book # No external code can directly set _available if not self.is_available: raise BookNotAvailableError(self.title) self._available -= 1 def return_book(self): \u0026#34;\u0026#34;\u0026#34;Increase available copies by 1. Raise if already at maximum.\u0026#34;\u0026#34;\u0026#34; if self._available \u0026gt;= self._total_copies: raise ValueError( f\u0026#34;All copies of \u0026#39;{self.title}\u0026#39; are already in the library.\u0026#34; ) self._available += 1 # CONCEPT: __str__ -- human-readable, shown to end users def __str__(self): status = f\u0026#34;{self._available} cop{\u0026#39;y\u0026#39; if self._available == 1 else \u0026#39;ies\u0026#39;} available\u0026#34; return f\u0026#34;{self.title} by {self.author} ({status})\u0026#34; # CONCEPT: __repr__ -- developer-readable, useful for debugging def __repr__(self): return (f\u0026#34;Book(title={self.title!r}, author={self.author!r}, \u0026#34; f\u0026#34;isbn={self.isbn!r}, available={self._available}/{self._total_copies})\u0026#34;) # ─── Member ──────────────────────────────────────────────────────────────── class Member: # CONCEPT: Class variable -- auto-incrementing ID counter _id_counter = 0 def __init__(self, name, max_books=3): Member._id_counter += 1 # CONCEPT: Instance variables self.name = name self.member_id = f\u0026#34;M{Member._id_counter:03d}\u0026#34; # M001, M002, ... self._borrowed = [] # private list -- accessed via methods only self.max_books = max_books def borrow_book(self, book): \u0026#34;\u0026#34;\u0026#34;Borrow a book. Raises BorrowLimitError or BookNotAvailableError.\u0026#34;\u0026#34;\u0026#34; # CONCEPT: Encapsulation -- this method controls the rules of borrowing if len(self._borrowed) \u0026gt;= self.max_books: raise BorrowLimitError(self.name, self.max_books) # Delegate to Book\u0026#39;s own borrow logic (Book owns its availability state) book.borrow() self._borrowed.append(book) print(f\u0026#34;{self.name} borrowed: {book.title}\u0026#34;) def return_book(self, book): \u0026#34;\u0026#34;\u0026#34;Return a borrowed book.\u0026#34;\u0026#34;\u0026#34; if book not in self._borrowed: raise ValueError(f\u0026#34;{self.name} has not borrowed \u0026#39;{book.title}\u0026#39;.\u0026#34;) book.return_book() self._borrowed.remove(book) print(f\u0026#34;{self.name} returned: {book.title}\u0026#34;) def borrowed_titles(self): \u0026#34;\u0026#34;\u0026#34;Return list of titles currently borrowed.\u0026#34;\u0026#34;\u0026#34; return [book.title for book in self._borrowed] def __str__(self): return (f\u0026#34;Member({self.name}, ID: {self.member_id}, \u0026#34; f\u0026#34;Books borrowed: {len(self._borrowed)}/{self.max_books})\u0026#34;) # ─── Library ─────────────────────────────────────────────────────────────── class Library: def __init__(self, name, city): self.name = name self.city = city self._books = {} # isbn -\u0026gt; Book (encapsulated collection) self._members = {} # member_id -\u0026gt; Member def add_book(self, book): self._books[book.isbn] = book def register_member(self, member): self._members[member.member_id] = member def search_by_title(self, title): \u0026#34;\u0026#34;\u0026#34;Return all books whose title contains the search string.\u0026#34;\u0026#34;\u0026#34; query = title.lower() return [b for b in self._books.values() if query in b.title.lower()] def search_by_author(self, author): \u0026#34;\u0026#34;\u0026#34;Return all books by a given author (case-insensitive).\u0026#34;\u0026#34;\u0026#34; query = author.lower() return [b for b in self._books.values() if query in b.author.lower()] def available_books(self): \u0026#34;\u0026#34;\u0026#34;Return all books with at least one available copy.\u0026#34;\u0026#34;\u0026#34; return [b for b in self._books.values() if b.is_available] def generate_report(self): \u0026#34;\u0026#34;\u0026#34;Print a summary of the library\u0026#39;s current state.\u0026#34;\u0026#34;\u0026#34; total_borrowed = sum( 1 for m in self._members.values() for _ in m._borrowed ) print(f\u0026#34;\\n{self.name} -- Report\u0026#34;) print(f\u0026#34; Total books : {len(self._books)} titles\u0026#34;) print(f\u0026#34; Total members : {len(self._members)}\u0026#34;) print(f\u0026#34; Books borrowed: {total_borrowed}\u0026#34;) # ─── Demo ────────────────────────────────────────────────────────────────── if __name__ == \u0026#34;__main__\u0026#34;: library = Library(\u0026#34;Pune Central Library\u0026#34;, \u0026#34;Pune\u0026#34;) b1 = Book(\u0026#34;The White Tiger\u0026#34;, \u0026#34;Aravind Adiga\u0026#34;, \u0026#34;ISBN001\u0026#34;, total_copies=2) b2 = Book(\u0026#34;A Suitable Boy\u0026#34;, \u0026#34;Vikram Seth\u0026#34;, \u0026#34;ISBN002\u0026#34;, total_copies=1) b3 = Book(\u0026#34;The God of Small Things\u0026#34;, \u0026#34;Arundhati Roy\u0026#34;, \u0026#34;ISBN003\u0026#34;, total_copies=3) library.add_book(b1) library.add_book(b2) library.add_book(b3) m1 = Member(\u0026#34;Aarav Sharma\u0026#34;) m2 = Member(\u0026#34;Priya Patel\u0026#34;) library.register_member(m1) library.register_member(m2) m1.borrow_book(b1) m1.borrow_book(b2) m2.borrow_book(b1) print(b1) print(b1.is_available) try: m2.borrow_book(b1) except BookNotAvailableError as e: print(f\u0026#34;Error: {e}\u0026#34;) m1.return_book(b2) library.generate_report() print(f\u0026#34;\\nTotal Book objects created: {Book.total_books_in_library}\u0026#34;) Solution 2 \u0026ndash; Employee Payroll System from abc import ABC, abstractmethod from datetime import date # ─── Abstract Base Class ─────────────────────────────────────────────────── # CONCEPT: Abstraction # Employee defines the CONTRACT that all employee types must follow. # You cannot instantiate Employee directly -- it exists only as a template. class Employee(ABC): VALID_DEPARTMENTS = { \u0026#34;Engineering\u0026#34;, \u0026#34;Product\u0026#34;, \u0026#34;Sales\u0026#34;, \u0026#34;HR\u0026#34;, \u0026#34;Marketing\u0026#34;, \u0026#34;Finance\u0026#34; } def __init__(self, name, employee_id, department): if not self.is_valid_department(department): raise ValueError( f\u0026#34;\u0026#39;{department}\u0026#39; is not a valid department. \u0026#34; f\u0026#34;Choose from: {sorted(self.VALID_DEPARTMENTS)}\u0026#34; ) # CONCEPT: Instance variables set in abstract base # All subclasses automatically have these self.name = name self.employee_id = employee_id self.department = department self.joining_date = date.today() # CONCEPT: abstractmethod -- forces every subclass to implement this # If a subclass does not implement calculate_pay(), Python raises TypeError @abstractmethod def calculate_pay(self) -\u0026gt; float: \u0026#34;\u0026#34;\u0026#34;Calculate and return the employee\u0026#39;s pay for this period.\u0026#34;\u0026#34;\u0026#34; pass @abstractmethod def employment_type(self) -\u0026gt; str: \u0026#34;\u0026#34;\u0026#34;Return a string describing the employment type.\u0026#34;\u0026#34;\u0026#34; pass # CONCEPT: Concrete method in abstract class # This method works for ALL employee types without modification. # It calls calculate_pay() and employment_type() which are overridden # in each subclass -- this is POLYMORPHISM in action inside the class. def generate_payslip(self): \u0026#34;\u0026#34;\u0026#34;Print a formatted payslip. Works for all employee types.\u0026#34;\u0026#34;\u0026#34; pay = self.calculate_pay() print(\u0026#34;=\u0026#34; * 44) print(\u0026#34;PAYSLIP\u0026#34;) print(f\u0026#34; Name : {self.name}\u0026#34;) print(f\u0026#34; ID : {self.employee_id}\u0026#34;) print(f\u0026#34; Department : {self.department}\u0026#34;) print(f\u0026#34; Type : {self.employment_type()}\u0026#34;) print(f\u0026#34; Joining Date : {self.joining_date.strftime(\u0026#39;%d %b %Y\u0026#39;)}\u0026#34;) print(f\u0026#34; Gross Pay : Rs.{pay:,.2f}\u0026#34;) print(\u0026#34;=\u0026#34; * 44) return pay # CONCEPT: classmethod -- alternative constructor # Each subclass overrides this to build from a dictionary @classmethod def from_dict(cls, data): raise NotImplementedError(\u0026#34;Subclasses must implement from_dict()\u0026#34;) # CONCEPT: staticmethod -- utility that belongs to the class # but needs no object or class state @staticmethod def is_valid_department(dept): return dept in Employee.VALID_DEPARTMENTS def __str__(self): return f\u0026#34;[{self.employment_type()}] {self.name} ({self.employee_id}) -- {self.department}\u0026#34; def __repr__(self): return f\u0026#34;{self.__class__.__name__}(name={self.name!r}, id={self.employee_id!r})\u0026#34; # ─── FullTimeEmployee ────────────────────────────────────────────────────── # CONCEPT: Inheritance # FullTimeEmployee inherits all attributes and methods from Employee, # then adds its own and overrides calculate_pay(). class FullTimeEmployee(Employee): def __init__(self, name, employee_id, department, monthly_salary, performance_rating=3.0): # CONCEPT: super() -- initialise the parent class first super().__init__(name, employee_id, department) # Additional instance variables specific to full-time employees self.monthly_salary = monthly_salary self.performance_rating = performance_rating # CONCEPT: Method overriding -- replacing the parent\u0026#39;s abstract method # with a concrete implementation specific to this type def calculate_pay(self): \u0026#34;\u0026#34;\u0026#34;Salary plus performance bonus if applicable.\u0026#34;\u0026#34;\u0026#34; pay = self.monthly_salary if self.performance_rating \u0026gt;= 4.5: pay *= 1.10 # 10% bonus elif self.performance_rating \u0026gt;= 4.0: pay *= 1.05 # 5% bonus return round(pay, 2) def employment_type(self): return \u0026#34;Full-Time\u0026#34; @classmethod def from_dict(cls, data): return cls( name=data[\u0026#34;name\u0026#34;], employee_id=data[\u0026#34;employee_id\u0026#34;], department=data[\u0026#34;department\u0026#34;], monthly_salary=data[\u0026#34;monthly_salary\u0026#34;], performance_rating=data.get(\u0026#34;performance_rating\u0026#34;, 3.0), ) # ─── ContractEmployee ────────────────────────────────────────────────────── # CONCEPT: Inheritance -- another subclass of Employee class ContractEmployee(Employee): def __init__(self, name, employee_id, department, hourly_rate, hours_worked): super().__init__(name, employee_id, department) self.hourly_rate = hourly_rate self.hours_worked = hours_worked # CONCEPT: Method overriding -- different pay calculation for contracts def calculate_pay(self): return round(self.hourly_rate * self.hours_worked, 2) def employment_type(self): return \u0026#34;Contract\u0026#34; @classmethod def from_dict(cls, data): return cls( name=data[\u0026#34;name\u0026#34;], employee_id=data[\u0026#34;employee_id\u0026#34;], department=data[\u0026#34;department\u0026#34;], hourly_rate=data[\u0026#34;hourly_rate\u0026#34;], hours_worked=data[\u0026#34;hours_worked\u0026#34;], ) # ─── Intern ──────────────────────────────────────────────────────────────── class Intern(Employee): MAX_STIPEND = 25000 MAX_DURATION = 6 # months def __init__(self, name, employee_id, department, monthly_stipend, duration_months): super().__init__(name, employee_id, department) if monthly_stipend \u0026gt; self.MAX_STIPEND: raise ValueError( f\u0026#34;Stipend cannot exceed Rs.{self.MAX_STIPEND:,}. Got: Rs.{monthly_stipend:,}\u0026#34; ) if duration_months \u0026gt; self.MAX_DURATION: raise ValueError( f\u0026#34;Internship cannot exceed {self.MAX_DURATION} months. Got: {duration_months}\u0026#34; ) self.monthly_stipend = monthly_stipend self.duration_months = duration_months # CONCEPT: Method overriding def calculate_pay(self): return self.monthly_stipend def employment_type(self): return \u0026#34;Intern\u0026#34; @classmethod def from_dict(cls, data): return cls( name=data[\u0026#34;name\u0026#34;], employee_id=data[\u0026#34;employee_id\u0026#34;], department=data[\u0026#34;department\u0026#34;], monthly_stipend=data[\u0026#34;monthly_stipend\u0026#34;], duration_months=data[\u0026#34;duration_months\u0026#34;], ) # ─── run_payroll ────────────────────────────────────────────────────────── # CONCEPT: Polymorphism # This function does not know or care whether it receives a FullTimeEmployee, # ContractEmployee, or Intern. It treats them ALL identically through the # shared Employee interface. When generate_payslip() is called, Python # automatically calls the right calculate_pay() for each type. def run_payroll(employees): \u0026#34;\u0026#34;\u0026#34;Process payroll for a mixed list of any Employee subclass objects.\u0026#34;\u0026#34;\u0026#34; print(\u0026#34;\\n\u0026#34; + \u0026#34;=\u0026#34; * 44) print(\u0026#34;PAYROLL RUN\u0026#34;) print(\u0026#34;=\u0026#34; * 44) totals = {} grand_total = 0.0 for employee in employees: pay = employee.generate_payslip() emp_type = employee.employment_type() if emp_type not in totals: totals[emp_type] = {\u0026#34;count\u0026#34;: 0, \u0026#34;amount\u0026#34;: 0.0} totals[emp_type][\u0026#34;count\u0026#34;] += 1 totals[emp_type][\u0026#34;amount\u0026#34;] += pay grand_total += pay print(\u0026#34;\\nPAYROLL SUMMARY\u0026#34;) print(\u0026#34;-\u0026#34; * 44) for emp_type, data in totals.items(): count = data[\u0026#34;count\u0026#34;] label = \u0026#34;employee\u0026#34; if count == 1 else \u0026#34;employees\u0026#34; print(f\u0026#34; {emp_type:\u0026lt;12}: {count} {label:\u0026lt;10} | Rs.{data[\u0026#39;amount\u0026#39;]:\u0026gt;10,.2f}\u0026#34;) print(\u0026#34;-\u0026#34; * 44) print(f\u0026#34; {\u0026#39;Grand Total\u0026#39;:\u0026lt;24} | Rs.{grand_total:\u0026gt;10,.2f}\u0026#34;) # ─── Demo ────────────────────────────────────────────────────────────────── if __name__ == \u0026#34;__main__\u0026#34;: employees = [ FullTimeEmployee(\u0026#34;Vikram Nair\u0026#34;, \u0026#34;E001\u0026#34;, \u0026#34;Engineering\u0026#34;, monthly_salary=95000, performance_rating=4.7), FullTimeEmployee(\u0026#34;Ananya Shah\u0026#34;, \u0026#34;E002\u0026#34;, \u0026#34;Product\u0026#34;, monthly_salary=85000, performance_rating=3.8), ContractEmployee(\u0026#34;Suresh Kumar\u0026#34;, \u0026#34;C001\u0026#34;, \u0026#34;Sales\u0026#34;, hourly_rate=800, hours_worked=160), Intern(\u0026#34;Divya Menon\u0026#34;, \u0026#34;I001\u0026#34;, \u0026#34;Engineering\u0026#34;, monthly_stipend=20000, duration_months=3), ] run_payroll(employees) # Demonstrate that Employee cannot be instantiated directly try: e = Employee(\u0026#34;Test\u0026#34;, \u0026#34;T001\u0026#34;, \u0026#34;Engineering\u0026#34;) except TypeError as err: print(f\u0026#34;\\nAbstraction enforced: {err}\u0026#34;) # Demonstrate @staticmethod print(f\u0026#34;\\n\u0026#39;Engineering\u0026#39; valid: {Employee.is_valid_department(\u0026#39;Engineering\u0026#39;)}\u0026#34;) print(f\u0026#34;\u0026#39;Accounts\u0026#39; valid: {Employee.is_valid_department(\u0026#39;Accounts\u0026#39;)}\u0026#34;) # Demonstrate @classmethod alternative constructor data = {\u0026#34;name\u0026#34;: \u0026#34;Meera Joshi\u0026#34;, \u0026#34;employee_id\u0026#34;: \u0026#34;E003\u0026#34;, \u0026#34;department\u0026#34;: \u0026#34;HR\u0026#34;, \u0026#34;monthly_salary\u0026#34;: 70000} emp = FullTimeEmployee.from_dict(data) print(f\u0026#34;\\nCreated from dict: {emp}\u0026#34;) Solution 3 \u0026ndash; Online Food Ordering System import json from datetime import datetime # ─── Custom Exception ────────────────────────────────────────────────────── class OrderError(Exception): pass # ─── MenuItem ────────────────────────────────────────────────────────────── class MenuItem: VALID_CATEGORIES = {\u0026#34;Starter\u0026#34;, \u0026#34;Main\u0026#34;, \u0026#34;Dessert\u0026#34;, \u0026#34;Beverage\u0026#34;} def __init__(self, name, price, category, is_available=True): # CONCEPT: Encapsulation -- price and category go through setters self.name = name self.price = price # calls the setter below self.category = category # calls the setter below self.is_available = is_available # CONCEPT: Property with validation -- encapsulates the price field @property def price(self): return self._price @price.setter def price(self, value): if not isinstance(value, (int, float)) or value \u0026lt;= 0: raise ValueError(f\u0026#34;Price must be a positive number. Got: {value}\u0026#34;) self._price = float(value) @property def category(self): return self._category @category.setter def category(self, value): if value not in self.VALID_CATEGORIES: raise ValueError( f\u0026#34;\u0026#39;{value}\u0026#39; is not a valid category. \u0026#34; f\u0026#34;Choose from: {self.VALID_CATEGORIES}\u0026#34; ) self._category = value # CONCEPT: Dunder methods -- makes MenuItem feel like a native Python type def __str__(self): return f\u0026#34;{self.name} -- Rs.{self._price:.0f} ({self._category})\u0026#34; def __repr__(self): return f\u0026#34;MenuItem(name={self.name!r}, price={self._price}, category={self._category!r})\u0026#34; def __eq__(self, other): \u0026#34;\u0026#34;\u0026#34;Two items are equal if same name and price.\u0026#34;\u0026#34;\u0026#34; if not isinstance(other, MenuItem): return NotImplemented return self.name == other.name and self._price == other._price def __lt__(self, other): \u0026#34;\u0026#34;\u0026#34;Order items by price -- enables sorted(menu_list).\u0026#34;\u0026#34;\u0026#34; if not isinstance(other, MenuItem): return NotImplemented return self._price \u0026lt; other._price def __hash__(self): # Required when __eq__ is defined and we want to use items in sets/dicts return hash((self.name, self._price)) # ─── CartItem ────────────────────────────────────────────────────────────── # CONCEPT: Composition -- CartItem HAS-A MenuItem class CartItem: def __init__(self, item, quantity=1): if not isinstance(item, MenuItem): raise TypeError(\u0026#34;item must be a MenuItem instance.\u0026#34;) self.item = item # composition: holds a reference to a MenuItem self.quantity = quantity # calls the setter @property def quantity(self): return self._quantity @quantity.setter def quantity(self, value): if not isinstance(value, int) or value \u0026lt; 1: raise ValueError(\u0026#34;Quantity must be a positive integer.\u0026#34;) self._quantity = value @property def subtotal(self): \u0026#34;\u0026#34;\u0026#34;Total price for this item x quantity.\u0026#34;\u0026#34;\u0026#34; return self.item.price * self._quantity def __str__(self): return (f\u0026#34;{self.item.name} x{self._quantity} \u0026#34; f\u0026#34;= Rs.{self.subtotal:.0f}\u0026#34;) # ─── Cart ────────────────────────────────────────────────────────────────── # CONCEPT: Composition -- Cart HAS-A list of CartItems class Cart: def __init__(self, customer_name, restaurant_name): self.customer_name = customer_name self.restaurant_name = restaurant_name self._items = [] # CONCEPT: Encapsulation -- private list def _find_item(self, menu_item): \u0026#34;\u0026#34;\u0026#34;Return the CartItem for a given MenuItem, or None.\u0026#34;\u0026#34;\u0026#34; for cart_item in self._items: if cart_item.item == menu_item: return cart_item return None def add_item(self, menu_item, quantity=1): \u0026#34;\u0026#34;\u0026#34;Add item to cart. If already present, increase quantity.\u0026#34;\u0026#34;\u0026#34; if not menu_item.is_available: raise OrderError(f\u0026#34;\u0026#39;{menu_item.name}\u0026#39; is not currently available.\u0026#34;) existing = self._find_item(menu_item) if existing: existing.quantity += quantity else: self._items.append(CartItem(menu_item, quantity)) def remove_item(self, menu_item): \u0026#34;\u0026#34;\u0026#34;Remove a MenuItem from the cart entirely.\u0026#34;\u0026#34;\u0026#34; cart_item = self._find_item(menu_item) if not cart_item: raise OrderError(f\u0026#34;\u0026#39;{menu_item.name}\u0026#39; is not in the cart.\u0026#34;) self._items.remove(cart_item) def update_quantity(self, menu_item, quantity): \u0026#34;\u0026#34;\u0026#34;Update the quantity for an existing item.\u0026#34;\u0026#34;\u0026#34; cart_item = self._find_item(menu_item) if not cart_item: raise OrderError(f\u0026#34;\u0026#39;{menu_item.name}\u0026#39; is not in the cart.\u0026#34;) cart_item.quantity = quantity def clear(self): self._items.clear() @property def total(self): return sum(ci.subtotal for ci in self._items) @property def item_count(self): return sum(ci.quantity for ci in self._items) @property def is_empty(self): return len(self._items) == 0 # CONCEPT: Dunder methods def __len__(self): \u0026#34;\u0026#34;\u0026#34;Number of distinct items in the cart.\u0026#34;\u0026#34;\u0026#34; return len(self._items) def __str__(self): if self.is_empty: return f\u0026#34;Cart for {self.customer_name} (empty)\u0026#34; lines = [f\u0026#34;Cart -- {self.customer_name} @ {self.restaurant_name}\u0026#34;] lines.append(\u0026#34;-\u0026#34; * 40) for ci in self._items: lines.append(f\u0026#34; {ci}\u0026#34;) lines.append(\u0026#34;-\u0026#34; * 40) lines.append(f\u0026#34; Total: Rs.{self.total:.2f}\u0026#34;) return \u0026#34;\\n\u0026#34;.join(lines) def __add__(self, other): \u0026#34;\u0026#34;\u0026#34;Merge two carts. Combines quantities for shared items.\u0026#34;\u0026#34;\u0026#34; if not isinstance(other, Cart): return NotImplemented merged = Cart( f\u0026#34;{self.customer_name} + {other.customer_name}\u0026#34;, self.restaurant_name ) for ci in self._items: merged.add_item(ci.item, ci.quantity) for ci in other._items: merged.add_item(ci.item, ci.quantity) return merged # CONCEPT: Context manager protocol # __enter__ and __exit__ make Cart usable with the \u0026#39;with\u0026#39; statement def __enter__(self): \u0026#34;\u0026#34;\u0026#34;Enter context -- return the cart itself.\u0026#34;\u0026#34;\u0026#34; return self def __exit__(self, exc_type, exc_val, exc_tb): \u0026#34;\u0026#34;\u0026#34;Exit context -- print summary, clear if empty.\u0026#34;\u0026#34;\u0026#34; if not self.is_empty: print(f\u0026#34;\\nCart session ended for {self.customer_name}.\u0026#34;) print(f\u0026#34; Items: {self.item_count} | Total: Rs.{self.total:.2f}\u0026#34;) else: print(f\u0026#34;Cart for {self.customer_name} is empty -- nothing to summarise.\u0026#34;) return False # do not suppress exceptions # ─── Order ───────────────────────────────────────────────────────────────── class Order: VALID_STATUSES = [ \u0026#34;Placed\u0026#34;, \u0026#34;Confirmed\u0026#34;, \u0026#34;Preparing\u0026#34;, \u0026#34;Out for Delivery\u0026#34;, \u0026#34;Delivered\u0026#34;, \u0026#34;Cancelled\u0026#34; ] _order_counter = 0 def __init__(self, customer_name, restaurant_name, items, delivery_address): Order._order_counter += 1 self.order_id = f\u0026#34;ORD-{Order._order_counter:04d}\u0026#34; self.customer_name = customer_name self.restaurant_name = restaurant_name self.items = list(items) # copy, not reference self.delivery_address = delivery_address self.order_time = datetime.now() self.status = \u0026#34;Placed\u0026#34; @property def subtotal(self): return sum(ci.subtotal for ci in self.items) @property def delivery_fee(self): return 0.0 if self.subtotal \u0026gt;= 300 else 40.0 @property def gst(self): return round(self.subtotal * 0.05, 2) @property def total(self): return round(self.subtotal + self.delivery_fee + self.gst, 2) def update_status(self, new_status): if new_status not in self.VALID_STATUSES: raise ValueError( f\u0026#34;\u0026#39;{new_status}\u0026#39; is not a valid status. \u0026#34; f\u0026#34;Choose from: {self.VALID_STATUSES}\u0026#34; ) self.status = new_status print(f\u0026#34;Order {self.order_id} status updated to: {self.status}\u0026#34;) # CONCEPT: classmethod as alternative constructor @classmethod def from_cart(cls, cart, delivery_address): \u0026#34;\u0026#34;\u0026#34;Create an Order from a Cart object.\u0026#34;\u0026#34;\u0026#34; if cart.is_empty: raise OrderError(\u0026#34;Cannot create an order from an empty cart.\u0026#34;) return cls( customer_name=cart.customer_name, restaurant_name=cart.restaurant_name, items=cart._items, delivery_address=delivery_address, ) def __str__(self): time_str = self.order_time.strftime(\u0026#34;%d %b %Y, %H:%M\u0026#34;) lines = [ \u0026#34;=\u0026#34; * 45, \u0026#34;ORDER RECEIPT\u0026#34;, f\u0026#34; Order ID : {self.order_id}\u0026#34;, f\u0026#34; Customer : {self.customer_name}\u0026#34;, f\u0026#34; Restaurant : {self.restaurant_name}\u0026#34;, f\u0026#34; Time : {time_str}\u0026#34;, f\u0026#34; Deliver to : {self.delivery_address}\u0026#34;, \u0026#34;-\u0026#34; * 45, ] for ci in self.items: lines.append(f\u0026#34; {ci.item.name:\u0026lt;20} x{ci.quantity:\u0026lt;2} Rs.{ci.subtotal:.0f}\u0026#34;) lines.extend([ \u0026#34;-\u0026#34; * 45, f\u0026#34; {\u0026#39;Subtotal\u0026#39;:\u0026lt;28}: Rs.{self.subtotal:.2f}\u0026#34;, f\u0026#34; {\u0026#39;Delivery Fee\u0026#39;:\u0026lt;28}: Rs.{self.delivery_fee:.2f}\u0026#34;, f\u0026#34; {\u0026#39;GST (5%)\u0026#39;:\u0026lt;28}: Rs.{self.gst:.2f}\u0026#34;, f\u0026#34; {\u0026#39;Total\u0026#39;:\u0026lt;28}: Rs.{self.total:.2f}\u0026#34;, \u0026#34;-\u0026#34; * 45, f\u0026#34; Status: {self.status}\u0026#34;, \u0026#34;=\u0026#34; * 45, ]) return \u0026#34;\\n\u0026#34;.join(lines) # ─── Demo ────────────────────────────────────────────────────────────────── if __name__ == \u0026#34;__main__\u0026#34;: dosa = MenuItem(\u0026#34;Masala Dosa\u0026#34;, 80, \u0026#34;Main\u0026#34;) coffee = MenuItem(\u0026#34;Filter Coffee\u0026#34;, 40, \u0026#34;Beverage\u0026#34;) idli = MenuItem(\u0026#34;Idli Sambar\u0026#34;, 60, \u0026#34;Main\u0026#34;) gulab = MenuItem(\u0026#34;Gulab Jamun\u0026#34;, 50, \u0026#34;Dessert\u0026#34;) vada = MenuItem(\u0026#34;Medu Vada\u0026#34;, 45, \u0026#34;Starter\u0026#34;) # CONCEPT: Context manager -- \u0026#39;with\u0026#39; ensures clean entry/exit with Cart(\u0026#34;Aarav Sharma\u0026#34;, \u0026#34;Udupi Palace\u0026#34;) as cart: cart.add_item(dosa, 2) cart.add_item(coffee, 1) cart.add_item(idli, 1) cart.add_item(dosa, 1) # increases dosa to 3, no duplicate print(f\u0026#34;Distinct items: {len(cart)}\u0026#34;) # 3 print(f\u0026#34;Total items : {cart.item_count}\u0026#34;) # 5 print(f\u0026#34;Cart total : Rs.{cart.total}\u0026#34;) # 340 order = Order.from_cart(cart, \u0026#34;42, Koregaon Park, Pune\u0026#34;) print(order) order.update_status(\u0026#34;Confirmed\u0026#34;) order.update_status(\u0026#34;Preparing\u0026#34;) # Sort menu by price (uses __lt__) menu = [dosa, coffee, idli, gulab, vada] print(\u0026#34;\\nMenu sorted by price:\u0026#34;) for item in sorted(menu): print(f\u0026#34; {item}\u0026#34;) # Combine two carts (uses __add__) cart2 = Cart(\u0026#34;Priya Patel\u0026#34;, \u0026#34;Udupi Palace\u0026#34;) cart2.add_item(gulab, 2) cart2.add_item(vada, 1) combined = cart + cart2 print(f\u0026#34;\\nCombined cart total: Rs.{combined.total}\u0026#34;) Solution 4 \u0026ndash; Student Grade Management System import json from datetime import datetime from dataclasses import dataclass, field from abc import ABC # ─── Mixins ──────────────────────────────────────────────────────────────── # CONCEPT: Mixins -- small, focused, reusable classes that add a specific # piece of behaviour. They are not meant to be used standalone. # This is a form of INHERITANCE used for composition of behaviours. class SerializableMixin: \u0026#34;\u0026#34;\u0026#34;Adds JSON serialisation to any class.\u0026#34;\u0026#34;\u0026#34; def to_dict(self): \u0026#34;\u0026#34;\u0026#34;Return a dict of all non-private attributes.\u0026#34;\u0026#34;\u0026#34; return { k: v for k, v in self.__dict__.items() if not k.startswith(\u0026#34;_\u0026#34;) } def to_json(self): \u0026#34;\u0026#34;\u0026#34;Return a JSON string representation.\u0026#34;\u0026#34;\u0026#34; def default_handler(obj): if isinstance(obj, datetime): return obj.isoformat() if hasattr(obj, \u0026#34;to_dict\u0026#34;): return obj.to_dict() return str(obj) return json.dumps(self.to_dict(), indent=2, default=default_handler) class TimestampMixin: \u0026#34;\u0026#34;\u0026#34;Adds creation and update timestamps to any class.\u0026#34;\u0026#34;\u0026#34; def __init__(self, *args, **kwargs): super().__init__(*args, **kwargs) self.created_at = datetime.now() self.updated_at = datetime.now() def touch(self): \u0026#34;\u0026#34;\u0026#34;Update the updated_at timestamp.\u0026#34;\u0026#34;\u0026#34; self.updated_at = datetime.now() # ─── GradeRecord ─────────────────────────────────────────────────────────── # CONCEPT: Dataclass -- reduces boilerplate for data-holding classes # @dataclass auto-generates __init__, __repr__, __eq__ # With order=True it also generates __lt__, __le__, __gt__, __ge__ @dataclass(order=True) class GradeRecord: \u0026#34;\u0026#34;\u0026#34;A single grade entry for one subject.\u0026#34;\u0026#34;\u0026#34; # sort_key is used for ordering -- we want to sort by percentage # field with init=False means it is not a constructor parameter sort_index: float = field(init=False, repr=False) subject: str marks_obtained: float max_marks: float = 100.0 exam_type: str = \u0026#34;Final\u0026#34; def __post_init__(self): \u0026#34;\u0026#34;\u0026#34;Called after __init__ -- compute sort_index.\u0026#34;\u0026#34;\u0026#34; self.sort_index = self.percentage @property def percentage(self): return round(self.marks_obtained / self.max_marks * 100, 2) @property def grade_letter(self): p = self.percentage if p \u0026gt;= 90: return \u0026#34;A+\u0026#34; if p \u0026gt;= 80: return \u0026#34;A\u0026#34; if p \u0026gt;= 70: return \u0026#34;B\u0026#34; if p \u0026gt;= 60: return \u0026#34;C\u0026#34; if p \u0026gt;= 50: return \u0026#34;D\u0026#34; return \u0026#34;F\u0026#34; # ─── Student ─────────────────────────────────────────────────────────────── # CONCEPT: Multiple inheritance -- Student inherits from both mixins # CONCEPT: __slots__ -- replaces __dict__ with fixed memory slots # This reduces memory per object significantly for large numbers of students class Student(SerializableMixin, TimestampMixin): # CONCEPT: __slots__ -- explicit list of allowed attributes # This reduces memory usage and slightly speeds up attribute access __slots__ = ( \u0026#34;name\u0026#34;, \u0026#34;student_id\u0026#34;, \u0026#34;department\u0026#34;, \u0026#34;year\u0026#34;, \u0026#34;_grade_records\u0026#34;, \u0026#34;created_at\u0026#34;, \u0026#34;updated_at\u0026#34; ) # Class-level counter for auto-generating IDs _counter = 0 def __init__(self, name, department, year): # CONCEPT: Cooperative multiple inheritance using super() # TimestampMixin.__init__ is called through the MRO chain super().__init__() Student._counter += 1 year_str = datetime.now().year self.name = name self.student_id = f\u0026#34;STU-{year_str}-{Student._counter:04d}\u0026#34; self.department = department self.year = year self._grade_records = [] # CONCEPT: Encapsulation -- private list def add_grade(self, grade_record): \u0026#34;\u0026#34;\u0026#34;Add a GradeRecord. Raises TypeError if wrong type.\u0026#34;\u0026#34;\u0026#34; if not isinstance(grade_record, GradeRecord): raise TypeError( f\u0026#34;Expected GradeRecord, got {type(grade_record).__name__}\u0026#34; ) self._grade_records.append(grade_record) self.touch() # update timestamp via TimestampMixin @property def grades(self): \u0026#34;\u0026#34;\u0026#34;Return a copy -- callers cannot modify the internal list.\u0026#34;\u0026#34;\u0026#34; return list(self._grade_records) @property def cgpa(self): \u0026#34;\u0026#34;\u0026#34;Average percentage on a 10-point scale.\u0026#34;\u0026#34;\u0026#34; if not self._grade_records: return 0.0 avg_pct = sum(g.percentage for g in self._grade_records) / len(self._grade_records) return round(avg_pct / 10, 2) @property def top_subject(self): \u0026#34;\u0026#34;\u0026#34;GradeRecord with the highest percentage.\u0026#34;\u0026#34;\u0026#34; return max(self._grade_records, key=lambda g: g.percentage) \\ if self._grade_records else None @property def failed_subjects(self): \u0026#34;\u0026#34;\u0026#34;List of GradeRecords where grade_letter is F.\u0026#34;\u0026#34;\u0026#34; return [g for g in self._grade_records if g.grade_letter == \u0026#34;F\u0026#34;] def generate_transcript(self): \u0026#34;\u0026#34;\u0026#34;Print a formatted academic transcript.\u0026#34;\u0026#34;\u0026#34; print(\u0026#34;\\n\u0026#34; + \u0026#34;=\u0026#34; * 46) print(\u0026#34;ACADEMIC TRANSCRIPT\u0026#34;) print(f\u0026#34; Name : {self.name}\u0026#34;) print(f\u0026#34; ID : {self.student_id}\u0026#34;) print(f\u0026#34; Department : {self.department}\u0026#34;) print(f\u0026#34; Year : {self.year}\u0026#34;) print(f\u0026#34; CGPA : {self.cgpa:.2f} / 10.00\u0026#34;) print(\u0026#34;-\u0026#34; * 46) print(f\u0026#34; {\u0026#39;Subject\u0026#39;:\u0026lt;24} {\u0026#39;Marks\u0026#39;:\u0026gt;7} {\u0026#39;Grade\u0026#39;}\u0026#34;) print(\u0026#34;-\u0026#34; * 46) for g in self._grade_records: marks_str = f\u0026#34;{g.marks_obtained:.0f}/{g.max_marks:.0f}\u0026#34; print(f\u0026#34; {g.subject:\u0026lt;24} {marks_str:\u0026gt;7} {g.grade_letter:\u0026lt;3} ({g.exam_type})\u0026#34;) print(\u0026#34;-\u0026#34; * 46) if self.top_subject: print(f\u0026#34; Top Subject: {self.top_subject.subject} \u0026#34; f\u0026#34;({self.top_subject.percentage:.1f}%)\u0026#34;) if self.failed_subjects: failed = \u0026#34;, \u0026#34;.join(g.subject for g in self.failed_subjects) print(f\u0026#34; Failed : {failed}\u0026#34;) print(\u0026#34;=\u0026#34; * 46) # CONCEPT: Dunder methods for natural Python behaviour def __str__(self): return (f\u0026#34;Student({self.name}, ID: {self.student_id}, \u0026#34; f\u0026#34;{self.department}, CGPA: {self.cgpa})\u0026#34;) def __repr__(self): return f\u0026#34;Student(name={self.name!r}, id={self.student_id!r})\u0026#34; def __eq__(self, other): \u0026#34;\u0026#34;\u0026#34;Two students are equal if they have the same student_id.\u0026#34;\u0026#34;\u0026#34; if not isinstance(other, Student): return NotImplemented return self.student_id == other.student_id def __lt__(self, other): \u0026#34;\u0026#34;\u0026#34;Students are ordered by CGPA -- enables sorted(students).\u0026#34;\u0026#34;\u0026#34; if not isinstance(other, Student): return NotImplemented return self.cgpa \u0026lt; other.cgpa def __hash__(self): return hash(self.student_id) def to_dict(self): \u0026#34;\u0026#34;\u0026#34;Override SerializableMixin.to_dict() to handle __slots__.\u0026#34;\u0026#34;\u0026#34; return { \u0026#34;name\u0026#34;: self.name, \u0026#34;student_id\u0026#34;: self.student_id, \u0026#34;department\u0026#34;: self.department, \u0026#34;year\u0026#34;: self.year, \u0026#34;cgpa\u0026#34;: self.cgpa, \u0026#34;grades\u0026#34;: [ { \u0026#34;subject\u0026#34;: g.subject, \u0026#34;marks\u0026#34;: g.marks_obtained, \u0026#34;max\u0026#34;: g.max_marks, \u0026#34;grade\u0026#34;: g.grade_letter, \u0026#34;exam_type\u0026#34;: g.exam_type, } for g in self._grade_records ], } # ─── Course ──────────────────────────────────────────────────────────────── # CONCEPT: Composition -- Course HAS-A list of Students class Course: def __init__(self, course_code, course_name, credits, instructor): self.course_code = course_code self.course_name = course_name self.credits = credits self.instructor = instructor self._enrolled_students = [] def enroll(self, student): if student not in self._enrolled_students: self._enrolled_students.append(student) def unenroll(self, student): if student in self._enrolled_students: self._enrolled_students.remove(student) @property def average_cgpa(self): if not self._enrolled_students: return 0.0 return round( sum(s.cgpa for s in self._enrolled_students) / len(self._enrolled_students), 2 ) @property def top_student(self): return max(self._enrolled_students, key=lambda s: s.cgpa) \\ if self._enrolled_students else None def __len__(self): return len(self._enrolled_students) def __contains__(self, student): \u0026#34;\u0026#34;\u0026#34;Supports: student in course\u0026#34;\u0026#34;\u0026#34; return student in self._enrolled_students def __str__(self): return f\u0026#34;{self.course_code}: {self.course_name} ({len(self)} students)\u0026#34; # ─── Department ──────────────────────────────────────────────────────────── # CONCEPT: Composition -- Department HAS-A dict of Courses and Students class Department: def __init__(self, name, head): self.name = name self.head = head self._courses = {} # course_code: Course self._students = {} # student_id: Student def add_course(self, course): self._courses[course.course_code] = course def add_student(self, student): self._students[student.student_id] = student def generate_report(self): \u0026#34;\u0026#34;\u0026#34;Print a complete department-wide report.\u0026#34;\u0026#34;\u0026#34; students = list(self._students.values()) if not students: print(f\u0026#34;\\n{self.name}: No students registered.\u0026#34;) return avg_cgpa = round(sum(s.cgpa for s in students) / len(students), 2) topper = max(students, key=lambda s: s.cgpa) at_risk = [s for s in students if s.cgpa \u0026lt; 5.0] # Grade distribution across all grade records all_grades = [g.grade_letter for s in students for g in s.grades] distribution = {} for letter in [\u0026#34;A+\u0026#34;, \u0026#34;A\u0026#34;, \u0026#34;B\u0026#34;, \u0026#34;C\u0026#34;, \u0026#34;D\u0026#34;, \u0026#34;F\u0026#34;]: distribution[letter] = all_grades.count(letter) print(f\u0026#34;\\n{\u0026#39;=\u0026#39; * 50}\u0026#34;) print(f\u0026#34; DEPARTMENT REPORT: {self.name.upper()}\u0026#34;) print(f\u0026#34; Head: {self.head}\u0026#34;) print(f\u0026#34;{\u0026#39;=\u0026#39; * 50}\u0026#34;) print(f\u0026#34; Total Students : {len(students)}\u0026#34;) print(f\u0026#34; Total Courses : {len(self._courses)}\u0026#34;) print(f\u0026#34; Avg CGPA : {avg_cgpa} / 10\u0026#34;) print(f\u0026#34; Topper : {topper.name} ({topper.cgpa})\u0026#34;) print(f\u0026#34; At Risk (\u0026lt;5.0) : {len(at_risk)}\u0026#34;) print(f\u0026#34;\\n Grade Distribution:\u0026#34;) for letter, count in distribution.items(): bar = \u0026#34;#\u0026#34; * count print(f\u0026#34; {letter:\u0026gt;2} : {bar:\u0026lt;20} ({count})\u0026#34;) print(f\u0026#34;{\u0026#39;=\u0026#39; * 50}\u0026#34;) # ─── Demo ────────────────────────────────────────────────────────────────── if __name__ == \u0026#34;__main__\u0026#34;: cs_dept = Department(\u0026#34;Computer Science\u0026#34;, \u0026#34;Dr. Ramesh Kumar\u0026#34;) python_course = Course(\u0026#34;CS101\u0026#34;, \u0026#34;Python Programming\u0026#34;, credits=4, instructor=\u0026#34;Prof. Priya Nair\u0026#34;) ml_course = Course(\u0026#34;CS301\u0026#34;, \u0026#34;Machine Learning\u0026#34;, credits=4, instructor=\u0026#34;Prof. Vikram Sharma\u0026#34;) cs_dept.add_course(python_course) cs_dept.add_course(ml_course) s1 = Student(\u0026#34;Aarav Sharma\u0026#34;, \u0026#34;Computer Science\u0026#34;, year=2) s2 = Student(\u0026#34;Priya Patel\u0026#34;, \u0026#34;Computer Science\u0026#34;, year=2) s3 = Student(\u0026#34;Rohan Verma\u0026#34;, \u0026#34;Computer Science\u0026#34;, year=3) s1.add_grade(GradeRecord(\u0026#34;Python Programming\u0026#34;, 88)) s1.add_grade(GradeRecord(\u0026#34;Machine Learning\u0026#34;, 91)) s1.add_grade(GradeRecord(\u0026#34;Data Structures\u0026#34;, 79, exam_type=\u0026#34;Midterm\u0026#34;)) s2.add_grade(GradeRecord(\u0026#34;Python Programming\u0026#34;, 95)) s2.add_grade(GradeRecord(\u0026#34;Machine Learning\u0026#34;, 88)) s2.add_grade(GradeRecord(\u0026#34;Data Structures\u0026#34;, 72)) s3.add_grade(GradeRecord(\u0026#34;Python Programming\u0026#34;, 45)) # failed s3.add_grade(GradeRecord(\u0026#34;Data Structures\u0026#34;, 58)) python_course.enroll(s1) python_course.enroll(s2) python_course.enroll(s3) ml_course.enroll(s1) ml_course.enroll(s2) cs_dept.add_student(s1) cs_dept.add_student(s2) cs_dept.add_student(s3) # Transcript s1.generate_transcript() # Serialisation (CONCEPT: Polymorphism through mixin) # to_json() works on any class that uses SerializableMixin print(\u0026#34;\\nStudent JSON:\u0026#34;) print(s1.to_json()) # Sorting students by CGPA (uses __lt__) all_students = [s1, s2, s3] print(\u0026#34;\\nRanking (highest CGPA first):\u0026#34;) for rank, student in enumerate(sorted(all_students, reverse=True), 1): print(f\u0026#34; {rank}. {student.name} -- CGPA: {student.cgpa}\u0026#34;) # __contains__ on Course print(f\u0026#34;\\nAarav in Python course : {s1 in python_course}\u0026#34;) print(f\u0026#34;Rohan in ML course : {s3 in ml_course}\u0026#34;) # Department report cs_dept.generate_report() Concept Coverage Summary Problem Encapsulation Inheritance Polymorphism Abstraction Other 1 \u0026ndash; Library Properties, private attrs, custom exceptions No No No Class variables, __str__, __repr__ 2 \u0026ndash; Payroll Property validation 3 subclasses, super() run_payroll() handles all types ABC, @abstractmethod @classmethod, @staticmethod 3 \u0026ndash; Food Properties with validation, private _items Context manager protocol Sorted via __lt__ No Composition, dunder suite, __add__ 4 \u0026ndash; Grades __slots__, private lists, properties Mixins, multiple inheritance to_json() via mixin works on any class No Dataclass, @dataclass(order=True), MRO All four pillars appear across the four problems. No single problem uses all four \u0026ndash; real systems rarely do. The skill is knowing which pillar solves which design problem.\nCodeverra OOP Assignment | learn.codeverra.com\n","permalink":"https://blogs.codeverra.com/blog/python-12-oop-practice/","summary":"\u003ch1 id=\"object-oriented-programming--assignment\"\u003eObject-Oriented Programming \u0026ndash; Assignment\u003c/h1\u003e\n\u003ch3 id=\"four-real-world-problems-to-solve-using-oop\"\u003eFour Real-World Problems to Solve Using OOP\u003c/h3\u003e\n\u003chr\u003e\n\u003cblockquote\u003e\n\u003cp\u003eThis assignment covers OOP Basics and Intermediate concepts.\nFor each problem, read the description carefully, understand what the\nsystem needs to do, and design your classes before writing any code.\nPlanning on paper first is not optional \u0026ndash; it is part of the exercise.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003chr\u003e\n\u003ch2 id=\"how-to-approach-each-problem\"\u003eHow to Approach Each Problem\u003c/h2\u003e\n\u003cp\u003eBefore writing a single line of code for any problem:\u003c/p\u003e","tags":["python"],"title":"OOP Practice - Problems and Solutions"},{"content":"🔰 Pattern 3: Prefix Sum Table of Contents What is Prefix Sum? Building the Intuition Step by Step The Range Sum Formula - Why It Works Common Variations Template Code Problem Set Problem 1: Range Sum Query – Immutable Problem 2: Running Sum of 1D Array Problem 3: Find Pivot Index Problem 4: Subarray Sum Equals K Problem 5: Contiguous Array Problem 6: Product of Array Except Self Problem 7: Subarray Sums Divisible by K Problem 8: Range Sum Query 2D – Immutable Problem 9: Continuous Subarray Sum Problem 10: Count Number of Nice Subarrays Key Takeaways \u0026amp; Summary What is Prefix Sum? Consider this scenario: you have an array of numbers and someone keeps asking you \u0026ldquo;what\u0026rsquo;s the sum of elements from index i to index j?\u0026rdquo; - not once, but hundreds of times with different i and j values.\nEach time, you could loop from i to j and add everything up. That\u0026rsquo;s O(n) per query. If someone asks 1000 questions on an array of size 10,000, that\u0026rsquo;s 10 million operations.\nBut what if you could answer every such query in O(1) - constant time - after just one pass of preprocessing?\nThat\u0026rsquo;s what prefix sum does.\nPrefix Sum is a technique where you precompute a cumulative sum array, so that the sum of any subarray can be calculated in O(1) using simple subtraction.\nBuilding the Intuition Step by Step Let\u0026rsquo;s build the idea from scratch. No shortcuts - just following the logic.\nStep 1: What does a prefix sum array look like? Given an array:\nnums = [3, 1, 4, 1, 5, 9] index: 0 1 2 3 4 5 The prefix sum array prefix[i] stores the sum of all elements from the start up to (and including) index i:\nprefix[0] = 3 = 3 prefix[1] = 3 + 1 = 4 prefix[2] = 3 + 1 + 4 = 8 prefix[3] = 3 + 1 + 4 + 1 = 9 prefix[4] = 3 + 1 + 4 + 1 + 5 = 14 prefix[5] = 3 + 1 + 4 + 1 + 5 + 9 = 23 So:\nnums = [3, 1, 4, 1, 5, 9] prefix = [3, 4, 8, 9, 14, 23] Each entry is just the previous entry plus the current element:\nprefix[i] = prefix[i-1] + nums[i] That\u0026rsquo;s it. One pass through the array and we\u0026rsquo;re done.\nStep 2: How does this help with range sums? Let\u0026rsquo;s say we want the sum of elements from index 2 to index 4: nums[2] + nums[3] + nums[4].\nWe already know:\nprefix[4] = nums[0] + nums[1] + nums[2] + nums[3] + nums[4] = 14 prefix[1] = nums[0] + nums[1] = 4 If we subtract:\nprefix[4] - prefix[1] = 14 - 4 = 10 Let\u0026rsquo;s verify: nums[2] + nums[3] + nums[4] = 4 + 1 + 5 = 10 ✅\nWhat just happened? prefix[4] contains the sum of everything from the start up to index 4. We don\u0026rsquo;t want the part before index 2, so we subtract prefix[1] (the sum of everything before index 2). What\u0026rsquo;s left is exactly the sum from index 2 to 4.\nprefix[4]: [3 + 1 + 4 + 1 + 5] = sum of indices 0..4 prefix[1]: [3 + 1] = sum of indices 0..1 difference: [4 + 1 + 5] = sum of indices 2..4 ✅ Step 3: The general formula Sum of nums[left..right] = prefix[right] - prefix[left - 1]\nThis works because:\nprefix[right] = sum from index 0 to right prefix[left - 1] = sum from index 0 to left - 1 Subtracting removes the part we don\u0026rsquo;t want, leaving the sum from left to right But wait - what if left = 0? Then left - 1 = -1, which is out of bounds.\nTwo ways to handle this:\nOption A: Special case for left = 0\nif left == 0: range_sum = prefix[right] else: range_sum = prefix[right] - prefix[left - 1] Option B: Add a dummy 0 at the beginning of the prefix array (preferred)\n# prefix[0] = 0 (dummy) # prefix[1] = nums[0] # prefix[2] = nums[0] + nums[1] # ... # prefix[i+1] = sum of nums[0..i] # Then: sum(nums[left..right]) = prefix[right + 1] - prefix[left] # No special cases needed! Let\u0026rsquo;s see Option B in action:\nnums = [3, 1, 4, 1, 5, 9] index: 0 1 2 3 4 5 prefix = [0, 3, 4, 8, 9, 14, 23] index: 0 1 2 3 4 5 6 Now prefix has length n + 1, and:\nprefix[0] = 0 (dummy - sum of zero elements) prefix[i] = sum of nums[0..i-1] Range sum from index left to right:\nsum = prefix[right + 1] - prefix[left] Example: sum from index 2 to 4:\nprefix[5] - prefix[2] = 14 - 4 = 10 ✅ Example: sum from index 0 to 2:\nprefix[3] - prefix[0] = 8 - 0 = 8 Verify: 3 + 1 + 4 = 8 ✅ No special cases. The dummy zero handles everything.\nStep 4: Visualizing the subtraction Here\u0026rsquo;s another way to see it. Think of the prefix sum as a running total on a number line:\nIndex: start 0 1 2 3 4 5 | | | | | | Prefix: 0 3 4 8 9 14 23 |______|____|____|____|____|____| To get sum(2..4): 0 3 4 8 9 14 23 |___________________|____|____| ↑ prefix[2] = 4 ↑ prefix[5] = 14 14 - 4 = 10 = sum of indices 2, 3, 4 You\u0026rsquo;re essentially reading two points on a cumulative curve and taking the difference.\nStep 5: The power - answering queries in O(1) Without prefix sum: Every range sum query takes O(n) - loop and add.\nWith prefix sum: One O(n) preprocessing step, then every query is O(1) - one subtraction.\nScenario Without Prefix Sum With Prefix Sum Build time None O(n) - one pass Per query O(n) - loop i to j O(1) - one subtraction 1,000 queries on array of 10,000 10,000,000 ops 10,000 + 1,000 = 11,000 ops The Range Sum Formula - Why It Works Let\u0026rsquo;s prove this clearly so there\u0026rsquo;s no doubt.\nDefinition:\nprefix[0] = 0 prefix[k] = nums[0] + nums[1] + ... + nums[k-1] We want: sum(nums[left..right]) = nums[left] + nums[left+1] + ... + nums[right]\nWe know:\nprefix[right + 1] = nums[0] + nums[1] + ... + nums[left-1] + nums[left] + ... + nums[right] |___________________________| |__________________________________| prefix[left] what we want Therefore:\nprefix[right + 1] = prefix[left] + sum(nums[left..right]) Rearranging: sum(nums[left..right]) = prefix[right + 1] - prefix[left] This is not a trick or a pattern to memorize. It\u0026rsquo;s just the definition of cumulative sums and basic subtraction.\nCommon Variations Prefix sum appears in many disguises. Here are the main variations you\u0026rsquo;ll encounter:\n1. Standard Prefix Sum (Range Sum Queries) As we just discussed. Precompute cumulative sums, answer range queries in O(1).\n2. Prefix Sum + HashMap (Count Subarrays with Target Sum) Instead of answering \u0026ldquo;what\u0026rsquo;s the sum from i to j?\u0026rdquo;, we ask \u0026ldquo;how many subarrays have sum equal to k?\u0026rdquo;\nThe key insight: if prefix[right] - prefix[left] = k, then the subarray from left to right-1 has sum k. So for each right, we need to count how many previous prefix values equal prefix[right] - k. A hashmap makes this O(1) per position.\nThis is the single most important variation. It shows up in Problems 4, 5, 7, 9, and 10 below.\n3. Prefix Product (Product Queries) Same idea but with multiplication instead of addition. Used in \u0026ldquo;Product of Array Except Self.\u0026rdquo;\n4. 2D Prefix Sum (Range Sum in a Matrix) Extends the concept to two dimensions. The inclusion-exclusion principle replaces simple subtraction.\n5. Prefix XOR, Prefix Count, Prefix Anything The concept generalizes to any operation where you can \u0026ldquo;undo\u0026rdquo; the prefix by some inverse operation (subtraction for sums, division for products, XOR for XOR).\nTemplate Code Template 1: Building a Prefix Sum Array def build_prefix_sum(nums): \u0026#34;\u0026#34;\u0026#34; Build a prefix sum array with a leading 0. prefix[0] = 0 prefix[i] = nums[0] + nums[1] + ... + nums[i-1] Range sum of nums[left..right] = prefix[right+1] - prefix[left] Time Complexity: O(n) - single pass Space Complexity: O(n) - storing the prefix array \u0026#34;\u0026#34;\u0026#34; n = len(nums) prefix = [0] * (n + 1) for i in range(n): prefix[i + 1] = prefix[i] + nums[i] return prefix # Usage: # prefix = build_prefix_sum(nums) # sum from index left to right (inclusive) = prefix[right + 1] - prefix[left] Template 2: Prefix Sum + HashMap (Count Subarrays with Given Sum) def count_subarrays_with_sum(nums, target): \u0026#34;\u0026#34;\u0026#34; Count the number of subarrays whose sum equals target. Core Idea: As we compute the running prefix sum, we ask: \u0026#34;Is there a previous prefix value such that current_prefix - previous_prefix = target?\u0026#34; That means: \u0026#34;Is (current_prefix - target) in our hashmap of seen prefix values?\u0026#34; Time Complexity: O(n) - single pass Space Complexity: O(n) - hashmap stores prefix sums \u0026#34;\u0026#34;\u0026#34; prefix_count = {0: 1} # prefix sum 0 has been seen once (before the array starts) current_sum = 0 count = 0 for num in nums: current_sum += num # How many times have we seen (current_sum - target) as a prefix sum? # Each such occurrence means there\u0026#39;s a subarray ending here with sum = target needed = current_sum - target if needed in prefix_count: count += prefix_count[needed] # Record this prefix sum prefix_count[current_sum] = prefix_count.get(current_sum, 0) + 1 return count Understanding the HashMap Template - A Detailed Example This is worth spending time on, because it\u0026rsquo;s the foundation for many problems.\nnums = [1, 2, 3, -3, 1, 2], target = 3 Let\u0026#39;s trace through: Step 0 (before starting): prefix_count = {0: 1}, current_sum = 0 \u0026#34;We\u0026#39;ve seen prefix sum 0 once - this represents the empty prefix.\u0026#34; Step 1: num = 1 current_sum = 1 needed = 1 - 3 = -2 → not in prefix_count → count stays 0 prefix_count = {0:1, 1:1} Step 2: num = 2 current_sum = 3 needed = 3 - 3 = 0 → prefix_count[0] = 1 → count += 1 → count = 1 prefix_count = {0:1, 1:1, 3:1} What did we find? current_sum=3 and a previous prefix of 0 exists. That means: sum from index 0 to index 1 = 3 - 0 = 3 ✅ Subarray: [1, 2] Step 3: num = 3 current_sum = 6 needed = 6 - 3 = 3 → prefix_count[3] = 1 → count += 1 → count = 2 prefix_count = {0:1, 1:1, 3:1, 6:1} Found: sum from some earlier point to index 2 = 3. Subarray: [3] (prefix went from 3 to 6, difference is 3) Step 4: num = -3 current_sum = 3 needed = 3 - 3 = 0 → prefix_count[0] = 1 → count += 1 → count = 3 prefix_count = {0:1, 1:1, 3:2, 6:1} Subarray: [1, 2, 3, -3] (prefix went from 0 to 3, difference is 3) Note: prefix_count[3] is now 2 - we\u0026#39;ve seen prefix sum 3 twice. Step 5: num = 1 current_sum = 4 needed = 4 - 3 = 1 → prefix_count[1] = 1 → count += 1 → count = 4 prefix_count = {0:1, 1:1, 3:2, 6:1, 4:1} Subarray: [2, 3, -3, 1] (prefix went from 1 to 4) Step 6: num = 2 current_sum = 6 needed = 6 - 3 = 3 → prefix_count[3] = 2 → count += 2 → count = 6 prefix_count = {0:1, 1:1, 3:2, 6:2, 4:1} TWO subarrays found at once! Prefix sum 3 appeared at two different points, and from each of those points to here, the sum is 3. Subarrays: [3, -3, 1, 2] and [1, 2] Final count: 6 Why do we initialize {0: 1}?\nThe 0 represents the prefix sum before the array starts. Without it, we\u0026rsquo;d miss subarrays that start at index 0. In Step 2 above, we found that current_sum = 3 and needed = 0. The prefix sum 0 exists because of this initialization - it means \u0026ldquo;the subarray starting from the very beginning sums to 3.\u0026rdquo;\nIf we forgot to put {0: 1}, we\u0026rsquo;d miss the subarray [1, 2]. Try it yourself and see.\nProblem Set Difficulty Progression # Problem Difficulty Key Concept 1 Range Sum Query – Immutable Easy Basic prefix sum 2 Running Sum of 1D Array Easy Building prefix sums 3 Find Pivot Index Easy Left sum vs right sum 4 Subarray Sum Equals K Medium Prefix sum + hashmap 5 Contiguous Array Medium Transform + prefix sum + hashmap 6 Product of Array Except Self Medium Prefix and suffix products 7 Subarray Sums Divisible by K Medium Prefix sum + modular arithmetic 8 Range Sum Query 2D – Immutable Medium 2D prefix sum 9 Continuous Subarray Sum Medium Prefix sum + modular + hashmap 10 Count Number of Nice Subarrays Medium Transform + prefix sum + hashmap Problem 1: Range Sum Query – Immutable LeetCode Link: https://leetcode.com/problems/range-sum-query-immutable/\nProblem Statement Implement the NumArray class:\nNumArray(nums) - initializes with the array. sumRange(left, right) - returns the sum of nums[left..right] inclusive. sumRange will be called many times.\nExample:\nnums = [-2, 0, 3, -5, 2, -1] sumRange(0, 2) → -2 + 0 + 3 = 1 sumRange(2, 5) → 3 + (-5) + 2 + (-1) = -1 sumRange(0, 5) → -2 + 0 + 3 + (-5) + 2 + (-1) = -3 Clarifying Questions \u0026amp; Constraints The array does not change after initialization (immutable). sumRange will be called up to 10⁴ times - it needs to be fast. 0 \u0026lt;= left \u0026lt;= right \u0026lt; nums.length Approach Discussion Approach 1: Brute Force - Sum on every query\nLoop from left to right and add elements each time. Time: O(n) per query, Space: O(1) ❌ With many queries, this is too slow. Approach 2: Prefix Sum (Optimal) ✅\nPrecompute the prefix sum array once in the constructor. Answer each query in O(1) using the formula. Time: O(n) initialization + O(1) per query, Space: O(n) This is the textbook use case for prefix sum - exactly the scenario we described in the intuition section.\nCode (Both Solutions) # ============================================================ # APPROACH 1: Brute Force - O(n) per query # ============================================================ class NumArray_Brute: \u0026#34;\u0026#34;\u0026#34; Store the array and compute the sum from scratch on every query. Time Complexity: __init__: O(1) sumRange: O(n) per call Space Complexity: O(1) extra (just storing the reference) \u0026#34;\u0026#34;\u0026#34; def __init__(self, nums: list[int]): self.nums = nums def sumRange(self, left: int, right: int) -\u0026gt; int: total = 0 for i in range(left, right + 1): total += nums[i] return total # ============================================================ # APPROACH 2: Prefix Sum - O(1) per query ✅ # ============================================================ class NumArray: \u0026#34;\u0026#34;\u0026#34; Precompute prefix sums for O(1) range queries. We use the \u0026#34;leading zero\u0026#34; convention: prefix[0] = 0 prefix[i] = nums[0] + nums[1] + ... + nums[i-1] Then: sumRange(left, right) = prefix[right + 1] - prefix[left] Time Complexity: __init__: O(n) - one pass to build prefix array sumRange: O(1) - single subtraction Space Complexity: O(n) - storing the prefix array \u0026#34;\u0026#34;\u0026#34; def __init__(self, nums: list[int]): n = len(nums) self.prefix = [0] * (n + 1) # Build the prefix sum array for i in range(n): self.prefix[i + 1] = self.prefix[i] + nums[i] def sumRange(self, left: int, right: int) -\u0026gt; int: # Sum of nums[left..right] = prefix[right+1] - prefix[left] return self.prefix[right + 1] - self.prefix[left] Edge Cases Single element range: sumRange(2, 2) → just nums[2]. Entire array: sumRange(0, n-1) → prefix[n] - prefix[0] = total sum. Negative numbers: Works fine - subtraction handles negatives correctly. Dry Run nums = [-2, 0, 3, -5, 2, -1] Building prefix: prefix[0] = 0 prefix[1] = 0 + (-2) = -2 prefix[2] = -2 + 0 = -2 prefix[3] = -2 + 3 = 1 prefix[4] = 1 + (-5) = -4 prefix[5] = -4 + 2 = -2 prefix[6] = -2 + (-1) = -3 prefix = [0, -2, -2, 1, -4, -2, -3] Query: sumRange(0, 2) = prefix[3] - prefix[0] = 1 - 0 = 1 ✅ Query: sumRange(2, 5) = prefix[6] - prefix[2] = -3 - (-2) = -1 ✅ Query: sumRange(0, 5) = prefix[6] - prefix[0] = -3 - 0 = -3 ✅ Problem 2: Running Sum of 1D Array LeetCode Link: https://leetcode.com/problems/running-sum-of-1d-array/\nProblem Statement Given an array nums, return the running sum where runningSum[i] = sum(nums[0]..nums[i]).\nExample:\nInput: nums = [1, 2, 3, 4] Output: [1, 3, 6, 10] Clarifying Questions \u0026amp; Constraints This is literally computing the prefix sum array (without the leading zero). 1 \u0026lt;= nums.length \u0026lt;= 1000 Approach Discussion Approach 1: Create a new array\nCompute cumulative sums into a new array. Time: O(n), Space: O(n) Approach 2: In-place modification (Optimal) ✅\nModify nums directly: nums[i] += nums[i-1]. Time: O(n), Space: O(1) Code (Both Solutions) # ============================================================ # APPROACH 1: New array - O(n) time, O(n) space # ============================================================ def runningSum_new(nums: list[int]) -\u0026gt; list[int]: \u0026#34;\u0026#34;\u0026#34; Build a new array of running sums. Time Complexity: O(n) Space Complexity: O(n) \u0026#34;\u0026#34;\u0026#34; result = [0] * len(nums) result[0] = nums[0] for i in range(1, len(nums)): result[i] = result[i - 1] + nums[i] return result # ============================================================ # APPROACH 2: In-place - O(n) time, O(1) space ✅ # ============================================================ def runningSum(nums: list[int]) -\u0026gt; list[int]: \u0026#34;\u0026#34;\u0026#34; Compute running sum by modifying the array in-place. Each element becomes the sum of itself and everything before it. Time Complexity: O(n) - single pass Space Complexity: O(1) - modifying in place (no extra array) \u0026#34;\u0026#34;\u0026#34; for i in range(1, len(nums)): nums[i] += nums[i - 1] return nums Edge Cases Single element: [5] → [5]. All zeros: [0, 0, 0] → [0, 0, 0]. Negative numbers: [-1, 2, -3] → [-1, 1, -2]. Dry Run Input: nums = [1, 2, 3, 4] i=1: nums[1] = 2 + nums[0] = 2 + 1 = 3 → [1, 3, 3, 4] i=2: nums[2] = 3 + nums[1] = 3 + 3 = 6 → [1, 3, 6, 4] i=3: nums[3] = 4 + nums[2] = 4 + 6 = 10 → [1, 3, 6, 10] Output: [1, 3, 6, 10] ✅ Problem 3: Find Pivot Index LeetCode Link: https://leetcode.com/problems/find-pivot-index/\nProblem Statement Given an array nums, find the pivot index where the sum of all elements to the left equals the sum of all elements to the right. The element at the pivot index is not included in either side.\nReturn the leftmost pivot index. If none exists, return -1.\nExample:\nInput: nums = [1, 7, 3, 6, 5, 6] Output: 3 (left sum = 1+7+3 = 11, right sum = 5+6 = 11) Clarifying Questions \u0026amp; Constraints \u0026ldquo;Left of index 0\u0026rdquo; is empty → left sum = 0. \u0026ldquo;Right of last index\u0026rdquo; is empty → right sum = 0. Return the leftmost pivot if there are multiple. Approach Discussion Approach 1: Brute Force\nFor each index, compute left sum and right sum separately. Time: O(n²), Space: O(1) Approach 2: Total Sum - Prefix Sum Logic (Optimal) ✅\nCompute the total sum. As we scan left to right, maintain a running left_sum. At each index i: right_sum = total_sum - left_sum - nums[i]. If left_sum == right_sum, we found the pivot. Time: O(n), Space: O(1) The insight: we don\u0026rsquo;t actually need to build a prefix sum array. We just need the running left sum and can derive the right sum from the total.\nCode (Optimal Solution) def pivotIndex(nums: list[int]) -\u0026gt; int: \u0026#34;\u0026#34;\u0026#34; Find the pivot index where left sum equals right sum. Instead of building a full prefix sum array, we use: - left_sum: running sum of elements to the left of current index - right_sum = total_sum - left_sum - nums[i] We check if left_sum == right_sum at each position. Time Complexity: O(n) - two passes (one for total, one for scanning) Space Complexity: O(1) - only a few variables \u0026#34;\u0026#34;\u0026#34; total_sum = sum(nums) left_sum = 0 for i in range(len(nums)): # Right sum = everything except left_sum and nums[i] right_sum = total_sum - left_sum - nums[i] if left_sum == right_sum: return i # Found the pivot # Add nums[i] to left_sum for the next iteration left_sum += nums[i] return -1 # No pivot found Edge Cases Pivot at index 0: [2, 1, -1] → left sum = 0, right sum = 1+(-1) = 0 → return 0. Pivot at last index: [-1, 1, 2] → left sum = -1+1 = 0, right sum = 0 → return 2. No pivot: [1, 2, 3] → no index works → return -1. Single element: [1] → left sum = 0, right sum = 0 → return 0. Dry Run Input: nums = [1, 7, 3, 6, 5, 6] total_sum = 28 i=0: left_sum=0, right_sum=28-0-1=27, 0≠27 → left_sum=1 i=1: left_sum=1, right_sum=28-1-7=20, 1≠20 → left_sum=8 i=2: left_sum=8, right_sum=28-8-3=17, 8≠17 → left_sum=11 i=3: left_sum=11, right_sum=28-11-6=11, 11==11 ✅ → return 3 Output: 3 ✅ Problem 4: Subarray Sum Equals K LeetCode Link: https://leetcode.com/problems/subarray-sum-equals-k/\nProblem Statement Given an integer array nums and an integer k, return the total number of subarrays whose sum equals k.\nExample:\nInput: nums = [1, 1, 1], k = 2 Output: 2 (subarrays: [1,1] starting at index 0, and [1,1] starting at index 1) Clarifying Questions \u0026amp; Constraints Subarrays are contiguous. Elements can be negative (so sliding window doesn\u0026rsquo;t work here!). We need the count, not the actual subarrays. 1 \u0026lt;= nums.length \u0026lt;= 2 × 10⁴ Approach Discussion Approach 1: Brute Force\nCheck every subarray. Time: O(n²), Space: O(1) Approach 2: Prefix Sum + HashMap (Optimal) ✅\nThis is exactly the pattern from Template 2 above. Track current_sum (running prefix sum). For each position, check how many times current_sum - k has appeared before. Time: O(n), Space: O(n) Why can\u0026rsquo;t we use sliding window? Sliding window works for \u0026ldquo;min/max length subarray with sum ≥ target\u0026rdquo; only when all elements are positive. With negatives, growing the window can decrease the sum and shrinking can increase it, so the window logic breaks. Prefix sum + hashmap handles negatives perfectly.\nCode (Both Solutions) # ============================================================ # APPROACH 1: Brute Force - O(n²) # ============================================================ def subarraySum_brute(nums: list[int], k: int) -\u0026gt; int: \u0026#34;\u0026#34;\u0026#34; Check every subarray. Time Complexity: O(n²) Space Complexity: O(1) \u0026#34;\u0026#34;\u0026#34; count = 0 n = len(nums) for i in range(n): current_sum = 0 for j in range(i, n): current_sum += nums[j] if current_sum == k: count += 1 return count # ============================================================ # APPROACH 2: Prefix Sum + HashMap - O(n) ✅ # ============================================================ def subarraySum(nums: list[int], k: int) -\u0026gt; int: \u0026#34;\u0026#34;\u0026#34; Count subarrays with sum equal to k using prefix sum + hashmap. Core Idea: If prefix_sum at index j minus prefix_sum at index i equals k, then the subarray from i+1 to j has sum k. So for each position j, we need: how many earlier prefix sums equal (current_prefix - k)? A hashmap gives us this in O(1). Why we initialize {0: 1}: This accounts for subarrays starting at index 0. If current_sum == k at some point, then current_sum - k = 0, and we need to find that prefix_sum 0 exists (it does - before the array starts). Time Complexity: O(n) - single pass Space Complexity: O(n) - hashmap of prefix sums \u0026#34;\u0026#34;\u0026#34; prefix_count = {0: 1} # prefix_sum → how many times we\u0026#39;ve seen it current_sum = 0 count = 0 for num in nums: # Extend the prefix sum current_sum += num # How many earlier prefix sums equal current_sum - k? needed = current_sum - k if needed in prefix_count: count += prefix_count[needed] # Record this prefix sum prefix_count[current_sum] = prefix_count.get(current_sum, 0) + 1 return count Edge Cases Single element equals k: nums=[3], k=3 → 1. Negative numbers: nums=[-1, -1, 1], k=0 → 1 (subarray [-1, 1]). k = 0: nums=[0, 0, 0] → 6 (every subarray of zeros sums to 0). No valid subarray: nums=[1, 2, 3], k=10 → 0. Dry Run Input: nums = [1, 2, 3], k = 3 prefix_count = {0: 1}, current_sum = 0, count = 0 num=1: current_sum = 1 needed = 1 - 3 = -2 → not in map → count = 0 prefix_count = {0:1, 1:1} num=2: current_sum = 3 needed = 3 - 3 = 0 → prefix_count[0] = 1 → count = 1 prefix_count = {0:1, 1:1, 3:1} Found: subarray [1, 2] sums to 3 num=3: current_sum = 6 needed = 6 - 3 = 3 → prefix_count[3] = 1 → count = 2 prefix_count = {0:1, 1:1, 3:1, 6:1} Found: subarray [3] sums to 3 Output: 2 ✅ (subarrays: [1,2] and [3]) Problem 5: Contiguous Array LeetCode Link: https://leetcode.com/problems/contiguous-array/\nProblem Statement Given a binary array nums, find the maximum length of a contiguous subarray with an equal number of 0s and 1s.\nExample:\nInput: nums = [0, 1, 0, 0, 1, 1, 0] Output: 6 (subarray [0, 1, 0, 0, 1, 1] or [1, 0, 0, 1, 1, 0] → 3 zeros and 3 ones) Clarifying Questions \u0026amp; Constraints The array contains only 0s and 1s. We want the longest such subarray. Approach Discussion Approach 1: Brute Force\nCheck every subarray, count 0s and 1s. Time: O(n²), Space: O(1) Approach 2: Transform + Prefix Sum + HashMap (Optimal) ✅\nThe trick is a clever transformation: treat 0s as -1s. Then \u0026ldquo;equal number of 0s and 1s\u0026rdquo; becomes \u0026ldquo;subarray sum equals 0.\u0026rdquo;\nWhy? If we have three 1s and three 0s (treated as -1s): 1 + 1 + 1 + (-1) + (-1) + (-1) = 0.\nNow the problem is: find the longest subarray with sum 0. This is a variation of the prefix sum + hashmap pattern.\nKey insight for \u0026ldquo;longest\u0026rdquo; vs \u0026ldquo;count\u0026rdquo;:\n\u0026ldquo;Count subarrays with sum k\u0026rdquo; → store the count of each prefix sum.\n\u0026ldquo;Longest subarray with sum k\u0026rdquo; → store the first occurrence of each prefix sum (earliest index). When we see the same prefix sum again, the subarray between those two positions has sum 0.\nTime: O(n), Space: O(n)\nCode (Optimal Solution) def findMaxLength(nums: list[int]) -\u0026gt; int: \u0026#34;\u0026#34;\u0026#34; Find the longest subarray with equal 0s and 1s. Transformation: Replace every 0 with -1. Now the problem becomes: find the longest subarray with sum = 0. If prefix_sum[i] == prefix_sum[j], then sum(nums[i+1..j]) = 0, meaning equal 0s and 1s in that range. We store the FIRST occurrence of each prefix sum. When we see the same prefix sum again, the distance is a candidate for the longest subarray. Why first occurrence? Because we want the LONGEST subarray, so we want the earliest start point. Time Complexity: O(n) - single pass Space Complexity: O(n) - hashmap stores prefix sums \u0026#34;\u0026#34;\u0026#34; # Map: prefix_sum → first index where this prefix_sum was seen first_seen = {0: -1} # prefix_sum 0 first seen at index -1 (before array starts) current_sum = 0 max_length = 0 for i in range(len(nums)): # Transform: treat 0 as -1 current_sum += 1 if nums[i] == 1 else -1 if current_sum in first_seen: # We\u0026#39;ve seen this prefix sum before! # The subarray from (first_seen[current_sum] + 1) to i has sum 0 length = i - first_seen[current_sum] max_length = max(max_length, length) else: # First time seeing this prefix sum - record the index first_seen[current_sum] = i return max_length Edge Cases All zeros or all ones: [0, 0, 0] → 0 (can never balance). Perfectly balanced: [0, 1] → 2. Already balanced: [0, 1, 0, 1] → 4. Empty-ish: [0] → 0. Dry Run Input: nums = [0, 1, 0, 0, 1, 1, 0] Transform 0→-1: [-1, 1, -1, -1, 1, 1, -1] first_seen = {0: -1}, current_sum = 0, max_length = 0 i=0: num=0 → current_sum = -1 -1 not in first_seen → first_seen[-1] = 0 i=1: num=1 → current_sum = 0 0 in first_seen (at index -1)! length = 1 - (-1) = 2, max_length = 2 i=2: num=0 → current_sum = -1 -1 in first_seen (at index 0)! length = 2 - 0 = 2, max_length = 2 i=3: num=0 → current_sum = -2 -2 not in first_seen → first_seen[-2] = 3 i=4: num=1 → current_sum = -1 -1 in first_seen (at index 0)! length = 4 - 0 = 4, max_length = 4 i=5: num=1 → current_sum = 0 0 in first_seen (at index -1)! length = 5 - (-1) = 6, max_length = 6 i=6: num=0 → current_sum = -1 -1 in first_seen (at index 0)! length = 6 - 0 = 6, max_length = 6 Output: 6 ✅ Problem 6: Product of Array Except Self LeetCode Link: https://leetcode.com/problems/product-of-array-except-self/\nProblem Statement Given an integer array nums, return an array answer where answer[i] is the product of all elements except nums[i]. You must solve it without using division and in O(n) time.\nExample:\nInput: nums = [1, 2, 3, 4] Output: [24, 12, 8, 6] Clarifying Questions \u0026amp; Constraints Cannot use division (so total_product / nums[i] is not allowed). Must run in O(n). Follow-up: Can you do it in O(1) extra space (not counting the output array)? Approach Discussion Approach 1: Division (Not Allowed)\nCompute total product, divide by each element. Problem: Division by zero if any element is 0. Also explicitly forbidden. Approach 2: Prefix and Suffix Products (Good)\nprefix_product[i] = product of all elements before index i. suffix_product[i] = product of all elements after index i. answer[i] = prefix_product[i] × suffix_product[i]. Time: O(n), Space: O(n) Approach 3: Single-Pass Optimization (Optimal) ✅\nFirst pass: build answer[i] as the prefix product (everything to the left). Second pass (right to left): multiply each answer[i] by the running suffix product. Time: O(n), Space: O(1) extra (output array doesn\u0026rsquo;t count). This is the prefix sum idea applied to multiplication: instead of cumulative sums, we compute cumulative products from both directions.\nCode (Both Approaches) # ============================================================ # APPROACH 2: Prefix and Suffix Arrays - O(n) time, O(n) space # ============================================================ def productExceptSelf_arrays(nums: list[int]) -\u0026gt; list[int]: \u0026#34;\u0026#34;\u0026#34; Build prefix product and suffix product arrays, then multiply. prefix[i] = product of nums[0] × nums[1] × ... × nums[i-1] suffix[i] = product of nums[i+1] × nums[i+2] × ... × nums[n-1] answer[i] = prefix[i] × suffix[i] Time Complexity: O(n) - three passes Space Complexity: O(n) - two extra arrays \u0026#34;\u0026#34;\u0026#34; n = len(nums) prefix = [1] * n suffix = [1] * n # Build prefix products (everything to the LEFT of i) for i in range(1, n): prefix[i] = prefix[i - 1] * nums[i - 1] # Build suffix products (everything to the RIGHT of i) for i in range(n - 2, -1, -1): suffix[i] = suffix[i + 1] * nums[i + 1] # Combine answer = [prefix[i] * suffix[i] for i in range(n)] return answer # ============================================================ # APPROACH 3: Two-Pass O(1) Extra Space - ✅ # ============================================================ def productExceptSelf(nums: list[int]) -\u0026gt; list[int]: \u0026#34;\u0026#34;\u0026#34; Compute product of array except self without division and with O(1) extra space. Pass 1 (left to right): Build prefix products into the answer array. answer[i] = product of all elements to the LEFT of i. Pass 2 (right to left): Multiply each answer[i] by the running suffix product. After this, answer[i] = left_product × right_product = everything except nums[i]. Time Complexity: O(n) - two passes Space Complexity: O(1) extra - only the output array and one variable \u0026#34;\u0026#34;\u0026#34; n = len(nums) answer = [1] * n # Pass 1: Fill answer with prefix products (left side) # answer[i] will contain the product of all elements BEFORE index i left_product = 1 for i in range(n): answer[i] = left_product left_product *= nums[i] # Pass 2: Multiply with suffix products (right side) # Multiply answer[i] by the product of all elements AFTER index i right_product = 1 for i in range(n - 1, -1, -1): answer[i] *= right_product right_product *= nums[i] return answer Edge Cases Contains zero: [1, 0, 3] → [0, 3, 0]. Multiple zeros: [0, 0, 3] → [0, 0, 0]. Two elements: [2, 3] → [3, 2]. Negative numbers: [-1, 2, -3] → [2×-3, -1×-3, -1×2] = [-6, 3, -2]. Dry Run Input: nums = [1, 2, 3, 4] Pass 1 (left to right): Build prefix products i=0: answer[0] = 1, left_product = 1*1 = 1 i=1: answer[1] = 1, left_product = 1*2 = 2 i=2: answer[2] = 2, left_product = 2*3 = 6 i=3: answer[3] = 6, left_product = 6*4 = 24 answer = [1, 1, 2, 6] (each entry = product of everything to its left) Pass 2 (right to left): Multiply by suffix products i=3: answer[3] = 6 * 1 = 6, right_product = 1*4 = 4 i=2: answer[2] = 2 * 4 = 8, right_product = 4*3 = 12 i=1: answer[1] = 1 * 12 = 12, right_product = 12*2 = 24 i=0: answer[0] = 1 * 24 = 24, right_product = 24*1 = 24 answer = [24, 12, 8, 6] ✅ Problem 7: Subarray Sums Divisible by K LeetCode Link: https://leetcode.com/problems/subarray-sums-divisible-by-k/\nProblem Statement Given an integer array nums and an integer k, return the number of subarrays whose sum is divisible by k.\nExample:\nInput: nums = [4, 5, 0, -2, -3, 1], k = 5 Output: 7 Subarrays: [4,5,0,-2,-3,1], [5], [5,0], [5,0,-2,-3], [0], [-2,-3], [4,5,0,-2,-3] Clarifying Questions \u0026amp; Constraints Elements can be negative. k is always positive. \u0026ldquo;Divisible by k\u0026rdquo; means the sum modulo k equals 0. Approach Discussion Approach 1: Brute Force\nCheck every subarray sum. Time: O(n²), Space: O(1) Approach 2: Prefix Sum + Modular Arithmetic + HashMap (Optimal) ✅\nThis is a variation of the \u0026ldquo;subarray sum equals k\u0026rdquo; pattern with a twist.\nKey math insight: If prefix[j] % k == prefix[i] % k, then (prefix[j] - prefix[i]) % k == 0, which means the subarray from i+1 to j has a sum divisible by k.\nSo instead of looking for prefix[j] - prefix[i] == target, we look for prefix[j] % k == prefix[i] % k. Group prefix sums by their remainder when divided by k.\nHandling negative remainders: In Python, -1 % 5 = 4 (Python handles this correctly). In some languages, you need ((prefix % k) + k) % k to ensure a non-negative remainder.\nTime: O(n), Space: O(k) Code (Both Solutions) # ============================================================ # APPROACH 1: Brute Force - O(n²) # ============================================================ def subarraysDivByK_brute(nums: list[int], k: int) -\u0026gt; int: \u0026#34;\u0026#34;\u0026#34; Check every subarray sum for divisibility by k. Time Complexity: O(n²) Space Complexity: O(1) \u0026#34;\u0026#34;\u0026#34; count = 0 n = len(nums) for i in range(n): current_sum = 0 for j in range(i, n): current_sum += nums[j] if current_sum % k == 0: count += 1 return count # ============================================================ # APPROACH 2: Prefix Sum + Modular Arithmetic - O(n) ✅ # ============================================================ def subarraysDivByK(nums: list[int], k: int) -\u0026gt; int: \u0026#34;\u0026#34;\u0026#34; Count subarrays with sum divisible by k using prefix sum remainders. Core Idea: If two prefix sums have the SAME remainder when divided by k, then the subarray between them has a sum divisible by k. Why? If prefix[j] % k == prefix[i] % k, then: (prefix[j] - prefix[i]) % k == 0 → sum of subarray from i+1 to j is divisible by k. So we count how many prefix sums share each remainder. If n prefix sums have the same remainder, we can choose any 2 of them to form a valid subarray → n*(n-1)/2 pairs. But we count incrementally. Time Complexity: O(n) - single pass Space Complexity: O(k) - at most k different remainders (0 to k-1) \u0026#34;\u0026#34;\u0026#34; # Map: remainder → how many prefix sums have this remainder remainder_count = {0: 1} # prefix_sum 0 has remainder 0 (seen once) current_sum = 0 count = 0 for num in nums: current_sum += num remainder = current_sum % k # In Python, this is always non-negative # How many previous prefix sums had the same remainder? if remainder in remainder_count: count += remainder_count[remainder] # Record this remainder remainder_count[remainder] = remainder_count.get(remainder, 0) + 1 return count Edge Cases All zeros: [0, 0, 0], k=5 → 6 (every subarray sums to 0, divisible by anything). Single element: [5], k=5 → 1. Negative numbers: [-1, 2, 9], k=2 → remainders are handled by Python\u0026rsquo;s modulo. k = 1: Every subarray has a sum divisible by 1 → n*(n+1)/2. Dry Run Input: nums = [4, 5, 0, -2, -3, 1], k = 5 remainder_count = {0: 1}, current_sum = 0, count = 0 num=4: current_sum = 4, remainder = 4%5 = 4 4 not in map → count = 0 map = {0:1, 4:1} num=5: current_sum = 9, remainder = 9%5 = 4 4 in map, count = 1 → count = 1 map = {0:1, 4:2} num=0: current_sum = 9, remainder = 9%5 = 4 4 in map (count=2), count += 2 → count = 3 map = {0:1, 4:3} num=-2: current_sum = 7, remainder = 7%5 = 2 2 not in map → count = 3 map = {0:1, 4:3, 2:1} num=-3: current_sum = 4, remainder = 4%5 = 4 4 in map (count=3), count += 3 → count = 6 map = {0:1, 4:4, 2:1} num=1: current_sum = 5, remainder = 5%5 = 0 0 in map (count=1), count += 1 → count = 7 map = {0:2, 4:4, 2:1} Output: 7 ✅ Problem 8: Range Sum Query 2D – Immutable LeetCode Link: https://leetcode.com/problems/range-sum-query-2d-immutable/\nProblem Statement Given a 2D matrix, handle multiple queries of the following type: calculate the sum of elements inside a rectangle defined by its upper-left corner (row1, col1) and lower-right corner (row2, col2).\nExample:\nMatrix: [3, 0, 1, 4, 2] [5, 6, 3, 2, 1] [1, 2, 0, 1, 5] [4, 1, 0, 1, 7] [1, 0, 3, 0, 5] sumRegion(2, 1, 4, 3) → 8 (sum of the rectangle from (2,1) to (4,3): 2+0+1 + 1+0+1 + 0+3+0 = 8) Clarifying Questions \u0026amp; Constraints The matrix does not change (immutable). sumRegion will be called many times. Extend the 1D prefix sum idea to 2D. Approach Discussion Approach 1: Brute Force\nFor each query, loop over the rectangle and sum. Time: O(m × n) per query. ❌ Approach 2: 2D Prefix Sum (Optimal) ✅\nBuild a 2D prefix sum matrix where prefix[i][j] = sum of all elements in the rectangle from (0,0) to (i-1, j-1). Use inclusion-exclusion to answer queries in O(1). Time: O(m × n) to build + O(1) per query. Space: O(m × n). Inclusion-Exclusion Principle:\nTo get the sum of a rectangle from (r1, c1) to (r2, c2):\nsum = prefix[r2+1][c2+1] (whole rectangle from origin to (r2,c2)) - prefix[r1][c2+1] (subtract the rows above) - prefix[r2+1][c1] (subtract the columns to the left) + prefix[r1][c1] (add back the corner we subtracted twice) This is exactly like the 1D formula but in two dimensions. In 1D, we subtract the prefix to the left. In 2D, we subtract the top and left regions, then add back the overlap (top-left corner) that was subtracted twice.\n+-----+-------+ | D | C | +-----+-------+ | B | query | +-----+-------+ sum(query) = sum(everything) - sum(C) - sum(B) + sum(D) (D was subtracted in both B and C, so add it back) Code (Optimal Solution) class NumMatrix: \u0026#34;\u0026#34;\u0026#34; 2D prefix sum for O(1) rectangular range queries. prefix[i][j] stores the sum of all elements in the rectangle from (0,0) to (i-1, j-1) in the original matrix. Time Complexity: __init__: O(m × n) to build the prefix matrix sumRegion: O(1) per query Space Complexity: O(m × n) for the prefix matrix \u0026#34;\u0026#34;\u0026#34; def __init__(self, matrix: list[list[int]]): if not matrix or not matrix[0]: return m, n = len(matrix), len(matrix[0]) # prefix has (m+1) rows and (n+1) cols, with a border of zeros self.prefix = [[0] * (n + 1) for _ in range(m + 1)] # Build the 2D prefix sum for i in range(1, m + 1): for j in range(1, n + 1): self.prefix[i][j] = ( matrix[i - 1][j - 1] # Current cell value + self.prefix[i - 1][j] # Sum of everything above + self.prefix[i][j - 1] # Sum of everything to the left - self.prefix[i - 1][j - 1] # Subtract overlap (counted twice) ) def sumRegion(self, row1: int, col1: int, row2: int, col2: int) -\u0026gt; int: \u0026#34;\u0026#34;\u0026#34; Return the sum of elements in the rectangle from (row1, col1) to (row2, col2). Uses inclusion-exclusion: sum = total - top - left + top_left_overlap \u0026#34;\u0026#34;\u0026#34; return ( self.prefix[row2 + 1][col2 + 1] # Full rectangle from origin - self.prefix[row1][col2 + 1] # Subtract rows above row1 - self.prefix[row2 + 1][col1] # Subtract columns left of col1 + self.prefix[row1][col1] # Add back the overlap ) Edge Cases Single cell: sumRegion(0, 0, 0, 0) → just that one cell. Entire matrix: sumRegion(0, 0, m-1, n-1) → prefix[m][n]. Single row/column: Works as a special case of the general formula. Dry Run Matrix: [3, 0, 1, 4, 2] [5, 6, 3, 2, 1] [1, 2, 0, 1, 5] [4, 1, 0, 1, 7] [1, 0, 3, 0, 5] Building prefix (showing the final result): [0, 0, 0, 0, 0, 0] [0, 3, 3, 4, 8, 10] [0, 8, 14, 18, 24, 27] [0, 9, 17, 21, 28, 36] [0, 13, 22, 26, 34, 49] [0, 14, 23, 30, 38, 58] Query: sumRegion(2, 1, 4, 3) = prefix[5][4] - prefix[2][4] - prefix[5][1] + prefix[2][1] = 38 - 24 - 14 + 8 = 8 ✅ Verify manually: Row 2: 2 + 0 + 1 = 3 Row 3: 1 + 0 + 1 = 2 Row 4: 0 + 3 + 0 = 3 Total: 3 + 2 + 3 = 8 ✅ Problem 9: Continuous Subarray Sum LeetCode Link: https://leetcode.com/problems/continuous-subarray-sum/\nProblem Statement Given an integer array nums and an integer k, return True if nums has a good subarray - a subarray of length at least 2 whose sum is a multiple of k.\nExample:\nInput: nums = [23, 2, 4, 6, 7], k = 6 Output: True (subarray [2, 4] has sum 6, which is 6×1) Clarifying Questions \u0026amp; Constraints Subarray must have length ≥ 2. A multiple of k includes 0 (since 0 = k × 0). k \u0026gt;= 1, elements are non-negative. Approach Discussion Approach 1: Brute Force\nCheck every subarray of length ≥ 2. Time: O(n²), Space: O(1) Approach 2: Prefix Sum + Modular Arithmetic + HashMap (Optimal) ✅\nSame concept as Problem 7 but with a twist: we need the subarray to have length ≥ 2.\nIf prefix[j] % k == prefix[i] % k and j - i \u0026gt;= 2, then the subarray from i+1 to j has sum divisible by k and length ≥ 2.\nStore the first index where each remainder was seen. If the same remainder appears again at an index ≥ 2 positions later, we found a valid subarray.\nTime: O(n), Space: O(min(n, k)) Code (Optimal Solution) def checkSubarraySum(nums: list[int], k: int) -\u0026gt; bool: \u0026#34;\u0026#34;\u0026#34; Check if there\u0026#39;s a subarray of length \u0026gt;= 2 with sum divisible by k. Strategy: Same as \u0026#34;subarray sums divisible by k\u0026#34; but we need length \u0026gt;= 2. Store the FIRST index where each remainder appears. When the same remainder appears again at index j, the subarray from (first_index + 1) to j has sum divisible by k. We check if j - first_index \u0026gt;= 2 (ensuring length \u0026gt;= 2). Why first index? We want the LONGEST gap to maximize our chance of length \u0026gt;= 2. First index gives the widest possible range. Time Complexity: O(n) - single pass Space Complexity: O(min(n, k)) - at most k distinct remainders \u0026#34;\u0026#34;\u0026#34; # Map: remainder → first index where this remainder appeared # Remainder 0 is first seen at \u0026#34;index -1\u0026#34; (before the array) remainder_first = {0: -1} current_sum = 0 for i in range(len(nums)): current_sum += nums[i] remainder = current_sum % k if remainder in remainder_first: # Same remainder seen before - check if the subarray is long enough if i - remainder_first[remainder] \u0026gt;= 2: return True # Don\u0026#39;t update the index - we want to keep the FIRST occurrence else: remainder_first[remainder] = i return False Edge Cases Length 1 subarray with sum divisible by k: [6], k=6 → False (length \u0026lt; 2). Two zeros: [0, 0], k=anything → True (sum 0 is a multiple of everything). k = 1: Any subarray of length ≥ 2 works (every integer sum is divisible by 1). Large k: [1, 2], k=100 → False (sum 3 is not divisible by 100). Dry Run Input: nums = [23, 2, 4, 6, 7], k = 6 remainder_first = {0: -1}, current_sum = 0 i=0: current_sum = 23, remainder = 23%6 = 5 5 not in map → remainder_first = {0:-1, 5:0} i=1: current_sum = 25, remainder = 25%6 = 1 1 not in map → remainder_first = {0:-1, 5:0, 1:1} i=2: current_sum = 29, remainder = 29%6 = 5 5 in map (first at index 0)! i - first = 2 - 0 = 2 ≥ 2 ✅ → return True The subarray is nums[1..2] = [2, 4], sum = 6, which is 6×1. ✅ Problem 10: Count Number of Nice Subarrays LeetCode Link: https://leetcode.com/problems/count-number-of-nice-subarrays/\nProblem Statement Given an array of integers nums and an integer k, return the number of nice subarrays - subarrays that contain exactly k odd numbers.\nExample:\nInput: nums = [1, 1, 2, 1, 1], k = 3 Output: 2 (subarrays: [1,1,2,1] and [1,2,1,1]) Clarifying Questions \u0026amp; Constraints We care about the count of odd numbers, not even numbers. Even numbers don\u0026rsquo;t affect the count but affect the subarray boundaries. 1 \u0026lt;= nums.length \u0026lt;= 50000 Approach Discussion Approach 1: Brute Force\nCheck every subarray, count odd numbers. Time: O(n²), Space: O(1) Approach 2: Transform + Prefix Sum + HashMap (Optimal) ✅\nTransform the problem: replace each number with 1 if odd, 0 if even. Now \u0026ldquo;exactly k odd numbers\u0026rdquo; becomes \u0026ldquo;subarray sum equals k.\u0026rdquo; This is exactly Problem 4 (Subarray Sum Equals K)!\nTime: O(n), Space: O(n) This is the same transformation trick we saw in Problem 5 (Contiguous Array), where we turned 0s into -1s. Here, we turn the \u0026ldquo;counting odds\u0026rdquo; problem into a \u0026ldquo;sum equals k\u0026rdquo; problem.\nCode (Optimal Solution) def numberOfSubarrays(nums: list[int], k: int) -\u0026gt; int: \u0026#34;\u0026#34;\u0026#34; Count subarrays with exactly k odd numbers. Transformation: Replace each number with 1 (odd) or 0 (even). Now the problem is: count subarrays with sum exactly k. This is identical to the \u0026#34;Subarray Sum Equals K\u0026#34; pattern. We don\u0026#39;t even need to physically transform the array - we just add (num % 2) to the running sum instead of the number itself. Time Complexity: O(n) - single pass Space Complexity: O(n) - hashmap of prefix sums \u0026#34;\u0026#34;\u0026#34; prefix_count = {0: 1} # prefix_sum 0 seen once (before array) current_sum = 0 # Counts odd numbers seen so far count = 0 for num in nums: # Add 1 if odd, 0 if even current_sum += num % 2 # How many earlier positions had (current_sum - k) odd numbers? needed = current_sum - k if needed in prefix_count: count += prefix_count[needed] prefix_count[current_sum] = prefix_count.get(current_sum, 0) + 1 return count Edge Cases All odd: [1, 1, 1], k=2 → 2 (subarrays [1,1] at indices 0-1 and 1-2). All even: [2, 4, 6], k=1 → 0 (no odd numbers at all). k = 0: Count subarrays with no odd numbers (all even). Single element: [1], k=1 → 1. Dry Run Input: nums = [1, 1, 2, 1, 1], k = 3 Transform: [1, 1, 0, 1, 1] (odd→1, even→0) Now count subarrays with sum = 3. prefix_count = {0: 1}, current_sum = 0, count = 0 num=1 (odd): current_sum = 1 needed = 1-3 = -2 → not in map → count = 0 map = {0:1, 1:1} num=1 (odd): current_sum = 2 needed = 2-3 = -1 → not in map → count = 0 map = {0:1, 1:1, 2:1} num=2 (even): current_sum = 2 needed = 2-3 = -1 → not in map → count = 0 map = {0:1, 1:1, 2:2} num=1 (odd): current_sum = 3 needed = 3-3 = 0 → map[0] = 1 → count = 1 map = {0:1, 1:1, 2:2, 3:1} Found: subarray from index 0 to 3 → [1, 1, 2, 1] has 3 odds ✅ num=1 (odd): current_sum = 4 needed = 4-3 = 1 → map[1] = 1 → count = 2 map = {0:1, 1:1, 2:2, 3:1, 4:1} Found: subarray from index 1 to 4 → [1, 2, 1, 1] has 3 odds ✅ Output: 2 ✅ Key Takeaways \u0026amp; Summary Quick Reference Table Problem Variation Time Space Core Trick Range Sum Query Basic prefix sum O(n) + O(1)/query O(n) prefix[r+1] - prefix[l] Running Sum Building prefix sums O(n) O(1) nums[i] += nums[i-1] Find Pivot Index Left sum vs right sum O(n) O(1) right_sum = total - left - nums[i] Subarray Sum = K Prefix + hashmap O(n) O(n) Count prefix - k in map Contiguous Array Transform (0→-1) + hashmap O(n) O(n) First occurrence of prefix sum Product Except Self Prefix \u0026amp; suffix products O(n) O(1) Two-pass: left products then right Sums Divisible by K Prefix + modular + hashmap O(n) O(k) Same remainder → divisible Range Sum 2D 2D prefix sum O(mn) + O(1)/query O(mn) Inclusion-exclusion Continuous Subarray Sum Modular + first index O(n) O(k) Same remainder, gap ≥ 2 Nice Subarrays Transform (odd→1) + hashmap O(n) O(n) Reduce to subarray sum = k The Three Flavors of Prefix Sum Problems What are you asked to find? 1. RANGE SUM QUERIES - \u0026#34;What\u0026#39;s the sum from index i to j?\u0026#34; └─ Build prefix array, answer in O(1) Formula: sum(i..j) = prefix[j+1] - prefix[i] 2. COUNT/FIND SUBARRAYS - \u0026#34;How many subarrays have sum = k / divisible by k?\u0026#34; └─ Prefix Sum + HashMap - For \u0026#34;sum = k\u0026#34;: store count of each prefix sum, look for (current - k) - For \u0026#34;divisible by k\u0026#34;: store count of each prefix sum % k (same remainder trick) - For \u0026#34;longest\u0026#34;: store first index of each prefix sum (not count) - Initialize {0: 1} (or {0: -1} for longest) 3. PREFIX/SUFFIX PRODUCTS - \u0026#34;Product of everything except self\u0026#34; └─ Two-pass: prefix products left-to-right, then suffix products right-to-left The HashMap Initialization - A Summary This trips up many people, so here\u0026rsquo;s a clear rule:\nProblem Type Initialize Why Count subarrays with sum = k {0: 1} Count 1 occurrence of prefix sum 0 (subarrays starting at index 0) Longest subarray with sum = k {0: -1} Prefix sum 0 first seen at index -1 (before array starts) Exists subarray with sum % k = 0 {0: -1} Remainder 0 first seen at index -1 The value in the map tells us:\nIf we\u0026rsquo;re counting → how many times this prefix sum appeared If we\u0026rsquo;re finding longest/exists → the first index where it appeared Common Transformation Tricks Several problems don\u0026rsquo;t look like prefix sum problems at first, but become one after a transformation:\nOriginal Problem Transformation Becomes Equal 0s and 1s 0 → -1 Subarray sum = 0 Exactly k odd numbers odd → 1, even → 0 Subarray sum = k At most k distinct (various) Sliding window (different pattern) The transformation step is often the hardest part. If a problem asks about counting specific elements in a subarray, try transforming it into a sum problem.\nWhat\u0026rsquo;s Next? With Prefix Sum covered, we\u0026rsquo;ve now completed three core patterns: Two Pointers, Sliding Window, and Prefix Sum. These three together handle a huge portion of array problems. Next up is Pattern 4: HashMap / Frequency Counting - a pattern that often works alongside prefix sums and shows up in almost every category of problems. Stay tuned!\n💡 Practice Tip: The \u0026ldquo;prefix sum + hashmap\u0026rdquo; combination is one of the highest-value patterns in interviews. If you master Problems 4, 5, 7, 9, and 10 from this set, you\u0026rsquo;ll be able to recognize and solve most subarray-sum variants. The key is always the same: \u0026ldquo;for the current prefix sum, how many (or where) have I seen current_sum - target before?\u0026rdquo;\n","permalink":"https://blogs.codeverra.com/blog/dsa-patterns-04-prefix-sum/","summary":"\u003ch1 id=\"-pattern-3-prefix-sum\"\u003e🔰 Pattern 3: Prefix Sum\u003c/h1\u003e\n\u003ch2 id=\"table-of-contents\"\u003eTable of Contents\u003c/h2\u003e\n\u003col\u003e\n\u003cli\u003e\u003ca href=\"#what-is-prefix-sum\"\u003eWhat is Prefix Sum?\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#building-the-intuition-step-by-step\"\u003eBuilding the Intuition Step by Step\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#the-range-sum-formula--why-it-works\"\u003eThe Range Sum Formula  -  Why It Works\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#common-variations\"\u003eCommon Variations\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#template-code\"\u003eTemplate Code\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#problem-set\"\u003eProblem Set\u003c/a\u003e\n\u003cul\u003e\n\u003cli\u003e\u003ca href=\"#problem-1-range-sum-query--immutable\"\u003eProblem 1: Range Sum Query – Immutable\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#problem-2-running-sum-of-1d-array\"\u003eProblem 2: Running Sum of 1D Array\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#problem-3-find-pivot-index\"\u003eProblem 3: Find Pivot Index\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#problem-4-subarray-sum-equals-k\"\u003eProblem 4: Subarray Sum Equals K\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#problem-5-contiguous-array\"\u003eProblem 5: Contiguous Array\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#problem-6-product-of-array-except-self\"\u003eProblem 6: Product of Array Except Self\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#problem-7-subarray-sums-divisible-by-k\"\u003eProblem 7: Subarray Sums Divisible by K\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#problem-8-range-sum-query-2d--immutable\"\u003eProblem 8: Range Sum Query 2D – Immutable\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#problem-9-continuous-subarray-sum\"\u003eProblem 9: Continuous Subarray Sum\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#problem-10-count-number-of-nice-subarrays\"\u003eProblem 10: Count Number of Nice Subarrays\u003c/a\u003e\u003c/li\u003e\n\u003c/ul\u003e\n\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#key-takeaways--summary\"\u003eKey Takeaways \u0026amp; Summary\u003c/a\u003e\u003c/li\u003e\n\u003c/ol\u003e\n\u003chr\u003e\n\u003ch2 id=\"what-is-prefix-sum\"\u003eWhat is Prefix Sum?\u003c/h2\u003e\n\u003cp\u003eConsider this scenario: you have an array of numbers and someone keeps asking you \u0026ldquo;what\u0026rsquo;s the sum of elements from index \u003ccode\u003ei\u003c/code\u003e to index \u003ccode\u003ej\u003c/code\u003e?\u0026rdquo;  -  not once, but hundreds of times with different \u003ccode\u003ei\u003c/code\u003e and \u003ccode\u003ej\u003c/code\u003e values.\u003c/p\u003e","tags":["dsa","dsa-patterns"],"title":"Prefix Sum Pattern  -  Complete Guide"},{"content":"Python Basics Masterclass Variables, Data Types, Strings, Operators, and Input/Output \u0026ndash; A Complete Foundation Who is this for? You are new to Python or just getting started. This is the very first thing to read and practice before moving on to collections, loops, functions, or anything else in the curriculum. Every concept is explained simply, with small Indian-context examples, before moving to the next.\nTable of Contents How Python Works Your First Python Program Comments Variables and Assignment Naming Rules and Conventions Python Keywords Numbers \u0026ndash; int, float, complex The math Module Booleans None \u0026ndash; The Absence of a Value Strings \u0026ndash; Complete Guide Operators Type Conversion Input and Output f-strings and String Formatting Practice Questions Solutions What Comes Next 1. How Python Works When you write Python code and run it, here is what happens behind the scenes:\nYour .py file (source code) | Python Interpreter | Bytecode (.pyc) | Python Virtual Machine (PVM) | Output on your screen Python is an interpreted language \u0026ndash; it reads and executes your code line by line, top to bottom. This is different from compiled languages like C++ where you compile the entire program first and then run it.\nThis means:\nYou see errors immediately at the line they happen You can run code interactively in Jupyter Notebook or the Python shell You do not need a compile step \u0026ndash; write, save, run Running Python # Run a script from terminal python my_script.py # Open interactive shell python # Run a single expression python -c \u0026#34;print(2 + 2)\u0026#34; 2. Your First Python Program print(\u0026#34;Namaste, World!\u0026#34;) Namaste, World! That is it. One line. No imports, no main function, no class required. Python\u0026rsquo;s simplicity is intentional.\n# A slightly more interesting first program name = \u0026#34;Aarav\u0026#34; city = \u0026#34;Bangalore\u0026#34; year = 2024 print(f\u0026#34;Hello! I am {name} from {city}.\u0026#34;) print(f\u0026#34;I started learning Python in {year}.\u0026#34;) Hello! I am Aarav from Bangalore. I started learning Python in 2024. 3. Comments Comments are notes in your code that Python completely ignores. They exist for humans reading the code.\nSingle-line comments # This is a comment print(\u0026#34;Hello\u0026#34;) # This is an inline comment # Always explain WHY, not just WHAT # BAD: x = x + 1 # add 1 to x # GOOD: x = x + 1 # increment page number before rendering Multi-line comments Python does not have a dedicated multi-line comment syntax. You can use multiple # lines or a multi-line string (triple quotes). The triple-quote version is technically a string that is not assigned to anything \u0026ndash; Python ignores it.\n# Line one of a long explanation # Line two of the same explanation # Line three \u0026#34;\u0026#34;\u0026#34; This is also used as a multi-line comment. Python evaluates it as a string but discards it because it is not assigned to a variable. \u0026#34;\u0026#34;\u0026#34; Docstrings \u0026ndash; documentation comments Docstrings are triple-quoted strings placed at the start of a function, class, or module. They are the official way to document your code and are accessible at runtime.\ndef add_gst(price): \u0026#34;\u0026#34;\u0026#34; Calculate price after adding 18% GST. Parameters: price (float): Base price of the product Returns: float: Price including GST \u0026#34;\u0026#34;\u0026#34; return round(price * 1.18, 2) print(add_gst.__doc__) # prints the docstring help(add_gst) # formatted help output Good comments and docstrings are a professional habit. Write them as if someone else will read your code tomorrow \u0026ndash; because they will, and that someone is often your future self.\n4. Variables and Assignment A variable is a named container that holds a value. In Python you do not declare variables \u0026ndash; you simply assign a value and Python creates the variable.\n# Assignment: name = value student_name = \u0026#34;Priya Sharma\u0026#34; age = 21 gpa = 8.7 is_enrolled = True How assignment works When you write age = 21, Python:\nCreates an integer object with value 21 in memory Creates a label age that points to that object The variable is just a label. You can move the label to a different object at any time.\nage = 21 # age points to integer 21 age = \u0026#34;twenty\u0026#34; # age now points to string \u0026#34;twenty\u0026#34; -- perfectly valid age = [1, 2, 3] # age now points to a list -- still valid This is what dynamic typing means \u0026ndash; the variable itself has no fixed type. The object it points to does.\nMultiple assignment # Assign the same value to multiple variables x = y = z = 0 # Assign multiple values in one line (tuple unpacking) name, city, score = \u0026#34;Rohan\u0026#34;, \u0026#34;Delhi\u0026#34;, 88 # Swap two variables -- elegant Python syntax a, b = 10, 20 a, b = b, a print(a, b) # 20 10 Augmented assignment operators score = 50 score += 10 # score = score + 10 --\u0026gt; 60 score -= 5 # score = score - 5 --\u0026gt; 55 score *= 2 # score = score * 2 --\u0026gt; 110 score //= 3 # score = score // 3 --\u0026gt; 36 score **= 2 # score = score ** 2 --\u0026gt; 1296 score %= 100 # score = score % 100 --\u0026gt; 96 Checking a variable\u0026rsquo;s type name = \u0026#34;Aarav\u0026#34; age = 21 gpa = 8.7 flag = True print(type(name)) # \u0026lt;class \u0026#39;str\u0026#39;\u0026gt; print(type(age)) # \u0026lt;class \u0026#39;int\u0026#39;\u0026gt; print(type(gpa)) # \u0026lt;class \u0026#39;float\u0026#39;\u0026gt; print(type(flag)) # \u0026lt;class \u0026#39;bool\u0026#39;\u0026gt; # isinstance() is the preferred way to check type in real code print(isinstance(age, int)) # True print(isinstance(gpa, float)) # True print(isinstance(name, str)) # True 5. Naming Rules and Conventions Rules \u0026ndash; these are enforced by Python # Valid names student_name = \u0026#34;Priya\u0026#34; _private = 42 CamelCase = True name2 = \u0026#34;Rohan\u0026#34; CONSTANT = 3.14 # Invalid names -- these cause SyntaxError 2name = \u0026#34;Aarav\u0026#34; # cannot start with a digit my-name = \u0026#34;Priya\u0026#34; # hyphens not allowed class = \u0026#34;10A\u0026#34; # cannot use a keyword my name = \u0026#34;Rohan\u0026#34; # spaces not allowed Conventions \u0026ndash; these are style standards (PEP 8) What Convention Example Regular variables snake_case student_name, total_marks Constants UPPER_SNAKE_CASE MAX_SCORE, TAX_RATE Functions snake_case calculate_gst(), get_name() Classes PascalCase StudentRecord, BankAccount Private variables leading underscore _internal_count \u0026ldquo;Don\u0026rsquo;t care\u0026rdquo; variable single underscore for _ in range(5): # Good naming -- self-documenting code monthly_revenue = 125000 gst_rate = 0.18 MAX_RETRIES = 3 # Bad naming -- cryptic mr = 125000 g = 0.18 x = 3 Naming well is one of the most valuable skills in programming. A good variable name makes the code read like a sentence and eliminates the need for a comment.\n6. Python Keywords Keywords are reserved words that have special meaning in Python. You cannot use them as variable names.\nimport keyword print(keyword.kwlist) [\u0026#39;False\u0026#39;, \u0026#39;None\u0026#39;, \u0026#39;True\u0026#39;, \u0026#39;and\u0026#39;, \u0026#39;as\u0026#39;, \u0026#39;assert\u0026#39;, \u0026#39;async\u0026#39;, \u0026#39;await\u0026#39;, \u0026#39;break\u0026#39;, \u0026#39;class\u0026#39;, \u0026#39;continue\u0026#39;, \u0026#39;def\u0026#39;, \u0026#39;del\u0026#39;, \u0026#39;elif\u0026#39;, \u0026#39;else\u0026#39;, \u0026#39;except\u0026#39;, \u0026#39;finally\u0026#39;, \u0026#39;for\u0026#39;, \u0026#39;from\u0026#39;, \u0026#39;global\u0026#39;, \u0026#39;if\u0026#39;, \u0026#39;import\u0026#39;, \u0026#39;in\u0026#39;, \u0026#39;is\u0026#39;, \u0026#39;lambda\u0026#39;, \u0026#39;nonlocal\u0026#39;, \u0026#39;not\u0026#39;, \u0026#39;or\u0026#39;, \u0026#39;pass\u0026#39;, \u0026#39;raise\u0026#39;, \u0026#39;return\u0026#39;, \u0026#39;try\u0026#39;, \u0026#39;while\u0026#39;, \u0026#39;with\u0026#39;, \u0026#39;yield\u0026#39;] You will learn what each of these does as you progress through the curriculum. For now, just know you cannot name a variable any of these words.\n7. Numbers \u0026ndash; int, float, complex Python has three numeric types built in.\nint \u0026ndash; integers # Positive and negative whole numbers -- no size limit in Python population = 1_428_627_663 # underscores for readability (Python 3.6+) temperature = -15 zero = 0 # Different bases binary = 0b1010 # binary -- value is 10 octal = 0o17 # octal -- value is 15 hex_val = 0xFF # hex -- value is 255 print(binary, octal, hex_val) # 10 15 255 # Convert to other bases print(bin(255)) # \u0026#39;0b11111111\u0026#39; print(oct(255)) # \u0026#39;0o377\u0026#39; print(hex(255)) # \u0026#39;0xff\u0026#39; float \u0026ndash; decimal numbers price = 299.99 pi = 3.14159 gst_rate = 0.18 # Scientific notation avogadro = 6.022e23 # 6.022 x 10^23 electron = 1.6e-19 # 1.6 x 10^-19 print(avogadro) # 6.022e+23 print(electron) # 1.6e-19 # Float precision -- a known quirk print(0.1 + 0.2) # 0.30000000000000004 (not exactly 0.3) print(round(0.1 + 0.2, 2)) # 0.3 (use round() for financial calculations) # For exact decimal arithmetic, use the decimal module from decimal import Decimal print(Decimal(\u0026#34;0.1\u0026#34;) + Decimal(\u0026#34;0.2\u0026#34;)) # 0.3 (exact) Arithmetic operators a, b = 17, 5 print(a + b) # 22 addition print(a - b) # 12 subtraction print(a * b) # 85 multiplication print(a / b) # 3.4 division -- always returns float print(a // b) # 3 floor division -- integer result, rounds down print(a % b) # 2 modulo -- remainder print(a ** b) # 1419857 exponentiation # Important: // always rounds towards negative infinity print( 17 // 5) # 3 print(-17 // 5) # -4 (not -3) print( 17 // -5) # -4 Operator precedence (BODMAS equivalent) # Python follows standard mathematical precedence # ** \u0026gt; unary - \u0026gt; * / // % \u0026gt; + - print(2 + 3 * 4) # 14, not 20 (multiplication before addition) print((2 + 3) * 4) # 20 (parentheses first) print(2 ** 3 ** 2) # 512 (** is right-associative: 2 ** (3**2) = 2**9) print(10 - 4 - 2) # 4 (left to right: (10-4)-2) complex \u0026ndash; complex numbers z1 = 3 + 4j z2 = 1 - 2j print(z1 + z2) # (4+2j) print(z1 * z2) # (11-2j) print(z1.real) # 3.0 print(z1.imag) # 4.0 print(abs(z1)) # 5.0 (magnitude: sqrt(3^2 + 4^2)) 8. The math Module The math module gives you mathematical functions beyond basic arithmetic.\nimport math # Constants print(math.pi) # 3.141592653589793 print(math.e) # 2.718281828459045 print(math.inf) # infinity print(math.tau) # 6.283... (2 * pi) # Rounding print(math.floor(4.9)) # 4 (round down) print(math.ceil(4.1)) # 5 (round up) print(round(4.5)) # 4 (built-in -- banker\u0026#39;s rounding) print(round(4.567, 2)) # 4.57 # Power and logarithm print(math.sqrt(144)) # 12.0 print(math.pow(2, 10)) # 1024.0 print(math.log(100, 10)) # 2.0 (log base 10 of 100) print(math.log(math.e)) # 1.0 (natural log) print(math.log2(1024)) # 10.0 # Absolute value print(abs(-42)) # 42 (built-in) print(math.fabs(-42)) # 42.0 (math module -- always returns float) # Trigonometry (input in radians) print(math.sin(math.pi / 2)) # 1.0 print(math.cos(0)) # 1.0 print(math.tan(math.pi / 4)) # 0.9999... (~1.0) # Convert degrees to radians and back print(math.degrees(math.pi)) # 180.0 print(math.radians(180)) # 3.14159... # Factorial and combinations print(math.factorial(5)) # 120 print(math.comb(10, 3)) # 120 (10 choose 3) print(math.perm(10, 3)) # 720 (10 P 3) # Greatest common divisor and least common multiple print(math.gcd(48, 18)) # 6 print(math.lcm(4, 6)) # 12 9. Booleans Booleans represent truth values. There are exactly two: True and False. Note the capital first letter \u0026ndash; Python is case-sensitive.\nis_logged_in = True has_paid = False is_eligible = True print(type(True)) # \u0026lt;class \u0026#39;bool\u0026#39;\u0026gt; print(type(False)) # \u0026lt;class \u0026#39;bool\u0026#39;\u0026gt; Booleans are integers In Python, bool is a subclass of int. True equals 1 and False equals 0.\nprint(True + True) # 2 print(True * 5) # 5 print(False + 10) # 10 print(True == 1) # True print(False == 0) # True # Useful trick: count True values in a list results = [True, False, True, True, False] print(sum(results)) # 3 (counts the Trues) Comparison operators \u0026ndash; always return a boolean score = 78 print(score \u0026gt; 60) # True print(score \u0026lt; 60) # False print(score == 78) # True -- equality check (note: ==, not =) print(score != 78) # False -- not equal print(score \u0026gt;= 78) # True -- greater than or equal print(score \u0026lt;= 77) # False -- less than or equal # Chained comparisons -- very Pythonic marks = 85 print(60 \u0026lt;= marks \u0026lt;= 100) # True -- between 60 and 100 inclusive print(0 \u0026lt; marks \u0026lt; 60) # False -- marks is not below 60 Logical operators age = 22 has_degree = True experience = 3 # and -- True only if BOTH are True print(age \u0026gt;= 21 and has_degree) # True print(age \u0026gt;= 21 and experience \u0026gt;= 5) # False # or -- True if AT LEAST ONE is True print(has_degree or experience \u0026gt;= 5) # True print(age \u0026lt; 18 or experience \u0026gt;= 5) # False # not -- reverses the boolean print(not has_degree) # False print(not False) # True # Short-circuit evaluation # and stops at the first False # or stops at the first True print(False and 1/0) # False (1/0 never evaluated -- short-circuit) print(True or 1/0) # True (1/0 never evaluated -- short-circuit) Truthy and Falsy values In Python, every value has a boolean meaning. This is used heavily in conditionals.\n# Falsy values -- evaluate to False in a boolean context # False, None, 0, 0.0, 0j, \u0026#34;\u0026#34;, [], (), {}, set() # Everything else is Truthy print(bool(0)) # False print(bool(\u0026#34;\u0026#34;)) # False print(bool([])) # False print(bool(None)) # False print(bool(1)) # True print(bool(\u0026#34;hi\u0026#34;)) # True print(bool([0])) # True -- list with one item, even if item is 0 # Used in conditions directly name = \u0026#34;\u0026#34; if name: print(f\u0026#34;Hello, {name}\u0026#34;) else: print(\u0026#34;No name provided\u0026#34;) # this runs because \u0026#34;\u0026#34; is falsy 10. None \u0026ndash; The Absence of a Value None is Python\u0026rsquo;s way of saying \u0026ldquo;nothing\u0026rdquo;, \u0026ldquo;missing\u0026rdquo;, or \u0026ldquo;not yet set\u0026rdquo;. It is not zero, not an empty string, not False \u0026ndash; it is the intentional absence of a value.\nresult = None response = None print(result) # None print(type(result)) # \u0026lt;class \u0026#39;NoneType\u0026#39;\u0026gt; print(result is None) # True -- use \u0026#39;is\u0026#39; to check for None, not == print(result == None) # True -- works but not recommended # Common patterns def find_student(name, students): for s in students: if s[\u0026#34;name\u0026#34;] == name: return s return None # explicit: did not find anything result = find_student(\u0026#34;Rohan\u0026#34;, [{\u0026#34;name\u0026#34;: \u0026#34;Priya\u0026#34;}]) if result is None: print(\u0026#34;Student not found\u0026#34;) # None as a default parameter (avoids the mutable default trap) def register(name, courses=None): if courses is None: courses = [] courses.append(\u0026#34;Python Basics\u0026#34;) return courses 11. Strings \u0026ndash; Complete Guide A string is a sequence of characters. In Python, strings are immutable \u0026ndash; once created, they cannot be changed in place. Any operation that seems to modify a string actually creates a new one.\nCreating strings # Single or double quotes -- both identical name1 = \u0026#39;Aarav Sharma\u0026#39; name2 = \u0026#34;Priya Patel\u0026#34; # Use the other quote type to include quotes inside sentence1 = \u0026#34;It\u0026#39;s a beautiful day in Mumbai.\u0026#34; sentence2 = \u0026#39;He said \u0026#34;Namaste\u0026#34; to everyone.\u0026#39; # Escape characters tab_eg = \u0026#34;Name:\\tAarav\u0026#34; # \\t = tab newline = \u0026#34;Line1\\nLine2\u0026#34; # \\n = newline backslash = \u0026#34;C:\\\\Users\\\\Aarav\u0026#34; # \\\\ = literal backslash raw_str = r\u0026#34;C:\\Users\\Aarav\u0026#34; # r\u0026#34;...\u0026#34; = raw string, no escapes # Triple quotes -- multi-line strings address = \u0026#34;\u0026#34;\u0026#34; 123, MG Road, Koramangala, Bangalore - 560001 \u0026#34;\u0026#34;\u0026#34; poem = \u0026#39;\u0026#39;\u0026#39; Itna na mujhse tu pyaar badha, ke main ek rooh awaara tha. \u0026#39;\u0026#39;\u0026#39; String indexing and slicing Strings are sequences \u0026ndash; each character has a position (index) starting from 0.\ncity = \u0026#34;Hyderabad\u0026#34; # 012345678 (positive indices) # -987654321 (negative indices) # Indexing print(city[0]) # H (first character) print(city[-1]) # d (last character) print(city[4]) # r print(city[-4]) # r (same character, from the end) # Slicing: string[start:stop:step] # start -- inclusive, stop -- exclusive print(city[0:5]) # Hyder (index 0 to 4) print(city[5:]) # abad (index 5 to end) print(city[:5]) # Hyder (start to index 4) print(city[:]) # Hyderabad (full copy) print(city[::2]) # Hdrbd (every 2nd character) print(city[::-1]) # darebadyH (reversed) # Strings are immutable -- you cannot change individual characters city[0] = \u0026#34;h\u0026#34; # TypeError: \u0026#39;str\u0026#39; object does not support item assignment String operators first = \u0026#34;Jai \u0026#34; second = \u0026#34;Hind\u0026#34; # Concatenation print(first + second) # Jai Hind # Repetition print(\u0026#34;Ha\u0026#34; * 3) # HaHaHa print(\u0026#34;-\u0026#34; * 30) # ------------------------------ (useful for separators) # Membership print(\u0026#34;Hind\u0026#34; in first + second) # True print(\u0026#34;bye\u0026#34; in first + second) # False print(\u0026#34;bye\u0026#34; not in \u0026#34;Hello\u0026#34;) # True String methods \u0026ndash; complete reference All string methods return a new string. They never modify the original.\nCase methods text = \u0026#34; hello, INDIA! \u0026#34; print(text.upper()) # \u0026#34; HELLO, INDIA! \u0026#34; print(text.lower()) # \u0026#34; hello, india! \u0026#34; print(text.title()) # \u0026#34; Hello, India! \u0026#34; print(text.capitalize()) # \u0026#34; hello, india! \u0026#34; (only first char of whole string) print(text.swapcase()) # \u0026#34; HELLO, india! \u0026#34; Stripping whitespace text = \u0026#34; Namaste \u0026#34; print(text.strip()) # \u0026#34;Namaste\u0026#34; (both ends) print(text.lstrip()) # \u0026#34;Namaste \u0026#34; (left only) print(text.rstrip()) # \u0026#34; Namaste\u0026#34; (right only) # Strip specific characters \u0026#34;***hello***\u0026#34;.strip(\u0026#34;*\u0026#34;) # \u0026#34;hello\u0026#34; \u0026#34;...ok...\u0026#34;.strip(\u0026#34;.\u0026#34;) # \u0026#34;ok\u0026#34; Finding and checking city = \u0026#34;Bangalore\u0026#34; print(city.find(\u0026#34;gal\u0026#34;)) # 3 (index of first occurrence, -1 if not found) print(city.find(\u0026#34;xyz\u0026#34;)) # -1 print(city.index(\u0026#34;gal\u0026#34;)) # 3 (same as find, but raises ValueError if not found) print(city.count(\u0026#34;a\u0026#34;)) # 2 (how many times \u0026#34;a\u0026#34; appears) print(city.startswith(\u0026#34;Ban\u0026#34;)) # True print(city.endswith(\u0026#34;lore\u0026#34;)) # True print(city.startswith(\u0026#34;Mum\u0026#34;)) # False # Type checking methods print(\u0026#34;hello123\u0026#34;.isalnum()) # True (only letters and digits) print(\u0026#34;hello\u0026#34;.isalpha()) # True (only letters) print(\u0026#34;12345\u0026#34;.isdigit()) # True (only digits) print(\u0026#34; \u0026#34;.isspace()) # True (only whitespace) print(\u0026#34;Hello World\u0026#34;.istitle()) # True (title case) print(\u0026#34;HELLO\u0026#34;.isupper()) # True print(\u0026#34;hello\u0026#34;.islower()) # True Replacing and splitting text = \u0026#34;Mumbai is the financial capital of India\u0026#34; # Replace all occurrences print(text.replace(\u0026#34;India\u0026#34;, \u0026#34;Bharat\u0026#34;)) # \u0026#34;Mumbai is the financial capital of Bharat\u0026#34; # Replace only first n occurrences print(\u0026#34;aaa\u0026#34;.replace(\u0026#34;a\u0026#34;, \u0026#34;b\u0026#34;, 2)) # \u0026#34;bba\u0026#34; # Split into a list print(text.split()) # splits on whitespace by default # [\u0026#39;Mumbai\u0026#39;, \u0026#39;is\u0026#39;, \u0026#39;the\u0026#39;, \u0026#39;financial\u0026#39;, \u0026#39;capital\u0026#39;, \u0026#39;of\u0026#39;, \u0026#39;India\u0026#39;] print(text.split(\u0026#34;,\u0026#34;)) # split on comma # [\u0026#39;Mumbai is the financial capital of India\u0026#39;] (no comma, so no split) csv_line = \u0026#34;Aarav,21,Delhi,Engineering\u0026#34; print(csv_line.split(\u0026#34;,\u0026#34;)) # [\u0026#39;Aarav\u0026#39;, \u0026#39;21\u0026#39;, \u0026#39;Delhi\u0026#39;, \u0026#39;Engineering\u0026#39;] # Split with limit print(\u0026#34;a:b:c:d\u0026#34;.split(\u0026#34;:\u0026#34;, 2)) # [\u0026#39;a\u0026#39;, \u0026#39;b\u0026#39;, \u0026#39;c:d\u0026#39;] (max 2 splits) # rsplit -- split from the right print(\u0026#34;a:b:c:d\u0026#34;.rsplit(\u0026#34;:\u0026#34;, 1)) # [\u0026#39;a:b:c\u0026#39;, \u0026#39;d\u0026#39;] # splitlines -- split on newline characters lines = \u0026#34;Line1\\nLine2\\nLine3\u0026#34; print(lines.splitlines()) # [\u0026#39;Line1\u0026#39;, \u0026#39;Line2\u0026#39;, \u0026#39;Line3\u0026#39;] Joining # join -- the opposite of split # join is called on the separator, with the list as the argument words = [\u0026#34;Jai\u0026#34;, \u0026#34;Hind\u0026#34;, \u0026#34;Zindabad\u0026#34;] result = \u0026#34; \u0026#34;.join(words) print(result) # \u0026#34;Jai Hind Zindabad\u0026#34; # Join with different separators print(\u0026#34;, \u0026#34;.join([\u0026#34;Delhi\u0026#34;, \u0026#34;Mumbai\u0026#34;, \u0026#34;Bangalore\u0026#34;])) # \u0026#34;Delhi, Mumbai, Bangalore\u0026#34; print(\u0026#34;-\u0026#34;.join([\u0026#34;2024\u0026#34;, \u0026#34;01\u0026#34;, \u0026#34;15\u0026#34;])) # \u0026#34;2024-01-15\u0026#34; # Join is much faster than concatenation in a loop # BAD: sentence = \u0026#34;\u0026#34; for word in words: sentence += word + \u0026#34; \u0026#34; # GOOD: sentence = \u0026#34; \u0026#34;.join(words) Padding and alignment text = \u0026#34;Aarav\u0026#34; print(text.ljust(10)) # \u0026#34;Aarav \u0026#34; (left-aligned, padded to width 10) print(text.rjust(10)) # \u0026#34; Aarav\u0026#34; (right-aligned) print(text.center(11)) # \u0026#34; Aarav \u0026#34; (centred) print(text.center(11, \u0026#34;-\u0026#34;)) # \u0026#34;---Aarav---\u0026#34; (centred with fill character) print(\u0026#34;42\u0026#34;.zfill(6)) # \u0026#34;000042\u0026#34; (pad with zeros on left) Encoding text = \u0026#34;Namaste\u0026#34; encoded = text.encode(\u0026#34;utf-8\u0026#34;) # bytes object: b\u0026#39;Namaste\u0026#39; decoded = encoded.decode(\u0026#34;utf-8\u0026#34;) # back to string: \u0026#34;Namaste\u0026#34; Useful built-in functions for strings text = \u0026#34;Bangalore\u0026#34; print(len(text)) # 9 -- length print(min(text)) # \u0026#39;B\u0026#39; -- character with lowest ASCII value print(max(text)) # \u0026#39;r\u0026#39; -- character with highest ASCII value print(sorted(text)) # sorted list of characters print(list(text)) # [\u0026#39;B\u0026#39;,\u0026#39;a\u0026#39;,\u0026#39;n\u0026#39;,\u0026#39;g\u0026#39;,\u0026#39;a\u0026#39;,\u0026#39;l\u0026#39;,\u0026#39;o\u0026#39;,\u0026#39;r\u0026#39;,\u0026#39;e\u0026#39;] # ord() and chr() -- convert between character and ASCII code print(ord(\u0026#34;A\u0026#34;)) # 65 print(ord(\u0026#34;a\u0026#34;)) # 97 print(chr(65)) # \u0026#39;A\u0026#39; print(chr(2309)) # Indian language character (Unicode) String immutability \u0026ndash; understanding what happens name = \u0026#34;Priya\u0026#34; # This does NOT modify \u0026#39;name\u0026#39; -- it creates a new string and reassigns the label name = name.upper() print(name) # \u0026#34;PRIYA\u0026#34; # The original string \u0026#34;Priya\u0026#34; is still in memory until garbage collected # The variable \u0026#39;name\u0026#39; now points to the new string \u0026#34;PRIYA\u0026#34; # This is why you must reassign the result city = \u0026#34;bangalore\u0026#34; city.title() # does nothing visible print(city) # still \u0026#34;bangalore\u0026#34; city = city.title() # now it works print(city) # \u0026#34;Bangalore\u0026#34; 12. Operators Arithmetic operators (covered in Section 7) print(10 + 3) # 13 print(10 - 3) # 7 print(10 * 3) # 30 print(10 / 3) # 3.333... print(10 // 3) # 3 print(10 % 3) # 1 print(10 ** 3) # 1000 Comparison operators (covered in Section 9) print(5 \u0026gt; 3) # True print(5 \u0026lt; 3) # False print(5 == 5) # True print(5 != 3) # True print(5 \u0026gt;= 5) # True print(5 \u0026lt;= 4) # False Logical operators (covered in Section 9) print(True and False) # False print(True or False) # True print(not True) # False Membership operators cities = [\u0026#34;Delhi\u0026#34;, \u0026#34;Mumbai\u0026#34;, \u0026#34;Bangalore\u0026#34;] print(\u0026#34;Mumbai\u0026#34; in cities) # True print(\u0026#34;Chennai\u0026#34; in cities) # False print(\u0026#34;Chennai\u0026#34; not in cities) # True # Works on strings too name = \u0026#34;Aarav Sharma\u0026#34; print(\u0026#34;Sharma\u0026#34; in name) # True print(\u0026#34;Patel\u0026#34; not in name) # True Identity operators # \u0026#39;is\u0026#39; checks if two variables point to the SAME object in memory # \u0026#39;==\u0026#39; checks if two variables have the SAME VALUE a = [1, 2, 3] b = [1, 2, 3] c = a print(a == b) # True -- same value print(a is b) # False -- different objects in memory print(a is c) # True -- same object (c is just another label for a) # None check -- always use \u0026#39;is\u0026#39;, never \u0026#39;==\u0026#39; result = None print(result is None) # True -- correct way print(result == None) # True -- works but not recommended (PEP 8) Bitwise operators Used for binary operations. Less common in everyday code but important for systems programming, flags, and some performance-critical work.\na = 0b1010 # 10 in decimal b = 0b1100 # 12 in decimal print(a \u0026amp; b) # 8 AND: 0b1000 print(a | b) # 14 OR: 0b1110 print(a ^ b) # 6 XOR: 0b0110 print(~a) # -11 NOT: flips all bits print(a \u0026lt;\u0026lt; 1) # 20 left shift by 1 (multiply by 2) print(a \u0026gt;\u0026gt; 1) # 5 right shift by 1 (divide by 2, floor) # Practical use: check if a number is even or odd def is_even(n): return n \u0026amp; 1 == 0 # last bit is 0 means even print(is_even(8)) # True print(is_even(7)) # False Operator precedence \u0026ndash; full order (high to low) 1. () Parentheses 2. ** Exponentiation (right to left) 3. +x, -x, ~x Unary plus, minus, bitwise NOT 4. *, /, //, % Multiplication, division, floor div, modulo 5. +, - Addition, subtraction 6. \u0026lt;\u0026lt;, \u0026gt;\u0026gt; Bitwise shift 7. \u0026amp; Bitwise AND 8. ^ Bitwise XOR 9. | Bitwise OR 10. ==, !=, \u0026gt;, \u0026gt;=, \u0026lt;, \u0026lt;=, Comparisons is, is not, in, not in 11. not Logical NOT 12. and Logical AND 13. or Logical OR # When in doubt, use parentheses result = (2 + 3) * (4 - 1) ** 2 # 45 -- clear result = 2 + 3 * 4 - 1 ** 2 # 13 -- requires knowing precedence 13. Type Conversion Implicit conversion (automatic) Python automatically converts between compatible types in some operations.\n# int + float --\u0026gt; float result = 5 + 2.0 print(result) # 7.0 print(type(result)) # \u0026lt;class \u0026#39;float\u0026#39;\u0026gt; # int + bool --\u0026gt; int print(10 + True) # 11 print(10 + False) # 10 Explicit conversion (casting) # int() print(int(3.9)) # 3 (truncates -- does NOT round) print(int(\u0026#34;42\u0026#34;)) # 42 print(int(True)) # 1 print(int(False)) # 0 print(int(\u0026#34;0xFF\u0026#34;, 16)) # 255 (hex string to int) print(int(\u0026#34;1010\u0026#34;, 2)) # 10 (binary string to int) # float() print(float(5)) # 5.0 print(float(\u0026#34;3.14\u0026#34;)) # 3.14 print(float(\u0026#34;inf\u0026#34;)) # inf # str() print(str(42)) # \u0026#34;42\u0026#34; print(str(3.14)) # \u0026#34;3.14\u0026#34; print(str(True)) # \u0026#34;True\u0026#34; print(str(None)) # \u0026#34;None\u0026#34; # bool() print(bool(0)) # False print(bool(1)) # True print(bool(\u0026#34;\u0026#34;)) # False print(bool(\u0026#34;hello\u0026#34;)) # True print(bool([])) # False print(bool([1,2])) # True # Conversion failures -- raise ValueError int(\u0026#34;hello\u0026#34;) # ValueError: invalid literal for int() float(\u0026#34;abc\u0026#34;) # ValueError: could not convert string to float # Safe conversion pattern def safe_int(value): try: return int(value) except (ValueError, TypeError): return None print(safe_int(\u0026#34;42\u0026#34;)) # 42 print(safe_int(\u0026#34;hello\u0026#34;)) # None print(safe_int(None)) # None 14. Input and Output print() \u0026ndash; in depth # Basic print print(\u0026#34;Hello, Aarav!\u0026#34;) # Multiple values -- separated by space by default print(\u0026#34;Name:\u0026#34;, \u0026#34;Priya\u0026#34;, \u0026#34;Age:\u0026#34;, 21) # Name: Priya Age: 21 # Change separator print(\u0026#34;2024\u0026#34;, \u0026#34;01\u0026#34;, \u0026#34;15\u0026#34;, sep=\u0026#34;-\u0026#34;) # 2024-01-15 print(\u0026#34;Delhi\u0026#34;, \u0026#34;Mumbai\u0026#34;, sep=\u0026#34; | \u0026#34;) # Delhi | Mumbai # Change end character (default is newline) print(\u0026#34;Loading\u0026#34;, end=\u0026#34;\u0026#34;) print(\u0026#34;...\u0026#34;) # Loading... (on same line) print(\u0026#34;A\u0026#34;, end=\u0026#34; \u0026#34;) print(\u0026#34;B\u0026#34;, end=\u0026#34; \u0026#34;) print(\u0026#34;C\u0026#34;) # A B C # Print to a file with open(\u0026#34;output.txt\u0026#34;, \u0026#34;w\u0026#34;) as f: print(\u0026#34;Hello, file!\u0026#34;, file=f) # Flush the output buffer immediately (useful for progress indicators) import time for i in range(5): print(f\u0026#34;\\rProgress: {i+1}/5\u0026#34;, end=\u0026#34;\u0026#34;, flush=True) time.sleep(0.5) input() \u0026ndash; reading user input input() always returns a string. You must convert it to the appropriate type.\n# Basic input name = input(\u0026#34;Enter your name: \u0026#34;) print(f\u0026#34;Hello, {name}!\u0026#34;) # Convert to number age = int(input(\u0026#34;Enter your age: \u0026#34;)) gpa = float(input(\u0026#34;Enter your GPA: \u0026#34;)) # Multiple inputs on one line x, y = input(\u0026#34;Enter two numbers separated by space: \u0026#34;).split() x, y = int(x), int(y) print(f\u0026#34;Sum: {x + y}\u0026#34;) # Or in one step a, b = map(int, input(\u0026#34;Enter two integers: \u0026#34;).split()) print(f\u0026#34;Product: {a * b}\u0026#34;) # Input with validation while True: age_str = input(\u0026#34;Enter a valid age (0-120): \u0026#34;) if age_str.isdigit() and 0 \u0026lt;= int(age_str) \u0026lt;= 120: age = int(age_str) break print(\u0026#34;Invalid input. Please try again.\u0026#34;) print(f\u0026#34;Your age is {age}\u0026#34;) 15. f-strings and String Formatting f-strings (formatted string literals, introduced in Python 3.6) are the modern and preferred way to embed values inside strings.\nBasic f-string syntax name = \u0026#34;Sneha\u0026#34; score = 91.5 city = \u0026#34;Hyderabad\u0026#34; print(f\u0026#34;Name: {name}, Score: {score}, City: {city}\u0026#34;) # Name: Sneha, Score: 91.5, City: Hyderabad # Expressions inside f-strings print(f\u0026#34;Score in percentage: {score / 100:.1%}\u0026#34;) # Score in percentage: 91.5% print(f\u0026#34;Double the score: {score * 2}\u0026#34;) # Double the score: 183.0 print(f\u0026#34;Name in uppercase: {name.upper()}\u0026#34;) # Name in uppercase: SNEHA Number formatting price = 1234567.89 discount = 0.1825 count = 42 # Thousands separator print(f\u0026#34;Price: Rs.{price:,}\u0026#34;) # Rs.1,234,567.89 print(f\u0026#34;Price: Rs.{price:,.2f}\u0026#34;) # Rs.1,234,567.89 (2 decimal places) # Decimal places print(f\u0026#34;Pi: {3.14159265:.3f}\u0026#34;) # Pi: 3.142 print(f\u0026#34;Pi: {3.14159265:.6f}\u0026#34;) # Pi: 3.141593 # Percentage print(f\u0026#34;Discount: {discount:.1%}\u0026#34;) # Discount: 18.3% print(f\u0026#34;Discount: {discount:.2%}\u0026#34;) # Discount: 18.25% # Integer with padding print(f\u0026#34;Count: {count:05d}\u0026#34;) # Count: 00042 print(f\u0026#34;Count: {count:\u0026gt;10d}\u0026#34;) # Count: 42 (right-aligned) print(f\u0026#34;Count: {count:\u0026lt;10d}\u0026#34;) # Count: 42 (left-aligned) print(f\u0026#34;Count: {count:^10d}\u0026#34;) # Count: 42 (centred) # Scientific notation print(f\u0026#34;Avogadro: {6.022e23:.3e}\u0026#34;) # Avogadro: 6.022e+23 String alignment in f-strings # Build a formatted table students = [(\u0026#34;Aarav\u0026#34;, 88), (\u0026#34;Priya\u0026#34;, 92), (\u0026#34;Rohan\u0026#34;, 75), (\u0026#34;Sneha\u0026#34;, 96)] print(f\u0026#34;{\u0026#39;Name\u0026#39;:\u0026lt;12} {\u0026#39;Score\u0026#39;:\u0026gt;6}\u0026#34;) print(\u0026#34;-\u0026#34; * 20) for name, score in students: print(f\u0026#34;{name:\u0026lt;12} {score:\u0026gt;6}\u0026#34;) Name Score -------------------- Aarav 88 Priya 92 Rohan 75 Sneha 96 Debugging with f-strings (Python 3.8+) x = 42 y = 3.14 # The = sign prints both the expression and its value print(f\u0026#34;{x = }\u0026#34;) # x = 42 print(f\u0026#34;{y = :.2f}\u0026#34;) # y = 3.14 print(f\u0026#34;{x * y = }\u0026#34;) # x * y = 131.88 Older formatting methods (still seen in legacy code) name = \u0026#34;Aarav\u0026#34; score = 88 # % formatting (old style, avoid in new code) print(\u0026#34;Name: %s, Score: %d\u0026#34; % (name, score)) print(\u0026#34;Price: Rs.%.2f\u0026#34; % 1234.567) # .format() method (Python 3, before f-strings) print(\u0026#34;Name: {}, Score: {}\u0026#34;.format(name, score)) print(\u0026#34;Name: {0}, Score: {1}\u0026#34;.format(name, score)) print(\u0026#34;Name: {n}, Score: {s}\u0026#34;.format(n=name, s=score)) Use f-strings for all new code. They are faster, more readable, and support the full formatting mini-language.\n16. Practice Questions Try every question on your own before reading the solutions in Section 17. Questions are ordered: Easy (1-6), Medium (7-12), Hard (13-18).\nEasy Q1 \u0026ndash; Variables and Types Create variables for the following information about a student and print each one with its type.\nName: \u0026ldquo;Arjun Verma\u0026rdquo; Age: 20 CGPA: 8.45 Is hostel resident: True Pending fee: None Expected output format:\nName : Arjun Verma | Type: \u0026lt;class \u0026#39;str\u0026#39;\u0026gt; Age : 20 | Type: \u0026lt;class \u0026#39;int\u0026#39;\u0026gt; ... Q2 \u0026ndash; Arithmetic A Zomato order has the following details. Calculate and print:\nSubtotal (sum of all item prices) Delivery fee (Rs.30 if subtotal \u0026lt; 200, else Rs.0) GST at 5% on subtotal Grand total (subtotal + delivery fee + GST), rounded to 2 decimal places item1 = 120 # Masala Dosa item2 = 80 # Filter Coffee item3 = 60 # Idli Sambar Q3 \u0026ndash; String Basics Given the string below, write code to:\nPrint the first and last character Print the string reversed Print the length Print the string in title case Count how many times the letter \u0026lsquo;a\u0026rsquo; appears (case-insensitive) text = \u0026#34;incredible india\u0026#34; Q4 \u0026ndash; String Methods Given the messy name string below, clean it up using string methods:\nStrip leading and trailing whitespace Convert to title case Replace any double spaces with single spaces Check if it starts with \u0026ldquo;Priya\u0026rdquo; name = \u0026#34; priya sharma \u0026#34; Q5 \u0026ndash; Type Conversion The following variables are all strings (as if read from user input). Convert each to the appropriate type, perform the required operation, and print the result.\nprice_str = \u0026#34;1299\u0026#34; quantity_str = \u0026#34;4\u0026#34; discount_str = \u0026#34;15\u0026#34; # percent Calculate total price before discount Apply the discount Print final amount rounded to 2 decimal places Q6 \u0026ndash; Booleans and Operators Given the voter eligibility rules (age \u0026gt;= 18 and is_citizen = True and not is_disqualified), write code to check eligibility for three people and print the result.\nperson1 = {\u0026#34;name\u0026#34;: \u0026#34;Aarav\u0026#34;, \u0026#34;age\u0026#34;: 22, \u0026#34;is_citizen\u0026#34;: True, \u0026#34;is_disqualified\u0026#34;: False} person2 = {\u0026#34;name\u0026#34;: \u0026#34;Priya\u0026#34;, \u0026#34;age\u0026#34;: 16, \u0026#34;is_citizen\u0026#34;: True, \u0026#34;is_disqualified\u0026#34;: False} person3 = {\u0026#34;name\u0026#34;: \u0026#34;Rohan\u0026#34;, \u0026#34;age\u0026#34;: 30, \u0026#34;is_citizen\u0026#34;: False, \u0026#34;is_disqualified\u0026#34;: False} Medium Q7 \u0026ndash; String Slicing Given the IFSC code string below, extract and print each component.\nifsc = \u0026#34;SBIN0001234\u0026#34; # Format: Bank code (4 chars) + \u0026#39;0\u0026#39; + Branch code (6 chars) Expected output:\nBank Code : SBIN Branch Code : 001234 Full IFSC : SBIN0001234 Is valid : True (length must be 11) Q8 \u0026ndash; String Operations Pipeline Take the raw sentence below and apply the following transformations in order:\nStrip whitespace Convert to lowercase Replace all commas with spaces Split into words Remove duplicate words while preserving order Join back with a single space Print the result and the word count sentence = \u0026#34; Delhi, Mumbai, Bangalore, Delhi, Chennai, Mumbai, Hyderabad \u0026#34; Q9 \u0026ndash; Number Operations Write code to:\nCheck if 2024 is a leap year using only arithmetic and logical operators Find the sum of digits of the number 987654 Reverse the digits of 12345 without converting to string Find the GCD of 252 and 105 using the math module Q10 \u0026ndash; f-string Formatting Given the sales data below, print a formatted table with right-aligned numbers, a thousands separator for revenue, and percentage sign for growth.\ndata = [ (\u0026#34;Electronics\u0026#34;, 845000, 12.4), (\u0026#34;Clothing\u0026#34;, 320000, -3.2), (\u0026#34;Books\u0026#34;, 95000, 8.7), (\u0026#34;Home\u0026#34;, 210000, 5.1), (\u0026#34;Beauty\u0026#34;, 145000, 22.8), ] # Columns: Category, Revenue (Rs.), Growth (%) Expected output:\nCategory Revenue Growth ------------------------------------ Electronics Rs.8,45,000 12.4% Clothing Rs.3,20,000 -3.2% ... Q11 \u0026ndash; Input Validation Write a program that:\nAsks the user to enter a mobile number Validates it: must be exactly 10 digits, all numeric, and start with 6, 7, 8, or 9 Keeps asking until a valid number is entered Prints \u0026ldquo;Valid mobile number: XXXXXXXXXX\u0026rdquo; (For testing without actual input, write the logic as a function validate_mobile(number_str) that returns True or False.)\nQ12 \u0026ndash; Operator Challenge Without using any if statements or loops, write expressions using only operators to:\nReturn the absolute value of a number (hint: use ** and **0.5, or think about abs()) Check if a number is divisible by both 3 and 7 Extract the tens digit from a 3-digit number (e.g., from 347, get 4) Check if a year is a century year (divisible by 100) but not a leap year (not divisible by 400) num = -42 value = 347 year = 1900 Hard Q13 \u0026ndash; String Parsing You are given a raw transaction log string. Parse it and extract the date, transaction ID, amount, and status without using any imports. Then print a clean summary.\nlog = \u0026#34;TXN|2024-03-15|TXN004521|RS.12500.00|SUCCESS|HDFC|NEFT\u0026#34; Expected output:\nDate : 2024-03-15 Transaction ID : TXN004521 Amount : Rs.12,500.00 Status : Success Bank : HDFC Mode : NEFT Q14 \u0026ndash; Caesar Cipher using String Methods Write a function caesar(text, shift) that encrypts a string by shifting each letter by shift positions in the alphabet. Preserve case and leave non-alphabetic characters unchanged. Then write decrypt(text, shift) that reverses it.\nmessage = \u0026#34;Namaste, India! Python hai sahi.\u0026#34; shift = 5 encrypted = caesar(message, shift) decrypted = decrypt(encrypted, shift) print(f\u0026#34;Original : {message}\u0026#34;) print(f\u0026#34;Encrypted: {encrypted}\u0026#34;) print(f\u0026#34;Decrypted: {decrypted}\u0026#34;) Q15 \u0026ndash; Number to Words Write a function number_to_words(n) that converts any integer from 0 to 9999 into its English word form. Use only string operations, arithmetic, and dictionaries.\nprint(number_to_words(0)) # \u0026#34;zero\u0026#34; print(number_to_words(15)) # \u0026#34;fifteen\u0026#34; print(number_to_words(100)) # \u0026#34;one hundred\u0026#34; print(number_to_words(342)) # \u0026#34;three hundred forty two\u0026#34; print(number_to_words(1999)) # \u0026#34;one thousand nine hundred ninety nine\u0026#34; print(number_to_words(9999)) # \u0026#34;nine thousand nine hundred ninety nine\u0026#34; Q16 \u0026ndash; String Statistics Write a function text_stats(text) that analyses a paragraph and returns a dictionary with:\nTotal characters (including spaces) Total characters (excluding spaces) Word count Sentence count (count \u0026ldquo;.\u0026rdquo;, \u0026ldquo;!\u0026rdquo;, \u0026ldquo;?\u0026rdquo;) Most frequent word (case-insensitive, excluding common words like \u0026ldquo;the\u0026rdquo;, \u0026ldquo;a\u0026rdquo;, \u0026ldquo;is\u0026rdquo;, \u0026ldquo;in\u0026rdquo;, \u0026ldquo;of\u0026rdquo;, \u0026ldquo;and\u0026rdquo;, \u0026ldquo;to\u0026rdquo;) Average word length para = \u0026#34;\u0026#34;\u0026#34; India is a land of diversity. It has many cultures, languages and traditions. From the mountains of Himalayas to the beaches of Kerala, India is truly incredible. The people of India are known for their warmth and hospitality. \u0026#34;\u0026#34;\u0026#34; Q17 \u0026ndash; Justified Text Formatter Write a function justify(text, width) that takes a paragraph and reformats it so every line is exactly width characters wide (full justification \u0026ndash; spaces are distributed evenly between words). The last line of each paragraph is left-aligned.\nparagraph = \u0026#34;Python is a high level general purpose programming language known for its simplicity and readability\u0026#34; width = 40 justify(paragraph, width) Expected output (approximately):\nPython is a high level general purpose programming language known for its simplicity and readability Q18 \u0026ndash; Full Pipeline You receive a messy CSV string of student exam records. Write a complete program that:\nParses each record into a dictionary Cleans all string fields (strip, title case) Converts numeric fields to correct types Calculates total and percentage (out of 500) Assigns grade: Distinction (\u0026gt;=85%), First Class (\u0026gt;=60%), Pass (\u0026gt;=40%), Fail (\u0026lt;40%) Prints a formatted report Prints summary stats: class average, highest scorer, number of failures raw_csv = \u0026#34;\u0026#34;\u0026#34; name, maths, science, english, history, cs aarav SHARMA, 88, 92, 85, 78, 95 PRIYA patel, 55, 48, 62, 50, 58 rohan VERMA, 72, 68, 75, 70, 65 SNEHA iyer, 95, 98, 92, 88, 97 karan SINGH, 35, 40, 38, 42, 30 meera NAIR, 78, 82, 80, 75, 84 \u0026#34;\u0026#34;\u0026#34; 17. Solutions Q1 \u0026ndash; Variables and Types name = \u0026#34;Arjun Verma\u0026#34; age = 20 cgpa = 8.45 is_hostel = True pending_fee = None fields = [ (\u0026#34;Name\u0026#34;, name), (\u0026#34;Age\u0026#34;, age), (\u0026#34;CGPA\u0026#34;, cgpa), (\u0026#34;Is hostel resident\u0026#34;, is_hostel), (\u0026#34;Pending fee\u0026#34;, pending_fee), ] for label, value in fields: print(f\u0026#34;{label:\u0026lt;22}: {str(value):\u0026lt;15} | Type: {type(value)}\u0026#34;) Q2 \u0026ndash; Arithmetic item1 = 120 item2 = 80 item3 = 60 subtotal = item1 + item2 + item3 delivery_fee = 0 if subtotal \u0026gt;= 200 else 30 gst = subtotal * 0.05 grand_total = round(subtotal + delivery_fee + gst, 2) print(f\u0026#34;Subtotal : Rs.{subtotal}\u0026#34;) print(f\u0026#34;Delivery fee : Rs.{delivery_fee}\u0026#34;) print(f\u0026#34;GST (5%) : Rs.{gst:.2f}\u0026#34;) print(f\u0026#34;Grand Total : Rs.{grand_total}\u0026#34;) Q3 \u0026ndash; String Basics text = \u0026#34;incredible india\u0026#34; print(\u0026#34;First character :\u0026#34;, text[0]) print(\u0026#34;Last character :\u0026#34;, text[-1]) print(\u0026#34;Reversed :\u0026#34;, text[::-1]) print(\u0026#34;Length :\u0026#34;, len(text)) print(\u0026#34;Title case :\u0026#34;, text.title()) print(\u0026#34;Count of \u0026#39;a\u0026#39; :\u0026#34;, text.lower().count(\u0026#34;a\u0026#34;)) Q4 \u0026ndash; String Methods name = \u0026#34; priya sharma \u0026#34; name = name.strip() name = name.title() while \u0026#34; \u0026#34; in name: name = name.replace(\u0026#34; \u0026#34;, \u0026#34; \u0026#34;) print(\u0026#34;Cleaned name :\u0026#34;, name) print(\u0026#34;Starts with Priya:\u0026#34;, name.startswith(\u0026#34;Priya\u0026#34;)) Q5 \u0026ndash; Type Conversion price_str = \u0026#34;1299\u0026#34; quantity_str = \u0026#34;4\u0026#34; discount_str = \u0026#34;15\u0026#34; price = int(price_str) quantity = int(quantity_str) discount = int(discount_str) subtotal = price * quantity discount_amount = subtotal * (discount / 100) final_amount = round(subtotal - discount_amount, 2) print(f\u0026#34;Subtotal : Rs.{subtotal:,}\u0026#34;) print(f\u0026#34;Discount ({discount}%) : Rs.{discount_amount:,.2f}\u0026#34;) print(f\u0026#34;Final Amount : Rs.{final_amount:,.2f}\u0026#34;) Q6 \u0026ndash; Booleans and Operators people = [ {\u0026#34;name\u0026#34;: \u0026#34;Aarav\u0026#34;, \u0026#34;age\u0026#34;: 22, \u0026#34;is_citizen\u0026#34;: True, \u0026#34;is_disqualified\u0026#34;: False}, {\u0026#34;name\u0026#34;: \u0026#34;Priya\u0026#34;, \u0026#34;age\u0026#34;: 16, \u0026#34;is_citizen\u0026#34;: True, \u0026#34;is_disqualified\u0026#34;: False}, {\u0026#34;name\u0026#34;: \u0026#34;Rohan\u0026#34;, \u0026#34;age\u0026#34;: 30, \u0026#34;is_citizen\u0026#34;: False, \u0026#34;is_disqualified\u0026#34;: False}, ] for p in people: eligible = (p[\u0026#34;age\u0026#34;] \u0026gt;= 18 and p[\u0026#34;is_citizen\u0026#34;] and not p[\u0026#34;is_disqualified\u0026#34;]) status = \u0026#34;Eligible to vote\u0026#34; if eligible else \u0026#34;Not eligible\u0026#34; print(f\u0026#34;{p[\u0026#39;name\u0026#39;]:\u0026lt;8}: {status}\u0026#34;) Q7 \u0026ndash; String Slicing ifsc = \u0026#34;SBIN0001234\u0026#34; bank_code = ifsc[:4] branch_code = ifsc[5:] is_valid = len(ifsc) == 11 print(f\u0026#34;Bank Code : {bank_code}\u0026#34;) print(f\u0026#34;Branch Code : {branch_code}\u0026#34;) print(f\u0026#34;Full IFSC : {ifsc}\u0026#34;) print(f\u0026#34;Is valid : {is_valid}\u0026#34;) Q8 \u0026ndash; String Operations Pipeline sentence = \u0026#34; Delhi, Mumbai, Bangalore, Delhi, Chennai, Mumbai, Hyderabad \u0026#34; result = sentence.strip().lower().replace(\u0026#34;,\u0026#34;, \u0026#34; \u0026#34;) words = result.split() seen = [] unique_words = [] for word in words: if word not in seen: seen.append(word) unique_words.append(word) final = \u0026#34; \u0026#34;.join(unique_words) print(\u0026#34;Result :\u0026#34;, final) print(\u0026#34;Word count:\u0026#34;, len(unique_words)) Q9 \u0026ndash; Number Operations import math # Leap year check year = 2024 is_leap = (year % 4 == 0 and year % 100 != 0) or (year % 400 == 0) print(f\u0026#34;{year} is a leap year: {is_leap}\u0026#34;) # Sum of digits of 987654 n = 987654 digit_sum = 0 temp = n while temp \u0026gt; 0: digit_sum += temp % 10 temp //= 10 print(f\u0026#34;Sum of digits of {n}: {digit_sum}\u0026#34;) # Reverse digits of 12345 num = 12345 reversed_num = 0 temp = num while temp \u0026gt; 0: reversed_num = reversed_num * 10 + temp % 10 temp //= 10 print(f\u0026#34;Reversed {num}: {reversed_num}\u0026#34;) # GCD print(f\u0026#34;GCD of 252 and 105: {math.gcd(252, 105)}\u0026#34;) Q10 \u0026ndash; f-string Formatting data = [ (\u0026#34;Electronics\u0026#34;, 845000, 12.4), (\u0026#34;Clothing\u0026#34;, 320000, -3.2), (\u0026#34;Books\u0026#34;, 95000, 8.7), (\u0026#34;Home\u0026#34;, 210000, 5.1), (\u0026#34;Beauty\u0026#34;, 145000, 22.8), ] print(f\u0026#34;{\u0026#39;Category\u0026#39;:\u0026lt;16} {\u0026#39;Revenue\u0026#39;:\u0026gt;14} {\u0026#39;Growth\u0026#39;:\u0026gt;8}\u0026#34;) print(\u0026#34;-\u0026#34; * 42) for category, revenue, growth in data: sign = \u0026#34;+\u0026#34; if growth \u0026gt;= 0 else \u0026#34;\u0026#34; print(f\u0026#34;{category:\u0026lt;16} Rs.{revenue:\u0026gt;10,} {sign}{growth:.1f}%\u0026#34;) Q11 \u0026ndash; Input Validation def validate_mobile(number_str): if len(number_str) != 10: return False if not number_str.isdigit(): return False if number_str[0] not in \u0026#34;6789\u0026#34;: return False return True # Test the function test_numbers = [\u0026#34;9876543210\u0026#34;, \u0026#34;1234567890\u0026#34;, \u0026#34;8765\u0026#34;, \u0026#34;abc1234567\u0026#34;, \u0026#34;7000000001\u0026#34;] for num in test_numbers: status = \u0026#34;Valid\u0026#34; if validate_mobile(num) else \u0026#34;Invalid\u0026#34; print(f\u0026#34;{num:\u0026lt;15}: {status}\u0026#34;) # For actual use with input(): # while True: # mobile = input(\u0026#34;Enter mobile number: \u0026#34;).strip() # if validate_mobile(mobile): # print(f\u0026#34;Valid mobile number: {mobile}\u0026#34;) # break # print(\u0026#34;Invalid. Must be 10 digits starting with 6, 7, 8, or 9.\u0026#34;) Q12 \u0026ndash; Operator Challenge num = -42 value = 347 year = 1900 # Absolute value without if absolute = (num ** 2) ** 0.5 print(f\u0026#34;Absolute value of {num}: {absolute}\u0026#34;) # or just abs(num) # Divisible by both 3 and 7 n = 42 div_3_and_7 = (n % 3 == 0) and (n % 7 == 0) print(f\u0026#34;{n} divisible by 3 and 7: {div_3_and_7}\u0026#34;) # Extract tens digit from 3-digit number tens_digit = (value // 10) % 10 print(f\u0026#34;Tens digit of {value}: {tens_digit}\u0026#34;) # Century year but not a leap year is_century_non_leap = (year % 100 == 0) and (year % 400 != 0) print(f\u0026#34;{year} is century but not leap: {is_century_non_leap}\u0026#34;) Q13 \u0026ndash; String Parsing log = \u0026#34;TXN|2024-03-15|TXN004521|RS.12500.00|SUCCESS|HDFC|NEFT\u0026#34; parts = log.split(\u0026#34;|\u0026#34;) date = parts[1] txn_id = parts[2] amount = float(parts[3].replace(\u0026#34;RS.\u0026#34;, \u0026#34;\u0026#34;).replace(\u0026#34;,\u0026#34;, \u0026#34;\u0026#34;)) status = parts[4].title() bank = parts[5] mode = parts[6] print(f\u0026#34;Date : {date}\u0026#34;) print(f\u0026#34;Transaction ID : {txn_id}\u0026#34;) print(f\u0026#34;Amount : Rs.{amount:,.2f}\u0026#34;) print(f\u0026#34;Status : {status}\u0026#34;) print(f\u0026#34;Bank : {bank}\u0026#34;) print(f\u0026#34;Mode : {mode}\u0026#34;) Q14 \u0026ndash; Caesar Cipher def caesar(text, shift): result = [] for char in text: if char.isalpha(): base = ord(\u0026#34;A\u0026#34;) if char.isupper() else ord(\u0026#34;a\u0026#34;) shifted = chr((ord(char) - base + shift) % 26 + base) result.append(shifted) else: result.append(char) return \u0026#34;\u0026#34;.join(result) def decrypt(text, shift): return caesar(text, -shift) message = \u0026#34;Namaste, India! Python hai sahi.\u0026#34; shift = 5 encrypted = caesar(message, shift) decrypted = decrypt(encrypted, shift) print(f\u0026#34;Original : {message}\u0026#34;) print(f\u0026#34;Encrypted: {encrypted}\u0026#34;) print(f\u0026#34;Decrypted: {decrypted}\u0026#34;) Q15 \u0026ndash; Number to Words def number_to_words(n): if n == 0: return \u0026#34;zero\u0026#34; ones = [\u0026#34;\u0026#34;, \u0026#34;one\u0026#34;, \u0026#34;two\u0026#34;, \u0026#34;three\u0026#34;, \u0026#34;four\u0026#34;, \u0026#34;five\u0026#34;, \u0026#34;six\u0026#34;, \u0026#34;seven\u0026#34;, \u0026#34;eight\u0026#34;, \u0026#34;nine\u0026#34;, \u0026#34;ten\u0026#34;, \u0026#34;eleven\u0026#34;, \u0026#34;twelve\u0026#34;, \u0026#34;thirteen\u0026#34;, \u0026#34;fourteen\u0026#34;, \u0026#34;fifteen\u0026#34;, \u0026#34;sixteen\u0026#34;, \u0026#34;seventeen\u0026#34;, \u0026#34;eighteen\u0026#34;, \u0026#34;nineteen\u0026#34;] tens = [\u0026#34;\u0026#34;, \u0026#34;\u0026#34;, \u0026#34;twenty\u0026#34;, \u0026#34;thirty\u0026#34;, \u0026#34;forty\u0026#34;, \u0026#34;fifty\u0026#34;, \u0026#34;sixty\u0026#34;, \u0026#34;seventy\u0026#34;, \u0026#34;eighty\u0026#34;, \u0026#34;ninety\u0026#34;] def below_100(num): if num \u0026lt; 20: return ones[num] elif num % 10 == 0: return tens[num // 10] else: return tens[num // 10] + \u0026#34; \u0026#34; + ones[num % 10] def below_1000(num): if num \u0026lt; 100: return below_100(num) else: rest = num % 100 hundreds = ones[num // 100] + \u0026#34; hundred\u0026#34; if rest == 0: return hundreds return hundreds + \u0026#34; \u0026#34; + below_100(rest) if n \u0026lt; 1000: return below_1000(n) else: thousands = ones[n // 1000] + \u0026#34; thousand\u0026#34; rest = n % 1000 if rest == 0: return thousands return thousands + \u0026#34; \u0026#34; + below_1000(rest) print(number_to_words(0)) print(number_to_words(15)) print(number_to_words(100)) print(number_to_words(342)) print(number_to_words(1999)) print(number_to_words(9999)) Q16 \u0026ndash; String Statistics def text_stats(text): stop_words = {\u0026#34;the\u0026#34;, \u0026#34;a\u0026#34;, \u0026#34;is\u0026#34;, \u0026#34;in\u0026#34;, \u0026#34;of\u0026#34;, \u0026#34;and\u0026#34;, \u0026#34;to\u0026#34;, \u0026#34;it\u0026#34;, \u0026#34;are\u0026#34;, \u0026#34;for\u0026#34;, \u0026#34;was\u0026#34;, \u0026#34;has\u0026#34;, \u0026#34;its\u0026#34;, \u0026#34;an\u0026#34;} total_chars = len(text) no_space_chars = len(text.replace(\u0026#34; \u0026#34;, \u0026#34;\u0026#34;).replace(\u0026#34;\\n\u0026#34;, \u0026#34;\u0026#34;)) words = text.split() word_count = len(words) sentence_count = sum(1 for ch in text if ch in \u0026#34;.!?\u0026#34;) freq = {} for word in words: clean = word.strip(\u0026#34;.,!?\\\u0026#34;\u0026#39;\u0026#34;).lower() if clean and clean not in stop_words: freq[clean] = freq.get(clean, 0) + 1 most_frequent = max(freq, key=freq.get) if freq else \u0026#34;\u0026#34; avg_word_len = round(sum(len(w) for w in words) / word_count, 2) if word_count else 0 return { \u0026#34;total_chars\u0026#34;: total_chars, \u0026#34;no_space_chars\u0026#34;: no_space_chars, \u0026#34;word_count\u0026#34;: word_count, \u0026#34;sentence_count\u0026#34;: sentence_count, \u0026#34;most_frequent\u0026#34;: most_frequent, \u0026#34;avg_word_len\u0026#34;: avg_word_len, } para = \u0026#34;\u0026#34;\u0026#34; India is a land of diversity. It has many cultures, languages and traditions. From the mountains of Himalayas to the beaches of Kerala, India is truly incredible. The people of India are known for their warmth and hospitality. \u0026#34;\u0026#34;\u0026#34; stats = text_stats(para) for key, value in stats.items(): print(f\u0026#34;{key:\u0026lt;20}: {value}\u0026#34;) Q17 \u0026ndash; Justified Text Formatter def justify(text, width): words = text.split() lines = [] current_line = [] current_len = 0 for word in words: if current_len + len(word) + len(current_line) \u0026lt;= width: current_line.append(word) current_len += len(word) else: lines.append(current_line) current_line = [word] current_len = len(word) lines.append(current_line) # last line output = [] for i, line in enumerate(lines): if i == len(lines) - 1 or len(line) == 1: output.append(\u0026#34; \u0026#34;.join(line)) else: total_chars = sum(len(w) for w in line) total_spaces = width - total_chars gaps = len(line) - 1 base_space = total_spaces // gaps extra = total_spaces % gaps result = \u0026#34;\u0026#34; for j, word in enumerate(line): result += word if j \u0026lt; gaps: spaces = base_space + (1 if j \u0026lt; extra else 0) result += \u0026#34; \u0026#34; * spaces output.append(result) for line in output: print(line) paragraph = \u0026#34;Python is a high level general purpose programming language known for its simplicity and readability\u0026#34; justify(paragraph, 40) Q18 \u0026ndash; Full Pipeline raw_csv = \u0026#34;\u0026#34;\u0026#34; name, maths, science, english, history, cs aarav SHARMA, 88, 92, 85, 78, 95 PRIYA patel, 55, 48, 62, 50, 58 rohan VERMA, 72, 68, 75, 70, 65 SNEHA iyer, 95, 98, 92, 88, 97 karan SINGH, 35, 40, 38, 42, 30 meera NAIR, 78, 82, 80, 75, 84 \u0026#34;\u0026#34;\u0026#34; lines = [l for l in raw_csv.strip().split(\u0026#34;\\n\u0026#34;) if l.strip()] headers = [h.strip() for h in lines[0].split(\u0026#34;,\u0026#34;)] subjects = headers[1:] students = [] for line in lines[1:]: parts = [p.strip() for p in line.split(\u0026#34;,\u0026#34;)] record = { \u0026#34;name\u0026#34;: parts[0].title(), \u0026#34;maths\u0026#34;: int(parts[1]), \u0026#34;science\u0026#34;: int(parts[2]), \u0026#34;english\u0026#34;: int(parts[3]), \u0026#34;history\u0026#34;: int(parts[4]), \u0026#34;cs\u0026#34;: int(parts[5]), } record[\u0026#34;total\u0026#34;] = sum(record[s] for s in subjects) record[\u0026#34;pct\u0026#34;] = round(record[\u0026#34;total\u0026#34;] / 500 * 100, 1) if record[\u0026#34;pct\u0026#34;] \u0026gt;= 85: record[\u0026#34;grade\u0026#34;] = \u0026#34;Distinction\u0026#34; elif record[\u0026#34;pct\u0026#34;] \u0026gt;= 60: record[\u0026#34;grade\u0026#34;] = \u0026#34;First Class\u0026#34; elif record[\u0026#34;pct\u0026#34;] \u0026gt;= 40: record[\u0026#34;grade\u0026#34;] = \u0026#34;Pass\u0026#34; else: record[\u0026#34;grade\u0026#34;] = \u0026#34;Fail\u0026#34; students.append(record) # Report print(\u0026#34;=\u0026#34; * 65) print(f\u0026#34;{\u0026#39;Name\u0026#39;:\u0026lt;18} {\u0026#39;Total\u0026#39;:\u0026gt;6} {\u0026#39;Pct\u0026#39;:\u0026gt;7} {\u0026#39;Grade\u0026#39;}\u0026#34;) print(\u0026#34;=\u0026#34; * 65) for s in students: print(f\u0026#34;{s[\u0026#39;name\u0026#39;]:\u0026lt;18} {s[\u0026#39;total\u0026#39;]:\u0026gt;6}/500 {s[\u0026#39;pct\u0026#39;]:\u0026gt;6}% {s[\u0026#39;grade\u0026#39;]}\u0026#34;) # Summary print(\u0026#34;=\u0026#34; * 65) avg_pct = round(sum(s[\u0026#34;pct\u0026#34;] for s in students) / len(students), 1) topper = max(students, key=lambda s: s[\u0026#34;pct\u0026#34;]) failures = sum(1 for s in students if s[\u0026#34;grade\u0026#34;] == \u0026#34;Fail\u0026#34;) print(f\u0026#34;\\nClass Average : {avg_pct}%\u0026#34;) print(f\u0026#34;Topper : {topper[\u0026#39;name\u0026#39;]} ({topper[\u0026#39;pct\u0026#39;]}%)\u0026#34;) print(f\u0026#34;Failures : {failures}\u0026#34;) 18. What Comes Next Now that you have a solid foundation in Python basics, here is where this fits in the full curriculum:\nStep Topic Status 1 Python Basics (this document) Complete 2 Python Collections \u0026ndash; List, Tuple, Dict, Set, String Next 3 Python Loops \u0026ndash; for, while, comprehensions Available 4 Python Functions \u0026ndash; all concepts Available 5 NumPy \u0026ndash; numerical computing Available 6 Pandas \u0026ndash; data analysis Available 7 Matplotlib \u0026ndash; data visualisation Available 8 Seaborn \u0026ndash; statistical visualisation Available Before moving to Collections: Make sure you are comfortable with:\nCreating and using variables of all basic types All string methods and slicing Arithmetic and comparison operators Type conversion f-string formatting Boolean logic and truthiness The collections blog (lists, tuples, dictionaries, sets) builds directly on strings because strings are sequences \u0026ndash; everything you learned about indexing and slicing applies there too.\nMade with care for Codeverra learners | codeverra.com\n","permalink":"https://blogs.codeverra.com/blog/python-01-python-basics/","summary":"\u003ch1 id=\"python-basics-masterclass\"\u003ePython Basics Masterclass\u003c/h1\u003e\n\u003ch3 id=\"variables-data-types-strings-operators-and-inputoutput--a-complete-foundation\"\u003eVariables, Data Types, Strings, Operators, and Input/Output \u0026ndash; A Complete Foundation\u003c/h3\u003e\n\u003chr\u003e\n\u003cblockquote\u003e\n\u003cp\u003eWho is this for?\nYou are new to Python or just getting started. This is the very first thing to read and practice\nbefore moving on to collections, loops, functions, or anything else in the curriculum.\nEvery concept is explained simply, with small Indian-context examples, before moving to the next.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003chr\u003e\n\u003ch2 id=\"table-of-contents\"\u003eTable of Contents\u003c/h2\u003e\n\u003col\u003e\n\u003cli\u003e\u003ca href=\"#1-how-python-works\"\u003eHow Python Works\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#2-your-first-python-program\"\u003eYour First Python Program\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#3-comments\"\u003eComments\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#4-variables-and-assignment\"\u003eVariables and Assignment\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#5-naming-rules-and-conventions\"\u003eNaming Rules and Conventions\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#6-python-keywords\"\u003ePython Keywords\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#7-numbers----int-float-complex\"\u003eNumbers \u0026ndash; int, float, complex\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#8-the-math-module\"\u003eThe math Module\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#9-booleans\"\u003eBooleans\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#10-none----the-absence-of-a-value\"\u003eNone \u0026ndash; The Absence of a Value\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#11-strings----complete-guide\"\u003eStrings \u0026ndash; Complete Guide\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#12-operators\"\u003eOperators\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#13-type-conversion\"\u003eType Conversion\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#14-input-and-output\"\u003eInput and Output\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#15-f-strings-and-string-formatting\"\u003ef-strings and String Formatting\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#16-practice-questions\"\u003ePractice Questions\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#17-solutions\"\u003eSolutions\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#18-what-comes-next\"\u003eWhat Comes Next\u003c/a\u003e\u003c/li\u003e\n\u003c/ol\u003e\n\u003chr\u003e\n\u003ch2 id=\"1-how-python-works\"\u003e1. How Python Works\u003c/h2\u003e\n\u003cp\u003eWhen you write Python code and run it, here is what happens behind the scenes:\u003c/p\u003e","tags":["python"],"title":"Python Basics Masterclass"},{"content":"Python Collections Lists, Tuples, Dictionaries, and Sets \u0026ndash; The Four Ways to Store Data Before We Begin \u0026ndash; Why Collections Are Central to Everything At some point you stop writing programs that work on one value at a time and start writing programs that work on many values at once. A list of students. A dictionary of city populations. A set of unique tags. A tuple representing a coordinate.\nCollections are the bridge between \u0026ldquo;I know Python basics\u0026rdquo; and \u0026ldquo;I can solve real problems.\u0026rdquo; Everything that follows in this curriculum \u0026ndash; loops, functions, Pandas, APIs \u0026ndash; assumes you are comfortable with collections.\nPython has four built-in collection types, each designed for a specific kind of data storage problem:\nList -- ordered, mutable, allows duplicates \u0026#34;a sequence of things I can change\u0026#34; Tuple -- ordered, immutable, allows duplicates \u0026#34;a fixed record that should not change\u0026#34; Dictionary -- key-value pairs, ordered (3.7+), mutable, unique keys \u0026#34;a lookup table\u0026#34; Set -- unordered, mutable, no duplicates \u0026#34;a bag of unique things\u0026#34; This guide covers each one completely \u0026ndash; how they work, all their methods, when to use them, and the patterns you will see in real code.\nTable of Contents Lists List Methods \u0026ndash; Complete Reference List Patterns and Idioms Tuples When to Use a Tuple vs a List Dictionaries Dictionary Methods \u0026ndash; Complete Reference Dictionary Patterns and Idioms Sets Set Operations \u0026ndash; Complete Reference Strings as Sequences Nested Collections Choosing the Right Collection The collections Module Summary and Key Takeaways 1. Lists A list is an ordered sequence of items. Items can be of any type and can be repeated. You can add, remove, and change items after the list is created. This is what \u0026ldquo;mutable\u0026rdquo; means \u0026ndash; it can be changed.\nCreating lists # Empty list students = [] scores = list() # List with items cities = [\u0026#34;Delhi\u0026#34;, \u0026#34;Mumbai\u0026#34;, \u0026#34;Bangalore\u0026#34;, \u0026#34;Chennai\u0026#34;] temperatures = [32.5, 28.0, 22.0, 35.0] mixed = [\u0026#34;Aarav\u0026#34;, 21, 8.9, True, None] # any types allowed nested = [[1, 2], [3, 4], [5, 6]] # lists inside lists # From another iterable numbers = list(range(1, 11)) # [1, 2, 3, 4, 5, 6, 7, 8, 9, 10] chars = list(\u0026#34;India\u0026#34;) # [\u0026#39;I\u0026#39;, \u0026#39;n\u0026#39;, \u0026#39;d\u0026#39;, \u0026#39;i\u0026#39;, \u0026#39;a\u0026#39;] Indexing \u0026ndash; accessing single elements ipl_teams = [\u0026#34;CSK\u0026#34;, \u0026#34;MI\u0026#34;, \u0026#34;RCB\u0026#34;, \u0026#34;KKR\u0026#34;, \u0026#34;DC\u0026#34;, \u0026#34;SRH\u0026#34;] # 0 1 2 3 4 5 (positive index) # -6 -5 -4 -3 -2 -1 (negative index) print(ipl_teams[0]) # CSK (first element) print(ipl_teams[2]) # RCB (third element) print(ipl_teams[-1]) # SRH (last element) print(ipl_teams[-2]) # DC (second to last) Slicing \u0026ndash; accessing a range of elements Slicing creates a new list. It never modifies the original.\n# Syntax: list[start:stop:step] # start -- inclusive (default: 0) # stop -- exclusive (default: end) # step -- how many to jump (default: 1) teams = [\u0026#34;CSK\u0026#34;, \u0026#34;MI\u0026#34;, \u0026#34;RCB\u0026#34;, \u0026#34;KKR\u0026#34;, \u0026#34;DC\u0026#34;, \u0026#34;SRH\u0026#34;, \u0026#34;GT\u0026#34;, \u0026#34;LSG\u0026#34;] print(teams[1:4]) # [\u0026#39;MI\u0026#39;, \u0026#39;RCB\u0026#39;, \u0026#39;KKR\u0026#39;] (index 1, 2, 3) print(teams[:3]) # [\u0026#39;CSK\u0026#39;, \u0026#39;MI\u0026#39;, \u0026#39;RCB\u0026#39;] (first 3) print(teams[5:]) # [\u0026#39;SRH\u0026#39;, \u0026#39;GT\u0026#39;, \u0026#39;LSG\u0026#39;] (from index 5 to end) print(teams[::2]) # [\u0026#39;CSK\u0026#39;, \u0026#39;RCB\u0026#39;, \u0026#39;DC\u0026#39;, \u0026#39;GT\u0026#39;] (every 2nd) print(teams[::-1]) # reversed list print(teams[-3:]) # last 3 elements # Slicing a copy copy = teams[:] # creates a shallow copy of the entire list Modifying lists players = [\u0026#34;Rohit\u0026#34;, \u0026#34;Kohli\u0026#34;, \u0026#34;Dhoni\u0026#34;, \u0026#34;Jadeja\u0026#34;, \u0026#34;Bumrah\u0026#34;] # Change an element players[2] = \u0026#34;Pant\u0026#34; print(players) # [\u0026#39;Rohit\u0026#39;, \u0026#39;Kohli\u0026#39;, \u0026#39;Pant\u0026#39;, \u0026#39;Jadeja\u0026#39;, \u0026#39;Bumrah\u0026#39;] # Change a slice players[1:3] = [\u0026#34;Virat\u0026#34;, \u0026#34;MS\u0026#34;, \u0026#34;SKY\u0026#34;] # can change length too print(players) # [\u0026#39;Rohit\u0026#39;, \u0026#39;Virat\u0026#39;, \u0026#39;MS\u0026#39;, \u0026#39;SKY\u0026#39;, \u0026#39;Jadeja\u0026#39;, \u0026#39;Bumrah\u0026#39;] # Delete an element del players[0] print(players) # [\u0026#39;Virat\u0026#39;, \u0026#39;MS\u0026#39;, \u0026#39;SKY\u0026#39;, \u0026#39;Jadeja\u0026#39;, \u0026#39;Bumrah\u0026#39;] Checking membership cities = [\u0026#34;Delhi\u0026#34;, \u0026#34;Mumbai\u0026#34;, \u0026#34;Bangalore\u0026#34;] print(\u0026#34;Mumbai\u0026#34; in cities) # True print(\u0026#34;Kolkata\u0026#34; in cities) # False print(\u0026#34;Kolkata\u0026#34; not in cities) # True 2. List Methods \u0026ndash; Complete Reference fruits = [\u0026#34;mango\u0026#34;, \u0026#34;banana\u0026#34;, \u0026#34;apple\u0026#34;, \u0026#34;mango\u0026#34;, \u0026#34;orange\u0026#34;] # ── Adding elements ────────────────────────────────────────────────────── fruits.append(\u0026#34;guava\u0026#34;) # add to end print(fruits) # [\u0026#39;mango\u0026#39;, \u0026#39;banana\u0026#39;, \u0026#39;apple\u0026#39;, \u0026#39;mango\u0026#39;, \u0026#39;orange\u0026#39;, \u0026#39;guava\u0026#39;] fruits.insert(2, \u0026#34;papaya\u0026#34;) # insert at index 2 print(fruits) # [\u0026#39;mango\u0026#39;, \u0026#39;banana\u0026#39;, \u0026#39;papaya\u0026#39;, \u0026#39;apple\u0026#39;, \u0026#39;mango\u0026#39;, \u0026#39;orange\u0026#39;, \u0026#39;guava\u0026#39;] fruits.extend([\u0026#34;grapes\u0026#34;, \u0026#34;kiwi\u0026#34;]) # add multiple items from another iterable # equivalent to: fruits += [\u0026#34;grapes\u0026#34;, \u0026#34;kiwi\u0026#34;] # ── Removing elements ──────────────────────────────────────────────────── fruits.remove(\u0026#34;banana\u0026#34;) # removes FIRST occurrence of this value # raises ValueError if not found popped = fruits.pop() # removes and returns the LAST element popped = fruits.pop(0) # removes and returns element at index 0 fruits.clear() # removes ALL elements # ── Searching ──────────────────────────────────────────────────────────── teams = [\u0026#34;CSK\u0026#34;, \u0026#34;MI\u0026#34;, \u0026#34;RCB\u0026#34;, \u0026#34;KKR\u0026#34;, \u0026#34;MI\u0026#34;, \u0026#34;CSK\u0026#34;] print(teams.index(\u0026#34;MI\u0026#34;)) # 1 -- index of FIRST occurrence # raises ValueError if not found print(teams.count(\u0026#34;CSK\u0026#34;)) # 2 -- how many times it appears # ── Sorting ────────────────────────────────────────────────────────────── scores = [45, 88, 72, 91, 60, 38, 55] scores.sort() # sort IN PLACE (modifies original) print(scores) # [38, 45, 55, 60, 72, 88, 91] scores.sort(reverse=True) # sort descending print(scores) # [91, 88, 72, 60, 55, 45, 38] # Sort by a key function students = [\u0026#34;Priya\u0026#34;, \u0026#34;Aarav\u0026#34;, \u0026#34;Rohan\u0026#34;, \u0026#34;Sneha\u0026#34;, \u0026#34;Karan\u0026#34;] students.sort(key=len) # sort by string length print(students) students.sort(key=str.lower) # sort case-insensitively # sorted() -- returns a NEW sorted list, does not modify original original = [5, 2, 8, 1, 9] new_list = sorted(original) # original unchanged print(original) # [5, 2, 8, 1, 9] print(new_list) # [1, 2, 5, 8, 9] # ── Other operations ───────────────────────────────────────────────────── nums = [1, 2, 3] nums.reverse() # reverses IN PLACE print(nums) # [3, 2, 1] nums_copy = nums.copy() # shallow copy (same as nums[:]) # ── Built-in functions ─────────────────────────────────────────────────── scores = [45, 88, 72, 91, 60] print(len(scores)) # 5 -- number of items print(min(scores)) # 45 -- minimum value print(max(scores)) # 91 -- maximum value print(sum(scores)) # 356 -- sum of all values print(sum(scores) / len(scores)) # 71.2 -- average 3. List Patterns and Idioms List comprehension The most important list technique. Creates a new list from an existing iterable in one readable line.\n# Basic comprehension: [expression for item in iterable] marks = [88, 45, 72, 95, 60, 38, 81] # Double all marks doubled = [m * 2 for m in marks] # Add 5% bonus to all with_bonus = [round(m * 1.05, 1) for m in marks] # String transformation cities = [\u0026#34;delhi\u0026#34;, \u0026#34;mumbai\u0026#34;, \u0026#34;bangalore\u0026#34;] proper = [city.title() for city in cities] # With condition: [expression for item in iterable if condition] passed = [m for m in marks if m \u0026gt;= 60] # [88, 72, 95, 60, 81] # Condition in the expression (if-else inside comprehension) results = [\u0026#34;Pass\u0026#34; if m \u0026gt;= 60 else \u0026#34;Fail\u0026#34; for m in marks] # Nested comprehension (flatten a 2D list) matrix = [[1, 2, 3], [4, 5, 6], [7, 8, 9]] flat = [val for row in matrix for val in row] # [1, 2, 3, 4, 5, 6, 7, 8, 9] Unpacking # Unpack list into variables first, second, third = [10, 20, 30] print(first, second, third) # 10 20 30 # Extended unpacking with * (star) first, *rest = [1, 2, 3, 4, 5] *start, last = [1, 2, 3, 4, 5] first, *middle, last = [1, 2, 3, 4, 5] print(first) # 1 print(rest) # [2, 3, 4, 5] print(last) # 5 print(middle) # [2, 3, 4] # Swap two variables (no temp variable needed) a, b = 10, 20 a, b = b, a print(a, b) # 20 10 Checking list contents marks = [88, 45, 72, 95, 60] # any() -- True if at least one element satisfies the condition print(any(m \u0026gt;= 90 for m in marks)) # True (95 \u0026gt;= 90) print(any(m \u0026gt;= 100 for m in marks)) # False # all() -- True if ALL elements satisfy the condition print(all(m \u0026gt;= 40 for m in marks)) # True (all \u0026gt;= 40) print(all(m \u0026gt;= 60 for m in marks)) # False (45 \u0026lt; 60) 4. Tuples A tuple is like a list, but immutable \u0026ndash; once created, it cannot be changed. You cannot add, remove, or modify elements.\nCreating tuples # Empty tuple empty = () empty = tuple() # Single item tuple -- the comma is required single = (42,) # without comma: (42) is just 42 in parentheses single = 42, # parentheses are optional # Multiple items coordinates = (28.6139, 77.2090) # Delhi lat/lon rgb_red = (255, 0, 0) student = (\u0026#34;Aarav Sharma\u0026#34;, 101, 8.9, \u0026#34;Computer Science\u0026#34;) # Parentheses are optional (but recommended for clarity) point = 3, 4 Accessing tuple elements student = (\u0026#34;Aarav Sharma\u0026#34;, 101, 8.9, \u0026#34;Computer Science\u0026#34;) print(student[0]) # Aarav Sharma print(student[-1]) # Computer Science print(student[1:3]) # (101, 8.9) -- slicing returns a tuple Tuple unpacking # Basic unpacking name, roll, cgpa, dept = (\u0026#34;Aarav Sharma\u0026#34;, 101, 8.9, \u0026#34;CS\u0026#34;) # Swap values a, b = 10, 20 a, b = b, a # Ignore values with _ name, _, cgpa, _ = (\u0026#34;Aarav\u0026#34;, 101, 8.9, \u0026#34;CS\u0026#34;) # _ is a convention for \u0026#34;I don\u0026#39;t need this value\u0026#34; # Multiple return values from a function (returns a tuple) def get_stats(numbers): return min(numbers), max(numbers), sum(numbers) / len(numbers) low, high, avg = get_stats([45, 88, 72, 91, 60]) print(f\u0026#34;Low: {low}, High: {high}, Avg: {avg}\u0026#34;) Tuple methods Tuples have only two methods (because they are immutable):\ncities = (\u0026#34;Delhi\u0026#34;, \u0026#34;Mumbai\u0026#34;, \u0026#34;Delhi\u0026#34;, \u0026#34;Bangalore\u0026#34;, \u0026#34;Delhi\u0026#34;) print(cities.count(\u0026#34;Delhi\u0026#34;)) # 3 print(cities.index(\u0026#34;Mumbai\u0026#34;)) # 1 -- index of first occurrence Tuples as dictionary keys Because tuples are immutable, they can be used as dictionary keys. Lists cannot.\n# Map city coordinates to city names locations = { (28.6139, 77.2090): \u0026#34;Delhi\u0026#34;, (19.0760, 72.8777): \u0026#34;Mumbai\u0026#34;, (12.9716, 77.5946): \u0026#34;Bangalore\u0026#34;, } lookup = (28.6139, 77.2090) print(locations[lookup]) # Delhi # Lists cannot be dict keys (unhashable) # {[1, 2]: \u0026#34;value\u0026#34;} # TypeError: unhashable type: \u0026#39;list\u0026#39; 5. When to Use a Tuple vs a List Use a TUPLE when: - The data represents a fixed record that should not change Example: GPS coordinates (lat, lon), an RGB colour (r, g, b) - Returning multiple values from a function - Using as a dictionary key - Performance matters -- tuples are slightly faster than lists - You want to signal to readers: \u0026#34;this data is not meant to change\u0026#34; Use a LIST when: - You need to add, remove, or modify elements - The number of elements changes over time - Order matters but the contents can change - You are building up a collection iteratively # Good uses of tuples MONTHS = (\u0026#34;Jan\u0026#34;, \u0026#34;Feb\u0026#34;, \u0026#34;Mar\u0026#34;, \u0026#34;Apr\u0026#34;, \u0026#34;May\u0026#34;, \u0026#34;Jun\u0026#34;, \u0026#34;Jul\u0026#34;, \u0026#34;Aug\u0026#34;, \u0026#34;Sep\u0026#34;, \u0026#34;Oct\u0026#34;, \u0026#34;Nov\u0026#34;, \u0026#34;Dec\u0026#34;) DB_CONFIG = (\u0026#34;localhost\u0026#34;, 5432, \u0026#34;mydb\u0026#34;) # host, port, name DELHI_COORDS = (28.6139, 77.2090) # Good uses of lists students = [\u0026#34;Aarav\u0026#34;, \u0026#34;Priya\u0026#34;, \u0026#34;Rohan\u0026#34;] # students join and leave scores = [] # accumulate during processing current_queue = [\u0026#34;task1\u0026#34;, \u0026#34;task2\u0026#34;] # items added and removed 6. Dictionaries A dictionary stores data as key-value pairs. You look up values by their key rather than by position. Keys must be unique and immutable (strings, numbers, tuples). Values can be anything.\nCreating dictionaries # Empty dictionary profile = {} profile = dict() # With initial data student = { \u0026#34;name\u0026#34;: \u0026#34;Aarav Sharma\u0026#34;, \u0026#34;roll\u0026#34;: 101, \u0026#34;cgpa\u0026#34;: 8.9, \u0026#34;courses\u0026#34;: [\u0026#34;Python\u0026#34;, \u0026#34;Data Science\u0026#34;, \u0026#34;SQL\u0026#34;], \u0026#34;active\u0026#34;: True, } # From a list of key-value pairs items = [(\u0026#34;name\u0026#34;, \u0026#34;Priya\u0026#34;), (\u0026#34;age\u0026#34;, 22), (\u0026#34;city\u0026#34;, \u0026#34;Mumbai\u0026#34;)] person = dict(items) # Using keyword arguments config = dict(host=\u0026#34;localhost\u0026#34;, port=5432, db=\u0026#34;mydb\u0026#34;) Accessing values student = {\u0026#34;name\u0026#34;: \u0026#34;Aarav\u0026#34;, \u0026#34;roll\u0026#34;: 101, \u0026#34;cgpa\u0026#34;: 8.9} # Direct access -- raises KeyError if key does not exist print(student[\u0026#34;name\u0026#34;]) # Aarav print(student[\u0026#34;roll\u0026#34;]) # 101 # .get() -- returns None (or a default) if key does not exist print(student.get(\u0026#34;cgpa\u0026#34;)) # 8.9 print(student.get(\u0026#34;email\u0026#34;)) # None print(student.get(\u0026#34;email\u0026#34;, \u0026#34;N/A\u0026#34;)) # N/A # Always use .get() when the key might not exist Modifying dictionaries student = {\u0026#34;name\u0026#34;: \u0026#34;Aarav\u0026#34;, \u0026#34;roll\u0026#34;: 101} # Add a new key student[\u0026#34;cgpa\u0026#34;] = 8.9 # Update an existing key student[\u0026#34;cgpa\u0026#34;] = 9.1 # Add/update multiple keys at once student.update({\u0026#34;city\u0026#34;: \u0026#34;Bangalore\u0026#34;, \u0026#34;year\u0026#34;: 2}) student.update(cgpa=9.2, active=True) # keyword argument form # Delete a key del student[\u0026#34;roll\u0026#34;] # Delete and return a value cgpa = student.pop(\u0026#34;cgpa\u0026#34;) # returns 9.2, removes key city = student.pop(\u0026#34;city\u0026#34;, \u0026#34;Unknown\u0026#34;) # safe pop -- returns \u0026#34;Unknown\u0026#34; if missing Checking keys config = {\u0026#34;host\u0026#34;: \u0026#34;localhost\u0026#34;, \u0026#34;port\u0026#34;: 5432} print(\u0026#34;host\u0026#34; in config) # True print(\u0026#34;user\u0026#34; in config) # False print(\u0026#34;user\u0026#34; not in config) # True # Iterating for key in config: # iterate over keys print(key, config[key]) for key in config.keys(): # same as above, explicit print(key) for value in config.values(): # iterate over values print(value) for key, value in config.items(): # iterate over key-value pairs print(f\u0026#34;{key}: {value}\u0026#34;) 7. Dictionary Methods \u0026ndash; Complete Reference inventory = {\u0026#34;Laptop\u0026#34;: 15, \u0026#34;Phone\u0026#34;: 42, \u0026#34;Tablet\u0026#34;: 8, \u0026#34;Monitor\u0026#34;: 20} # ── Accessing ───────────────────────────────────────────────────────────── print(inventory.get(\u0026#34;Laptop\u0026#34;, 0)) # 15 print(inventory.get(\u0026#34;Printer\u0026#34;, 0)) # 0 (default) # setdefault -- add key with default ONLY if key does not exist inventory.setdefault(\u0026#34;Keyboard\u0026#34;, 0) # adds \u0026#34;Keyboard\u0026#34;: 0 inventory.setdefault(\u0026#34;Laptop\u0026#34;, 999) # does NOT update -- Laptop already exists print(inventory[\u0026#34;Laptop\u0026#34;]) # still 15 # ── Views ──────────────────────────────────────────────────────────────── print(inventory.keys()) # dict_keys([\u0026#39;Laptop\u0026#39;, \u0026#39;Phone\u0026#39;, ...]) print(inventory.values()) # dict_values([15, 42, ...]) print(inventory.items()) # dict_items([(\u0026#39;Laptop\u0026#39;, 15), ...]) # Convert to list if you need to index or slice key_list = list(inventory.keys()) # ── Removing ───────────────────────────────────────────────────────────── removed = inventory.pop(\u0026#34;Tablet\u0026#34;) # returns 8, removes key removed = inventory.pop(\u0026#34;Headphones\u0026#34;, 0) # safe -- returns 0 if missing last = inventory.popitem() # removes and returns last inserted pair inventory.clear() # removes all pairs # ── Merging ────────────────────────────────────────────────────────────── defaults = {\u0026#34;timeout\u0026#34;: 30, \u0026#34;retries\u0026#34;: 3, \u0026#34;debug\u0026#34;: False} overrides = {\u0026#34;timeout\u0026#34;: 60, \u0026#34;debug\u0026#34;: True} # update() -- modifies in place merged = dict(defaults) merged.update(overrides) print(merged) # {\u0026#39;timeout\u0026#39;: 60, \u0026#39;retries\u0026#39;: 3, \u0026#39;debug\u0026#39;: True} # | operator (Python 3.9+) -- creates new dict merged = defaults | overrides print(merged) # |= operator (Python 3.9+) -- update in place defaults |= overrides # ── Copying ────────────────────────────────────────────────────────────── shallow = inventory.copy() # top-level copy import copy deep = copy.deepcopy(inventory) # full independent copy 8. Dictionary Patterns and Idioms Dictionary comprehension # Basic: {key_expr: value_expr for item in iterable} students = [\u0026#34;Aarav\u0026#34;, \u0026#34;Priya\u0026#34;, \u0026#34;Rohan\u0026#34;, \u0026#34;Sneha\u0026#34;] marks = [88, 92, 78, 95] grade_dict = {name: mark for name, mark in zip(students, marks)} print(grade_dict) # {\u0026#39;Aarav\u0026#39;: 88, \u0026#39;Priya\u0026#39;: 92, \u0026#39;Rohan\u0026#39;: 78, \u0026#39;Sneha\u0026#39;: 95} # With condition passing = {name: mark for name, mark in grade_dict.items() if mark \u0026gt;= 80} # Transform values with_grade = { name: (\u0026#34;A\u0026#34; if m \u0026gt;= 85 else \u0026#34;B\u0026#34; if m \u0026gt;= 70 else \u0026#34;C\u0026#34;) for name, m in grade_dict.items() } # Invert a dictionary inverted = {v: k for k, v in grade_dict.items()} Using setdefault for grouping # Group employees by department employees = [ (\u0026#34;Aarav\u0026#34;, \u0026#34;Engineering\u0026#34;), (\u0026#34;Priya\u0026#34;, \u0026#34;HR\u0026#34;), (\u0026#34;Rohan\u0026#34;, \u0026#34;Engineering\u0026#34;), (\u0026#34;Sneha\u0026#34;, \u0026#34;Marketing\u0026#34;), (\u0026#34;Karan\u0026#34;, \u0026#34;HR\u0026#34;), ] by_department = {} for name, dept in employees: by_department.setdefault(dept, []).append(name) print(by_department) # {\u0026#39;Engineering\u0026#39;: [\u0026#39;Aarav\u0026#39;, \u0026#39;Rohan\u0026#39;], \u0026#39;HR\u0026#39;: [\u0026#39;Priya\u0026#39;, \u0026#39;Karan\u0026#39;], \u0026#39;Marketing\u0026#39;: [\u0026#39;Sneha\u0026#39;]} Using a dictionary as a lookup table (replacing if-elif) # Instead of: def get_day_name(day_num): if day_num == 1: return \u0026#34;Monday\u0026#34; elif day_num == 2: return \u0026#34;Tuesday\u0026#34; elif day_num == 3: return \u0026#34;Wednesday\u0026#34; # ... etc # Use a dictionary: DAY_NAMES = { 1: \u0026#34;Monday\u0026#34;, 2: \u0026#34;Tuesday\u0026#34;, 3: \u0026#34;Wednesday\u0026#34;, 4: \u0026#34;Thursday\u0026#34;, 5: \u0026#34;Friday\u0026#34;, 6: \u0026#34;Saturday\u0026#34;, 7: \u0026#34;Sunday\u0026#34; } def get_day_name(day_num): return DAY_NAMES.get(day_num, \u0026#34;Invalid day\u0026#34;) print(get_day_name(5)) # Friday print(get_day_name(9)) # Invalid day Counting frequencies # Manually count word frequencies words = \u0026#34;the cat sat on the mat the cat wore a hat\u0026#34;.split() freq = {} for word in words: freq[word] = freq.get(word, 0) + 1 print(sorted(freq.items(), key=lambda x: x[1], reverse=True)) # [(\u0026#39;the\u0026#39;, 3), (\u0026#39;cat\u0026#39;, 2), ...] # Using collections.Counter (cleaner) from collections import Counter freq = Counter(words) print(freq.most_common(3)) 9. Sets A set is an unordered collection of unique elements. Duplicates are automatically removed. Sets are extremely fast for membership testing and support mathematical set operations.\nCreating sets # From a literal cities = {\u0026#34;Delhi\u0026#34;, \u0026#34;Mumbai\u0026#34;, \u0026#34;Bangalore\u0026#34;, \u0026#34;Chennai\u0026#34;} # From a list (removes duplicates) tags = set([\u0026#34;python\u0026#34;, \u0026#34;data\u0026#34;, \u0026#34;python\u0026#34;, \u0026#34;ml\u0026#34;, \u0026#34;data\u0026#34;]) print(tags) # {\u0026#39;python\u0026#39;, \u0026#39;data\u0026#39;, \u0026#39;ml\u0026#39;} -- order not guaranteed # Empty set -- MUST use set(), not {} empty_set = set() empty_dict = {} # this creates an empty DICT, not a set! Basic set operations cities = {\u0026#34;Delhi\u0026#34;, \u0026#34;Mumbai\u0026#34;, \u0026#34;Bangalore\u0026#34;} # Membership (O(1) -- very fast, unlike list which is O(n)) print(\u0026#34;Mumbai\u0026#34; in cities) # True print(\u0026#34;Kolkata\u0026#34; in cities) # False # Add and remove cities.add(\u0026#34;Chennai\u0026#34;) # add one element cities.add(\u0026#34;Mumbai\u0026#34;) # adding a duplicate does nothing cities.remove(\u0026#34;Delhi\u0026#34;) # remove -- raises KeyError if not found cities.discard(\u0026#34;Kolkata\u0026#34;) # safe remove -- no error if not found popped = cities.pop() # remove and return an ARBITRARY element cities.clear() # remove all elements 10. Set Operations \u0026ndash; Complete Reference Sets support all the mathematical set operations from set theory.\n# Two teams competing in IPL finals team_a = {\u0026#34;Rohit\u0026#34;, \u0026#34;Virat\u0026#34;, \u0026#34;Bumrah\u0026#34;, \u0026#34;Jadeja\u0026#34;, \u0026#34;Pant\u0026#34;} team_b = {\u0026#34;Dhoni\u0026#34;, \u0026#34;Jadeja\u0026#34;, \u0026#34;Bumrah\u0026#34;, \u0026#34;Ashwin\u0026#34;, \u0026#34;Kohli\u0026#34;} # Union -- all players from EITHER team (no duplicates) all_players = team_a | team_b all_players = team_a.union(team_b) print(all_players) # {\u0026#39;Rohit\u0026#39;, \u0026#39;Virat\u0026#39;, \u0026#39;Bumrah\u0026#39;, \u0026#39;Jadeja\u0026#39;, \u0026#39;Pant\u0026#39;, \u0026#39;Dhoni\u0026#39;, \u0026#39;Ashwin\u0026#39;, \u0026#39;Kohli\u0026#39;} # Intersection -- players in BOTH teams both_teams = team_a \u0026amp; team_b both_teams = team_a.intersection(team_b) print(both_teams) # {\u0026#39;Bumrah\u0026#39;, \u0026#39;Jadeja\u0026#39;} # Difference -- players in team_a but NOT in team_b only_a = team_a - team_b only_a = team_a.difference(team_b) print(only_a) # {\u0026#39;Rohit\u0026#39;, \u0026#39;Virat\u0026#39;, \u0026#39;Pant\u0026#39;} # Symmetric difference -- players in EXACTLY ONE team (not both) exclusive = team_a ^ team_b exclusive = team_a.symmetric_difference(team_b) print(exclusive) # {\u0026#39;Rohit\u0026#39;, \u0026#39;Virat\u0026#39;, \u0026#39;Pant\u0026#39;, \u0026#39;Dhoni\u0026#39;, \u0026#39;Ashwin\u0026#39;, \u0026#39;Kohli\u0026#39;} # Subset -- is team_a a subset of a larger set? squad = {\u0026#34;Rohit\u0026#34;, \u0026#34;Virat\u0026#34;, \u0026#34;Bumrah\u0026#34;, \u0026#34;Jadeja\u0026#34;, \u0026#34;Pant\u0026#34;, \u0026#34;Dhoni\u0026#34;, \u0026#34;Hardik\u0026#34;} playing11 = {\u0026#34;Rohit\u0026#34;, \u0026#34;Virat\u0026#34;, \u0026#34;Bumrah\u0026#34;} print(playing11.issubset(squad)) # True -- all in playing11 are in squad print(squad.issuperset(playing11)) # True -- squad contains all of playing11 print(team_a.isdisjoint(team_b)) # False -- they share elements # In-place set operations team_a |= {\u0026#34;Hardik\u0026#34;, \u0026#34;Surya\u0026#34;} # add elements from another set team_a \u0026amp;= team_b # keep only elements in both team_a -= {\u0026#34;Pant\u0026#34;} # remove elements in second set Set comprehension # {expression for item in iterable if condition} marks = [88, 45, 72, 95, 60, 38, 72, 88] # Unique passing marks unique_passing = {m for m in marks if m \u0026gt;= 60} print(unique_passing) # {88, 72, 95, 60} -- no duplicates, unordered # First letters of city names cities = [\u0026#34;Delhi\u0026#34;, \u0026#34;Mumbai\u0026#34;, \u0026#34;Bangalore\u0026#34;, \u0026#34;Chennai\u0026#34;, \u0026#34;Darjeeling\u0026#34;] first_letters = {city[0] for city in cities} print(first_letters) # {\u0026#39;D\u0026#39;, \u0026#39;M\u0026#39;, \u0026#39;B\u0026#39;, \u0026#39;C\u0026#39;} Frozenset \u0026ndash; immutable sets # frozenset is immutable -- can be used as a dict key or in a set tags1 = frozenset([\u0026#34;python\u0026#34;, \u0026#34;beginner\u0026#34;]) tags2 = frozenset([\u0026#34;python\u0026#34;, \u0026#34;advanced\u0026#34;]) # Can be dict keys course_map = { frozenset([\u0026#34;python\u0026#34;, \u0026#34;beginner\u0026#34;]): \u0026#34;Python 101\u0026#34;, frozenset([\u0026#34;python\u0026#34;, \u0026#34;advanced\u0026#34;]): \u0026#34;Python OOP\u0026#34;, } print(course_map[frozenset([\u0026#34;python\u0026#34;, \u0026#34;beginner\u0026#34;])]) # Python 101 11. Strings as Sequences Strings share many behaviours with lists and tuples because they are also sequences. Everything you learned about indexing and slicing applies.\nname = \u0026#34;Hyderabad\u0026#34; # Indexing print(name[0]) # H print(name[-1]) # d # Slicing print(name[0:5]) # Hyder print(name[::-1]) # darebadyH (reversed) print(name[::2]) # Hdrd # Membership print(\u0026#34;Hyde\u0026#34; in name) # True print(\u0026#34;x\u0026#34; not in name) # True # Iteration for char in name: print(char, end=\u0026#34; \u0026#34;) # H y d e r a b a d # Length print(len(name)) # 9 Key difference from lists: strings are immutable name = \u0026#34;Aarav\u0026#34; name[0] = \u0026#34;a\u0026#34; # TypeError -- strings cannot be modified # To change a string, create a new one name = name.replace(\u0026#34;A\u0026#34;, \u0026#34;a\u0026#34;, 1) # \u0026#34;aarav\u0026#34; # or name = \u0026#34;a\u0026#34; + name[1:] # \u0026#34;aarav\u0026#34; 12. Nested Collections Real-world data is almost always nested. API responses, database records, configuration files \u0026ndash; all involve collections within collections.\nList of dictionaries (the most common pattern) # This pattern represents what you will get from APIs, databases, and CSV files students = [ {\u0026#34;name\u0026#34;: \u0026#34;Aarav Sharma\u0026#34;, \u0026#34;roll\u0026#34;: 101, \u0026#34;marks\u0026#34;: [88, 92, 75, 90, 85]}, {\u0026#34;name\u0026#34;: \u0026#34;Priya Patel\u0026#34;, \u0026#34;roll\u0026#34;: 102, \u0026#34;marks\u0026#34;: [95, 88, 91, 87, 93]}, {\u0026#34;name\u0026#34;: \u0026#34;Rohan Verma\u0026#34;, \u0026#34;roll\u0026#34;: 103, \u0026#34;marks\u0026#34;: [65, 70, 60, 75, 68]}, ] # Access print(students[0][\u0026#34;name\u0026#34;]) # Aarav Sharma print(students[1][\u0026#34;marks\u0026#34;][2]) # 91 (Priya\u0026#39;s third mark) # Process all students for student in students: avg = sum(student[\u0026#34;marks\u0026#34;]) / len(student[\u0026#34;marks\u0026#34;]) grade = \u0026#34;Pass\u0026#34; if avg \u0026gt;= 60 else \u0026#34;Fail\u0026#34; print(f\u0026#34;{student[\u0026#39;name\u0026#39;]}: {avg:.1f} -- {grade}\u0026#34;) # List comprehension on nested data all_names = [s[\u0026#34;name\u0026#34;] for s in students] averages = [sum(s[\u0026#34;marks\u0026#34;]) / len(s[\u0026#34;marks\u0026#34;]) for s in students] toppers = [s[\u0026#34;name\u0026#34;] for s in students if sum(s[\u0026#34;marks\u0026#34;]) / len(s[\u0026#34;marks\u0026#34;]) \u0026gt;= 85] Dictionary of lists # Employees grouped by department by_dept = { \u0026#34;Engineering\u0026#34;: [\u0026#34;Aarav\u0026#34;, \u0026#34;Rohan\u0026#34;, \u0026#34;Meera\u0026#34;], \u0026#34;HR\u0026#34;: [\u0026#34;Priya\u0026#34;, \u0026#34;Karan\u0026#34;], \u0026#34;Sales\u0026#34;: [\u0026#34;Sneha\u0026#34;, \u0026#34;Arjun\u0026#34;, \u0026#34;Divya\u0026#34;], } # Access print(by_dept[\u0026#34;Engineering\u0026#34;]) # [\u0026#39;Aarav\u0026#39;, \u0026#39;Rohan\u0026#39;, \u0026#39;Meera\u0026#39;] print(by_dept[\u0026#34;Engineering\u0026#34;][0]) # Aarav # Iterate for dept, members in by_dept.items(): print(f\u0026#34;{dept}: {len(members)} members -- {\u0026#39;, \u0026#39;.join(members)}\u0026#34;) # Add to a department by_dept[\u0026#34;Engineering\u0026#34;].append(\u0026#34;Vikram\u0026#34;) # Safe access finance_team = by_dept.get(\u0026#34;Finance\u0026#34;, []) # returns [] if Finance not present Deeply nested JSON-style data # Simulating an API response order = { \u0026#34;order_id\u0026#34;: \u0026#34;ORD001\u0026#34;, \u0026#34;customer\u0026#34;: { \u0026#34;name\u0026#34;: \u0026#34;Aarav Sharma\u0026#34;, \u0026#34;city\u0026#34;: \u0026#34;Bangalore\u0026#34;, \u0026#34;contact\u0026#34;: { \u0026#34;email\u0026#34;: \u0026#34;aarav@example.com\u0026#34;, \u0026#34;phone\u0026#34;: \u0026#34;9876543210\u0026#34; } }, \u0026#34;items\u0026#34;: [ {\u0026#34;product\u0026#34;: \u0026#34;Laptop\u0026#34;, \u0026#34;qty\u0026#34;: 1, \u0026#34;price\u0026#34;: 65000}, {\u0026#34;product\u0026#34;: \u0026#34;Mouse\u0026#34;, \u0026#34;qty\u0026#34;: 2, \u0026#34;price\u0026#34;: 850}, {\u0026#34;product\u0026#34;: \u0026#34;Keyboard\u0026#34;, \u0026#34;qty\u0026#34;: 1, \u0026#34;price\u0026#34;: 2200}, ], \u0026#34;status\u0026#34;: \u0026#34;Processing\u0026#34; } # Navigating nested data customer_name = order[\u0026#34;customer\u0026#34;][\u0026#34;name\u0026#34;] customer_email = order[\u0026#34;customer\u0026#34;][\u0026#34;contact\u0026#34;][\u0026#34;email\u0026#34;] first_item = order[\u0026#34;items\u0026#34;][0][\u0026#34;product\u0026#34;] total = sum(item[\u0026#34;qty\u0026#34;] * item[\u0026#34;price\u0026#34;] for item in order[\u0026#34;items\u0026#34;]) print(f\u0026#34;Customer: {customer_name}\u0026#34;) print(f\u0026#34;Email : {customer_email}\u0026#34;) print(f\u0026#34;Total : Rs.{total:,}\u0026#34;) # Safe navigation with .get() phone = (order .get(\u0026#34;customer\u0026#34;, {}) .get(\u0026#34;contact\u0026#34;, {}) .get(\u0026#34;phone\u0026#34;, \u0026#34;Not provided\u0026#34;)) 13. Choosing the Right Collection This decision affects the readability, correctness, and performance of your code. Here is the complete decision guide:\nDo you need to look up values by a key (name, ID, etc.)? YES --\u0026gt; Dictionary Do you need uniqueness? (no duplicates, fast membership test) YES --\u0026gt; Set (or frozenset if it must be immutable) Do you need an ordered sequence? YES --\u0026gt; Continue below NO --\u0026gt; Set or Dictionary Will the sequence change (add/remove/modify)? YES --\u0026gt; List NO --\u0026gt; Tuple Is it fixed data that represents a record (like a coordinate or RGB value)? YES --\u0026gt; Tuple Is it a collection that will grow or shrink over time? YES --\u0026gt; List # Choosing well student_id = 101 # int (not a collection) student_name = \u0026#34;Aarav Sharma\u0026#34; # str (a sequence) coordinates = (28.6139, 77.2090) # tuple (fixed, immutable record) tags = {\u0026#34;python\u0026#34;, \u0026#34;data\u0026#34;, \u0026#34;ml\u0026#34;} # set (unique, no order needed) history = [\u0026#34;Python\u0026#34;, \u0026#34;SQL\u0026#34;, \u0026#34;Pandas\u0026#34;] # list (ordered, can change) profile = {\u0026#34;name\u0026#34;: \u0026#34;Aarav\u0026#34;, \u0026#34;age\u0026#34;: 21} # dict (key-value lookup) Performance comparison import time n = 1_000_000 data_list = list(range(n)) data_set = set(range(n)) data_dict = {i: i for i in range(n)} # Membership check: list vs set vs dict target = 999_999 # List: O(n) -- must check every element start = time.perf_counter() target in data_list print(f\u0026#34;List : {time.perf_counter() - start:.6f}s\u0026#34;) # ~0.01s # Set: O(1) -- hash lookup start = time.perf_counter() target in data_set print(f\u0026#34;Set : {time.perf_counter() - start:.6f}s\u0026#34;) # ~0.000001s # Dict: O(1) -- hash lookup start = time.perf_counter() target in data_dict print(f\u0026#34;Dict : {time.perf_counter() - start:.6f}s\u0026#34;) # ~0.000001s For large membership checks, always use a set or dict, never a list.\n14. The collections Module Python\u0026rsquo;s collections module provides specialised container types that extend the built-in ones.\nCounter Counts occurrences of elements. Returns a dictionary-like object.\nfrom collections import Counter # Count IPL match results results = [\u0026#34;CSK\u0026#34;, \u0026#34;MI\u0026#34;, \u0026#34;CSK\u0026#34;, \u0026#34;RCB\u0026#34;, \u0026#34;MI\u0026#34;, \u0026#34;CSK\u0026#34;, \u0026#34;KKR\u0026#34;, \u0026#34;MI\u0026#34;, \u0026#34;MI\u0026#34;, \u0026#34;CSK\u0026#34;] wins = Counter(results) print(wins) # Counter({\u0026#39;CSK\u0026#39;: 4, \u0026#39;MI\u0026#39;: 4, \u0026#39;KKR\u0026#39;: 1, \u0026#39;RCB\u0026#39;: 1}) print(wins.most_common(2)) # [(\u0026#39;CSK\u0026#39;, 4), (\u0026#39;MI\u0026#39;, 4)] print(wins[\u0026#34;CSK\u0026#34;]) # 4 print(wins[\u0026#34;DC\u0026#34;]) # 0 (no KeyError for missing keys) # Count characters in a string letter_freq = Counter(\u0026#34;Bangalore\u0026#34;) print(letter_freq.most_common(3)) # [(\u0026#39;a\u0026#39;, 2), (\u0026#39;l\u0026#39;, 2), (\u0026#39;B\u0026#39;, 1)] defaultdict A dictionary that automatically creates a default value for missing keys.\nfrom collections import defaultdict # Default value is an empty list by_dept = defaultdict(list) employees = [(\u0026#34;Aarav\u0026#34;, \u0026#34;Engineering\u0026#34;), (\u0026#34;Priya\u0026#34;, \u0026#34;HR\u0026#34;), (\u0026#34;Rohan\u0026#34;, \u0026#34;Engineering\u0026#34;), (\u0026#34;Karan\u0026#34;, \u0026#34;HR\u0026#34;)] for name, dept in employees: by_dept[dept].append(name) # no need to check if key exists print(dict(by_dept)) # {\u0026#39;Engineering\u0026#39;: [\u0026#39;Aarav\u0026#39;, \u0026#39;Rohan\u0026#39;], \u0026#39;HR\u0026#39;: [\u0026#39;Priya\u0026#39;, \u0026#39;Karan\u0026#39;]} # Default value is 0 (useful for counting) word_count = defaultdict(int) for word in \u0026#34;the cat sat on the mat\u0026#34;.split(): word_count[word] += 1 # no KeyError on first increment namedtuple A tuple where each position has a name. Readable, immutable, memory-efficient.\nfrom collections import namedtuple # Create a type Student = namedtuple(\u0026#34;Student\u0026#34;, [\u0026#34;name\u0026#34;, \u0026#34;roll\u0026#34;, \u0026#34;cgpa\u0026#34;, \u0026#34;dept\u0026#34;]) # Create instances s1 = Student(\u0026#34;Aarav Sharma\u0026#34;, 101, 8.9, \u0026#34;CS\u0026#34;) s2 = Student(name=\u0026#34;Priya Patel\u0026#34;, roll=102, cgpa=9.2, dept=\u0026#34;IT\u0026#34;) # Access by name (readable) or index (still works) print(s1.name) # Aarav Sharma print(s1.cgpa) # 8.9 print(s1[0]) # Aarav Sharma (tuple indexing still works) # Unpack like a tuple name, roll, cgpa, dept = s1 # Cannot modify (immutable) # s1.cgpa = 9.0 # AttributeError # Convert to dict print(s1._asdict()) # {\u0026#39;name\u0026#39;: \u0026#39;Aarav Sharma\u0026#39;, \u0026#39;roll\u0026#39;: 101, ...} deque A double-ended queue. Fast O(1) appends and pops from BOTH ends. Lists have O(n) for insert(0) and pop(0).\nfrom collections import deque # Use as a queue (FIFO) queue = deque() queue.append(\u0026#34;task1\u0026#34;) # add to right queue.append(\u0026#34;task2\u0026#34;) queue.append(\u0026#34;task3\u0026#34;) print(queue.popleft()) # \u0026#34;task1\u0026#34; -- remove from left (FIFO order) # Use as a stack (LIFO) stack = deque() stack.append(\u0026#34;page1\u0026#34;) stack.append(\u0026#34;page2\u0026#34;) print(stack.pop()) # \u0026#34;page2\u0026#34; -- remove from right (LIFO order) # appendleft and extendleft queue.appendleft(\u0026#34;urgent_task\u0026#34;) # add to front -- O(1) # Bounded deque -- automatically discards oldest when full recent = deque(maxlen=3) for i in range(6): recent.append(i) print(recent) # deque([3, 4, 5], maxlen=3) OrderedDict Dictionaries in Python 3.7+ maintain insertion order, so OrderedDict is rarely needed. But it has one unique method:\nfrom collections import OrderedDict od = OrderedDict([(\u0026#34;a\u0026#34;, 1), (\u0026#34;b\u0026#34;, 2), (\u0026#34;c\u0026#34;, 3)]) # move_to_end -- not available on regular dict od.move_to_end(\u0026#34;a\u0026#34;) # move \u0026#34;a\u0026#34; to the end od.move_to_end(\u0026#34;c\u0026#34;, last=False) # move \u0026#34;c\u0026#34; to the front 15. Summary and Key Takeaways The four collections at a glance LIST Ordered, mutable, duplicates allowed Use for: sequences that change over time Literal: [1, 2, 3] TUPLE Ordered, immutable, duplicates allowed Use for: fixed records, multiple return values, dict keys Literal: (1, 2, 3) or 1, 2, 3 DICTIONARY Key-value pairs, ordered (3.7+), mutable, unique keys Use for: lookup by key, grouping, counting, config Literal: {\u0026#34;key\u0026#34;: \u0026#34;value\u0026#34;} SET Unordered, mutable, no duplicates Use for: uniqueness, fast membership, set math Literal: {1, 2, 3} -- but set() for empty set! The most important operations to know # List lst.append(x) # add to end lst.insert(i, x) # insert at position lst.remove(x) # remove first occurrence lst.pop(i) # remove and return at index sorted(lst, key=...) # return sorted copy [expr for x in lst if cond] # comprehension # Tuple t[i] # index access a, b, c = t # unpack t.count(x) # count occurrences # Dictionary d.get(k, default) # safe access d.setdefault(k, v) # add only if missing d.update(other) # merge for k, v in d.items() # iterate key-value pairs {k: expr for k, v in d.items()} # comprehension # Set s.add(x) # add element s.discard(x) # safe remove s | other # union s \u0026amp; other # intersection s - other # difference s ^ other # symmetric difference x in s # O(1) membership test Common mistakes to avoid # 1. Empty set vs empty dict empty_set = set() # correct empty_set = {} # WRONG -- this is an empty dict # 2. Modifying a list while iterating over it for item in my_list: my_list.remove(item) # unpredictable behaviour # Correct: iterate over a copy for item in my_list[:]: my_list.remove(item) # Or: build a new list my_list = [item for item in my_list if keep_condition(item)] # 3. Forgetting that sort() modifies in place and returns None sorted_list = my_list.sort() # sorted_list is None! my_list.sort() # sort in place sorted_list = sorted(my_list) # create a new sorted list # 4. Shallow copy vs deep copy with nested structures original = {\u0026#34;scores\u0026#34;: [88, 92]} shallow = original.copy() shallow[\u0026#34;scores\u0026#34;].append(100) print(original[\u0026#34;scores\u0026#34;]) # [88, 92, 100] -- original was affected! import copy deep = copy.deepcopy(original) deep[\u0026#34;scores\u0026#34;].append(999) print(original[\u0026#34;scores\u0026#34;]) # [88, 92, 100] -- original unaffected # 5. Using list for membership checks on large data # O(n) lookup -- slow for large lists if name in large_name_list: # slow if large_name_list has 100,000 items # Convert to set first -- O(1) lookup name_set = set(large_name_list) if name in name_set: # fast Made with care for Codeverra learners | codeverra.com\n","permalink":"https://blogs.codeverra.com/blog/python-05-collections-theory/","summary":"\u003ch1 id=\"python-collections\"\u003ePython Collections\u003c/h1\u003e\n\u003ch3 id=\"lists-tuples-dictionaries-and-sets--the-four-ways-to-store-data\"\u003eLists, Tuples, Dictionaries, and Sets \u0026ndash; The Four Ways to Store Data\u003c/h3\u003e\n\u003chr\u003e\n\u003ch2 id=\"before-we-begin--why-collections-are-central-to-everything\"\u003eBefore We Begin \u0026ndash; Why Collections Are Central to Everything\u003c/h2\u003e\n\u003cp\u003eAt some point you stop writing programs that work on one value at a time\nand start writing programs that work on many values at once. A list of\nstudents. A dictionary of city populations. A set of unique tags.\nA tuple representing a coordinate.\u003c/p\u003e\n\u003cp\u003eCollections are the bridge between \u0026ldquo;I know Python basics\u0026rdquo; and \u0026ldquo;I can solve\nreal problems.\u0026rdquo; Everything that follows in this curriculum \u0026ndash; loops,\nfunctions, Pandas, APIs \u0026ndash; assumes you are comfortable with collections.\u003c/p\u003e","tags":["python"],"title":"Python Collections - Theory"},{"content":"Python Learning Roadmap The Complete Codeverra Curriculum \u0026ndash; Your Navigation Hub This document is the master index for everything published at Codeverra. Every blog, masterclass, and practice workbook links back here. Bookmark this page. Return to it whenever you finish a topic and want to know what is next.\nHow to Use This Roadmap The curriculum is divided into four phases. Each phase builds on the previous one.\nDo not skip phases. A student who rushes into Pandas without knowing loops and functions will struggle badly. Each topic has a blog (concept explanation), a practice workbook (questions), and a solutions file. The estimated time per phase assumes consistent practice of 1 to 2 hours per day. Phase 1 \u0026ndash; Python Foundations Estimated time: 4 to 6 weeks This is where everything starts. Master these before touching any library or framework. Every data scientist, backend engineer, ML engineer, and data engineer uses these concepts every day.\n1.1 Python Basics What you will learn: How Python works, variables, all basic data types (int, float, bool, None), the complete string reference (all methods, slicing, formatting), operators (arithmetic, comparison, logical, membership, identity, bitwise), type conversion, input/output, and f-strings.\nWhy it matters: Everything else in Python is built on these primitives. You cannot understand a list until you understand what a variable is. You cannot understand Pandas until you understand strings.\nResource Description 01_python_basics.md Full masterclass covering all concepts with Indian-context examples Practice questions 18 questions \u0026ndash; Easy to Hard \u0026ndash; with full solutions in the same file Key concepts checklist:\nVariables and dynamic typing int, float, complex and the math module Strings \u0026ndash; creation, indexing, slicing, all methods Booleans and truthiness None and its correct usage All operator types and precedence Type conversion \u0026ndash; implicit and explicit print() and input() in depth f-strings and number formatting 1.2 Control Flow What you will learn: if, elif, else conditionals. Nested conditions. The ternary (one-line if). match-case (Python 3.10+).\nWhy it matters: Programs that cannot make decisions cannot do anything useful. Control flow is how you add logic.\nResource Description 02_control_flow.md Coming soon Key concepts checklist:\nif, elif, else syntax and indentation Nested conditionals One-line ternary: value = a if condition else b match-case statement (Python 3.10+) Common patterns: guard clauses, early returns 1.3 Loops What you will learn: for loops, while loops, range(), break, continue, pass, else on loops, enumerate(), zip(), nested loops, list comprehensions, dictionary and set comprehensions.\nWhy it matters: Loops are how you process collections of data. Everything in data science involves iterating over rows, columns, lists, and dictionaries.\nResource Description 03_loops.md Full masterclass \u0026ndash; 18 sections loops_practice_solutions.md 20 practice questions with full solutions Key concepts checklist:\nfor loop over all iterable types while loop and when to use it break vs continue vs pass else on for and while enumerate() and zip() Nested loops and performance intuition List, dict, and set comprehensions 1.4 Python Collections What you will learn: List, Tuple, Dictionary, Set \u0026ndash; all methods, operations, and patterns. When to use each one.\nWhy it matters: Collections are how you store and organise data in Python. Pandas DataFrames are built on top of these. SQL results come back as lists of dictionaries. APIs return JSON which becomes nested dicts and lists.\nResource Description 04_collections.md Full masterclass with strings, lists, tuples, dicts, sets collections_practice.md 40+ questions across all 5 collection types with solutions Key concepts checklist:\nList: indexing, slicing, all methods, nested lists, comprehensions Tuple: immutability, unpacking, namedtuple, as dict keys Dictionary: all methods, nested dicts, dict comprehensions, merging Set: operations (union, intersection, difference, symmetric difference), frozenset String: (covered in 1.1) \u0026ndash; fits here too as a sequence type When to use which collection Phase 2 \u0026ndash; Core Python Estimated time: 4 to 6 weeks With foundations in place, you now learn the tools that professional Python developers use every day.\n2.1 Functions What you will learn: Defining functions, parameters and arguments, default values, *args and **kwargs, return values, scope and the LEGB rule, global and nonlocal, lambda functions, higher order functions, map/filter/reduce, closures, decorators, recursion, and generator functions.\nWhy it matters: Functions are how you organise code. Decorators are used in every web framework. Closures underpin many design patterns. Generators are essential for processing large datasets efficiently.\nResource Description 05_functions.md Full masterclass \u0026ndash; 23 sections covering all concepts functions_practice_solutions.md 20 practice questions with full solutions Key concepts checklist:\ndef, parameters, arguments, return values Default parameters and the mutable default trap *args and **kwargs Keyword-only and positional-only parameters LEGB scope rule global and nonlocal Docstrings and type annotations Lambda functions Higher order functions map(), filter(), reduce() Closures and function factories Decorators with @syntax and functools.wraps Decorators with arguments Chaining decorators Recursion and the base case Generator functions and yield 2.2 Exception Handling What you will learn: try, except, else, finally, raise, custom exceptions, exception hierarchy.\nWhy it matters: Real programs fail. Exception handling is how you make programs fail gracefully instead of crashing and losing data.\nResource Description 06_exceptions.md Coming soon Key concepts checklist:\ntry, except, else, finally Catching specific exceptions vs bare except Raising exceptions with raise Creating custom exception classes Exception chaining Context managers and the with statement 2.3 File Handling What you will learn: Reading and writing text files, CSV files, JSON files, the with statement, file modes, pathlib.\nWhy it matters: All real data lives in files. Before you learn Pandas, you need to understand what a file actually is and how Python reads it.\nResource Description 07_file_handling.md Coming soon Key concepts checklist:\nopen(), read(), write(), close() The with statement and why it matters File modes: r, w, a, rb, wb Reading line by line Reading and writing CSV manually Reading and writing JSON with the json module pathlib for modern path handling 2.4 Modules and Packages What you will learn: import statement, creating your own modules, pip, virtual environments, the standard library.\nWhy it matters: Every Python project you work on will use external libraries. Understanding how imports and packages work prevents confusing errors and makes you self-sufficient.\nResource Description 08_modules.md Coming soon Key concepts checklist:\nimport, from \u0026hellip; import, import \u0026hellip; as Creating your own module name == \u0026ldquo;main\u0026rdquo; pip install and requirements.txt Virtual environments (venv) Key standard library modules: os, sys, datetime, collections, itertools, functools 2.5 Object-Oriented Programming What you will learn: Classes, objects, init, instance vs class variables, methods, inheritance, encapsulation, polymorphism, special (dunder) methods.\nWhy it matters: Every Python library you use (Pandas, Scikit-learn, Django, FastAPI) is built with classes. You do not need to build complex class hierarchies in everyday data work, but you need to understand what an object is, what methods are, and why df.groupby(\u0026quot;city\u0026quot;) works.\nResource Description 09_oop.md Coming soon Key concepts checklist:\nclass keyword and init self and instance attributes Instance methods, class methods, static methods Inheritance and super() Method overriding Encapsulation with _ and __ Dunder methods: str, repr, len, eq, lt Properties with @property Dataclasses (Python 3.7+) Phase 3 \u0026ndash; Data Tools Estimated time: 6 to 10 weeks This is where Python connects to real-world data work. These are the libraries used by data analysts, data scientists, and ML engineers every day.\n3.1 NumPy What you will learn: N-dimensional arrays, all array operations, indexing and slicing, boolean masking, broadcasting, universal functions, aggregations, linear algebra, random number generation, saving and loading arrays.\nWhy it matters: NumPy is the foundation of every numerical computing library in Python. Pandas, Scikit-learn, TensorFlow, and PyTorch all use NumPy arrays underneath.\nResource Description numpy_masterclass.md Full masterclass \u0026ndash; 19 sections with 15 practice questions numpy_masterclass.md (Section 20) Full solutions to all 15 questions Key concepts checklist:\nndarray creation and data types Indexing, slicing, fancy indexing Boolean masking and np.where Array math \u0026ndash; element-wise and axis-based Universal functions (ufuncs) Aggregation: sum, mean, std, min, max along axes Reshaping, transpose, stacking, splitting Broadcasting rules Random number generation with np.random Linear algebra: dot, inv, solve, eig Sorting and searching Copies vs views 3.2 Pandas What you will learn: Series and DataFrame, reading/writing data, exploring datasets, selecting and filtering, adding and modifying columns, handling missing data, string operations, datetime operations, apply/map, GroupBy, merging and joining, pivot tables, reshaping, and performance tips.\nWhy it matters: Pandas is the primary tool for data manipulation in Python. Every data analyst and data scientist uses it daily. It is the bridge between raw data and analysis.\nResource Description pandas_masterclass.md Full masterclass \u0026ndash; 22 sections with 13 practice questions pandas_masterclass.md (Section 23) Full solutions to all 13 questions Key concepts checklist:\nSeries and DataFrame \u0026ndash; creation and structure read_csv, read_excel, read_json, read_sql head, tail, info, describe, value_counts loc and iloc \u0026ndash; label vs position indexing Boolean filtering, isin, between, query Adding, renaming, dropping columns Sorting values and index Handling missing data \u0026ndash; dropna, fillna, interpolate String operations with .str accessor Datetime operations with .dt accessor apply(), map(), DataFrame.map() GroupBy \u0026ndash; agg, transform, filter merge() \u0026ndash; all four join types concat() for stacking pivot_table and crosstab melt() and stack/unstack Category dtype for memory efficiency 3.3 Matplotlib What you will learn: The full Matplotlib API \u0026ndash; every major chart type (line, bar, scatter, histogram, pie, box, heatmap, area), all customisations (colours, typography, axes, grid, spines, legend, annotations), subplots and gridspec, saving figures, and a complete dataset analysis project.\nWhy it matters: Every chart you will ever present \u0026ndash; in a report, a notebook, a dashboard \u0026ndash; will be built on Matplotlib, even if you use Seaborn or Pandas .plot() on top of it.\nResource Description matplotlib_masterclass.md Full masterclass \u0026ndash; 19 sections with 15 practice questions and solutions Key concepts checklist:\nFigure and Axes \u0026ndash; the two-object model pyplot vs object-oriented interface Line, bar, horizontal bar, scatter, histogram, pie, box, heatmap, area charts Subplots with nrows/ncols and gridspec All customisation options \u0026ndash; colour, font, ticks, grid, spines, legend Annotations and reference lines Colour maps Saving at print quality 3.4 Seaborn What you will learn: All Seaborn plot types (lineplot, barplot, countplot, histplot, kdeplot, ecdfplot, boxplot, violinplot, stripplot, swarmplot, boxenplot, pointplot, scatterplot, regplot, heatmap, pairplot, FacetGrid, catplot, lmplot), themes, palettes, axes-level vs figure-level API, and combining with Matplotlib.\nWhy it matters: Seaborn produces statistical charts \u0026ndash; confidence intervals, regression lines, density curves \u0026ndash; with much less code than Matplotlib. It is the standard for exploratory data analysis.\nResource Description seaborn_masterclass.md Full masterclass \u0026ndash; 26 sections with 15 practice questions and solutions Key concepts checklist:\nAxes-level vs figure-level functions Statistical aggregation built into plots hue, size, style for multi-dimensional encoding FacetGrid for automatic subplot grids All distribution plots \u0026ndash; hist, kde, ecdf All categorical plots \u0026ndash; box, violin, strip, point, bar Regression plots \u0026ndash; regplot, lmplot Heatmaps and pairplots Themes, styles, contexts, palettes 3.5 SQL with Python What you will learn: SQL fundamentals, all query types, joins, aggregations, window functions, connecting Python to databases with SQLAlchemy, using pandas with SQL, and working with real Kaggle datasets.\nWhy it matters: Most data in the world lives in relational databases. SQL is the language of data. Every data analyst, data scientist, and data engineer needs it. Python and SQL together are far more powerful than either alone.\nResource Description SQL with Python course Modules 1-3 available at learn.codeverra.com Key concepts checklist:\nSELECT, WHERE, ORDER BY, LIMIT Aggregate functions: COUNT, SUM, AVG, MIN, MAX GROUP BY and HAVING All JOIN types: INNER, LEFT, RIGHT, FULL OUTER Subqueries and CTEs Window functions: ROW_NUMBER, RANK, LAG, LEAD Connecting Python to SQLite, PostgreSQL pd.read_sql() and df.to_sql() Real dataset analysis with IPL, Zomato, Kaggle data Phase 4 \u0026ndash; Specialisation Estimated time: 3 to 12 months depending on path After completing Phases 1-3, you have enough to choose a direction. Each path below builds on the same core. Pick the one that aligns with your goal.\nPath A \u0026ndash; Data Analyst Goal: Turn raw data into business decisions. Build reports, dashboards, and data stories.\nCore additional skills:\nAdvanced Pandas and SQL Statistical thinking: distributions, hypothesis testing, correlation Matplotlib and Seaborn for storytelling charts Power BI or Tableau for business dashboards Excel integration with openpyxl Typical first job titles: Data Analyst, Business Analyst, Analytics Engineer\nPath B \u0026ndash; Data Scientist Goal: Build predictive models and extract deeper insight from data.\nCore additional skills:\nStatistics: probability, distributions, inference, regression Scikit-learn: preprocessing, model training, evaluation, pipelines Feature engineering Model selection and cross-validation Experiment design and A/B testing Plotly for interactive visualisation Typical first job titles: Data Scientist, ML Researcher (junior), Analyst\nPath C \u0026ndash; Machine Learning Engineer Goal: Build, train, and deploy machine learning models at scale in production.\nCore additional skills:\nDeep learning with PyTorch or TensorFlow MLflow for experiment tracking Docker and containerisation FastAPI for model serving Cloud platforms: AWS SageMaker, GCP Vertex AI, Azure ML MLOps: CI/CD for ML pipelines Typical first job titles: ML Engineer, Applied Scientist, AI Engineer\nPath D \u0026ndash; AI / LLM Engineer Goal: Build applications powered by large language models. One of the fastest-growing roles as of 2024.\nCore additional skills:\nLangChain and LlamaIndex OpenAI, Anthropic, and open-source model APIs Retrieval-Augmented Generation (RAG) Vector databases: Pinecone, Chroma, Weaviate Prompt engineering and evaluation Fine-tuning with PEFT and LoRA FastAPI for building LLM-powered APIs Typical first job titles: LLM Engineer, AI Engineer, Prompt Engineer\nPath E \u0026ndash; Backend Engineer Goal: Build the APIs and server-side logic that power web and mobile applications.\nCore additional skills:\nFastAPI or Django for web frameworks Pydantic for data validation PostgreSQL and Redis SQLAlchemy ORM Docker and docker-compose REST API design Authentication: JWT, OAuth 2.0 Deployment: Linux, Nginx, cloud platforms Typical first job titles: Backend Engineer, Python Developer, API Developer\nPath F \u0026ndash; Data Engineer Goal: Build the pipelines that move, transform, and store data at scale.\nCore additional skills:\nApache Spark with PySpark Apache Airflow for pipeline orchestration dbt for data transformation Apache Kafka for streaming Cloud data warehouses: BigQuery, Redshift, Snowflake ETL pipeline design patterns SQL at scale Typical first job titles: Data Engineer, Analytics Engineer, Pipeline Engineer\nCurriculum Progress Tracker Use this table to track what you have completed.\n# Topic Blog Practice Status 1.1 Python Basics 01_python_basics.md In blog 1.2 Control Flow 02_control_flow.md Coming soon 1.3 Loops 03_loops.md loops_practice_solutions.md 1.4 Collections 04_collections.md collections_practice.md 2.1 Functions 05_functions.md functions_practice_solutions.md 2.2 Exception Handling 06_exceptions.md Coming soon 2.3 File Handling 07_file_handling.md Coming soon 2.4 Modules and Packages 08_modules.md Coming soon 2.5 OOP 09_oop.md Coming soon 3.1 NumPy numpy_masterclass.md In masterclass 3.2 Pandas pandas_masterclass.md In masterclass 3.3 Matplotlib matplotlib_masterclass.md In masterclass 3.4 Seaborn seaborn_masterclass.md In masterclass 3.5 SQL with Python learn.codeverra.com Course exercises How Long Will This Take? Phase Consistent learner (1-2 hrs/day) Intensive learner (4+ hrs/day) Phase 1 4 to 6 weeks 2 to 3 weeks Phase 2 4 to 6 weeks 2 to 3 weeks Phase 3 6 to 10 weeks 3 to 5 weeks Phase 4 3 to 12 months 2 to 6 months Total to first job 6 to 18 months 4 to 12 months The range is wide because it depends on your starting point, how much you practice, and the quality of your projects. The most important variable is not how many tutorials you watch \u0026ndash; it is how many problems you solve yourself.\nThe Most Important Advice on This Page Reading a tutorial and understanding it feels like learning. It is not. Real learning only happens when you sit down with a blank file and build something from memory.\nAfter every blog post, close it and try to reproduce the key examples from scratch. After every section, solve the practice questions before reading the solutions. After every phase, build a small project that uses everything you learned.\nThe students who finish this curriculum in 6 months are not more intelligent. They practice more consistently and they build things.\nMade with care for Codeverra learners | codeverra.com | learn.codeverra.com\n","permalink":"https://blogs.codeverra.com/blog/python-00-python-roadmap/","summary":"\u003ch1 id=\"python-learning-roadmap\"\u003ePython Learning Roadmap\u003c/h1\u003e\n\u003ch3 id=\"the-complete-codeverra-curriculum--your-navigation-hub\"\u003eThe Complete Codeverra Curriculum \u0026ndash; Your Navigation Hub\u003c/h3\u003e\n\u003chr\u003e\n\u003cblockquote\u003e\n\u003cp\u003eThis document is the master index for everything published at Codeverra.\nEvery blog, masterclass, and practice workbook links back here.\nBookmark this page. Return to it whenever you finish a topic and want to know what is next.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003chr\u003e\n\u003ch2 id=\"how-to-use-this-roadmap\"\u003eHow to Use This Roadmap\u003c/h2\u003e\n\u003cp\u003eThe curriculum is divided into four phases. Each phase builds on the previous one.\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003eDo not skip phases. A student who rushes into Pandas without knowing loops and functions will struggle badly.\u003c/li\u003e\n\u003cli\u003eEach topic has a blog (concept explanation), a practice workbook (questions), and a solutions file.\u003c/li\u003e\n\u003cli\u003eThe estimated time per phase assumes consistent practice of 1 to 2 hours per day.\u003c/li\u003e\n\u003c/ul\u003e\n\u003chr\u003e\n\u003ch2 id=\"phase-1--python-foundations\"\u003ePhase 1 \u0026ndash; Python Foundations\u003c/h2\u003e\n\u003ch3 id=\"estimated-time-4-to-6-weeks\"\u003eEstimated time: 4 to 6 weeks\u003c/h3\u003e\n\u003cp\u003eThis is where everything starts. Master these before touching any library or framework.\nEvery data scientist, backend engineer, ML engineer, and data engineer uses these concepts every day.\u003c/p\u003e","tags":["python","learning-roadmap"],"title":"Python Learning Roadmap"},{"content":"🔁 Python Loops Masterclass Complete Command Over for and while Loops — From Basics to Patterns Who is this for? You know Python basics — variables, conditionals, functions. Now you want to truly master loops — not just use them, but think in them. This guide covers every loop concept in Python, with simple examples before every idea.\n📌 Table of Contents Why Loops Exist The for Loop Iterating Over Different Data Types The range() Function The while Loop break — Exit the Loop Early continue — Skip to the Next Iteration pass — Do Nothing (Placeholder) else with Loops enumerate() — Loop with an Index zip() — Loop Over Multiple Sequences Nested Loops List Comprehensions Dictionary and Set Comprehensions Loop Patterns Every Programmer Uses Common Loop Mistakes and How to Avoid Them Practice Questions What We Covered + What\u0026rsquo;s Next 1. Why Loops Exist Imagine you need to print a welcome message for 500 students. Without loops:\nprint(\u0026#34;Welcome, Aarav!\u0026#34;) print(\u0026#34;Welcome, Priya!\u0026#34;) print(\u0026#34;Welcome, Rohan!\u0026#34;) # ... 497 more lines With a loop:\nstudents = [\u0026#34;Aarav\u0026#34;, \u0026#34;Priya\u0026#34;, \u0026#34;Rohan\u0026#34;, \u0026#34;Sneha\u0026#34;, \u0026#34;Karan\u0026#34;] for student in students: print(f\u0026#34;Welcome, {student}!\u0026#34;) Welcome, Aarav! Welcome, Priya! Welcome, Rohan! Welcome, Sneha! Welcome, Karan! That\u0026rsquo;s the entire point of a loop — repeat an action for every item in a sequence, or until a condition changes.\nPython has two types of loops:\nfor loop — when you know what you\u0026rsquo;re iterating over (a list, string, range, etc.) while loop — when you repeat as long as a condition is True 2. The for Loop Basic Syntax for variable in sequence: # code to run for each item The variable takes the value of each item in the sequence, one at a time. The loop runs once per item.\n# Runs 5 times — variable \u0026#39;city\u0026#39; changes each iteration cities = [\u0026#34;Delhi\u0026#34;, \u0026#34;Mumbai\u0026#34;, \u0026#34;Bangalore\u0026#34;, \u0026#34;Chennai\u0026#34;, \u0026#34;Kolkata\u0026#34;] for city in cities: print(city) Delhi Mumbai Bangalore Chennai Kolkata The loop body is indented Everything indented under for is part of the loop. The first unindented line is after the loop.\nprices = [100, 250, 80, 430] for price in prices: discounted = price * 0.9 # inside loop print(f\u0026#34;₹{price} → ₹{discounted}\u0026#34;) # inside loop print(\u0026#34;Done!\u0026#34;) # outside loop — runs once ₹100 → ₹90.0 ₹250 → ₹225.0 ₹80 → ₹72.0 ₹430 → ₹387.0 Done! 3. Iterating Over Different Data Types A for loop works on anything iterable — any object you can step through item by item.\n3.1 Iterating Over a List ipl_teams = [\u0026#34;CSK\u0026#34;, \u0026#34;MI\u0026#34;, \u0026#34;RCB\u0026#34;, \u0026#34;KKR\u0026#34;, \u0026#34;DC\u0026#34;, \u0026#34;SRH\u0026#34;] for team in ipl_teams: print(f\u0026#34;Team: {team}\u0026#34;) 3.2 Iterating Over a String A string is a sequence of characters — you can loop over each character.\nname = \u0026#34;INDIA\u0026#34; for char in name: print(char) I N D I A # Count vowels in a word word = \u0026#34;Hyderabad\u0026#34; vowels = 0 for char in word.lower(): if char in \u0026#34;aeiou\u0026#34;: vowels += 1 print(f\u0026#34;Vowels in \u0026#39;{word}\u0026#39;: {vowels}\u0026#34;) # Vowels in \u0026#39;Hyderabad\u0026#39;: 4 3.3 Iterating Over a Tuple Tuples work exactly like lists in a loop.\ncoordinates = (28.6, 77.2) # Delhi\u0026#39;s lat/long lat, lon = coordinates # unpacking (not a loop) # Loop over a list of coordinate tuples cities = [(\u0026#34;Delhi\u0026#34;, 28.6, 77.2), (\u0026#34;Mumbai\u0026#34;, 19.0, 72.8), (\u0026#34;Bangalore\u0026#34;, 12.9, 77.5)] for city, lat, lon in cities: print(f\u0026#34;{city} is at {lat}°N, {lon}°E\u0026#34;) Delhi is at 28.6°N, 77.2°E Mumbai is at 19.0°N, 72.8°E Bangalore is at 12.9°N, 77.5°E This is called tuple unpacking in a loop — Python automatically unpacks each tuple into the variables you name.\n3.4 Iterating Over a Dictionary Looping over a dictionary gives you keys by default.\nplayer_scores = { \u0026#34;Rohit\u0026#34;: 45, \u0026#34;Kohli\u0026#34;: 82, \u0026#34;Dhoni\u0026#34;: 37, \u0026#34;Hardik\u0026#34;: 55, } # Loop over keys (default) for player in player_scores: print(player) # Rohit, Kohli, Dhoni, Hardik # Loop over values for score in player_scores.values(): print(score) # 45, 82, 37, 55 # Loop over key-value pairs — most common for player, score in player_scores.items(): print(f\u0026#34;{player} scored {score} runs\u0026#34;) Rohit scored 45 runs Kohli scored 82 runs Dhoni scored 37 runs Hardik scored 55 runs # Real use case: update all values updated_scores = {} for player, score in player_scores.items(): updated_scores[player] = score + 10 # bonus 10 runs print(updated_scores) # {\u0026#39;Rohit\u0026#39;: 55, \u0026#39;Kohli\u0026#39;: 92, \u0026#39;Dhoni\u0026#39;: 47, \u0026#39;Hardik\u0026#39;: 65} 3.5 Iterating Over a Set Sets are unordered — the loop works, but order is not guaranteed.\nunique_cities = {\u0026#34;Delhi\u0026#34;, \u0026#34;Mumbai\u0026#34;, \u0026#34;Bangalore\u0026#34;, \u0026#34;Delhi\u0026#34;, \u0026#34;Mumbai\u0026#34;} for city in unique_cities: print(city) # Output order varies — sets don\u0026#39;t preserve insertion order 3.6 Iterating Over a Range range() produces a sequence of numbers — covered in detail in the next section.\nfor i in range(5): print(i) # 0, 1, 2, 3, 4 4. The range() Function range() is how you loop a specific number of times or generate a sequence of numbers.\nThree Forms of range() range(stop) # 0 to stop-1 range(start, stop) # start to stop-1 range(start, stop, step) # start to stop-1, jumping by step # Count from 0 to 4 for i in range(5): print(i) # 0 1 2 3 4 # Count from 1 to 5 for i in range(1, 6): print(i) # 1 2 3 4 5 # Count even numbers from 2 to 10 for i in range(2, 11, 2): print(i) # 2 4 6 8 10 # Count backwards from 10 to 1 for i in range(10, 0, -1): print(i) # 10 9 8 7 6 5 4 3 2 1 range() in Practice # Print multiplication table for 7 for i in range(1, 11): print(f\u0026#34;7 × {i} = {7 * i}\u0026#34;) 7 × 1 = 7 7 × 2 = 14 ... 7 × 10 = 70 # Use range with len() to access list items by index students = [\u0026#34;Aarav\u0026#34;, \u0026#34;Priya\u0026#34;, \u0026#34;Rohan\u0026#34;, \u0026#34;Sneha\u0026#34;] for i in range(len(students)): print(f\u0026#34;Roll #{i+1}: {students[i]}\u0026#34;) Roll #1: Aarav Roll #2: Priya Roll #3: Rohan Roll #4: Sneha Tip: for item in list is cleaner when you just need values. Use range(len(list)) only when you need the index AND need to modify the list in-place.\n5. The while Loop A while loop keeps running as long as its condition is True. Use it when you don\u0026rsquo;t know in advance how many iterations you need.\nBasic Syntax while condition: # code to run while condition is True # Countdown from 5 count = 5 while count \u0026gt; 0: print(f\u0026#34;T-minus {count}...\u0026#34;) count -= 1 # IMPORTANT: update the variable, or loop runs forever print(\u0026#34;Liftoff! 🚀\u0026#34;) T-minus 5... T-minus 4... T-minus 3... T-minus 2... T-minus 1... Liftoff! 🚀 while for User Input The classic use case for while — keep asking until valid input is given.\n# Ask until the user enters a positive number number = -1 while number \u0026lt;= 0: number = int(input(\u0026#34;Enter a positive number: \u0026#34;)) if number \u0026lt;= 0: print(\u0026#34;That\u0026#39;s not positive. Try again.\u0026#34;) print(f\u0026#34;Great! You entered {number}.\u0026#34;) while for Searching # Find first score above 80 scores = [45, 62, 58, 84, 91, 70] i = 0 while i \u0026lt; len(scores): if scores[i] \u0026gt; 80: print(f\u0026#34;First score above 80: {scores[i]} at index {i}\u0026#34;) break i += 1 Infinite Loop with break Sometimes the cleanest structure is while True with an explicit break.\n# ATM-style menu while True: print(\u0026#34;\\n--- MENU ---\u0026#34;) print(\u0026#34;1. Check Balance\u0026#34;) print(\u0026#34;2. Withdraw\u0026#34;) print(\u0026#34;3. Exit\u0026#34;) choice = input(\u0026#34;Enter choice: \u0026#34;) if choice == \u0026#34;1\u0026#34;: print(\u0026#34;Balance: ₹10,000\u0026#34;) elif choice == \u0026#34;2\u0026#34;: print(\u0026#34;Withdrawing...\u0026#34;) elif choice == \u0026#34;3\u0026#34;: print(\u0026#34;Thank you. Goodbye!\u0026#34;) break else: print(\u0026#34;Invalid choice. Try again.\u0026#34;) 6. break — Exit the Loop Early break immediately exits the loop, skipping all remaining iterations — even if there are more items to go through.\n# Stop as soon as you find a failing score scores = [88, 76, 92, 45, 81, 67] for score in scores: if score \u0026lt; 50: print(f\u0026#34;Found a failing score: {score}. Stopping.\u0026#34;) break print(f\u0026#34;Score {score} — OK\u0026#34;) Score 88 — OK Score 76 — OK Score 92 — OK Found a failing score: 45. Stopping. # Search for a name in a list names = [\u0026#34;Aarav\u0026#34;, \u0026#34;Priya\u0026#34;, \u0026#34;Rohan\u0026#34;, \u0026#34;Sneha\u0026#34;, \u0026#34;Karan\u0026#34;] target = \u0026#34;Rohan\u0026#34; for i, name in enumerate(names): if name == target: print(f\u0026#34;Found \u0026#39;{target}\u0026#39; at position {i}\u0026#34;) break else: print(f\u0026#34;\u0026#39;{target}\u0026#39; not found\u0026#34;) # else runs only if break was never hit break in while loop attempts = 0 password = \u0026#34;india@123\u0026#34; while True: attempt = input(\u0026#34;Enter password: \u0026#34;) attempts += 1 if attempt == password: print(\u0026#34;Access granted!\u0026#34;) break if attempts \u0026gt;= 3: print(\u0026#34;Too many failed attempts. Locked.\u0026#34;) break print(f\u0026#34;Wrong password. {3 - attempts} attempt(s) left.\u0026#34;) 7. continue — Skip to the Next Iteration continue skips the rest of the current iteration and jumps straight to the next one. The loop does not exit — it just moves on.\n# Print only even numbers — skip odd ones for i in range(1, 11): if i % 2 != 0: continue # skip this iteration print(i) 2 4 6 8 10 # Skip students who haven\u0026#39;t submitted students = [\u0026#34;Aarav\u0026#34;, \u0026#34;Priya\u0026#34;, None, \u0026#34;Sneha\u0026#34;, None, \u0026#34;Karan\u0026#34;] for student in students: if student is None: continue # skip missing entries print(f\u0026#34;Processing submission for {student}\u0026#34;) Processing submission for Aarav Processing submission for Priya Processing submission for Sneha Processing submission for Karan # Skip negative numbers when computing sum numbers = [10, -3, 25, -7, 8, -1, 14] total = 0 for n in numbers: if n \u0026lt; 0: continue total += n print(f\u0026#34;Sum of positives: {total}\u0026#34;) # 57 break vs continue — side by side break → EXIT the loop entirely continue → SKIP this iteration, keep looping numbers = [1, 2, 3, 4, 5, 6, 7, 8] # break at 5 → prints 1 2 3 4 for n in numbers: if n == 5: break print(n) # continue at 5 → prints 1 2 3 4 6 7 8 for n in numbers: if n == 5: continue print(n) 8. pass — Do Nothing (Placeholder) pass is a no-op — it literally does nothing. It\u0026rsquo;s used when Python\u0026rsquo;s syntax requires a code block, but you have nothing to write yet.\n# Placeholder while building logic for i in range(5): pass # TODO: add logic later # You can also use it in conditionals for score in [88, 45, 92, 38, 76]: if score \u0026gt;= 60: pass # handle passing students later else: print(f\u0026#34;Failed: {score}\u0026#34;) # Common use: stub out a loop you\u0026#39;ll fill in later cities = [\u0026#34;Delhi\u0026#34;, \u0026#34;Mumbai\u0026#34;, \u0026#34;Bangalore\u0026#34;] for city in cities: pass # will add city-specific processing here pass is not for skipping items (that\u0026rsquo;s continue). It\u0026rsquo;s for writing syntactically valid code when the body is intentionally empty.\n9. else with Loops Python has a unique feature: a for or while loop can have an else block. The else runs only if the loop completed normally — i.e., was never stopped by a break.\n# Search for a city — else confirms \u0026#34;not found\u0026#34; cities = [\u0026#34;Delhi\u0026#34;, \u0026#34;Mumbai\u0026#34;, \u0026#34;Bangalore\u0026#34;, \u0026#34;Chennai\u0026#34;] target = \u0026#34;Kolkata\u0026#34; for city in cities: if city == target: print(f\u0026#34;Found: {target}\u0026#34;) break else: print(f\u0026#34;{target} not found in the list\u0026#34;) # runs because break was never hit Kolkata not found in the list # Check if a number is prime def is_prime(n): if n \u0026lt; 2: return False for i in range(2, n): if n % i == 0: print(f\u0026#34;{n} is not prime (divisible by {i})\u0026#34;) break else: print(f\u0026#34;{n} is prime!\u0026#34;) # only reaches here if no divisor was found is_prime(7) # 7 is prime! is_prime(12) # 12 is not prime (divisible by 2) else with while attempts = 3 password = \u0026#34;india@123\u0026#34; while attempts \u0026gt; 0: guess = input(\u0026#34;Password: \u0026#34;) if guess == password: print(\u0026#34;Correct!\u0026#34;) break attempts -= 1 else: # runs only if while condition became False (ran out of attempts without break) print(\u0026#34;Account locked after too many attempts.\u0026#34;) 10. enumerate() — Loop with an Index When you need both the position and the value while looping, use enumerate(). It\u0026rsquo;s cleaner than managing a counter variable yourself.\nplayers = [\u0026#34;Rohit\u0026#34;, \u0026#34;Kohli\u0026#34;, \u0026#34;Pant\u0026#34;, \u0026#34;Jadeja\u0026#34;, \u0026#34;Bumrah\u0026#34;] # Without enumerate — messy i = 0 for player in players: print(f\u0026#34;{i}: {player}\u0026#34;) i += 1 # With enumerate — clean for i, player in enumerate(players): print(f\u0026#34;{i}: {player}\u0026#34;) 0: Rohit 1: Kohli 2: Pant 3: Jadeja 4: Bumrah Custom start index # Start counting from 1 (like a rank or roll number) for rank, player in enumerate(players, start=1): print(f\u0026#34;Rank #{rank}: {player}\u0026#34;) Rank #1: Rohit Rank #2: Kohli Rank #3: Pant Rank #4: Jadeja Rank #5: Bumrah Real use case — find index of a value marks = [78, 92, 65, 88, 45, 91] for i, mark in enumerate(marks): if mark \u0026lt; 50: print(f\u0026#34;Student at index {i} failed with {mark} marks\u0026#34;) Student at index 4 failed with 45 marks Real use case — add numbered labels to a list menu_items = [\u0026#34;Masala Dosa\u0026#34;, \u0026#34;Idli Sambar\u0026#34;, \u0026#34;Vada\u0026#34;, \u0026#34;Filter Coffee\u0026#34;] for num, item in enumerate(menu_items, start=1): print(f\u0026#34; {num}. {item}\u0026#34;) 1. Masala Dosa 2. Idli Sambar 3. Vada 4. Filter Coffee 11. zip() — Loop Over Multiple Sequences zip() lets you loop over two or more sequences at the same time, pairing up their items by position.\nstudents = [\u0026#34;Aarav\u0026#34;, \u0026#34;Priya\u0026#34;, \u0026#34;Rohan\u0026#34;, \u0026#34;Sneha\u0026#34;] marks = [85, 92, 78, 95] for student, mark in zip(students, marks): print(f\u0026#34;{student}: {mark}\u0026#34;) Aarav: 85 Priya: 92 Rohan: 78 Sneha: 95 zip() with three sequences names = [\u0026#34;Aarav\u0026#34;, \u0026#34;Priya\u0026#34;, \u0026#34;Rohan\u0026#34;] maths = [85, 92, 78] science = [90, 88, 82] for name, m, s in zip(names, maths, science): total = m + s print(f\u0026#34;{name} — Maths: {m}, Science: {s}, Total: {total}\u0026#34;) Aarav — Maths: 85, Science: 90, Total: 175 Priya — Maths: 92, Science: 88, Total: 180 Rohan — Maths: 78, Science: 82, Total: 160 zip() stops at the shortest sequence a = [1, 2, 3, 4, 5] b = [\u0026#34;a\u0026#34;, \u0026#34;b\u0026#34;, \u0026#34;c\u0026#34;] for x, y in zip(a, b): print(x, y) # 1 a # 2 b # 3 c ← stops here because b is exhausted zip() to create a dictionary keys = [\u0026#34;name\u0026#34;, \u0026#34;city\u0026#34;, \u0026#34;score\u0026#34;] values = [\u0026#34;Kohli\u0026#34;, \u0026#34;Delhi\u0026#34;, 82] result = dict(zip(keys, values)) print(result) # {\u0026#39;name\u0026#39;: \u0026#39;Kohli\u0026#39;, \u0026#39;city\u0026#39;: \u0026#39;Delhi\u0026#39;, \u0026#39;score\u0026#39;: 82} enumerate() + zip() together students = [\u0026#34;Aarav\u0026#34;, \u0026#34;Priya\u0026#34;, \u0026#34;Rohan\u0026#34;] marks = [85, 92, 78] for i, (student, mark) in enumerate(zip(students, marks), start=1): print(f\u0026#34;{i}. {student}: {mark}\u0026#34;) 1. Aarav: 85 2. Priya: 92 3. Rohan: 78 12. Nested Loops A nested loop is a loop inside another loop. The inner loop completes all its iterations for each single iteration of the outer loop.\n# Multiplication table for 1 to 3 for i in range(1, 4): # outer loop: 3 times for j in range(1, 4): # inner loop: 3 times per outer print(f\u0026#34;{i} × {j} = {i*j}\u0026#34;) print(\u0026#34;---\u0026#34;) 1 × 1 = 1 1 × 2 = 2 1 × 3 = 3 --- 2 × 1 = 2 2 × 2 = 4 2 × 3 = 6 --- 3 × 1 = 3 3 × 2 = 6 3 × 3 = 9 --- Nested loop on a 2D list (matrix) # Marks of 3 students in 3 subjects marks_matrix = [ [78, 85, 90], # Aarav [92, 76, 88], # Priya [65, 70, 80], # Rohan ] student_names = [\u0026#34;Aarav\u0026#34;, \u0026#34;Priya\u0026#34;, \u0026#34;Rohan\u0026#34;] subjects = [\u0026#34;Maths\u0026#34;, \u0026#34;Science\u0026#34;, \u0026#34;English\u0026#34;] for i, row in enumerate(marks_matrix): print(f\u0026#34;\\n{student_names[i]}\u0026#39;s marks:\u0026#34;) for j, mark in enumerate(row): print(f\u0026#34; {subjects[j]}: {mark}\u0026#34;) Pattern Printing with Nested Loops Pattern printing is a classic way to build intuition for nested loops.\n# Right triangle of stars for i in range(1, 6): for j in range(i): print(\u0026#34;*\u0026#34;, end=\u0026#34; \u0026#34;) print() * * * * * * * * * * * * * * * # Number triangle for i in range(1, 6): for j in range(1, i + 1): print(j, end=\u0026#34; \u0026#34;) print() 1 1 2 1 2 3 1 2 3 4 1 2 3 4 5 break inside nested loops break only exits the innermost loop it\u0026rsquo;s in.\n# Stop searching as soon as target is found matrix = [[1, 2, 3], [4, 5, 6], [7, 8, 9]] target = 5 found = False for row in matrix: for val in row: if val == target: print(f\u0026#34;Found {target}!\u0026#34;) found = True break # exits inner loop only if found: break # now exit outer loop too 13. List Comprehensions List comprehension is a compact, readable way to build a new list using a loop — all in one line.\nBasic Syntax new_list = [expression for item in iterable] # With a condition (filter) new_list = [expression for item in iterable if condition] Side-by-side comparison # Traditional loop squares = [] for i in range(1, 6): squares.append(i ** 2) # List comprehension — same result, one line squares = [i ** 2 for i in range(1, 6)] print(squares) # [1, 4, 9, 16, 25] With a condition # Only even numbers evens = [i for i in range(1, 11) if i % 2 == 0] print(evens) # [2, 4, 6, 8, 10] # Students who passed marks = [78, 45, 92, 38, 65, 88] passed = [m for m in marks if m \u0026gt;= 60] print(passed) # [78, 92, 65, 88] With transformation # Apply 10% GST to all prices prices = [100, 250, 480, 1200, 3000] after_gst = [round(p * 1.1, 2) for p in prices] print(after_gst) # [110.0, 275.0, 528.0, 1320.0, 3300.0] # Convert all city names to uppercase cities = [\u0026#34;delhi\u0026#34;, \u0026#34;mumbai\u0026#34;, \u0026#34;bangalore\u0026#34;, \u0026#34;chennai\u0026#34;] upper_cities = [city.upper() for city in cities] With if-else (transformation based on condition) marks = [78, 45, 92, 38, 65, 88] # Assign pass/fail label for each mark result = [\u0026#34;Pass\u0026#34; if m \u0026gt;= 60 else \u0026#34;Fail\u0026#34; for m in marks] print(result) # [\u0026#39;Pass\u0026#39;, \u0026#39;Fail\u0026#39;, \u0026#39;Pass\u0026#39;, \u0026#39;Fail\u0026#39;, \u0026#39;Pass\u0026#39;, \u0026#39;Pass\u0026#39;] Nested list comprehension # Flatten a 2D list into 1D matrix = [[1, 2, 3], [4, 5, 6], [7, 8, 9]] flat = [val for row in matrix for val in row] print(flat) # [1, 2, 3, 4, 5, 6, 7, 8, 9] When to use comprehensions: For simple, readable transformations and filters. If the logic needs multiple lines or nested conditions, a regular loop is clearer.\n14. Dictionary and Set Comprehensions The same idea as list comprehensions, but for dictionaries and sets.\nDictionary Comprehension # Basic: {key: value for item in iterable} students = [\u0026#34;Aarav\u0026#34;, \u0026#34;Priya\u0026#34;, \u0026#34;Rohan\u0026#34;, \u0026#34;Sneha\u0026#34;] marks = [85, 92, 78, 95] # Create a dict mapping name → marks grade_dict = {student: mark for student, mark in zip(students, marks)} print(grade_dict) # {\u0026#39;Aarav\u0026#39;: 85, \u0026#39;Priya\u0026#39;: 92, \u0026#39;Rohan\u0026#39;: 78, \u0026#39;Sneha\u0026#39;: 95} # Square of numbers from 1 to 5 squares = {n: n**2 for n in range(1, 6)} print(squares) # {1: 1, 2: 4, 3: 9, 4: 16, 5: 25} # With a condition — only passing students passed = {name: m for name, m in zip(students, marks) if m \u0026gt;= 80} print(passed) # {\u0026#39;Aarav\u0026#39;: 85, \u0026#39;Priya\u0026#39;: 92, \u0026#39;Sneha\u0026#39;: 95} # Invert a dictionary (swap keys and values) original = {\u0026#34;a\u0026#34;: 1, \u0026#34;b\u0026#34;: 2, \u0026#34;c\u0026#34;: 3} inverted = {v: k for k, v in original.items()} print(inverted) # {1: \u0026#39;a\u0026#39;, 2: \u0026#39;b\u0026#39;, 3: \u0026#39;c\u0026#39;} Set Comprehension # Like list comprehension, but uses {} and guarantees uniqueness cities = [\u0026#34;Delhi\u0026#34;, \u0026#34;Mumbai\u0026#34;, \u0026#34;Delhi\u0026#34;, \u0026#34;Bangalore\u0026#34;, \u0026#34;Mumbai\u0026#34;, \u0026#34;Chennai\u0026#34;] unique_cities = {city for city in cities} print(unique_cities) # {\u0026#39;Delhi\u0026#39;, \u0026#39;Mumbai\u0026#39;, \u0026#39;Bangalore\u0026#39;, \u0026#39;Chennai\u0026#39;} # Only long city names long_names = {city for city in cities if len(city) \u0026gt; 5} print(long_names) # {\u0026#39;Mumbai\u0026#39;, \u0026#39;Bangalore\u0026#39;, \u0026#39;Chennai\u0026#39;} 15. Loop Patterns Every Programmer Uses These are the building blocks you\u0026rsquo;ll see in real code over and over again.\nPattern 1 — Accumulator Build up a result by adding to it each iteration.\n# Sum all odd numbers from 1 to 20 total = 0 for i in range(1, 21): if i % 2 != 0: total += i print(total) # 100 # Concatenate strings words = [\u0026#34;Jai\u0026#34;, \u0026#34;Hind\u0026#34;, \u0026#34;Zindabad\u0026#34;] sentence = \u0026#34;\u0026#34; for word in words: sentence += word + \u0026#34; \u0026#34; print(sentence.strip()) # Jai Hind Zindabad Pattern 2 — Find Maximum / Minimum Manually scores = [88, 45, 92, 73, 65, 100, 58] maximum = scores[0] for score in scores: if score \u0026gt; maximum: maximum = score print(f\u0026#34;Highest score: {maximum}\u0026#34;) # 100 Pattern 3 — Counting feedback = [\u0026#34;good\u0026#34;, \u0026#34;bad\u0026#34;, \u0026#34;good\u0026#34;, \u0026#34;excellent\u0026#34;, \u0026#34;bad\u0026#34;, \u0026#34;good\u0026#34;] count_good = 0 for f in feedback: if f == \u0026#34;good\u0026#34;: count_good += 1 print(f\u0026#34;Good ratings: {count_good}\u0026#34;) # 3 Pattern 4 — Building a New List orders = [(\u0026#34;Laptop\u0026#34;, 65000), (\u0026#34;Mouse\u0026#34;, 850), (\u0026#34;Monitor\u0026#34;, 18000), (\u0026#34;Keyboard\u0026#34;, 2200)] expensive = [] for product, price in orders: if price \u0026gt; 5000: expensive.append(product) print(expensive) # [\u0026#39;Laptop\u0026#39;, \u0026#39;Monitor\u0026#39;] Pattern 5 — Group by Category (Dictionary of Lists) employees = [ (\u0026#34;Aarav\u0026#34;, \u0026#34;Engineering\u0026#34;), (\u0026#34;Priya\u0026#34;, \u0026#34;HR\u0026#34;), (\u0026#34;Rohan\u0026#34;, \u0026#34;Engineering\u0026#34;), (\u0026#34;Sneha\u0026#34;, \u0026#34;Marketing\u0026#34;), (\u0026#34;Karan\u0026#34;, \u0026#34;HR\u0026#34;), (\u0026#34;Meera\u0026#34;, \u0026#34;Engineering\u0026#34;), ] departments = {} for name, dept in employees: if dept not in departments: departments[dept] = [] departments[dept].append(name) print(departments) # { # \u0026#39;Engineering\u0026#39;: [\u0026#39;Aarav\u0026#39;, \u0026#39;Rohan\u0026#39;, \u0026#39;Meera\u0026#39;], # \u0026#39;HR\u0026#39;: [\u0026#39;Priya\u0026#39;, \u0026#39;Karan\u0026#39;], # \u0026#39;Marketing\u0026#39;: [\u0026#39;Sneha\u0026#39;] # } Pattern 6 — Running Total / Cumulative # Cumulative sales by month monthly = [12, 18, 15, 22, 19, 25] cumulative = [] running_total = 0 for sales in monthly: running_total += sales cumulative.append(running_total) print(cumulative) # [12, 30, 45, 67, 86, 111] Pattern 7 — Filtering and Transforming Together transactions = [ {\u0026#34;city\u0026#34;: \u0026#34;Mumbai\u0026#34;, \u0026#34;amount\u0026#34;: 15000, \u0026#34;status\u0026#34;: \u0026#34;success\u0026#34;}, {\u0026#34;city\u0026#34;: \u0026#34;Delhi\u0026#34;, \u0026#34;amount\u0026#34;: 8200, \u0026#34;status\u0026#34;: \u0026#34;failed\u0026#34;}, {\u0026#34;city\u0026#34;: \u0026#34;Bangalore\u0026#34;, \u0026#34;amount\u0026#34;: 22000, \u0026#34;status\u0026#34;: \u0026#34;success\u0026#34;}, {\u0026#34;city\u0026#34;: \u0026#34;Chennai\u0026#34;, \u0026#34;amount\u0026#34;: 4500, \u0026#34;status\u0026#34;: \u0026#34;success\u0026#34;}, {\u0026#34;city\u0026#34;: \u0026#34;Pune\u0026#34;, \u0026#34;amount\u0026#34;: 1200, \u0026#34;status\u0026#34;: \u0026#34;failed\u0026#34;}, ] # Total of all successful transactions above ₹5000 total = 0 for t in transactions: if t[\u0026#34;status\u0026#34;] == \u0026#34;success\u0026#34; and t[\u0026#34;amount\u0026#34;] \u0026gt; 5000: total += t[\u0026#34;amount\u0026#34;] print(f\u0026#34;Total successful high-value transactions: ₹{total}\u0026#34;) # ₹37,000 Pattern 8 — Sliding Window # Moving average over 3 months monthly_sales = [10, 14, 12, 18, 22, 19, 25, 28] for i in range(len(monthly_sales) - 2): window = monthly_sales[i:i+3] avg = sum(window) / 3 print(f\u0026#34;Months {i+1}-{i+3}: avg = {avg:.1f}\u0026#34;) Months 1-3: avg = 12.0 Months 2-4: avg = 14.7 Months 3-5: avg = 17.3 Months 4-6: avg = 19.7 Months 5-7: avg = 22.0 Months 6-8: avg = 24.0 16. Common Loop Mistakes and How to Avoid Them Mistake 1 — Forgetting to update the while variable (infinite loop) # WRONG — runs forever i = 0 while i \u0026lt; 5: print(i) # forgot: i += 1 # CORRECT i = 0 while i \u0026lt; 5: print(i) i += 1 Mistake 2 — Modifying a list while iterating over it # WRONG — unpredictable behavior numbers = [1, 2, 3, 4, 5] for n in numbers: if n % 2 == 0: numbers.remove(n) # modifying while iterating # CORRECT — iterate over a copy for n in numbers[:]: # numbers[:] is a copy if n % 2 == 0: numbers.remove(n) # OR — better: build a new list numbers = [n for n in numbers if n % 2 != 0] Mistake 3 — Using range(len()) when you don\u0026rsquo;t need the index # UNNECESSARY names = [\u0026#34;Aarav\u0026#34;, \u0026#34;Priya\u0026#34;, \u0026#34;Rohan\u0026#34;] for i in range(len(names)): print(names[i]) # CLEANER for name in names: print(name) Mistake 4 — Off-by-one errors with range() # Prints 0 to 9, NOT 1 to 10 for i in range(10): print(i) # Prints 1 to 10 for i in range(1, 11): print(i) Mistake 5 — Assuming dict order in older Python # In Python 3.7+, dicts maintain insertion order — this is fine # In older versions (pre-3.7), iteration order was not guaranteed # Always use Python 3.7+ for predictable dict iteration Mistake 6 — Misplacing the accumulator reset # WRONG — reset inside the loop, total is always 0 at end of each run for month in range(1, 4): total = 0 # ← reset INSIDE loop for sale in [100, 200, 300]: total += sale print(total) # CORRECT — reset outside total = 0 # ← reset OUTSIDE loop for sale in [100, 200, 300]: total += sale print(total) # 600 17. Practice Questions Sharpen your skills. Questions go from easy to hard — don\u0026rsquo;t jump ahead.\n🟢 Easy Q1 — Basic for Loop Print all numbers from 1 to 20 that are divisible by 3.\nQ2 — String Loop Write a loop that counts the number of uppercase letters in the string below.\ntext = \u0026#34;Welcome To India, The Land Of DiversiTy\u0026#34; Expected output: Uppercase letters: 6\nQ3 — List Loop with Condition Given the list below, print only the names that have more than 4 characters.\nnames = [\u0026#34;Ali\u0026#34;, \u0026#34;Priya\u0026#34;, \u0026#34;Raj\u0026#34;, \u0026#34;Sneha\u0026#34;, \u0026#34;Om\u0026#34;, \u0026#34;Vikram\u0026#34;, \u0026#34;Ria\u0026#34;] Q4 — while Loop Write a while loop that keeps dividing a number by 2 until it goes below 1. Print the value at each step.\nnumber = 64 Expected output: 64 → 32.0 → 16.0 → 8.0 → 4.0 → 2.0 → 1.0\nQ5 — Dictionary Loop Given the dictionary below, print each item\u0026rsquo;s name and its price after a 15% discount.\nmenu = { \u0026#34;Masala Chai\u0026#34;: 20, \u0026#34;Vada Pav\u0026#34;: 15, \u0026#34;Samosa\u0026#34;: 10, \u0026#34;Filter Coffee\u0026#34;: 25, \u0026#34;Idli\u0026#34;: 30, } 🟡 Medium Q6 — enumerate() + Logic Given a list of cricket scores, use enumerate() to print each score with its ball number (starting from 1). Then print the ball number on which the highest score was hit.\nball_scores = [1, 0, 4, 6, 1, 2, 6, 0, 4, 3] Q7 — zip() and Comparison You have two lists: last year\u0026rsquo;s sales and this year\u0026rsquo;s sales for 6 cities. Use zip() to print, for each city, whether sales went up, went down, or stayed the same.\ncities = [\u0026#34;Delhi\u0026#34;, \u0026#34;Mumbai\u0026#34;, \u0026#34;Bangalore\u0026#34;, \u0026#34;Chennai\u0026#34;, \u0026#34;Hyderabad\u0026#34;, \u0026#34;Pune\u0026#34;] last_year = [120, 145, 98, 110, 85, 76] this_year = [135, 140, 115, 110, 92, 88] Q8 — break and continue Together Given the list of transactions below, skip any transaction with status = \u0026quot;pending\u0026quot;, and stop processing entirely if you encounter a status = \u0026quot;fraud\u0026quot;. Print all valid processed transactions.\ntransactions = [ {\u0026#34;id\u0026#34;: 1, \u0026#34;amount\u0026#34;: 5000, \u0026#34;status\u0026#34;: \u0026#34;success\u0026#34;}, {\u0026#34;id\u0026#34;: 2, \u0026#34;amount\u0026#34;: 1200, \u0026#34;status\u0026#34;: \u0026#34;pending\u0026#34;}, {\u0026#34;id\u0026#34;: 3, \u0026#34;amount\u0026#34;: 8800, \u0026#34;status\u0026#34;: \u0026#34;success\u0026#34;}, {\u0026#34;id\u0026#34;: 4, \u0026#34;amount\u0026#34;: 3200, \u0026#34;status\u0026#34;: \u0026#34;fraud\u0026#34;}, {\u0026#34;id\u0026#34;: 5, \u0026#34;amount\u0026#34;: 4400, \u0026#34;status\u0026#34;: \u0026#34;success\u0026#34;}, ] Q9 — Nested Loop Pattern Print the following pattern using nested loops:\n1 2 2 3 3 3 4 4 4 4 5 5 5 5 5 Q10 — List Comprehension Using a single list comprehension, create a list of all words from the sentence below that are longer than 3 characters and start with a vowel.\nsentence = \u0026#34;An elephant ate orange apples in an orchard outside the city\u0026#34; Q11 — Dictionary Comprehension Given the two lists below, create a dictionary where each student\u0026rsquo;s name maps to their grade (\u0026ldquo;A\u0026rdquo; if marks ≥ 80, \u0026ldquo;B\u0026rdquo; if 60–79, \u0026ldquo;C\u0026rdquo; otherwise).\nstudents = [\u0026#34;Aarav\u0026#34;, \u0026#34;Priya\u0026#34;, \u0026#34;Rohan\u0026#34;, \u0026#34;Sneha\u0026#34;, \u0026#34;Karan\u0026#34;, \u0026#34;Meera\u0026#34;] marks = [88, 55, 73, 91, 62, 47] 🔴 Hard Q12 — Loop + Data Grouping Given the list of orders below, write a loop to build a summary dictionary showing:\nTotal revenue per city Number of orders per city Then print each city\u0026rsquo;s summary in a readable format. orders = [ {\u0026#34;city\u0026#34;: \u0026#34;Mumbai\u0026#34;, \u0026#34;amount\u0026#34;: 1500}, {\u0026#34;city\u0026#34;: \u0026#34;Delhi\u0026#34;, \u0026#34;amount\u0026#34;: 3200}, {\u0026#34;city\u0026#34;: \u0026#34;Mumbai\u0026#34;, \u0026#34;amount\u0026#34;: 800}, {\u0026#34;city\u0026#34;: \u0026#34;Bangalore\u0026#34;, \u0026#34;amount\u0026#34;: 4500}, {\u0026#34;city\u0026#34;: \u0026#34;Delhi\u0026#34;, \u0026#34;amount\u0026#34;: 1100}, {\u0026#34;city\u0026#34;: \u0026#34;Mumbai\u0026#34;, \u0026#34;amount\u0026#34;: 2200}, {\u0026#34;city\u0026#34;: \u0026#34;Bangalore\u0026#34;, \u0026#34;amount\u0026#34;: 3800}, {\u0026#34;city\u0026#34;: \u0026#34;Delhi\u0026#34;, \u0026#34;amount\u0026#34;: 900}, ] Expected output format:\nMumbai → Orders: 3, Revenue: ₹4500 Delhi → Orders: 3, Revenue: ₹5200 Bangalore → Orders: 2, Revenue: ₹8300 Q13 — while + Validation Logic Write a number guessing game. The program picks a secret number between 1 and 50. The user gets 6 attempts. After each wrong guess, tell the user if the answer is higher or lower. If they exhaust all attempts, reveal the number. (For testing, hardcode the secret number as 33.)\nQ14 — Nested Loop + Star Pattern Print a diamond shape of stars for a given size n. For n = 4:\n* * * * * * * * * * * * * * * * * * * * Q15 — Full Pipeline You are given a dataset of students. Write a complete program using loops to:\nCalculate each student\u0026rsquo;s total marks and percentage Assign grade: \u0026ldquo;Distinction\u0026rdquo; (≥85%), \u0026ldquo;First Class\u0026rdquo; (≥60%), \u0026ldquo;Pass\u0026rdquo; (≥40%), \u0026ldquo;Fail\u0026rdquo; (\u0026lt;40%) Count how many students got each grade Find the topper (highest percentage) Print the full report students = [ {\u0026#34;name\u0026#34;: \u0026#34;Aarav\u0026#34;, \u0026#34;maths\u0026#34;: 88, \u0026#34;science\u0026#34;: 92, \u0026#34;english\u0026#34;: 85, \u0026#34;history\u0026#34;: 78, \u0026#34;cs\u0026#34;: 95}, {\u0026#34;name\u0026#34;: \u0026#34;Priya\u0026#34;, \u0026#34;maths\u0026#34;: 55, \u0026#34;science\u0026#34;: 48, \u0026#34;english\u0026#34;: 62, \u0026#34;history\u0026#34;: 50, \u0026#34;cs\u0026#34;: 58}, {\u0026#34;name\u0026#34;: \u0026#34;Rohan\u0026#34;, \u0026#34;maths\u0026#34;: 72, \u0026#34;science\u0026#34;: 68, \u0026#34;english\u0026#34;: 75, \u0026#34;history\u0026#34;: 70, \u0026#34;cs\u0026#34;: 65}, {\u0026#34;name\u0026#34;: \u0026#34;Sneha\u0026#34;, \u0026#34;maths\u0026#34;: 95, \u0026#34;science\u0026#34;: 98, \u0026#34;english\u0026#34;: 92, \u0026#34;history\u0026#34;: 88, \u0026#34;cs\u0026#34;: 97}, {\u0026#34;name\u0026#34;: \u0026#34;Karan\u0026#34;, \u0026#34;maths\u0026#34;: 35, \u0026#34;science\u0026#34;: 40, \u0026#34;english\u0026#34;: 38, \u0026#34;history\u0026#34;: 42, \u0026#34;cs\u0026#34;: 30}, {\u0026#34;name\u0026#34;: \u0026#34;Meera\u0026#34;, \u0026#34;maths\u0026#34;: 78, \u0026#34;science\u0026#34;: 82, \u0026#34;english\u0026#34;: 80, \u0026#34;history\u0026#34;: 75, \u0026#34;cs\u0026#34;: 84}, {\u0026#34;name\u0026#34;: \u0026#34;Arjun\u0026#34;, \u0026#34;maths\u0026#34;: 45, \u0026#34;science\u0026#34;: 55, \u0026#34;english\u0026#34;: 50, \u0026#34;history\u0026#34;: 48, \u0026#34;cs\u0026#34;: 52}, ] Expected output format:\n===== STUDENT REPORT ===== Aarav | Total: 438/500 | 87.6% | Distinction Priya | Total: 273/500 | 54.6% | Pass ... ===== GRADE SUMMARY ===== Distinction : 2 First Class : 2 Pass : 2 Fail : 1 ===== TOPPER ===== Sneha with 94.0% 18. What We Covered + What\u0026rsquo;s Next ✅ What This Masterclass Covered Concept What It Does for loop Iterate over any sequence — list, string, tuple, dict, set, range Iterating data types Lists, strings, tuples, dicts (.keys, .values, .items), sets range() Generate number sequences with start, stop, step while loop Repeat while a condition is True break Exit the loop immediately continue Skip current iteration, keep looping pass Syntactic placeholder — do nothing else with loops Runs only if loop completed without a break enumerate() Loop with index + value together zip() Loop over multiple sequences in parallel Nested loops Loops inside loops — for matrices, patterns, combinations List comprehensions Build lists in one readable line Dict comprehensions Build dicts in one readable line Set comprehensions Build unique sets in one line Loop patterns Accumulator, max/min, grouping, sliding window, filtering Common mistakes Infinite loops, modifying while iterating, off-by-one 🚀 What to Explore Next 1. Itertools The itertools module gives you powerful loop tools: chain, product, combinations, permutations, groupby, islice. Essential once you\u0026rsquo;re doing serious iteration work.\n2. Generators Instead of building an entire list in memory, generators produce items one at a time using yield. Much more memory-efficient for large sequences. range() itself is a generator.\n3. Map, Filter, Reduce Functional programming alternatives to loops. map(func, list), filter(func, list), and functools.reduce(). Often replaced by comprehensions in modern Python, but good to know.\n4. Recursion Sometimes the cleanest way to repeat something is to have a function call itself. Recursion is loops taken to a different level — essential for trees, graphs, and divide-and-conquer algorithms.\n5. DSA with Loops Loops are the backbone of algorithms — sorting (bubble, insertion, selection), searching (linear, binary), and dynamic programming all rely on mastering iteration patterns.\nFinal thought: Every experienced Python developer has one thing in common — they\u0026rsquo;ve written a lot of loops. The patterns in Section 15 will show up in every data pipeline, web app, and algorithm you ever write.\nDo the practice questions. Especially Q12 to Q15. The questions that make you think for 10 minutes are the ones that actually build the skill. 💪\nMade with ❤️ for Codeverra learners | codeverra.com\n","permalink":"https://blogs.codeverra.com/blog/python-03-loops-masterclass/","summary":"\u003ch1 id=\"-python-loops-masterclass\"\u003e🔁 Python Loops Masterclass\u003c/h1\u003e\n\u003ch3 id=\"complete-command-over-for-and-while-loops--from-basics-to-patterns\"\u003e\u003cem\u003eComplete Command Over for and while Loops — From Basics to Patterns\u003c/em\u003e\u003c/h3\u003e\n\u003chr\u003e\n\u003cblockquote\u003e\n\u003cp\u003e\u003cstrong\u003eWho is this for?\u003c/strong\u003e\nYou know Python basics — variables, conditionals, functions.\nNow you want to truly master loops — not just use them, but \u003cem\u003ethink\u003c/em\u003e in them.\nThis guide covers every loop concept in Python, with simple examples before every idea.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003chr\u003e\n\u003ch2 id=\"-table-of-contents\"\u003e📌 Table of Contents\u003c/h2\u003e\n\u003col\u003e\n\u003cli\u003e\u003ca href=\"#1-why-loops-exist\"\u003eWhy Loops Exist\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#2-the-for-loop\"\u003eThe for Loop\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#3-iterating-over-different-data-types\"\u003eIterating Over Different Data Types\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#4-the-range-function\"\u003eThe range() Function\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#5-the-while-loop\"\u003eThe while Loop\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#6-break--exit-the-loop-early\"\u003ebreak — Exit the Loop Early\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#7-continue--skip-to-the-next-iteration\"\u003econtinue — Skip to the Next Iteration\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#8-pass--do-nothing-placeholder\"\u003epass — Do Nothing (Placeholder)\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#9-else-with-loops\"\u003eelse with Loops\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#10-enumerate--loop-with-an-index\"\u003eenumerate() — Loop with an Index\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#11-zip--loop-over-multiple-sequences\"\u003ezip() — Loop Over Multiple Sequences\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#12-nested-loops\"\u003eNested Loops\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#13-list-comprehensions\"\u003eList Comprehensions\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#14-dictionary-and-set-comprehensions\"\u003eDictionary and Set Comprehensions\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#15-loop-patterns-every-programmer-uses\"\u003eLoop Patterns Every Programmer Uses\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#16-common-loop-mistakes-and-how-to-avoid-them\"\u003eCommon Loop Mistakes and How to Avoid Them\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#17-practice-questions\"\u003ePractice Questions\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#18-what-we-covered--whats-next\"\u003eWhat We Covered + What\u0026rsquo;s Next\u003c/a\u003e\u003c/li\u003e\n\u003c/ol\u003e\n\u003chr\u003e\n\u003ch2 id=\"1-why-loops-exist\"\u003e1. Why Loops Exist\u003c/h2\u003e\n\u003cp\u003eImagine you need to print a welcome message for 500 students. Without loops:\u003c/p\u003e","tags":["python"],"title":"Python Loops Masterclass"},{"content":"Python Refresher for DSA Table of Contents What You Will Learn in This Blog Python Built-in Data Types for DSA Lists Dictionaries Sets Tuples The Collections Module deque defaultdict Counter List Comprehensions Important Built-in Functions for DSA Sorting in Python String Operations heapq \u0026ndash; The Built-in Min Heap Common Python Patterns in DSA Summary Key Takeaways Next Steps What You Will Learn in This Blog The Python built-in data types you will use constantly in DSA How lists, dictionaries, sets, and tuples work and when to use each Important Python built-in functions and their complexities The collections module \u0026ndash; deque, defaultdict, Counter List comprehensions and when to use them Common Python patterns that appear repeatedly in DSA problems This is not a complete Python tutorial. It covers exactly what you need to follow the rest of this guide comfortably. If you are already confident with all of the above, you can skim this and move on.\nA note on time complexity notation\nThroughout this blog you will see notations like O(1), O(n), and O(n log n) next to operations. These describe how the time or memory an operation takes grows as the input gets larger. They are covered in full detail in the next blog \u0026ndash; Complexity Analysis. For now, here is just enough to follow along:\nNotation Plain English O(1) Takes the same time regardless of input size \u0026ndash; constant O(n) Time grows linearly with input size O(n log n) Slightly worse than linear \u0026ndash; typical of good sorting algorithms O(n^2) Time grows quadratically \u0026ndash; usually something to avoid for large inputs You do not need to deeply understand these yet. Just use the table as a reference as you read. Everything will become clear in the next blog.\nPython Built-in Data Types for DSA Python gives you several built-in data types that map directly to fundamental data structures. Understanding how they work under the hood will make your complexity analysis much more accurate later.\nLists A Python list is a dynamic array. It can hold elements of any type, grows automatically when you add to it, and supports indexing.\n# Creating a list numbers = [10, 20, 30, 40, 50] # Indexing -- accessing elements by position print(numbers[0]) # 10 -- first element print(numbers[-1]) # 50 -- last element (negative indexing counts from the end) print(numbers[-2]) # 40 -- second from last Slicing Slicing lets you extract a portion of a list. The syntax is list[start:end:step] where end is exclusive.\nnumbers = [10, 20, 30, 40, 50] print(numbers[1:4]) # [20, 30, 40] -- index 1 up to but not including index 4 print(numbers[:3]) # [10, 20, 30] -- from start up to index 3 print(numbers[2:]) # [30, 40, 50] -- from index 2 to end print(numbers[::2]) # [10, 30, 50] -- every second element print(numbers[::-1]) # [50, 40, 30, 20, 10] -- reversed list Common List Operations and Their Time Complexity Note on complexity notation: You will see terms like O(1), O(n), and O(n log n) in the tables below. These come from Big O notation, which is covered in full detail in the next blog \u0026ndash; Complexity Analysis. For now, read them loosely as follows: O(1) means the operation takes the same time regardless of input size, O(n) means the time grows with the size of the input, and O(n log n) means slightly worse than O(n). Once you read the next blog, come back and these tables will make complete sense.\nThis table is important. Knowing these complexities will directly affect the choices you make when solving problems.\nOperation Example Time Complexity Notes Access by index nums[i] O(1) Direct memory access Append to end nums.append(x) O(1) amortised Occasionally O(n) when resizing Pop from end nums.pop() O(1) Insert at position nums.insert(i, x) O(n) Shifts all elements after i Delete by index del nums[i] O(n) Shifts elements Search for value x in nums O(n) Scans the whole list Get length len(nums) O(1) Stored internally Slice nums[i:j] O(k) k is the size of the slice Sort nums.sort() O(n log n) Timsort Reverse nums.reverse() O(n) In-place nums = [3, 1, 4, 1, 5, 9, 2, 6] # Append -- O(1) amortised nums.append(7) print(nums) # [3, 1, 4, 1, 5, 9, 2, 6, 7] # Pop from end -- O(1) last = nums.pop() print(last) # 7 # Insert at index -- O(n) because elements shift nums.insert(0, 99) print(nums) # [99, 3, 1, 4, 1, 5, 9, 2, 6] # Check membership -- O(n) print(5 in nums) # True # Sort in place -- O(n log n) nums.sort() print(nums) # [1, 1, 2, 3, 4, 5, 6, 9, 99] # Sort and return a new list -- O(n log n) sorted_nums = sorted(nums, reverse=True) print(sorted_nums) # [99, 9, 6, 5, 4, 3, 2, 1, 1] Key point: Inserting or deleting at the beginning or middle of a list is O(n) because Python has to shift all elements. This matters a lot when choosing between a list and a deque later.\nDictionaries A Python dictionary is a hash map. It stores key-value pairs and gives you O(1) average-case lookup, insertion, and deletion by key.\n# Creating a dictionary student = { \u0026#34;name\u0026#34;: \u0026#34;Alice\u0026#34;, \u0026#34;age\u0026#34;: 22, \u0026#34;grade\u0026#34;: \u0026#34;A\u0026#34; } # Accessing a value by key -- O(1) average print(student[\u0026#34;name\u0026#34;]) # Alice # Safe access with .get() -- returns None if key not found instead of raising an error print(student.get(\u0026#34;score\u0026#34;)) # None print(student.get(\u0026#34;score\u0026#34;, 0)) # 0 -- default value if key not found # Adding and updating -- O(1) average student[\u0026#34;score\u0026#34;] = 95 student[\u0026#34;age\u0026#34;] = 23 # Deleting -- O(1) average del student[\u0026#34;grade\u0026#34;] # Check if a key exists -- O(1) average print(\u0026#34;name\u0026#34; in student) # True print(\u0026#34;grade\u0026#34; in student) # False Iterating Over a Dictionary scores = {\u0026#34;Alice\u0026#34;: 90, \u0026#34;Bob\u0026#34;: 85, \u0026#34;Carol\u0026#34;: 92} # Iterate over keys for name in scores: print(name) # Iterate over values for score in scores.values(): print(score) # Iterate over key-value pairs -- you will use this constantly for name, score in scores.items(): print(f\u0026#34;{name}: {score}\u0026#34;) Dictionary Time Complexity Operation Time Complexity Notes Access by key O(1) average O(n) worst case \u0026ndash; rare with good hash function Insert / Update O(1) average Delete O(1) average Search by key O(1) average Iterate O(n) n is number of key-value pairs When to use a dictionary in DSA: Any time you need fast lookup by a key \u0026ndash; frequency counting, grouping elements, caching results, mapping one value to another.\nSets A Python set is a hash set. It stores unique elements and gives you O(1) average-case membership testing, insertion, and deletion.\n# Creating a set visited = {1, 2, 3, 4, 5} # Add an element -- O(1) average visited.add(6) # Remove an element -- O(1) average visited.remove(3) # raises KeyError if not found visited.discard(3) # safe -- does nothing if not found # Membership test -- O(1) average (this is much faster than doing x in list) print(4 in visited) # True print(10 in visited) # False # Convert a list to a set to remove duplicates nums = [1, 2, 2, 3, 3, 3, 4] unique = set(nums) print(unique) # {1, 2, 3, 4} Set Operations a = {1, 2, 3, 4} b = {3, 4, 5, 6} print(a | b) # Union: {1, 2, 3, 4, 5, 6} print(a \u0026amp; b) # Intersection: {3, 4} print(a - b) # Difference: {1, 2} print(a ^ b) # Symmetric difference: {1, 2, 5, 6} -- elements in one but not both When to use a set in DSA: Any time you need to track what you have already seen, eliminate duplicates, or check membership quickly.\nTuples A tuple is an immutable list. Once created, it cannot be changed. You will use tuples in DSA to represent fixed pairs or groups of values \u0026ndash; coordinates, edges in a graph, keys in a dictionary.\n# Creating a tuple point = (3, 7) edge = (0, 1, 5) # node_from, node_to, weight # Accessing elements -- same as list print(point[0]) # 3 print(point[1]) # 7 # Tuples can be used as dictionary keys because they are immutable distances = {} distances[(0, 1)] = 5 distances[(1, 2)] = 3 # Tuple unpacking -- very commonly used in DSA x, y = point node_from, node_to, weight = edge print(x, y) # 3 7 Lists vs Tuples: Use a list when the collection will change. Use a tuple when the values are fixed and you want to use them as a key or want to signal that they should not change.\nThe Collections Module Python\u0026rsquo;s collections module gives you specialised data structures that come up constantly in DSA problems. You do not need to build these from scratch \u0026ndash; Python gives them to you.\ndeque (Double-Ended Queue) A deque lets you append and pop from both ends in O(1). A regular Python list can only pop from the end efficiently \u0026ndash; popping from the front is O(n). Use a deque whenever you need a queue.\nfrom collections import deque # Create a deque q = deque() # Append to right -- O(1) q.append(1) q.append(2) q.append(3) print(q) # deque([1, 2, 3]) # Append to left -- O(1) q.appendleft(0) print(q) # deque([0, 1, 2, 3]) # Pop from right -- O(1) q.pop() print(q) # deque([0, 1, 2]) # Pop from left -- O(1) \u0026lt;-- this is what makes deque better than list for queues q.popleft() print(q) # deque([1, 2]) You will use deque every time you implement BFS (Breadth-First Search).\ndefaultdict A defaultdict is a dictionary that automatically creates a default value for a key when you access it for the first time. This removes the need to check if a key exists before using it.\nfrom collections import defaultdict # Without defaultdict -- you have to check and initialise word_count = {} words = [\u0026#34;apple\u0026#34;, \u0026#34;banana\u0026#34;, \u0026#34;apple\u0026#34;, \u0026#34;cherry\u0026#34;, \u0026#34;banana\u0026#34;, \u0026#34;apple\u0026#34;] for word in words: if word not in word_count: word_count[word] = 0 word_count[word] += 1 # With defaultdict -- cleaner and less error-prone word_count = defaultdict(int) # default value is 0 for word in words: word_count[word] += 1 print(dict(word_count)) # {\u0026#39;apple\u0026#39;: 3, \u0026#39;banana\u0026#39;: 2, \u0026#39;cherry\u0026#39;: 1} # defaultdict with list -- very useful for grouping from collections import defaultdict graph = defaultdict(list) # default value is an empty list graph[0].append(1) graph[0].append(2) graph[1].append(3) print(dict(graph)) # {0: [1, 2], 1: [3]} You will use defaultdict(list) constantly when building adjacency lists for graphs.\nCounter Counter counts the frequency of elements in an iterable. It is a dictionary subclass where keys are elements and values are counts.\nfrom collections import Counter nums = [1, 2, 2, 3, 3, 3, 4, 4, 4, 4] count = Counter(nums) print(count) # Counter({4: 4, 3: 3, 2: 2, 1: 1}) # Most common elements print(count.most_common(2)) # [(4, 4), (3, 3)] -- top 2 most frequent # Works on strings too s = \u0026#34;hello world\u0026#34; char_count = Counter(s) print(char_count) # Counter({\u0026#39;l\u0026#39;: 3, \u0026#39;o\u0026#39;: 2, \u0026#39;h\u0026#39;: 1, \u0026#39;e\u0026#39;: 1, \u0026#39; \u0026#39;: 1, \u0026#39;w\u0026#39;: 1, \u0026#39;r\u0026#39;: 1, \u0026#39;d\u0026#39;: 1}) List Comprehensions List comprehensions are a concise way to create lists. They appear everywhere in Python DSA solutions and you need to be comfortable reading and writing them.\n# Basic pattern: [expression for item in iterable] # Instead of this: squares = [] for i in range(10): squares.append(i * i) # Write this: squares = [i * i for i in range(10)] print(squares) # [0, 1, 4, 9, 16, 25, 36, 49, 64, 81] # With a condition: [expression for item in iterable if condition] evens = [i for i in range(20) if i % 2 == 0] print(evens) # [0, 2, 4, 6, 8, 10, 12, 14, 16, 18] # Nested list comprehension -- creating a 2D grid grid = [[0 for _ in range(3)] for _ in range(3)] print(grid) # [[0, 0, 0], [0, 0, 0], [0, 0, 0]] Note: The _ variable is a Python convention meaning \u0026ldquo;I do not care about this variable, I just need the loop to run n times.\u0026rdquo;\nImportant Built-in Functions for DSA These are the Python built-ins you will reach for constantly. Know them well.\n# range -- generates a sequence of numbers for i in range(5): # 0, 1, 2, 3, 4 pass for i in range(2, 8): # 2, 3, 4, 5, 6, 7 pass for i in range(0, 10, 2): # 0, 2, 4, 6, 8 pass for i in range(10, 0, -1): # 10, 9, 8, ..., 1 -- counting down pass # enumerate -- gives you index and value together fruits = [\u0026#34;apple\u0026#34;, \u0026#34;banana\u0026#34;, \u0026#34;cherry\u0026#34;] for i, fruit in enumerate(fruits): print(i, fruit) # 0 apple # 1 banana # 2 cherry # zip -- iterates over two lists in parallel names = [\u0026#34;Alice\u0026#34;, \u0026#34;Bob\u0026#34;, \u0026#34;Carol\u0026#34;] scores = [90, 85, 92] for name, score in zip(names, scores): print(f\u0026#34;{name}: {score}\u0026#34;) # min and max print(min(3, 1, 4, 1, 5)) # 1 print(max([3, 1, 4, 1, 5])) # 5 # min/max with a key function -- very useful in DSA points = [(1, 5), (3, 2), (2, 8)] print(min(points, key=lambda p: p[1])) # (3, 2) -- point with smallest y value # abs -- absolute value print(abs(-7)) # 7 # sum print(sum([1, 2, 3, 4, 5])) # 15 # any and all nums = [1, 2, 3, 4, 5] print(any(n \u0026gt; 4 for n in nums)) # True -- at least one element \u0026gt; 4 print(all(n \u0026gt; 0 for n in nums)) # True -- all elements \u0026gt; 0 print(all(n \u0026gt; 2 for n in nums)) # False -- not all elements \u0026gt; 2 Sorting in Python Python\u0026rsquo;s sort is stable and runs in O(n log n). You will use it constantly.\n# Sort a list in place nums = [3, 1, 4, 1, 5, 9, 2, 6] nums.sort() # ascending, in place nums.sort(reverse=True) # descending, in place # sorted() returns a new list, original is unchanged nums = [3, 1, 4, 1, 5, 9] new_nums = sorted(nums) print(nums) # [3, 1, 4, 1, 5, 9] -- unchanged print(new_nums) # [1, 1, 3, 4, 5, 9] -- new sorted list # Sorting with a key -- sort by a computed value words = [\u0026#34;banana\u0026#34;, \u0026#34;apple\u0026#34;, \u0026#34;fig\u0026#34;, \u0026#34;cherry\u0026#34;] words.sort(key=len) # sort by length print(words) # [\u0026#39;fig\u0026#39;, \u0026#39;apple\u0026#39;, \u0026#39;banana\u0026#39;, \u0026#39;cherry\u0026#39;] # Sort a list of tuples by second element pairs = [(1, 3), (2, 1), (3, 2)] pairs.sort(key=lambda x: x[1]) print(pairs) # [(2, 1), (3, 2), (1, 3)] # Sort by multiple criteria -- sort by first element, then second pairs = [(2, 3), (1, 2), (2, 1), (1, 5)] pairs.sort(key=lambda x: (x[0], x[1])) print(pairs) # [(1, 2), (1, 5), (2, 1), (2, 3)] String Operations Strings in Python are immutable. Every time you modify a string, Python creates a new one. This has important implications for performance.\ns = \u0026#34;hello world\u0026#34; # Common string methods print(s.upper()) # HELLO WORLD print(s.lower()) # hello world print(s.split()) # [\u0026#39;hello\u0026#39;, \u0026#39;world\u0026#39;] print(s.split(\u0026#39;o\u0026#39;)) # [\u0026#39;hell\u0026#39;, \u0026#39; w\u0026#39;, \u0026#39;rld\u0026#39;] print(s.strip()) # removes leading/trailing whitespace print(s.replace(\u0026#39;l\u0026#39;, \u0026#39;r\u0026#39;)) # herro worrd print(s.startswith(\u0026#39;hel\u0026#39;)) # True print(s.endswith(\u0026#39;rld\u0026#39;)) # True print(s.count(\u0026#39;l\u0026#39;)) # 3 print(s.find(\u0026#39;world\u0026#39;)) # 6 -- index of first occurrence, -1 if not found # Joining -- the right way to build a string from a list # This is O(n) -- efficient words = [\u0026#34;hello\u0026#34;, \u0026#34;world\u0026#34;, \u0026#34;python\u0026#34;] result = \u0026#34; \u0026#34;.join(words) print(result) # hello world python # String building in a loop -- the wrong way # Each += creates a new string -- O(n^2) overall result = \u0026#34;\u0026#34; for word in words: result += word # slow for large inputs # The right way for string building in a loop parts = [] for word in words: parts.append(word) result = \u0026#34;\u0026#34;.join(parts) # join at the end -- O(n) # Check if a character is alphanumeric print(\u0026#34;a\u0026#34;.isalpha()) # True print(\u0026#34;1\u0026#34;.isdigit()) # True print(\u0026#34;a1\u0026#34;.isalnum()) # True print(\u0026#34; \u0026#34;.isspace()) # True # ord and chr -- converting between characters and ASCII values print(ord(\u0026#39;a\u0026#39;)) # 97 print(ord(\u0026#39;z\u0026#39;)) # 122 print(chr(97)) # a # You will use ord() a lot when working with character frequency problems heapq \u0026ndash; The Built-in Min Heap Python\u0026rsquo;s heapq module gives you a min heap. You will use this for problems involving the K largest or smallest elements, priority queues, and graph algorithms.\nimport heapq nums = [5, 3, 8, 1, 9, 2] # heapify converts a list into a heap in-place -- O(n) heapq.heapify(nums) print(nums) # [1, 3, 2, 5, 9, 8] -- heap order (not fully sorted) # Push an element -- O(log n) heapq.heappush(nums, 4) # Pop the smallest element -- O(log n) smallest = heapq.heappop(nums) print(smallest) # 1 # Peek at smallest without removing -- O(1) print(nums[0]) # Python only has a min heap. For a max heap, negate the values max_heap = [] for n in [5, 3, 8, 1, 9]: heapq.heappush(max_heap, -n) # store negated largest = -heapq.heappop(max_heap) # negate back when popping print(largest) # 9 Common Python Patterns in DSA These patterns appear so frequently in DSA solutions that they are worth calling out explicitly.\n# Pattern 1: Initialise a result variable max_val = float(\u0026#39;-inf\u0026#39;) # negative infinity -- useful when looking for maximum min_val = float(\u0026#39;inf\u0026#39;) # positive infinity -- useful when looking for minimum # Pattern 2: Two pointer setup left, right = 0, len(nums) - 1 while left \u0026lt; right: # do something left += 1 right -= 1 # Pattern 3: Sliding window skeleton left = 0 for right in range(len(nums)): # expand window by including nums[right] while # window condition is violated: # shrink window from left left += 1 # update result # Pattern 4: Frequency map from collections import Counter freq = Counter(nums) # or manually: freq = {} for n in nums: freq[n] = freq.get(n, 0) + 1 # Pattern 5: Graph as adjacency list from collections import defaultdict graph = defaultdict(list) for u, v in edges: graph[u].append(v) graph[v].append(u) # for undirected graph # Pattern 6: BFS skeleton from collections import deque queue = deque([start]) visited = set([start]) while queue: node = queue.popleft() for neighbour in graph[node]: if neighbour not in visited: visited.add(neighbour) queue.append(neighbour) Summary Data Type What it is Key strength When to use in DSA list Dynamic array Fast index access, append General purpose, arrays, stacks dict Hash map O(1) key lookup Frequency counting, memoisation, mapping set Hash set O(1) membership test Tracking visited, deduplication tuple Immutable list Can be used as dict key Coordinates, edges, fixed groups deque Double-ended queue O(1) append/pop both ends BFS, sliding window defaultdict Dict with default Cleaner grouping Graphs, frequency maps Counter Frequency dict Count elements instantly Anagram, frequency problems heapq Min heap O(log n) push/pop, O(1) min peek Top K, priority queue, Dijkstra Key Takeaways Python lists are dynamic arrays. Appending to the end is O(1) amortised but inserting or deleting in the middle is O(n). Dictionaries and sets give you O(1) average-case lookup. Use them aggressively in DSA to reduce time complexity. For queues, always use collections.deque not a list. Popping from the front of a list is O(n). String concatenation in a loop is O(n^2). Always accumulate parts in a list and join at the end. Python only has a min heap via heapq. Negate values to simulate a max heap. The patterns at the end of this blog \u0026ndash; two pointers, sliding window, BFS skeleton \u0026ndash; will appear again and again throughout this guide. Next Steps Next blog: [Complexity Analysis] \u0026ndash; now that you know the Python tools, you will learn how to measure exactly how efficient any piece of code is Reference: Codeverra DSA Sheet ","permalink":"https://blogs.codeverra.com/blog/dsa-02-python-refresher/","summary":"\u003ch1 id=\"python-refresher-for-dsa\"\u003ePython Refresher for DSA\u003c/h1\u003e\n\u003chr\u003e\n\u003ch2 id=\"table-of-contents\"\u003eTable of Contents\u003c/h2\u003e\n\u003cul\u003e\n\u003cli\u003e\u003ca href=\"#what-you-will-learn-in-this-blog\"\u003eWhat You Will Learn in This Blog\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#python-built-in-data-types-for-dsa\"\u003ePython Built-in Data Types for DSA\u003c/a\u003e\n\u003cul\u003e\n\u003cli\u003e\u003ca href=\"#lists\"\u003eLists\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#dictionaries\"\u003eDictionaries\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#sets\"\u003eSets\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#tuples\"\u003eTuples\u003c/a\u003e\u003c/li\u003e\n\u003c/ul\u003e\n\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#the-collections-module\"\u003eThe Collections Module\u003c/a\u003e\n\u003cul\u003e\n\u003cli\u003e\u003ca href=\"#deque-double-ended-queue\"\u003edeque\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#defaultdict\"\u003edefaultdict\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#counter\"\u003eCounter\u003c/a\u003e\u003c/li\u003e\n\u003c/ul\u003e\n\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#list-comprehensions\"\u003eList Comprehensions\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#important-built-in-functions-for-dsa\"\u003eImportant Built-in Functions for DSA\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#sorting-in-python\"\u003eSorting in Python\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#string-operations\"\u003eString Operations\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#heapq----the-built-in-min-heap\"\u003eheapq \u0026ndash; The Built-in Min Heap\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#common-python-patterns-in-dsa\"\u003eCommon Python Patterns in DSA\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#summary\"\u003eSummary\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#key-takeaways\"\u003eKey Takeaways\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#next-steps\"\u003eNext Steps\u003c/a\u003e\u003c/li\u003e\n\u003c/ul\u003e\n\u003chr\u003e\n\u003ch2 id=\"what-you-will-learn-in-this-blog\"\u003eWhat You Will Learn in This Blog\u003c/h2\u003e\n\u003cul\u003e\n\u003cli\u003eThe Python built-in data types you will use constantly in DSA\u003c/li\u003e\n\u003cli\u003eHow lists, dictionaries, sets, and tuples work and when to use each\u003c/li\u003e\n\u003cli\u003eImportant Python built-in functions and their complexities\u003c/li\u003e\n\u003cli\u003eThe collections module \u0026ndash; deque, defaultdict, Counter\u003c/li\u003e\n\u003cli\u003eList comprehensions and when to use them\u003c/li\u003e\n\u003cli\u003eCommon Python patterns that appear repeatedly in DSA problems\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003eThis is not a complete Python tutorial. It covers exactly what you need to follow the rest of this guide comfortably. If you are already confident with all of the above, you can skim this and move on.\u003c/p\u003e","tags":["dsa","python"],"title":"Python Refresher for DSA"},{"content":"Regular Expressions in Python Pattern Matching, Text Processing, and the re Module Before We Begin \u0026ndash; Why Regular Expressions Matter Here is a real problem. You have a CSV file with 50,000 rows of customer data. The phone number column is a mess:\n+91-9876543210 9876 543 210 (091) 9876-543210 91-98765-43210 9876543210 +919876543210 All of these are the same phone number in different formats. You need to extract just the 10-digit number from all of them.\nWithout regular expressions, you would write a long, fragile function with many string operations and if statements that still misses edge cases.\nWith regular expressions, you write one pattern and extract all the numbers in a single pass.\nRegular expressions (regex) are a language for describing patterns in text. They are used for:\nValidation: Is this a valid email address? Is this a valid PIN code? Extraction: Pull all phone numbers from a block of text. Cleaning: Remove all HTML tags from a scraped webpage. Splitting: Split a sentence on any punctuation, not just a single character. Replacement: Replace all date formats in a document with a standard format. They appear in every domain \u0026ndash; data engineering (parsing logs), data science (cleaning text data for NLP), backend development (URL routing, input validation), and automation (processing files and documents).\nThe syntax looks intimidating at first. That is normal. You learn it pattern by pattern and it becomes readable quickly.\nTable of Contents How Regular Expressions Work The re Module \u0026ndash; Core Functions Character Classes Quantifiers \u0026ndash; How Many? Anchors \u0026ndash; Where in the String? Groups and Capturing Alternation and Special Characters Lookahead and Lookbehind Flags \u0026ndash; Modifying Behaviour Compiled Patterns \u0026ndash; re.compile Common Real-World Patterns Data Cleaning with Regex Practice Questions Solutions Summary and Key Takeaways 1. How Regular Expressions Work A regular expression is a pattern string that describes what you are looking for in a body of text.\nimport re text = \u0026#34;My phone number is 9876543210 and my PIN is 560001.\u0026#34; pattern = r\u0026#34;\\d{10}\u0026#34; # \\d means \u0026#34;a digit\u0026#34;, {10} means \u0026#34;exactly 10 of them\u0026#34; match = re.search(pattern, text) print(match.group()) # 9876543210 The raw string prefix r\u0026quot;\u0026hellip;\u0026quot; Always use raw strings (r\u0026quot;...\u0026quot;) for regex patterns. Without the r, Python processes backslash sequences (\\n, \\t) before the regex engine sees them. With r, backslashes are passed literally to the regex engine.\n# Without r -- Python processes \\d as invalid escape pattern = \u0026#34;\\d+\u0026#34; # might produce a DeprecationWarning or unexpected behaviour # With r -- regex engine gets the literal characters \\d+ pattern = r\u0026#34;\\d+\u0026#34; # always use this Regex patterns are matched left to right The engine scans the text from left to right and tries to match the pattern at each position. When it finds a match, it reports it (or all of them, if you ask for all matches).\n2. The re Module \u0026ndash; Core Functions import re text = \u0026#34;Aarav scored 88 marks. Priya scored 95. Rohan scored 72.\u0026#34; re.search() \u0026ndash; find first match anywhere match = re.search(r\u0026#34;\\d+\u0026#34;, text) if match: print(match.group()) # \u0026#34;88\u0026#34; -- first number found print(match.start()) # 13 -- index where match starts print(match.end()) # 15 -- index where match ends print(match.span()) # (13, 15) -- (start, end) tuple else: print(\u0026#34;No match found\u0026#34;) re.findall() \u0026ndash; return all non-overlapping matches as a list numbers = re.findall(r\u0026#34;\\d+\u0026#34;, text) print(numbers) # [\u0026#39;88\u0026#39;, \u0026#39;95\u0026#39;, \u0026#39;72\u0026#39;] -- always returns strings # With groups, returns list of tuples pairs = re.findall(r\u0026#34;(\\w+) scored (\\d+)\u0026#34;, text) print(pairs) # [(\u0026#39;Aarav\u0026#39;, \u0026#39;88\u0026#39;), (\u0026#39;Priya\u0026#39;, \u0026#39;95\u0026#39;), (\u0026#39;Rohan\u0026#39;, \u0026#39;72\u0026#39;)] re.finditer() \u0026ndash; return iterator of match objects # Better than findall when you need match positions or other match data for match in re.finditer(r\u0026#34;\\d+\u0026#34;, text): print(f\u0026#34;Found {match.group()} at position {match.start()}-{match.end()}\u0026#34;) re.match() \u0026ndash; match only at the START of the string # re.match() only checks the beginning of the string print(re.match(r\u0026#34;\\d+\u0026#34;, \u0026#34;123 abc\u0026#34;)) # match -- starts with digits print(re.match(r\u0026#34;\\d+\u0026#34;, \u0026#34;abc 123\u0026#34;)) # None -- does not start with digits # re.search() checks anywhere in the string print(re.search(r\u0026#34;\\d+\u0026#34;, \u0026#34;abc 123\u0026#34;)) # match -- finds 123 inside re.fullmatch() \u0026ndash; match must cover the ENTIRE string # Useful for validation print(re.fullmatch(r\u0026#34;\\d{10}\u0026#34;, \u0026#34;9876543210\u0026#34;)) # match -- exactly 10 digits print(re.fullmatch(r\u0026#34;\\d{10}\u0026#34;, \u0026#34;98765432\u0026#34;)) # None -- only 8 digits print(re.fullmatch(r\u0026#34;\\d{10}\u0026#34;, \u0026#34;9876543210 \u0026#34;)) # None -- trailing space re.sub() \u0026ndash; find and replace # re.sub(pattern, replacement, string, count=0) text = \u0026#34;Phone: 98765-43210, Alt: 87654-32109\u0026#34; # Replace with a fixed string cleaned = re.sub(r\u0026#34;-\u0026#34;, \u0026#34;\u0026#34;, text) print(cleaned) # \u0026#34;Phone: 9876543210, Alt: 8765432109\u0026#34; # Replace using a function def mask_number(match): number = match.group() return \u0026#34;X\u0026#34; * (len(number) - 4) + number[-4:] masked = re.sub(r\u0026#34;\\d{10}\u0026#34;, mask_number, \u0026#34;Contact: 9876543210\u0026#34;) print(masked) # Contact: XXXXXX3210 # Limit replacements text2 = \u0026#34;aaa bbb aaa ccc aaa\u0026#34; result = re.sub(r\u0026#34;aaa\u0026#34;, \u0026#34;ZZZ\u0026#34;, text2, count=2) print(result) # \u0026#34;ZZZ bbb ZZZ ccc aaa\u0026#34; (only first 2 replaced) re.split() \u0026ndash; split on a pattern # Split on any punctuation or whitespace text = \u0026#34;Delhi,Mumbai;Bangalore Hyderabad|Chennai\u0026#34; parts = re.split(r\u0026#34;[,;|\\s]+\u0026#34;, text) print(parts) # [\u0026#39;Delhi\u0026#39;, \u0026#39;Mumbai\u0026#39;, \u0026#39;Bangalore\u0026#39;, \u0026#39;Hyderabad\u0026#39;, \u0026#39;Chennai\u0026#39;] # Split a sentence on sentence-ending punctuation sentences = re.split(r\u0026#34;[.!?]+\\s*\u0026#34;, \u0026#34;Hello! How are you? I am fine. Thanks.\u0026#34;) print(sentences) # [\u0026#39;Hello\u0026#39;, \u0026#39;How are you\u0026#39;, \u0026#39;I am fine\u0026#39;, \u0026#39;Thanks\u0026#39;, \u0026#39;\u0026#39;] 3. Character Classes Character classes match a single character from a set.\nBuilt-in shorthand classes \\d Any digit: 0-9 \\D Any NON-digit \\w Any word character: letters, digits, underscore [a-zA-Z0-9_] \\W Any NON-word character \\s Any whitespace: space, tab, newline, etc. \\S Any NON-whitespace . Any character EXCEPT newline (unless re.DOTALL is set) text = \u0026#34;Aarav scored 88/100 in Python (4 credits)\u0026#34; print(re.findall(r\u0026#34;\\d+\u0026#34;, text)) # [\u0026#39;88\u0026#39;, \u0026#39;100\u0026#39;, \u0026#39;4\u0026#39;] -- all numbers print(re.findall(r\u0026#34;\\w+\u0026#34;, text)) # all words (letters+digits) print(re.findall(r\u0026#34;\\s\u0026#34;, text)) # all whitespace characters print(re.findall(r\u0026#34;\\D+\u0026#34;, text)) # sequences of non-digit characters Custom character classes with [] # [abc] -- match a, b, or c # [a-z] -- match any lowercase letter # [A-Z] -- match any uppercase letter # [0-9] -- match any digit (same as \\d) # [a-zA-Z] -- match any letter # [^abc] -- match any character EXCEPT a, b, c (^ inside [] negates) text = \u0026#34;Zomato Order #AB-2024-001: Rs.450.00\u0026#34; # Extract only letters letters = re.findall(r\u0026#34;[a-zA-Z]+\u0026#34;, text) print(letters) # [\u0026#39;Zomato\u0026#39;, \u0026#39;Order\u0026#39;, \u0026#39;AB\u0026#39;, \u0026#39;Rs\u0026#39;] # Extract only digits digits = re.findall(r\u0026#34;[0-9]+\u0026#34;, text) print(digits) # [\u0026#39;2024\u0026#39;, \u0026#39;001\u0026#39;, \u0026#39;450\u0026#39;, \u0026#39;00\u0026#39;] # Extract alphanumeric sequences alphanum = re.findall(r\u0026#34;[a-zA-Z0-9]+\u0026#34;, text) print(alphanum) # [\u0026#39;Zomato\u0026#39;, \u0026#39;Order\u0026#39;, \u0026#39;AB\u0026#39;, \u0026#39;2024\u0026#39;, \u0026#39;001\u0026#39;, \u0026#39;Rs\u0026#39;, \u0026#39;450\u0026#39;, \u0026#39;00\u0026#39;] # Match vowels vowels = re.findall(r\u0026#34;[aeiouAEIOU]\u0026#34;, \u0026#34;Bangalore\u0026#34;) print(vowels) # [\u0026#39;a\u0026#39;, \u0026#39;a\u0026#39;, \u0026#39;o\u0026#39;, \u0026#39;e\u0026#39;] # Match non-vowels consonants = re.findall(r\u0026#34;[^aeiouAEIOU\\s]\u0026#34;, \u0026#34;Bangalore\u0026#34;) print(consonants) # [\u0026#39;B\u0026#39;, \u0026#39;n\u0026#39;, \u0026#39;g\u0026#39;, \u0026#39;l\u0026#39;, \u0026#39;r\u0026#39;] 4. Quantifiers \u0026ndash; How Many? Quantifiers say how many times the preceding element must appear.\n? Zero or one (optional) * Zero or more + One or more {n} Exactly n times {n,} At least n times {n,m} Between n and m times (inclusive) text = \u0026#34;colour color colouur colooor\u0026#34; # ? -- zero or one \u0026#39;u\u0026#39; print(re.findall(r\u0026#34;colou?r\u0026#34;, text)) # [\u0026#39;colour\u0026#39;, \u0026#39;color\u0026#39;] -- matches with or without \u0026#39;u\u0026#39; # * -- zero or more \u0026#39;u\u0026#39; print(re.findall(r\u0026#34;colou*r\u0026#34;, text)) # [\u0026#39;colour\u0026#39;, \u0026#39;color\u0026#39;, \u0026#39;colouur\u0026#39;, \u0026#39;colooor\u0026#39;... wait, let us look at this more carefully # \u0026#39;colooor\u0026#39; doesn\u0026#39;t match because o* needs o\u0026#39;s between colo and r, not u\u0026#39;s # Actually the pattern \u0026#34;colou*r\u0026#34; matches: colo + zero or more u\u0026#39;s + r # + -- one or more digits phone = \u0026#34;Call 9876543210 or 022-12345678\u0026#34; print(re.findall(r\u0026#34;\\d+\u0026#34;, phone)) # [\u0026#39;9876543210\u0026#39;, \u0026#39;022\u0026#39;, \u0026#39;12345678\u0026#39;] # {n} -- exact count print(re.findall(r\u0026#34;\\d{10}\u0026#34;, phone)) # [\u0026#39;9876543210\u0026#39;] -- exactly 10 digits # {n,m} -- range print(re.findall(r\u0026#34;\\d{3,5}\u0026#34;, \u0026#34;12 123 1234 12345 123456\u0026#34;)) # [\u0026#39;123\u0026#39;, \u0026#39;1234\u0026#39;, \u0026#39;12345\u0026#39;, \u0026#39;12345\u0026#39;] -- 3 to 5 digits # Note: \u0026#39;123456\u0026#39; matches as \u0026#39;12345\u0026#39; (first 5 digits) then \u0026#39;6\u0026#39; is left Greedy vs lazy matching By default, quantifiers are greedy \u0026ndash; they match as much as possible. Adding ? after a quantifier makes it lazy \u0026ndash; it matches as little as possible.\nhtml = \u0026#34;\u0026lt;b\u0026gt;Bold\u0026lt;/b\u0026gt; and \u0026lt;i\u0026gt;Italic\u0026lt;/i\u0026gt;\u0026#34; # Greedy -- matches from first \u0026lt; to LAST \u0026gt; greedy = re.findall(r\u0026#34;\u0026lt;.+\u0026gt;\u0026#34;, html) print(greedy) # [\u0026#39;\u0026lt;b\u0026gt;Bold\u0026lt;/b\u0026gt; and \u0026lt;i\u0026gt;Italic\u0026lt;/i\u0026gt;\u0026#39;] -- one huge match # Lazy -- matches from \u0026lt; to the NEXT \u0026gt; lazy = re.findall(r\u0026#34;\u0026lt;.+?\u0026gt;\u0026#34;, html) print(lazy) # [\u0026#39;\u0026lt;b\u0026gt;\u0026#39;, \u0026#39;\u0026lt;/b\u0026gt;\u0026#39;, \u0026#39;\u0026lt;i\u0026gt;\u0026#39;, \u0026#39;\u0026lt;/i\u0026gt;\u0026#39;] -- four separate matches # Greedy quantifiers: * + ? {n,m} # Lazy quantifiers: *? +? ?? {n,m}? 5. Anchors \u0026ndash; Where in the String? Anchors do not match characters \u0026ndash; they match positions.\n^ Start of string (or start of line with re.MULTILINE) $ End of string (or end of line with re.MULTILINE) \\b Word boundary (between \\w and \\W) \\B NOT a word boundary # ^ -- must be at the start print(re.search(r\u0026#34;^\\d{4}\u0026#34;, \u0026#34;2024 is the year\u0026#34;)) # match -- starts with digits print(re.search(r\u0026#34;^\\d{4}\u0026#34;, \u0026#34;The year is 2024\u0026#34;)) # None -- not at start # $ -- must be at the end print(re.search(r\u0026#34;\\d{4}$\u0026#34;, \u0026#34;The year is 2024\u0026#34;)) # match -- ends with digits print(re.search(r\u0026#34;\\d{4}$\u0026#34;, \u0026#34;2024 is the year\u0026#34;)) # None -- not at end # Both -- must match the ENTIRE string (similar to fullmatch) print(re.search(r\u0026#34;^\\d{4}$\u0026#34;, \u0026#34;2024\u0026#34;)) # match print(re.search(r\u0026#34;^\\d{4}$\u0026#34;, \u0026#34;20245\u0026#34;)) # None # \\b -- word boundary text = \u0026#34;cat concatenate category cats\u0026#34; print(re.findall(r\u0026#34;\\bcat\\b\u0026#34;, text)) # [\u0026#39;cat\u0026#39;] -- only the exact word print(re.findall(r\u0026#34;cat\u0026#34;, text)) # [\u0026#39;cat\u0026#39;, \u0026#39;cat\u0026#39;, \u0026#39;cat\u0026#39;, \u0026#39;cat\u0026#39;] # -- matches inside other words too Practical anchor use # Validate that a string is a valid 6-digit Indian PIN code def is_valid_pin(pin: str) -\u0026gt; bool: return bool(re.fullmatch(r\u0026#34;[1-9]\\d{5}\u0026#34;, pin)) # Starts with 1-9 (no Indian PIN starts with 0) # Followed by exactly 5 more digits print(is_valid_pin(\u0026#34;560001\u0026#34;)) # True print(is_valid_pin(\u0026#34;060001\u0026#34;)) # False -- starts with 0 print(is_valid_pin(\u0026#34;56000\u0026#34;)) # False -- only 5 digits # Validate a line starts with a date lines = [ \u0026#34;2024-03-15 Payment received\u0026#34;, \u0026#34;Error: Payment failed\u0026#34;, \u0026#34;2024-03-16 Refund processed\u0026#34;, ] date_lines = [l for l in lines if re.match(r\u0026#34;^\\d{4}-\\d{2}-\\d{2}\u0026#34;, l)] print(date_lines) # [\u0026#39;2024-03-15 Payment received\u0026#39;, \u0026#39;2024-03-16 Refund processed\u0026#39;] 6. Groups and Capturing Parentheses () create groups. Groups:\nCapture the matched text so you can extract it separately Allow you to apply quantifiers to a sequence Enable backreferences text = \u0026#34;Aarav scored 88. Priya scored 95. Rohan scored 72.\u0026#34; # Without groups -- matches the whole pattern matches = re.findall(r\u0026#34;\\w+ scored \\d+\u0026#34;, text) print(matches) # [\u0026#39;Aarav scored 88\u0026#39;, \u0026#39;Priya scored 95\u0026#39;, \u0026#39;Rohan scored 72\u0026#39;] # With groups -- returns tuples of captured groups matches = re.findall(r\u0026#34;(\\w+) scored (\\d+)\u0026#34;, text) print(matches) # [(\u0026#39;Aarav\u0026#39;, \u0026#39;88\u0026#39;), (\u0026#39;Priya\u0026#39;, \u0026#39;95\u0026#39;), (\u0026#39;Rohan\u0026#39;, \u0026#39;72\u0026#39;)] # Extract just the scores for name, score in matches: print(f\u0026#34;{name}: {int(score)}\u0026#34;) Accessing groups in match objects log_line = \u0026#34;2024-03-15 14:30:22 ERROR payments.billing Payment failed for order 1042\u0026#34; pattern = r\u0026#34;(\\d{4}-\\d{2}-\\d{2}) (\\d{2}:\\d{2}:\\d{2}) (\\w+) (\\S+) (.+)\u0026#34; match = re.search(pattern, log_line) if match: print(match.group(0)) # entire match print(match.group(1)) # 2024-03-15 print(match.group(2)) # 14:30:22 print(match.group(3)) # ERROR print(match.group(4)) # payments.billing print(match.group(5)) # Payment failed for order 1042 print(match.groups()) # tuple of all groups Named groups \u0026ndash; r\u0026quot;(?Ppattern)\u0026quot; log_line = \u0026#34;2024-03-15 14:30:22 ERROR payments.billing Payment failed\u0026#34; pattern = ( r\u0026#34;(?P\u0026lt;date\u0026gt;\\d{4}-\\d{2}-\\d{2})\\s+\u0026#34; r\u0026#34;(?P\u0026lt;time\u0026gt;\\d{2}:\\d{2}:\\d{2})\\s+\u0026#34; r\u0026#34;(?P\u0026lt;level\u0026gt;\\w+)\\s+\u0026#34; r\u0026#34;(?P\u0026lt;module\u0026gt;\\S+)\\s+\u0026#34; r\u0026#34;(?P\u0026lt;message\u0026gt;.+)\u0026#34; ) match = re.search(pattern, log_line) if match: print(match.group(\u0026#34;date\u0026#34;)) # 2024-03-15 print(match.group(\u0026#34;level\u0026#34;)) # ERROR print(match.group(\u0026#34;message\u0026#34;)) # Payment failed print(match.groupdict()) # dict of all named groups Non-capturing groups \u0026ndash; r\u0026quot;(?:pattern)\u0026quot; When you need grouping for structure but do not need to capture:\n# You want to match \u0026#34;colour\u0026#34; or \u0026#34;color\u0026#34; but not capture the group text = \u0026#34;American color and British colour\u0026#34; # (?:...) groups without capturing words = re.findall(r\u0026#34;colo(?:u?)r\u0026#34;, text) print(words) # [\u0026#39;color\u0026#39;, \u0026#39;colour\u0026#39;] # Without non-capturing group, findall would return the captured \u0026#39;u\u0026#39; parts words2 = re.findall(r\u0026#34;colo(u?)r\u0026#34;, text) print(words2) # [\u0026#39;\u0026#39;, \u0026#39;u\u0026#39;] -- only the captured group content returned! 7. Alternation and Special Characters Alternation with | # Match \u0026#34;cat\u0026#34; or \u0026#34;dog\u0026#34; or \u0026#34;bird\u0026#34; animals = re.findall(r\u0026#34;cat|dog|bird\u0026#34;, \u0026#34;I have a cat and a dog but no bird\u0026#34;) print(animals) # [\u0026#39;cat\u0026#39;, \u0026#39;dog\u0026#39;, \u0026#39;bird\u0026#39;] # Use with groups for more complex alternation # Match Mumbai or Delhi or Bangalore as whole words cities = re.findall(r\u0026#34;\\b(?:Mumbai|Delhi|Bangalore)\\b\u0026#34;, \u0026#34;Flight from Delhi to Mumbai via Bangalore\u0026#34;) print(cities) # [\u0026#39;Delhi\u0026#39;, \u0026#39;Mumbai\u0026#39;, \u0026#39;Bangalore\u0026#39;] Escaping special characters These characters have special meaning in regex and must be escaped with \\ if you want to match them literally:\n. ^ $ * + ? { } [ ] \\ | ( ) # Match a literal dot (.) text = \u0026#34;Price: Rs.450.00\u0026#34; prices = re.findall(r\u0026#34;\\d+\\.\\d+\u0026#34;, text) print(prices) # [\u0026#39;450.00\u0026#39;] # Without escape, . matches ANY character prices2 = re.findall(r\u0026#34;\\d+.\\d+\u0026#34;, text) print(prices2) # [\u0026#39;450.00\u0026#39;] -- happens to work here but is wrong # Match a literal pipe character parts = re.split(r\u0026#34;\\|\u0026#34;, \u0026#34;Delhi|Mumbai|Bangalore\u0026#34;) print(parts) # [\u0026#39;Delhi\u0026#39;, \u0026#39;Mumbai\u0026#39;, \u0026#39;Bangalore\u0026#39;] # Match parentheses text = \u0026#34;Score (out of 100): 88\u0026#34; match = re.search(r\u0026#34;\\(out of (\\d+)\\)\u0026#34;, text) if match: print(match.group(1)) # 100 8. Lookahead and Lookbehind Lookahead and lookbehind are zero-width assertions \u0026ndash; they check for a pattern without including it in the match.\n(?=pattern) Positive lookahead -- what FOLLOWS the match (?!pattern) Negative lookahead -- what does NOT follow (?\u0026lt;=pattern) Positive lookbehind -- what PRECEDES the match (?\u0026lt;!pattern) Negative lookbehind -- what does NOT precede # Positive lookahead: find numbers followed by \u0026#34;kg\u0026#34; text = \u0026#34;I have 5kg of rice, 2 apples, and 10kg of flour\u0026#34; numbers_before_kg = re.findall(r\u0026#34;\\d+(?=kg)\u0026#34;, text) print(numbers_before_kg) # [\u0026#39;5\u0026#39;, \u0026#39;10\u0026#39;] -- only numbers before \u0026#34;kg\u0026#34; # Positive lookbehind: find numbers preceded by \u0026#34;Rs.\u0026#34; text = \u0026#34;Order total: Rs.450, Delivery: Rs.30, Tax: Rs.22\u0026#34; amounts = re.findall(r\u0026#34;(?\u0026lt;=Rs\\.)\\d+\u0026#34;, text) print(amounts) # [\u0026#39;450\u0026#39;, \u0026#39;30\u0026#39;, \u0026#39;22\u0026#39;] # Negative lookahead: find \u0026#34;Python\u0026#34; NOT followed by \u0026#34;3\u0026#34; text = \u0026#34;Python is great. Python 3 is even better. Learn Python.\u0026#34; matches = re.findall(r\u0026#34;Python(?! 3)\u0026#34;, text) print(matches) # [\u0026#39;Python\u0026#39;, \u0026#39;Python\u0026#39;] -- not the \u0026#34;Python 3\u0026#34; instance # Practical: extract values from key=value pairs config = \u0026#34;timeout=30, retries=3, debug=false, host=localhost\u0026#34; values = re.findall(r\u0026#34;(?\u0026lt;=timeout=)\\d+|(?\u0026lt;=retries=)\\d+\u0026#34;, config) print(values) # [\u0026#39;30\u0026#39;, \u0026#39;3\u0026#39;] # Better approach: extract all key=value pairs pairs = re.findall(r\u0026#34;(\\w+)=(\\w+)\u0026#34;, config) print(pairs) # [(\u0026#39;timeout\u0026#39;, \u0026#39;30\u0026#39;), (\u0026#39;retries\u0026#39;, \u0026#39;3\u0026#39;), (\u0026#39;debug\u0026#39;, \u0026#39;false\u0026#39;), (\u0026#39;host\u0026#39;, \u0026#39;localhost\u0026#39;)] 9. Flags \u0026ndash; Modifying Behaviour Flags change how the regex engine interprets patterns.\nimport re # re.IGNORECASE (re.I) -- case-insensitive matching text = \u0026#34;Python PYTHON python PyThOn\u0026#34; matches = re.findall(r\u0026#34;python\u0026#34;, text, re.IGNORECASE) print(matches) # [\u0026#39;Python\u0026#39;, \u0026#39;PYTHON\u0026#39;, \u0026#39;python\u0026#39;, \u0026#39;PyThOn\u0026#39;] # re.MULTILINE (re.M) -- ^ and $ match start/end of each LINE text = \u0026#34;\u0026#34;\u0026#34;First line Second line Third line\u0026#34;\u0026#34;\u0026#34; print(re.findall(r\u0026#34;^\\w+\u0026#34;, text)) # [\u0026#39;First\u0026#39;] -- only first line print(re.findall(r\u0026#34;^\\w+\u0026#34;, text, re.M)) # [\u0026#39;First\u0026#39;, \u0026#39;Second\u0026#39;, \u0026#39;Third\u0026#39;] # re.DOTALL (re.S) -- . matches newlines too text = \u0026#34;Name: Aarav\\nCity: Bangalore\\nScore: 88\u0026#34; match = re.search(r\u0026#34;Name: .+Score\u0026#34;, text) print(match) # None -- . does not cross newlines match = re.search(r\u0026#34;Name: .+Score\u0026#34;, text, re.DOTALL) print(match.group() if match else None) # Name: Aarav\\nCity: Bangalore\\nScore # re.VERBOSE (re.X) -- allows whitespace and comments in pattern # Makes complex patterns readable phone_pattern = re.compile(r\u0026#34;\u0026#34;\u0026#34; (?:\\+91|0)? # optional country code [-.\\s]? # optional separator ([6-9]\\d{9}) # 10-digit mobile number starting with 6-9 \u0026#34;\u0026#34;\u0026#34;, re.VERBOSE) print(phone_pattern.findall(\u0026#34;+91-9876543210\u0026#34;)) # [\u0026#39;9876543210\u0026#39;] # Combine flags with | (pipe) matches = re.findall(r\u0026#34;^python\u0026#34;, text, re.IGNORECASE | re.MULTILINE) 10. Compiled Patterns \u0026ndash; re.compile When you use the same pattern many times, compile it first. re.compile() pre-processes the pattern once, making repeated use faster.\nimport re # Without compile -- pattern re-processed each time for text in large_list_of_texts: matches = re.findall(r\u0026#34;\\d{10}\u0026#34;, text) # pattern compiled on each call # With compile -- pattern compiled once phone_pattern = re.compile(r\u0026#34;\\d{10}\u0026#34;) for text in large_list_of_texts: matches = phone_pattern.findall(text) # uses pre-compiled pattern # Compiled pattern has the same methods as re module functions pattern = re.compile(r\u0026#34;(\\w+)\\s+scored\\s+(\\d+)\u0026#34;, re.IGNORECASE) pattern.search(\u0026#34;Aarav scored 88\u0026#34;) pattern.findall(\u0026#34;Aarav scored 88. Priya scored 95.\u0026#34;) pattern.match(\u0026#34;Scored 88 Aarav\u0026#34;) # None -- wrong order pattern.sub(\u0026#34;REDACTED\u0026#34;, \u0026#34;Aarav scored 88\u0026#34;) pattern.split(\u0026#34;info scored 88 more info\u0026#34;) 11. Common Real-World Patterns A reference collection of patterns you will use regularly.\nimport re # ── Indian context ─────────────────────────────────────────────────────── # Mobile number (Indian -- starts with 6-9, 10 digits) MOBILE = re.compile(r\u0026#34;(?:\\+91|91|0)?[-.\\s]?([6-9]\\d{9})\u0026#34;) # Indian PIN code (6 digits, does not start with 0) PIN_CODE = re.compile(r\u0026#34;\\b[1-9]\\d{5}\\b\u0026#34;) # Indian PAN card (AAAAA1234A format) PAN = re.compile(r\u0026#34;\\b[A-Z]{5}\\d{4}[A-Z]\\b\u0026#34;) # Indian Aadhaar number (12 digits, often formatted as XXXX XXXX XXXX) AADHAAR = re.compile(r\u0026#34;\\b\\d{4}\\s?\\d{4}\\s?\\d{4}\\b\u0026#34;) # ── General ────────────────────────────────────────────────────────────── # Email address EMAIL = re.compile(r\u0026#34;\\b[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\\.[A-Z|a-z]{2,}\\b\u0026#34;) # URL URL = re.compile(r\u0026#34;https?://(?:www\\.)?[-a-zA-Z0-9@:%._+~#=]{1,256}\u0026#34; r\u0026#34;\\.[a-zA-Z0-9()]{1,6}\\b[-a-zA-Z0-9()@:%_+.~#?\u0026amp;=/]*\u0026#34;) # IPv4 address IPV4 = re.compile(r\u0026#34;\\b(?:(?:25[0-5]|2[0-4]\\d|[01]?\\d\\d?)\\.){3}\u0026#34; r\u0026#34;(?:25[0-5]|2[0-4]\\d|[01]?\\d\\d?)\\b\u0026#34;) # Date formats DATE_ISO = re.compile(r\u0026#34;\\b\\d{4}-\\d{2}-\\d{2}\\b\u0026#34;) # 2024-03-15 DATE_IN = re.compile(r\u0026#34;\\b\\d{2}/\\d{2}/\\d{4}\\b\u0026#34;) # 15/03/2024 DATE_LONG = re.compile(r\u0026#34;\\b\\d{1,2}\\s+(?:Jan|Feb|Mar|Apr|May|Jun|\u0026#34; r\u0026#34;Jul|Aug|Sep|Oct|Nov|Dec)[a-z]*\\s+\\d{4}\\b\u0026#34;, re.IGNORECASE) # 15 March 2024 # Time (24-hour) TIME_24 = re.compile(r\u0026#34;\\b([01]?\\d|2[0-3]):([0-5]\\d)(?::([0-5]\\d))?\\b\u0026#34;) # Password strength check components HAS_UPPER = re.compile(r\u0026#34;[A-Z]\u0026#34;) HAS_LOWER = re.compile(r\u0026#34;[a-z]\u0026#34;) HAS_DIGIT = re.compile(r\u0026#34;\\d\u0026#34;) HAS_SPECIAL = re.compile(r\u0026#34;[!@#$%^\u0026amp;*(),.?\\\u0026#34;:{}|\u0026lt;\u0026gt;]\u0026#34;) # ── Usage examples ─────────────────────────────────────────────────────── texts = [ \u0026#34;Contact Aarav at aarav@codeverra.com or +91-9876543210\u0026#34;, \u0026#34;Visit us at https://learn.codeverra.com\u0026#34;, \u0026#34;Office PIN: 560001, PAN: ABCDE1234F\u0026#34;, \u0026#34;DOB: 15 March 2024\u0026#34;, ] for text in texts: emails = EMAIL.findall(text) phones = MOBILE.findall(text) urls = URL.findall(text) pins = PIN_CODE.findall(text) dates = DATE_LONG.findall(text) if emails: print(f\u0026#34;Emails : {emails}\u0026#34;) if phones: print(f\u0026#34;Phones : {phones}\u0026#34;) if urls: print(f\u0026#34;URLs : {urls}\u0026#34;) if pins: print(f\u0026#34;PINs : {pins}\u0026#34;) if dates: print(f\u0026#34;Dates : {dates}\u0026#34;) 12. Data Cleaning with Regex This is where regex earns its place in data science work.\nimport re import pandas as pd # ── Phone number normalisation ──────────────────────────────────────────── def normalise_phone(phone: str) -\u0026gt; str | None: \u0026#34;\u0026#34;\u0026#34; Normalise an Indian mobile number to 10 digits. Handles formats like: +91-9876543210, 09876543210, 9876 543 210 Returns None if not a valid mobile number. \u0026#34;\u0026#34;\u0026#34; # Remove all non-digit characters digits = re.sub(r\u0026#34;\\D\u0026#34;, \u0026#34;\u0026#34;, phone) # Remove country code if present if len(digits) == 12 and digits.startswith(\u0026#34;91\u0026#34;): digits = digits[2:] elif len(digits) == 11 and digits.startswith(\u0026#34;0\u0026#34;): digits = digits[1:] # Validate: 10 digits starting with 6-9 if re.fullmatch(r\u0026#34;[6-9]\\d{9}\u0026#34;, digits): return digits return None phones = [\u0026#34;+91-9876543210\u0026#34;, \u0026#34;09876543210\u0026#34;, \u0026#34;9876 543 210\u0026#34;, \u0026#34;98765432\u0026#34;, \u0026#34;+44-9876543210\u0026#34;, \u0026#34;9876543210\u0026#34;] for phone in phones: result = normalise_phone(phone) print(f\u0026#34;{phone:\u0026lt;20} --\u0026gt; {result}\u0026#34;) # ── Text cleaning for NLP ───────────────────────────────────────────────── def clean_text(text: str) -\u0026gt; str: \u0026#34;\u0026#34;\u0026#34; Clean raw text for NLP processing. - Remove HTML tags - Remove URLs - Remove extra whitespace - Keep only letters and spaces \u0026#34;\u0026#34;\u0026#34; text = re.sub(r\u0026#34;\u0026lt;[^\u0026gt;]+\u0026gt;\u0026#34;, \u0026#34;\u0026#34;, text) # remove HTML tags text = re.sub(r\u0026#34;https?://\\S+\u0026#34;, \u0026#34;\u0026#34;, text) # remove URLs text = re.sub(r\u0026#34;[^a-zA-Z\\s]\u0026#34;, \u0026#34;\u0026#34;, text) # keep only letters text = re.sub(r\u0026#34;\\s+\u0026#34;, \u0026#34; \u0026#34;, text) # collapse whitespace return text.strip().lower() raw = \u0026#34;\u0026#34;\u0026#34; \u0026lt;p\u0026gt;Check out our website at https://codeverra.com! Python is GREAT for data science \u0026amp; ML. Call us: 9876543210.\u0026lt;/p\u0026gt; \u0026#34;\u0026#34;\u0026#34; print(clean_text(raw)) # \u0026#34;check out our website at python is great for data science ml call us\u0026#34; # ── Log file parsing ────────────────────────────────────────────────────── def parse_log_line(line: str) -\u0026gt; dict | None: \u0026#34;\u0026#34;\u0026#34; Parse a structured log line into components. Format: 2024-03-15 14:30:22,123 ERROR module.name Message here \u0026#34;\u0026#34;\u0026#34; pattern = re.compile( r\u0026#34;(?P\u0026lt;date\u0026gt;\\d{4}-\\d{2}-\\d{2})\\s+\u0026#34; r\u0026#34;(?P\u0026lt;time\u0026gt;\\d{2}:\\d{2}:\\d{2}),(?P\u0026lt;ms\u0026gt;\\d{3})\\s+\u0026#34; r\u0026#34;(?P\u0026lt;level\u0026gt;DEBUG|INFO|WARNING|ERROR|CRITICAL)\\s+\u0026#34; r\u0026#34;(?P\u0026lt;module\u0026gt;\\S+)\\s+\u0026#34; r\u0026#34;(?P\u0026lt;message\u0026gt;.+)\u0026#34; ) match = pattern.fullmatch(line.strip()) if not match: return None return match.groupdict() logs = [ \u0026#34;2024-03-15 14:30:22,123 ERROR payments.billing Payment failed for order 1042\u0026#34;, \u0026#34;2024-03-15 14:30:23,456 INFO data_loader Loaded 500 records\u0026#34;, \u0026#34;Not a valid log line\u0026#34;, \u0026#34;2024-03-15 14:30:24,789 WARNING analytics Missing values in column price\u0026#34;, ] for line in logs: parsed = parse_log_line(line) if parsed: print(f\u0026#34;[{parsed[\u0026#39;level\u0026#39;]}] {parsed[\u0026#39;module\u0026#39;]}: {parsed[\u0026#39;message\u0026#39;]}\u0026#34;) else: print(f\u0026#34;Unparseable: {line[:40]}\u0026#34;) # ── CSV field extraction ────────────────────────────────────────────────── def extract_product_info(description: str) -\u0026gt; dict: \u0026#34;\u0026#34;\u0026#34; Extract structured info from product descriptions like: \u0026#34;Laptop 16GB RAM 512GB SSD 15.6 inch Rs.65000\u0026#34; \u0026#34;\u0026#34;\u0026#34; result = {} ram_match = re.search(r\u0026#34;(\\d+)GB\\s+RAM\u0026#34;, description, re.IGNORECASE) ssd_match = re.search(r\u0026#34;(\\d+)GB\\s+SSD\u0026#34;, description, re.IGNORECASE) inch_match = re.search(r\u0026#34;(\\d+\\.?\\d*)\\s*inch\u0026#34;, description, re.IGNORECASE) price_match = re.search(r\u0026#34;Rs\\.(\\d+(?:,\\d+)*)\u0026#34;, description, re.IGNORECASE) if ram_match: result[\u0026#34;ram_gb\u0026#34;] = int(ram_match.group(1)) if ssd_match: result[\u0026#34;ssd_gb\u0026#34;] = int(ssd_match.group(1)) if inch_match: result[\u0026#34;screen\u0026#34;] = float(inch_match.group(1)) if price_match: result[\u0026#34;price\u0026#34;] = int(price_match.group(1).replace(\u0026#34;,\u0026#34;, \u0026#34;\u0026#34;)) return result descriptions = [ \u0026#34;Laptop 16GB RAM 512GB SSD 15.6 inch Rs.65000\u0026#34;, \u0026#34;MacBook 8GB RAM 256GB SSD 13.3 inch Rs.1,10,000\u0026#34;, \u0026#34;Gaming laptop 32GB RAM 1024GB SSD 17.3 inch Rs.1,50,000\u0026#34;, ] for desc in descriptions: print(extract_product_info(desc)) 13. Practice Questions Try each question before reading the solution. Questions ordered: Easy (1-5), Medium (6-10), Hard (11-15).\nEasy Q1 \u0026ndash; Basic search Write a function has_number(text) that returns True if the text contains at least one number (digit sequence).\nhas_number(\u0026#34;Hello World\u0026#34;) # False has_number(\u0026#34;I scored 88 marks\u0026#34;) # True has_number(\u0026#34;Room 404\u0026#34;) # True Q2 \u0026ndash; Extract all Write a function extract_emails(text) that returns a list of all email addresses found in the text.\ntext = \u0026#34;Contact aarav@codeverra.com or support@codeverra.com for help\u0026#34; extract_emails(text) # [\u0026#39;aarav@codeverra.com\u0026#39;, \u0026#39;support@codeverra.com\u0026#39;] Q3 \u0026ndash; Validation Write a function is_valid_pin(pin) that returns True if the string is a valid Indian PIN code (6 digits, first digit 1-9).\nis_valid_pin(\u0026#34;560001\u0026#34;) # True is_valid_pin(\u0026#34;060001\u0026#34;) # False is_valid_pin(\u0026#34;56001\u0026#34;) # False is_valid_pin(\u0026#34;5600011\u0026#34;) # False Q4 \u0026ndash; Substitution Write a function mask_card(text) that replaces all 16-digit credit card numbers in a string with \u0026ldquo;XXXX-XXXX-XXXX-XXXX\u0026rdquo;.\ntext = \u0026#34;Card 4532015112830366 was charged Rs.5000\u0026#34; mask_card(text) # \u0026#34;Card XXXX-XXXX-XXXX-XXXX was charged Rs.5000\u0026#34; Q5 \u0026ndash; Split Write a function split_csv_line(line) that splits a CSV line correctly, handling multiple consecutive delimiters (,, ;, or |) as one separator.\nsplit_csv_line(\u0026#34;Delhi,,Mumbai;Bangalore||Chennai\u0026#34;) # [\u0026#39;Delhi\u0026#39;, \u0026#39;Mumbai\u0026#39;, \u0026#39;Bangalore\u0026#39;, \u0026#39;Chennai\u0026#39;] Medium Q6 \u0026ndash; Groups Write a function parse_date(text) that extracts dates in the format DD-MM-YYYY or DD/MM/YYYY and returns them as a list of dicts with keys day, month, year.\nparse_date(\u0026#34;Event on 15-03-2024 and 22/04/2024\u0026#34;) # [{\u0026#39;day\u0026#39;: \u0026#39;15\u0026#39;, \u0026#39;month\u0026#39;: \u0026#39;03\u0026#39;, \u0026#39;year\u0026#39;: \u0026#39;2024\u0026#39;}, # {\u0026#39;day\u0026#39;: \u0026#39;22\u0026#39;, \u0026#39;month\u0026#39;: \u0026#39;04\u0026#39;, \u0026#39;year\u0026#39;: \u0026#39;2024\u0026#39;}] Q7 \u0026ndash; Named groups Write a function parse_transaction(line) that parses transaction log lines in this format and returns a dict:\nTXN20240315143022-001 | CREDIT | Rs.5000.00 | Aarav Sharma | UPI The dict should have keys: txn_id, type, amount, name, method.\nQ8 \u0026ndash; Lookahead Write a function extract_quantities(text) that extracts numbers that are immediately followed by a unit (kg, g, L, ml, km) but returns only the number, not the unit.\nextract_quantities(\u0026#34;I need 500g sugar, 2kg flour, and 1L milk for 5km drive\u0026#34;) # [(\u0026#39;500\u0026#39;, \u0026#39;g\u0026#39;), (\u0026#39;2\u0026#39;, \u0026#39;kg\u0026#39;), (\u0026#39;1\u0026#39;, \u0026#39;L\u0026#39;), (\u0026#39;5\u0026#39;, \u0026#39;km\u0026#39;)] # Return as list of (number, unit) tuples Q9 \u0026ndash; Multiline You have a multiline string of student results. Extract each student\u0026rsquo;s name and their total marks. Use re.MULTILINE.\nresults = \u0026#34;\u0026#34;\u0026#34; RESULT CARD Name: Aarav Sharma Total: 445/500 Name: Priya Patel Total: 472/500 Name: Rohan Verma Total: 398/500 \u0026#34;\u0026#34;\u0026#34; Expected output:\n[(\u0026#39;Aarav Sharma\u0026#39;, \u0026#39;445\u0026#39;), (\u0026#39;Priya Patel\u0026#39;, \u0026#39;472\u0026#39;), (\u0026#39;Rohan Verma\u0026#39;, \u0026#39;398\u0026#39;)] Q10 \u0026ndash; Password validator Write a function check_password(password) that returns a list of requirements the password fails. Use regex for each check.\nRequirements:\nAt least 8 characters At least one uppercase letter At least one lowercase letter At least one digit At least one special character from !@#$%^\u0026amp;*() Hard Q11 \u0026ndash; Log parser Write a complete log parser that reads a multi-line log string and returns a list of parsed records. Each record should have: timestamp, level, module, message. Count records by level.\nQ12 \u0026ndash; HTML tag stripper Write a function strip_html(html) that removes all HTML tags but preserves the text content. Handle nested tags, self-closing tags, and tags with attributes.\nhtml = \u0026#39;\u0026lt;p class=\u0026#34;intro\u0026#34;\u0026gt;Hello \u0026lt;b\u0026gt;World\u0026lt;/b\u0026gt;! Visit \u0026lt;a href=\u0026#34;url\u0026#34;\u0026gt;here\u0026lt;/a\u0026gt;.\u0026lt;/p\u0026gt;\u0026#39; strip_html(html) # \u0026#34;Hello World! Visit here.\u0026#34; Q13 \u0026ndash; Data normalisation pipeline Given a DataFrame column of messy address strings, write a function that uses regex to extract: flat number, building/area, city, and PIN code.\naddresses = [ \u0026#34;Flat 4B, Prestige Towers, Koramangala, Bangalore - 560034\u0026#34;, \u0026#34;42/A Indiranagar, Bangalore 560038\u0026#34;, \u0026#34;House No. 15, MG Road, Delhi - 110001\u0026#34;, ] Q14 \u0026ndash; Template engine Write a simple template substitution function render(template, context) that replaces {{variable_name}} placeholders with values from a dictionary. Handle missing variables gracefully.\ntemplate = \u0026#34;Dear {{name}}, your order {{order_id}} of Rs.{{amount}} is {{status}}.\u0026#34; context = {\u0026#34;name\u0026#34;: \u0026#34;Aarav\u0026#34;, \u0026#34;order_id\u0026#34;: \u0026#34;ORD001\u0026#34;, \u0026#34;amount\u0026#34;: \u0026#34;5000\u0026#34;} render(template, context) # \u0026#34;Dear Aarav, your order ORD001 of Rs.5000 is [MISSING:status].\u0026#34; Q15 \u0026ndash; Tokeniser Write a simple tokeniser that splits code-like text into tokens: integers, floats, strings (in quotes), operators (+,-,*,/,=,==,!=), identifiers (variable names), and keywords (if, else, for, while, return).\ncode = \u0026#39;if score \u0026gt;= 88: grade = \u0026#34;A\u0026#34;\u0026#39; tokens = tokenise(code) # [(\u0026#39;KEYWORD\u0026#39;, \u0026#39;if\u0026#39;), (\u0026#39;IDENTIFIER\u0026#39;, \u0026#39;score\u0026#39;), (\u0026#39;OPERATOR\u0026#39;, \u0026#39;\u0026gt;=\u0026#39;), # (\u0026#39;INTEGER\u0026#39;, \u0026#39;88\u0026#39;), (\u0026#39;OPERATOR\u0026#39;, \u0026#39;:\u0026#39;), (\u0026#39;IDENTIFIER\u0026#39;, \u0026#39;grade\u0026#39;), # (\u0026#39;OPERATOR\u0026#39;, \u0026#39;=\u0026#39;), (\u0026#39;STRING\u0026#39;, \u0026#39;\u0026#34;A\u0026#34;\u0026#39;)] 14. Solutions Q1 import re def has_number(text: str) -\u0026gt; bool: return bool(re.search(r\u0026#34;\\d+\u0026#34;, text)) Q2 def extract_emails(text: str) -\u0026gt; list[str]: return re.findall(r\u0026#34;\\b[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\\.[A-Za-z]{2,}\\b\u0026#34;, text) Q3 def is_valid_pin(pin: str) -\u0026gt; bool: return bool(re.fullmatch(r\u0026#34;[1-9]\\d{5}\u0026#34;, pin)) Q4 def mask_card(text: str) -\u0026gt; str: return re.sub(r\u0026#34;\\b\\d{16}\\b\u0026#34;, \u0026#34;XXXX-XXXX-XXXX-XXXX\u0026#34;, text) Q5 def split_csv_line(line: str) -\u0026gt; list[str]: return [part for part in re.split(r\u0026#34;[,;|]+\u0026#34;, line) if part] Q6 def parse_date(text: str) -\u0026gt; list[dict]: pattern = r\u0026#34;(\\d{2})[/-](\\d{2})[/-](\\d{4})\u0026#34; return [ {\u0026#34;day\u0026#34;: m.group(1), \u0026#34;month\u0026#34;: m.group(2), \u0026#34;year\u0026#34;: m.group(3)} for m in re.finditer(pattern, text) ] Q7 def parse_transaction(line: str) -\u0026gt; dict | None: pattern = re.compile( r\u0026#34;(?P\u0026lt;txn_id\u0026gt;\\S+)\\s*\\|\\s*\u0026#34; r\u0026#34;(?P\u0026lt;type\u0026gt;\\w+)\\s*\\|\\s*\u0026#34; r\u0026#34;Rs\\.(?P\u0026lt;amount\u0026gt;[\\d.]+)\\s*\\|\\s*\u0026#34; r\u0026#34;(?P\u0026lt;name\u0026gt;[^|]+?)\\s*\\|\\s*\u0026#34; r\u0026#34;(?P\u0026lt;method\u0026gt;\\w+)\u0026#34; ) match = pattern.search(line) if not match: return None data = match.groupdict() data[\u0026#34;amount\u0026#34;] = float(data[\u0026#34;amount\u0026#34;]) return data line = \u0026#34;TXN20240315143022-001 | CREDIT | Rs.5000.00 | Aarav Sharma | UPI\u0026#34; print(parse_transaction(line)) Q8 def extract_quantities(text: str) -\u0026gt; list[tuple[str, str]]: pattern = r\u0026#34;(\\d+(?:\\.\\d+)?)(kg|g|L|ml|km)\\b\u0026#34; return re.findall(pattern, text) Q9 def extract_results(results: str) -\u0026gt; list[tuple[str, str]]: pattern = r\u0026#34;Name:\\s+(.+?)\\s+Total:\\s+(\\d+)/500\u0026#34; return re.findall(pattern, results, re.MULTILINE) results = \u0026#34;\u0026#34;\u0026#34; RESULT CARD Name: Aarav Sharma Total: 445/500 Name: Priya Patel Total: 472/500 Name: Rohan Verma Total: 398/500 \u0026#34;\u0026#34;\u0026#34; print(extract_results(results)) Q10 def check_password(password: str) -\u0026gt; list[str]: errors = [] if len(password) \u0026lt; 8: errors.append(\u0026#34;At least 8 characters required\u0026#34;) if not re.search(r\u0026#34;[A-Z]\u0026#34;, password): errors.append(\u0026#34;At least one uppercase letter required\u0026#34;) if not re.search(r\u0026#34;[a-z]\u0026#34;, password): errors.append(\u0026#34;At least one lowercase letter required\u0026#34;) if not re.search(r\u0026#34;\\d\u0026#34;, password): errors.append(\u0026#34;At least one digit required\u0026#34;) if not re.search(r\u0026#34;[!@#$%^\u0026amp;*()]\u0026#34;, password): errors.append(\u0026#34;At least one special character required\u0026#34;) return errors Q11 def parse_logs(log_text: str) -\u0026gt; list[dict]: pattern = re.compile( r\u0026#34;(?P\u0026lt;timestamp\u0026gt;\\d{4}-\\d{2}-\\d{2}\\s\\d{2}:\\d{2}:\\d{2})\\s+\u0026#34; r\u0026#34;(?P\u0026lt;level\u0026gt;DEBUG|INFO|WARNING|ERROR|CRITICAL)\\s+\u0026#34; r\u0026#34;(?P\u0026lt;module\u0026gt;\\S+)\\s+\u0026#34; r\u0026#34;(?P\u0026lt;message\u0026gt;.+)\u0026#34; ) records = [] for line in log_text.strip().split(\u0026#34;\\n\u0026#34;): match = pattern.search(line.strip()) if match: records.append(match.groupdict()) from collections import Counter levels = Counter(r[\u0026#34;level\u0026#34;] for r in records) print(\u0026#34;Level counts:\u0026#34;, dict(levels)) return records Q12 def strip_html(html: str) -\u0026gt; str: # Remove all HTML tags text = re.sub(r\u0026#34;\u0026lt;[^\u0026gt;]+\u0026gt;\u0026#34;, \u0026#34;\u0026#34;, html) # Collapse multiple spaces text = re.sub(r\u0026#34;\\s+\u0026#34;, \u0026#34; \u0026#34;, text) return text.strip() Q13 def parse_address(address: str) -\u0026gt; dict: result = {} flat_match = re.search( r\u0026#34;(?:Flat|House\\s+No\\.?)\\s*([A-Za-z0-9/]+)\u0026#34;, address, re.IGNORECASE ) pin_match = re.search(r\u0026#34;\\b\\d{6}\\b\u0026#34;, address) city_match = re.search( r\u0026#34;(?:Bangalore|Delhi|Mumbai|Hyderabad|Chennai|Pune)\u0026#34;, address, re.IGNORECASE ) if flat_match: result[\u0026#34;flat\u0026#34;] = flat_match.group(1) if pin_match: result[\u0026#34;pin\u0026#34;] = pin_match.group() if city_match: result[\u0026#34;city\u0026#34;] = city_match.group() return result addresses = [ \u0026#34;Flat 4B, Prestige Towers, Koramangala, Bangalore - 560034\u0026#34;, \u0026#34;42/A Indiranagar, Bangalore 560038\u0026#34;, \u0026#34;House No. 15, MG Road, Delhi - 110001\u0026#34;, ] for addr in addresses: print(parse_address(addr)) Q14 def render(template: str, context: dict) -\u0026gt; str: def replace(match): key = match.group(1).strip() return str(context.get(key, f\u0026#34;[MISSING:{key}]\u0026#34;)) return re.sub(r\u0026#34;\\{\\{(.+?)\\}\\}\u0026#34;, replace, template) Q15 def tokenise(code: str) -\u0026gt; list[tuple[str, str]]: KEYWORDS = {\u0026#34;if\u0026#34;, \u0026#34;else\u0026#34;, \u0026#34;elif\u0026#34;, \u0026#34;for\u0026#34;, \u0026#34;while\u0026#34;, \u0026#34;return\u0026#34;, \u0026#34;def\u0026#34;, \u0026#34;class\u0026#34;, \u0026#34;import\u0026#34;, \u0026#34;from\u0026#34;, \u0026#34;in\u0026#34;, \u0026#34;not\u0026#34;, \u0026#34;and\u0026#34;, \u0026#34;or\u0026#34;} token_spec = [ (\u0026#34;FLOAT\u0026#34;, r\u0026#34;\\d+\\.\\d+\u0026#34;), (\u0026#34;INTEGER\u0026#34;, r\u0026#34;\\d+\u0026#34;), (\u0026#34;STRING\u0026#34;, r\u0026#39;\u0026#34;[^\u0026#34;]*\u0026#34;|\\\u0026#39;[^\\\u0026#39;]*\\\u0026#39;\u0026#39;), (\u0026#34;OPERATOR\u0026#34;, r\u0026#34;==|!=|\u0026gt;=|\u0026lt;=|[+\\-*/=\u0026lt;\u0026gt;:]\u0026#34;), (\u0026#34;IDENTIFIER\u0026#34;, r\u0026#34;[a-zA-Z_]\\w*\u0026#34;), (\u0026#34;SKIP\u0026#34;, r\u0026#34;[\\s,]+\u0026#34;), ] master_pattern = re.compile( \u0026#34;|\u0026#34;.join(f\u0026#34;(?P\u0026lt;{name}\u0026gt;{pattern})\u0026#34; for name, pattern in token_spec) ) tokens = [] for match in master_pattern.finditer(code): kind = match.lastgroup value = match.group() if kind == \u0026#34;SKIP\u0026#34;: continue if kind == \u0026#34;IDENTIFIER\u0026#34; and value in KEYWORDS: kind = \u0026#34;KEYWORD\u0026#34; tokens.append((kind, value)) return tokens code = \u0026#39;if score \u0026gt;= 88: grade = \u0026#34;A\u0026#34;\u0026#39; for token in tokenise(code): print(token) 15. Summary and Key Takeaways The core functions re.search(pattern, text) # find first match anywhere -- returns match or None re.findall(pattern, text) # return all matches as list of strings (or tuples) re.finditer(pattern, text) # return iterator of match objects re.match(pattern, text) # match at start only re.fullmatch(pattern, text) # match must cover entire string re.sub(pattern, repl, text) # find and replace re.split(pattern, text) # split on pattern re.compile(pattern, flags) # compile for repeated use The essential pattern vocabulary . any char except newline \\d digit ^ start of string \\D non-digit $ end of string \\w word char (letter/digit/_) \\b word boundary \\W non-word char ? zero or one \\s whitespace * zero or more \\S non-whitespace + one or more [abc] any of a, b, c {n} exactly n [^abc] not a, b, or c {n,m} n to m [a-z] any lowercase letter () capture group (?:) non-capturing group | alternation (?P\u0026lt;name\u0026gt;) named group The rules to remember Always use raw strings: r\u0026#34;\\d+\u0026#34; not \u0026#34;\\d+\u0026#34; Test your patterns at: regex101.com or pythex.org Use re.compile() when you use the same pattern more than twice Use non-capturing (?:) when you need grouping but not capturing Add ? after quantifiers to make them lazy: *? +? {n,m}? Use \\b for word boundaries to avoid matching inside words Use re.VERBOSE for complex patterns with comments Never use re.match() thinking it searches -- it only matches at the start Made with care for Codeverra learners | codeverra.com\n","permalink":"https://blogs.codeverra.com/blog/python-17-regex-guide/","summary":"\u003ch1 id=\"regular-expressions-in-python\"\u003eRegular Expressions in Python\u003c/h1\u003e\n\u003ch3 id=\"pattern-matching-text-processing-and-the-re-module\"\u003ePattern Matching, Text Processing, and the re Module\u003c/h3\u003e\n\u003chr\u003e\n\u003ch2 id=\"before-we-begin--why-regular-expressions-matter\"\u003eBefore We Begin \u0026ndash; Why Regular Expressions Matter\u003c/h2\u003e\n\u003cp\u003eHere is a real problem. You have a CSV file with 50,000 rows of customer\ndata. The phone number column is a mess:\u003c/p\u003e\n\u003cpre tabindex=\"0\"\u003e\u003ccode\u003e+91-9876543210\n9876 543 210\n(091) 9876-543210\n91-98765-43210\n9876543210\n+919876543210\n\u003c/code\u003e\u003c/pre\u003e\u003cp\u003eAll of these are the same phone number in different formats. You need to\nextract just the 10-digit number from all of them.\u003c/p\u003e","tags":["python"],"title":"Regular Expressions in Python"},{"content":"🔰 Pattern 2: Sliding Window Table of Contents What is the Sliding Window Technique? When to Use Sliding Window? Types of Sliding Window Template Code Problem Set Problem 1: Maximum Sum Subarray of Size K Problem 2: Longest Substring Without Repeating Characters Problem 3: Maximum Average Subarray I Problem 4: Minimum Size Subarray Sum Problem 5: Permutation in String Problem 6: Longest Repeating Character Replacement Problem 7: Fruit Into Baskets Problem 8: Minimum Window Substring Problem 9: Sliding Window Maximum Problem 10: Substring with Concatenation of All Words Key Takeaways \u0026amp; Summary What is the Sliding Window Technique? In the Two Pointers pattern, we learned how two indices can traverse an array efficiently. Sliding Window builds on that idea - it maintains a range (window) between two pointers and cares about the elements inside that range.\nLet\u0026rsquo;s see why this matters with a concrete problem.\nProblem: Given an array [2, 1, 5, 1, 3, 2], find the maximum sum of any 3 consecutive elements.\nBrute force: For every starting index, sum up 3 elements.\n[2, 1, 5, 1, 3, 2] ------ sum = 2+1+5 = 8 ------ sum = 1+5+1 = 7 ------ sum = 5+1+3 = 9 ------ sum = 1+3+2 = 6 Max = 9 Each sum takes O(k) work, and there are O(n) positions → O(n × k) total.\nSliding window insight: When the window slides one position to the right, most of the elements stay the same. We only lose the leftmost element and gain one new element on the right.\nWindow [2, 1, 5] → sum = 8 Slide: drop 2, add 1 Window [1, 5, 1] → sum = 8 - 2 + 1 = 7 (reused the previous sum!) Slide: drop 1, add 3 Window [5, 1, 3] → sum = 7 - 1 + 3 = 9 Slide: drop 5, add 2 Window [1, 3, 2] → sum = 9 - 5 + 2 = 6 Max = 9 Instead of recalculating the entire sum each time, we update it by subtracting the element that left the window and adding the element that entered. This turns O(n × k) into O(n).\nSliding Window is a technique where you maintain a window (a contiguous subarray or substring) defined by two pointers, and slide it across the data - expanding or shrinking the window as needed - while efficiently tracking some property of the elements inside (sum, count, frequency, etc.).\nWhy is it useful? Avoids redundant work: Instead of recomputing from scratch, you update incrementally. Turns O(n × k) or O(n²) into O(n): One pass through the data. Naturally handles \u0026ldquo;contiguous subarray/substring\u0026rdquo; problems: If the problem asks about consecutive elements, think sliding window. When to Use Sliding Window? Look for these signals in a problem:\nSignal Example \u0026ldquo;Contiguous subarray\u0026rdquo; or \u0026ldquo;substring\u0026rdquo; \u0026ldquo;Find the longest substring that\u0026hellip;\u0026rdquo; \u0026ldquo;Of size k\u0026rdquo; or \u0026ldquo;of length k\u0026rdquo; \u0026ldquo;Maximum sum of subarray of size k\u0026rdquo; \u0026ldquo;Minimum/maximum length subarray with some condition\u0026rdquo; \u0026ldquo;Shortest subarray with sum ≥ target\u0026rdquo; Involves frequency counting in a range \u0026ldquo;Check if string contains a permutation of another\u0026rdquo; \u0026ldquo;At most k distinct\u0026rdquo; or \u0026ldquo;at most k changes\u0026rdquo; \u0026ldquo;Longest substring with at most 2 distinct characters\u0026rdquo; Sliding Window vs Two Pointers - What\u0026rsquo;s the Difference? Both use two pointers, so the line can feel blurry. The key distinction:\nTwo Pointers: You care about the elements at the two pointer positions (e.g., nums[left] + nums[right]). Sliding Window: You care about all elements between the two pointers - the window as a whole (e.g., sum of the window, frequency map of the window). In practice, if you\u0026rsquo;re maintaining some running state (a sum, a hashmap, a count) that represents the content of a range, you\u0026rsquo;re using a sliding window.\nTypes of Sliding Window 1. Fixed-Size Window The window size k is given. You slide a window of exactly k elements across the array.\nArray: [a, b, c, d, e, f, g] k = 3 [-----] window 1 [-----] window 2 [-----] window 3 [-----] window 4 [-----] window 5 How it works:\nBuild the first window (indices 0 to k-1). Slide: remove the leftmost element, add the next right element. Track the best result at each position. Use when: The problem specifies a fixed window size (\u0026ldquo;subarray of size k\u0026rdquo;, \u0026ldquo;every k consecutive elements\u0026rdquo;).\n2. Variable-Size Window (Expanding/Shrinking) The window size changes dynamically. You expand the right end to include more elements, and shrink from the left when some condition is violated.\nArray: [a, b, c, d, e, f, g] [--] expand right [-----] expand right [--------] condition violated! [-----] shrink left [--------] expand right [-----] shrink left How it works:\nExpand the window by moving right forward. When the window violates the constraint, shrink by moving left forward. At each step, check if the current window is the best answer so far. Use when: You\u0026rsquo;re looking for the longest/shortest subarray or substring that satisfies some condition.\nThe Shrinking Decision The hardest part of variable-size windows is knowing when to shrink. Here\u0026rsquo;s the general rule:\nFinding the longest valid window → Shrink when the window becomes invalid. Keep it as large as possible. Finding the shortest valid window → Shrink when the window becomes valid. Try to make it smaller. Template Code Template 1: Fixed-Size Window def fixed_window(arr, k): \u0026#34;\u0026#34;\u0026#34; Template for fixed-size sliding window. Time Complexity: O(n) - one pass through the array Space Complexity: O(1) - only tracking the window state \u0026#34;\u0026#34;\u0026#34; n = len(arr) # Step 1: Build the first window (indices 0 to k-1) window_state = 0 # could be sum, product, count, etc. for i in range(k): window_state += arr[i] # or whatever operation best = window_state # Step 2: Slide the window from position k to n-1 for right in range(k, n): # Add the new element entering the window window_state += arr[right] # Remove the element leaving the window left_leaving = right - k window_state -= arr[left_leaving] # Update the result best = max(best, window_state) # or min, or whatever return best Template 2: Variable-Size Window (Longest) def variable_window_longest(arr): \u0026#34;\u0026#34;\u0026#34; Template for variable-size window - finding the LONGEST valid window. Expand right always, shrink left only when window becomes invalid. Time Complexity: O(n) - each element is added and removed at most once Space Complexity: depends on what state you track (often O(k) for a hashmap) \u0026#34;\u0026#34;\u0026#34; left = 0 window_state = {} # or sum, count, etc. best = 0 for right in range(len(arr)): # EXPAND: Add arr[right] to window state # (update your hashmap, sum, count, etc.) # SHRINK: While the window is INVALID, remove from left while window_is_invalid(): # Remove arr[left] from window state left += 1 # UPDATE: Current window [left..right] is valid best = max(best, right - left + 1) return best Template 3: Variable-Size Window (Shortest) def variable_window_shortest(arr, target): \u0026#34;\u0026#34;\u0026#34; Template for variable-size window - finding the SHORTEST valid window. Expand right always, shrink left while window REMAINS valid. Time Complexity: O(n) Space Complexity: depends on state tracking \u0026#34;\u0026#34;\u0026#34; left = 0 window_state = 0 best = float(\u0026#39;inf\u0026#39;) for right in range(len(arr)): # EXPAND: Add arr[right] to window state window_state += arr[right] # SHRINK: While the window is VALID, try to make it smaller while window_is_valid(): best = min(best, right - left + 1) # Remove arr[left] from window state window_state -= arr[left] left += 1 return best if best != float(\u0026#39;inf\u0026#39;) else 0 Problem Set Difficulty Progression # Problem Difficulty Key Concept 1 Max Sum Subarray of Size K Easy Fixed window basics 2 Longest Substring Without Repeating Characters Medium Variable window + hashmap 3 Maximum Average Subarray I Easy Fixed window with division 4 Minimum Size Subarray Sum Medium Variable window - shortest 5 Permutation in String Medium Fixed window + frequency matching 6 Longest Repeating Character Replacement Medium Variable window + character count 7 Fruit Into Baskets Medium Variable window - at most k distinct 8 Minimum Window Substring Hard Variable window - shortest with frequency 9 Sliding Window Maximum Hard Window + monotonic deque 10 Substring with Concatenation of All Words Hard Fixed window + word-level matching Problem 1: Maximum Sum Subarray of Size K LeetCode Link: This is a classic problem available on many platforms. On LeetCode, it maps to https://leetcode.com/problems/maximum-average-subarray-i/ (same logic, just divide by k at the end).\nProblem Statement Given an array of integers nums and an integer k, find the maximum sum of any contiguous subarray of size k.\nExample:\nInput: nums = [2, 1, 5, 1, 3, 2], k = 3 Output: 9 (subarray [5, 1, 3]) Clarifying Questions \u0026amp; Constraints 1 \u0026lt;= k \u0026lt;= len(nums) - there\u0026rsquo;s always at least one window. Elements can be negative. Approach Discussion Approach 1: Brute Force\nFor each starting index i, sum up k elements: sum(nums[i:i+k]). Time: O(n × k) - summing k elements for each of n positions. Space: O(1) ❌ Redundant: most of the sum stays the same between adjacent windows. Approach 2: Fixed-Size Sliding Window (Optimal) ✅\nCompute the sum of the first k elements. Slide: subtract the element going out, add the element coming in. Time: O(n) - single pass. Space: O(1) Code (Both Solutions) # ============================================================ # APPROACH 1: Brute Force - O(n × k) # ============================================================ def max_sum_brute(nums: list[int], k: int) -\u0026gt; int: \u0026#34;\u0026#34;\u0026#34; Check every subarray of size k and return the maximum sum. Time Complexity: O(n × k) - for each position, sum k elements Space Complexity: O(1) \u0026#34;\u0026#34;\u0026#34; max_sum = float(\u0026#39;-inf\u0026#39;) for i in range(len(nums) - k + 1): # Sum the subarray from index i to i+k-1 current_sum = 0 for j in range(i, i + k): current_sum += nums[j] max_sum = max(max_sum, current_sum) return max_sum # ============================================================ # APPROACH 2: Fixed-Size Sliding Window - O(n) ✅ # ============================================================ def max_sum_sliding_window(nums: list[int], k: int) -\u0026gt; int: \u0026#34;\u0026#34;\u0026#34; Find the maximum sum of any contiguous subarray of size k. Strategy: 1. Compute the sum of the first window (indices 0 to k-1). 2. Slide the window one step at a time: - Subtract the element that leaves (left side) - Add the element that enters (right side) 3. Track the maximum sum seen. Time Complexity: O(n) - single pass through the array Space Complexity: O(1) - only a few variables \u0026#34;\u0026#34;\u0026#34; # Step 1: Compute the sum of the first window window_sum = sum(nums[:k]) max_sum = window_sum # Step 2: Slide the window from index k to the end for right in range(k, len(nums)): # The element leaving the window is at index (right - k) # The element entering the window is at index (right) window_sum += nums[right] - nums[right - k] max_sum = max(max_sum, window_sum) return max_sum Edge Cases k equals array length: Only one window - return the total sum. All negative: [-3, -2, -5], k=2 → return -5 (least negative). Single element windows: k=1 → return the maximum element. Dry Run Input: nums = [2, 1, 5, 1, 3, 2], k = 3 Step 1: First window = nums[0:3] = [2, 1, 5] window_sum = 8, max_sum = 8 Step 2: right = 3 window_sum = 8 + nums[3] - nums[0] = 8 + 1 - 2 = 7 max_sum = max(8, 7) = 8 Step 3: right = 4 window_sum = 7 + nums[4] - nums[1] = 7 + 3 - 1 = 9 max_sum = max(8, 9) = 9 Step 4: right = 5 window_sum = 9 + nums[5] - nums[2] = 9 + 2 - 5 = 6 max_sum = max(9, 6) = 9 Output: 9 ✅ (subarray [5, 1, 3]) Problem 2: Longest Substring Without Repeating Characters LeetCode Link: https://leetcode.com/problems/longest-substring-without-repeating-characters/\nProblem Statement Given a string s, find the length of the longest substring without repeating characters.\nExample:\nInput: s = \u0026#34;abcabcbb\u0026#34; Output: 3 (substring \u0026#34;abc\u0026#34;) Clarifying Questions \u0026amp; Constraints The string can contain letters, digits, symbols, and spaces. An empty string has length 0. We need a substring (contiguous), not a subsequence. Approach Discussion Approach 1: Brute Force\nCheck every substring, verify it has no duplicates. Time: O(n³) - O(n²) substrings × O(n) to check each. Space: O(min(n, 26)) for the character set. ❌ Far too slow. Approach 2: Sliding Window + HashSet (Good)\nExpand right to include characters. If a duplicate is found, shrink from left one step at a time until the duplicate is removed. Time: O(2n) = O(n) - in the worst case, each character is added and removed once. Space: O(min(n, charset_size)) Approach 3: Sliding Window + HashMap (Optimal) ✅\nInstead of shrinking one step at a time, store the last seen index of each character. When we find a duplicate, jump left directly to last_seen[char] + 1. Time: O(n) - single pass, no inner shrinking loop. Space: O(min(n, charset_size)) Code (Approaches 2 and 3) # ============================================================ # APPROACH 2: Sliding Window + HashSet - O(n) # ============================================================ def lengthOfLongestSubstring_set(s: str) -\u0026gt; int: \u0026#34;\u0026#34;\u0026#34; Use a set to track characters in the current window. When a duplicate is found, shrink from the left until it\u0026#39;s removed. Time Complexity: O(n) - each character is added and removed from the set at most once (left and right each traverse the string once → 2n operations max) Space Complexity: O(min(n, m)) where m is the size of the character set \u0026#34;\u0026#34;\u0026#34; char_set = set() left = 0 max_length = 0 for right in range(len(s)): # If s[right] is already in the window, shrink from the left while s[right] in char_set: char_set.remove(s[left]) left += 1 # Add the new character to the window char_set.add(s[right]) # Update the maximum length max_length = max(max_length, right - left + 1) return max_length # ============================================================ # APPROACH 3: Sliding Window + HashMap - O(n) ✅ (Optimized) # ============================================================ def lengthOfLongestSubstring(s: str) -\u0026gt; int: \u0026#34;\u0026#34;\u0026#34; Use a hashmap to store the last seen index of each character. When a duplicate is found, jump \u0026#39;left\u0026#39; directly past the previous occurrence. Why is this better than the set approach? With a set, when we find a duplicate, we shrink left one step at a time. With a hashmap, we can jump left directly to the right position. Example: s = \u0026#34;abcba\u0026#34; - Set approach: when we hit the second \u0026#39;b\u0026#39; at index 3, we remove \u0026#39;a\u0026#39;, then \u0026#39;b\u0026#39; (2 steps) - HashMap approach: we see \u0026#39;b\u0026#39; was last at index 1, so jump left to index 2 (1 step) Time Complexity: O(n) - single pass, right pointer visits each character exactly once Space Complexity: O(min(n, m)) where m is the character set size \u0026#34;\u0026#34;\u0026#34; last_seen = {} # char → most recent index where it appeared left = 0 max_length = 0 for right in range(len(s)): char = s[right] # If we\u0026#39;ve seen this character before AND it\u0026#39;s inside our current window if char in last_seen and last_seen[char] \u0026gt;= left: # Jump left to one position past the previous occurrence left = last_seen[char] + 1 # Update the last seen position of this character last_seen[char] = right # Update the maximum length max_length = max(max_length, right - left + 1) return max_length Edge Cases Empty string: \u0026quot;\u0026quot; → 0. All same characters: \u0026quot;aaaa\u0026quot; → 1. All unique: \u0026quot;abcdef\u0026quot; → 6 (the whole string). Single character: \u0026quot;a\u0026quot; → 1. Spaces and symbols: \u0026quot;a b c\u0026quot; → 3 (\u0026quot;a b\u0026quot; or \u0026quot; b \u0026quot; etc., space counts as a character). Dry Run Input: s = \u0026#34;abcabcbb\u0026#34; Using Approach 3 (HashMap): last_seen = {}, left = 0, max_length = 0 right=0: char=\u0026#39;a\u0026#39;, not seen → last_seen={\u0026#39;a\u0026#39;:0}, max_length=max(0, 0-0+1)=1 right=1: char=\u0026#39;b\u0026#39;, not seen → last_seen={\u0026#39;a\u0026#39;:0,\u0026#39;b\u0026#39;:1}, max_length=max(1, 1-0+1)=2 right=2: char=\u0026#39;c\u0026#39;, not seen → last_seen={\u0026#39;a\u0026#39;:0,\u0026#39;b\u0026#39;:1,\u0026#39;c\u0026#39;:2}, max_length=max(2, 2-0+1)=3 right=3: char=\u0026#39;a\u0026#39;, seen at 0, 0 \u0026gt;= left(0) → left=1 last_seen={\u0026#39;a\u0026#39;:3,\u0026#39;b\u0026#39;:1,\u0026#39;c\u0026#39;:2}, max_length=max(3, 3-1+1)=3 right=4: char=\u0026#39;b\u0026#39;, seen at 1, 1 \u0026gt;= left(1) → left=2 last_seen={\u0026#39;a\u0026#39;:3,\u0026#39;b\u0026#39;:4,\u0026#39;c\u0026#39;:2}, max_length=max(3, 4-2+1)=3 right=5: char=\u0026#39;c\u0026#39;, seen at 2, 2 \u0026gt;= left(2) → left=3 last_seen={\u0026#39;a\u0026#39;:3,\u0026#39;b\u0026#39;:4,\u0026#39;c\u0026#39;:5}, max_length=max(3, 5-3+1)=3 right=6: char=\u0026#39;b\u0026#39;, seen at 4, 4 \u0026gt;= left(3) → left=5 last_seen={\u0026#39;a\u0026#39;:3,\u0026#39;b\u0026#39;:6,\u0026#39;c\u0026#39;:5}, max_length=max(3, 6-5+1)=2 right=7: char=\u0026#39;b\u0026#39;, seen at 6, 6 \u0026gt;= left(5) → left=7 last_seen={\u0026#39;a\u0026#39;:3,\u0026#39;b\u0026#39;:7,\u0026#39;c\u0026#39;:5}, max_length=max(3, 7-7+1)=3 → stays 3 Output: 3 ✅ Problem 3: Maximum Average Subarray I LeetCode Link: https://leetcode.com/problems/maximum-average-subarray-i/\nProblem Statement Given an integer array nums and an integer k, find a contiguous subarray of length k that has the maximum average value and return this value.\nExample:\nInput: nums = [1, 12, -5, -6, 50, 3], k = 4 Output: 12.75 (subarray [12, -5, -6, 50] → sum=51, avg=51/4=12.75) Clarifying Questions \u0026amp; Constraints 1 \u0026lt;= k \u0026lt;= nums.length Elements can be negative. Result should be a floating point number. Approach Discussion Approach 1: Brute Force\nFor each starting index, sum up k elements and compute average. Time: O(n × k), Space: O(1) Approach 2: Fixed-Size Sliding Window (Optimal) ✅\nIdentical to \u0026ldquo;Max Sum Subarray of Size K\u0026rdquo; - just divide the max sum by k at the end. No need to divide at every step since max(sum/k) = max(sum) / k when k is constant. Time: O(n), Space: O(1) Code (Optimal Solution) def findMaxAverage(nums: list[int], k: int) -\u0026gt; float: \u0026#34;\u0026#34;\u0026#34; Find the maximum average of any contiguous subarray of size k. Since k is constant, maximizing the average is the same as maximizing the sum. We find the max sum and divide by k at the end. Time Complexity: O(n) - single pass after building the first window Space Complexity: O(1) - only tracking window sum and max sum \u0026#34;\u0026#34;\u0026#34; # Build the first window window_sum = sum(nums[:k]) max_sum = window_sum # Slide the window for right in range(k, len(nums)): window_sum += nums[right] - nums[right - k] max_sum = max(max_sum, window_sum) # Divide at the end (not at each step - avoids floating point operations in the loop) return max_sum / k Edge Cases k equals array length: Average of the entire array. All negative: [-1, -2, -3], k=2 → -1.5. k = 1: Maximum element in the array. Dry Run Input: nums = [1, 12, -5, -6, 50, 3], k = 4 Step 1: First window = [1, 12, -5, -6] → window_sum = 2, max_sum = 2 Step 2: right=4 → window_sum = 2 + 50 - 1 = 51, max_sum = 51 Window: [12, -5, -6, 50] Step 3: right=5 → window_sum = 51 + 3 - 12 = 42, max_sum = 51 Window: [-5, -6, 50, 3] Result: 51 / 4 = 12.75 ✅ Problem 4: Minimum Size Subarray Sum LeetCode Link: https://leetcode.com/problems/minimum-size-subarray-sum/\nProblem Statement Given an array of positive integers nums and a positive integer target, return the minimum length of a subarray whose sum is greater than or equal to target. If no such subarray exists, return 0.\nExample:\nInput: target = 7, nums = [2, 3, 1, 2, 4, 3] Output: 2 (subarray [4, 3] has sum 7 ≥ 7) Clarifying Questions \u0026amp; Constraints All numbers are positive (this is important - it guarantees that adding more elements always increases the sum). Return 0 if no valid subarray exists. We want the shortest valid subarray. Approach Discussion Approach 1: Brute Force\nTry every subarray, check if its sum ≥ target, track the shortest. Time: O(n²), Space: O(1) Approach 2: Variable-Size Sliding Window - Shortest (Optimal) ✅\nExpand right to grow the window sum. Once the sum ≥ target (window is valid), try to shrink from the left to find the minimum length. This works because all numbers are positive - shrinking the window always decreases the sum. Time: O(n), Space: O(1) Why does this only work with positive numbers? If there were negative numbers, shrinking from the left might actually increase the sum (removing a negative), which breaks the logic. For arrays with negatives, you\u0026rsquo;d need a different approach (e.g., prefix sum + binary search → O(n log n)).\nCode (Both Solutions) # ============================================================ # APPROACH 1: Brute Force - O(n²) # ============================================================ def minSubArrayLen_brute(target: int, nums: list[int]) -\u0026gt; int: \u0026#34;\u0026#34;\u0026#34; Try every subarray starting from each index. Time Complexity: O(n²) - two nested loops Space Complexity: O(1) \u0026#34;\u0026#34;\u0026#34; n = len(nums) min_length = float(\u0026#39;inf\u0026#39;) for i in range(n): current_sum = 0 for j in range(i, n): current_sum += nums[j] if current_sum \u0026gt;= target: min_length = min(min_length, j - i + 1) break # No point continuing - longer subarrays won\u0026#39;t be shorter return min_length if min_length != float(\u0026#39;inf\u0026#39;) else 0 # ============================================================ # APPROACH 2: Variable-Size Sliding Window - O(n) ✅ # ============================================================ def minSubArrayLen(target: int, nums: list[int]) -\u0026gt; int: \u0026#34;\u0026#34;\u0026#34; Find the shortest subarray with sum \u0026gt;= target using a variable-size window. Strategy (Shortest valid window): - Expand right to increase the window sum. - Once sum \u0026gt;= target (valid), record the length, then SHRINK from left to see if we can find an even shorter valid window. - Keep shrinking as long as the window remains valid. Time Complexity: O(n) - left and right each move at most n times Space Complexity: O(1) - only a few variables \u0026#34;\u0026#34;\u0026#34; left = 0 window_sum = 0 min_length = float(\u0026#39;inf\u0026#39;) for right in range(len(nums)): # EXPAND: Add the new element to the window window_sum += nums[right] # SHRINK: While the window is valid (sum \u0026gt;= target), try to make it smaller while window_sum \u0026gt;= target: # Current window [left..right] is valid - record its length min_length = min(min_length, right - left + 1) # Remove the left element and shrink window_sum -= nums[left] left += 1 return min_length if min_length != float(\u0026#39;inf\u0026#39;) else 0 Edge Cases No valid subarray: target=100, nums=[1, 2, 3] → return 0. Single element ≥ target: target=5, nums=[5, 1, 2] → return 1. Entire array needed: target=15, nums=[1, 2, 3, 4, 5] → return 5. Dry Run Input: target = 7, nums = [2, 3, 1, 2, 4, 3] left=0, window_sum=0, min_length=∞ right=0: window_sum = 2 (\u0026lt; 7, don\u0026#39;t shrink) right=1: window_sum = 5 (\u0026lt; 7) right=2: window_sum = 6 (\u0026lt; 7) right=3: window_sum = 8 (≥ 7!) → min_length = min(∞, 3-0+1) = 4, window: [2,3,1,2] → shrink: window_sum = 8-2=6, left=1 (\u0026lt; 7, stop shrinking) right=4: window_sum = 6+4 = 10 (≥ 7!) → min_length = min(4, 4-1+1) = 4, window: [3,1,2,4] → shrink: window_sum = 10-3=7, left=2 (≥ 7!) → min_length = min(4, 4-2+1) = 3, window: [1,2,4] → shrink: window_sum = 7-1=6, left=3 (\u0026lt; 7, stop) right=5: window_sum = 6+3 = 9 (≥ 7!) → min_length = min(3, 5-3+1) = 3, window: [2,4,3] → shrink: window_sum = 9-2=7, left=4 (≥ 7!) → min_length = min(3, 5-4+1) = 2, window: [4,3] → shrink: window_sum = 7-4=3, left=5 (\u0026lt; 7, stop) Output: 2 ✅ (subarray [4, 3]) Problem 5: Permutation in String LeetCode Link: https://leetcode.com/problems/permutation-in-string/\nProblem Statement Given two strings s1 and s2, return True if s2 contains a permutation of s1.\nIn other words, check if any substring of s2 is an anagram of s1.\nExample:\nInput: s1 = \u0026#34;ab\u0026#34;, s2 = \u0026#34;eidbaooo\u0026#34; Output: True (s2 contains \u0026#34;ba\u0026#34; which is a permutation of \u0026#34;ab\u0026#34;) Clarifying Questions \u0026amp; Constraints Both strings consist of lowercase English letters. A permutation of s1 means same characters, same frequency, any order. 1 \u0026lt;= s1.length, s2.length \u0026lt;= 10^4 Approach Discussion Approach 1: Generate All Permutations\nGenerate all permutations of s1, check if any exists in s2. Time: O(n! × m) - there are n! permutations, checking each in s2 takes O(m). ❌ Factorial time - completely impractical. Approach 2: Sort + Fixed Window (Decent)\nSort s1. For every window of size len(s1) in s2, sort the window and compare. Time: O(m × n log n) where m = len(s2), n = len(s1) ❌ Better but still not great. Approach 3: Fixed Window + Frequency Count (Optimal) ✅\nA permutation has the exact same character frequencies as the original. Maintain a frequency count for a window of size len(s1) sliding over s2. Compare the window\u0026rsquo;s frequency count with s1\u0026rsquo;s frequency count. Time: O(m) where m = len(s2) - O(26) comparison at each step is constant. Space: O(1) - frequency arrays of size 26. Code (Optimal Solution) from collections import Counter def checkInclusion(s1: str, s2: str) -\u0026gt; bool: \u0026#34;\u0026#34;\u0026#34; Check if any substring of s2 is a permutation (anagram) of s1. Strategy: - A permutation of s1 has the same character frequencies as s1. - Slide a window of size len(s1) over s2. - Maintain a frequency count for the window. - If the window\u0026#39;s frequency matches s1\u0026#39;s frequency → found a permutation. Optimization: Instead of comparing full frequency maps each time, we track how many characters have matching counts. When all 26 characters match, we\u0026#39;ve found a permutation. Time Complexity: O(n + m) where n = len(s1), m = len(s2) - Building s1\u0026#39;s frequency: O(n) - Sliding the window: O(m), with O(1) work per step Space Complexity: O(1) - frequency arrays of size 26 (constant) \u0026#34;\u0026#34;\u0026#34; n, m = len(s1), len(s2) # If s1 is longer than s2, no permutation can exist if n \u0026gt; m: return False # Frequency counts for s1 and the current window in s2 s1_count = Counter(s1) window_count = Counter() # Track how many distinct characters have matching frequencies # When matches == number of distinct characters in s1_count, we found it matches = 0 # Total distinct characters we need to match required = len(s1_count) for right in range(m): # ADD the new character to the window char = s2[right] window_count[char] += 1 # Check if this character\u0026#39;s count now matches s1 if char in s1_count: if window_count[char] == s1_count[char]: matches += 1 # If we just went FROM matching TO not matching (one over) elif window_count[char] == s1_count[char] + 1: matches -= 1 # REMOVE the character that\u0026#39;s leaving the window (once window \u0026gt; n) if right \u0026gt;= n: left_char = s2[right - n] # Check BEFORE removing: was this character matching? if left_char in s1_count: if window_count[left_char] == s1_count[left_char]: matches -= 1 elif window_count[left_char] == s1_count[left_char] + 1: matches += 1 window_count[left_char] -= 1 # If all required characters match → permutation found if matches == required: return True return False A Simpler (But Slightly Less Efficient) Version The optimized version above can be tricky to get right. Here\u0026rsquo;s a cleaner version that\u0026rsquo;s still O(m):\nfrom collections import Counter def checkInclusion_simple(s1: str, s2: str) -\u0026gt; bool: \u0026#34;\u0026#34;\u0026#34; Simpler version: compare full frequency counters. Since we only have 26 lowercase letters, comparing two Counters is O(26) = O(1), so the overall complexity is still O(m). Time Complexity: O(m × 26) = O(m) - comparing counters is O(26) per step Space Complexity: O(1) - counters hold at most 26 entries \u0026#34;\u0026#34;\u0026#34; n, m = len(s1), len(s2) if n \u0026gt; m: return False s1_count = Counter(s1) window_count = Counter(s2[:n]) # First window if s1_count == window_count: return True for right in range(n, m): # Add new character window_count[s2[right]] += 1 # Remove old character left_char = s2[right - n] window_count[left_char] -= 1 if window_count[left_char] == 0: del window_count[left_char] # Clean up zero counts for comparison if s1_count == window_count: return True return False Edge Cases s1 longer than s2: s1=\u0026quot;abc\u0026quot;, s2=\u0026quot;ab\u0026quot; → False. Exact match: s1=\u0026quot;abc\u0026quot;, s2=\u0026quot;abc\u0026quot; → True. Single character: s1=\u0026quot;a\u0026quot;, s2=\u0026quot;a\u0026quot; → True. No match: s1=\u0026quot;ab\u0026quot;, s2=\u0026quot;cccc\u0026quot; → False. Dry Run Input: s1 = \u0026#34;ab\u0026#34;, s2 = \u0026#34;eidbaooo\u0026#34; Using the simpler version: s1_count = {\u0026#39;a\u0026#39;: 1, \u0026#39;b\u0026#39;: 1}, n = 2 First window \u0026#34;ei\u0026#34;: window_count = {\u0026#39;e\u0026#39;:1, \u0026#39;i\u0026#39;:1} → ≠ s1_count right=2: add \u0026#39;d\u0026#39;, remove \u0026#39;e\u0026#39; → window \u0026#34;id\u0026#34; → {\u0026#39;i\u0026#39;:1, \u0026#39;d\u0026#39;:1} → ≠ right=3: add \u0026#39;b\u0026#39;, remove \u0026#39;i\u0026#39; → window \u0026#34;db\u0026#34; → {\u0026#39;d\u0026#39;:1, \u0026#39;b\u0026#39;:1} → ≠ right=4: add \u0026#39;a\u0026#39;, remove \u0026#39;d\u0026#39; → window \u0026#34;ba\u0026#34; → {\u0026#39;b\u0026#39;:1, \u0026#39;a\u0026#39;:1} → == s1_count ✅ Output: True ✅ Problem 6: Longest Repeating Character Replacement LeetCode Link: https://leetcode.com/problems/longest-repeating-character-replacement/\nProblem Statement Given a string s and an integer k, you can choose any character of the string and change it to any other uppercase character. You can perform this operation at most k times. Return the length of the longest substring containing the same letter after performing at most k replacements.\nExample:\nInput: s = \u0026#34;AABABBA\u0026#34;, k = 1 Output: 4 (\u0026#34;AABABBA\u0026#34; → change one B → \u0026#34;AAAAABA\u0026#34; or similar, longest run = 4) Clarifying Questions \u0026amp; Constraints String contains only uppercase English letters. 0 \u0026lt;= k \u0026lt;= len(s) We want the longest window where we can make all characters the same with at most k changes. Approach Discussion Key Insight: In any window, the number of characters we need to change = window_length - count_of_most_frequent_char. If this is ≤ k, the window is valid (we can make all characters the same with at most k changes).\nApproach 1: Brute Force\nFor each substring, check if it can be made uniform with ≤ k changes. Time: O(n² × 26), Space: O(1) Approach 2: Variable-Size Sliding Window (Optimal) ✅\nExpand right. Track frequency of each character in the window. The window is valid if: window_length - max_frequency \u0026lt;= k. If invalid, shrink from left. Time: O(n), Space: O(26) = O(1) A subtle point about max_frequency: When we shrink the window, the true max frequency might decrease, but we don\u0026rsquo;t bother updating it downward. Why? Because we only care about finding a longer valid window than our current best. The max_frequency only needs to increase (or stay the same) for that to happen. This is a common optimization in sliding window problems.\nCode (Optimal Solution) def characterReplacement(s: str, k: int) -\u0026gt; int: \u0026#34;\u0026#34;\u0026#34; Find the longest substring where all characters can be made the same with at most k replacements. Core Idea: For a window to be valid: (window_size - max_freq_in_window) \u0026lt;= k - max_freq_in_window = count of the most frequent character in the window - (window_size - max_freq_in_window) = characters that need to be changed Time Complexity: O(n) - each character is visited at most twice (right adds, left removes) Space Complexity: O(26) = O(1) - frequency array for uppercase letters \u0026#34;\u0026#34;\u0026#34; freq = {} # Character frequencies in the current window max_freq = 0 # Highest frequency of any single character in the window left = 0 max_length = 0 for right in range(len(s)): # EXPAND: Add s[right] to the window char = s[right] freq[char] = freq.get(char, 0) + 1 # Update max_freq if this character now has the highest count max_freq = max(max_freq, freq[char]) # Window size = right - left + 1 # Characters to change = window_size - max_freq window_size = right - left + 1 # SHRINK: If the window is invalid (too many characters need changing) if window_size - max_freq \u0026gt; k: # Remove the leftmost character freq[s[left]] -= 1 left += 1 # Note: we DON\u0026#39;T decrease max_freq here. # It might be stale, but that\u0026#39;s okay - we only need max_freq # to increase to find a LONGER valid window. # UPDATE: Record the current window length max_length = max(max_length, right - left + 1) return max_length Edge Cases k = 0: Longest run of identical characters (no changes allowed). k ≥ len(s): Return len(s) (change everything). All same characters: \u0026quot;AAAA\u0026quot;, k=2 → 4 (already uniform). Single character: \u0026quot;A\u0026quot;, k=0 → 1. Dry Run Input: s = \u0026#34;AABABBA\u0026#34;, k = 1 left=0, max_freq=0, freq={} right=0: char=\u0026#39;A\u0026#39;, freq={\u0026#39;A\u0026#39;:1}, max_freq=1 window=1, changes=1-1=0 ≤ 1 ✅, max_length=1 right=1: char=\u0026#39;A\u0026#39;, freq={\u0026#39;A\u0026#39;:2}, max_freq=2 window=2, changes=2-2=0 ≤ 1 ✅, max_length=2 right=2: char=\u0026#39;B\u0026#39;, freq={\u0026#39;A\u0026#39;:2,\u0026#39;B\u0026#39;:1}, max_freq=2 window=3, changes=3-2=1 ≤ 1 ✅, max_length=3 right=3: char=\u0026#39;A\u0026#39;, freq={\u0026#39;A\u0026#39;:3,\u0026#39;B\u0026#39;:1}, max_freq=3 window=4, changes=4-3=1 ≤ 1 ✅, max_length=4 right=4: char=\u0026#39;B\u0026#39;, freq={\u0026#39;A\u0026#39;:3,\u0026#39;B\u0026#39;:2}, max_freq=3 window=5, changes=5-3=2 \u0026gt; 1 ❌ → remove s[0]=\u0026#39;A\u0026#39;: freq={\u0026#39;A\u0026#39;:2,\u0026#39;B\u0026#39;:2}, left=1 max_length=4 right=5: char=\u0026#39;B\u0026#39;, freq={\u0026#39;A\u0026#39;:2,\u0026#39;B\u0026#39;:3}, max_freq=3 window=5, changes=5-3=2 \u0026gt; 1 ❌ → remove s[1]=\u0026#39;A\u0026#39;: freq={\u0026#39;A\u0026#39;:1,\u0026#39;B\u0026#39;:3}, left=2 max_length=4 right=6: char=\u0026#39;A\u0026#39;, freq={\u0026#39;A\u0026#39;:2,\u0026#39;B\u0026#39;:3}, max_freq=3 window=5, changes=5-3=2 \u0026gt; 1 ❌ → remove s[2]=\u0026#39;B\u0026#39;: freq={\u0026#39;A\u0026#39;:2,\u0026#39;B\u0026#39;:2}, left=3 max_length=4 Output: 4 ✅ Problem 7: Fruit Into Baskets LeetCode Link: https://leetcode.com/problems/fruit-into-baskets/\nProblem Statement You have a row of trees, each producing one type of fruit (fruits[i]). You have two baskets, and each basket can only hold one type of fruit. Starting from any tree, you pick fruits from consecutive trees moving right. You stop when you encounter a third type of fruit.\nReturn the maximum number of fruits you can pick.\nTranslation: Find the longest contiguous subarray with at most 2 distinct elements.\nExample:\nInput: fruits = [1, 2, 1, 2, 3] Output: 4 (subarray [1, 2, 1, 2] - only 2 types) Clarifying Questions \u0026amp; Constraints This is the classic \u0026ldquo;longest subarray with at most k distinct elements\u0026rdquo; problem, where k = 2. 1 \u0026lt;= fruits.length \u0026lt;= 10^5 Approach Discussion Approach 1: Brute Force\nCheck every subarray, count distinct elements, track the longest with ≤ 2. Time: O(n²), Space: O(n) Approach 2: Variable-Size Sliding Window (Optimal) ✅\nMaintain a frequency map of fruit types in the window. Expand right to add fruits. When the window has more than 2 distinct types, shrink from left. Time: O(n), Space: O(1) - at most 3 entries in the map before shrinking. Code (Optimal Solution) from collections import defaultdict def totalFruit(fruits: list[int]) -\u0026gt; int: \u0026#34;\u0026#34;\u0026#34; Find the longest subarray with at most 2 distinct elements. This is the general \u0026#34;at most K distinct\u0026#34; sliding window pattern with K = 2. Strategy: - Expand right, adding fruit types to a frequency map. - When we have more than 2 distinct types, shrink from left until we\u0026#39;re back to 2 types. - Track the maximum window size throughout. Time Complexity: O(n) - each element is added and removed at most once Space Complexity: O(1) - the frequency map has at most 3 entries at any point \u0026#34;\u0026#34;\u0026#34; freq = defaultdict(int) # fruit_type → count in current window left = 0 max_fruits = 0 for right in range(len(fruits)): # EXPAND: Add the new fruit to the window freq[fruits[right]] += 1 # SHRINK: While we have more than 2 distinct types while len(freq) \u0026gt; 2: left_fruit = fruits[left] freq[left_fruit] -= 1 if freq[left_fruit] == 0: del freq[left_fruit] # Remove the type entirely left += 1 # UPDATE: Current window is valid (≤ 2 types) max_fruits = max(max_fruits, right - left + 1) return max_fruits Generalizing to \u0026ldquo;At Most K Distinct\u0026rdquo; def longest_with_k_distinct(arr: list[int], k: int) -\u0026gt; int: \u0026#34;\u0026#34;\u0026#34; General version: longest subarray with at most K distinct elements. The Fruit Into Baskets problem is just this with k=2. Time Complexity: O(n) Space Complexity: O(k) \u0026#34;\u0026#34;\u0026#34; freq = defaultdict(int) left = 0 max_length = 0 for right in range(len(arr)): freq[arr[right]] += 1 while len(freq) \u0026gt; k: freq[arr[left]] -= 1 if freq[arr[left]] == 0: del freq[arr[left]] left += 1 max_length = max(max_length, right - left + 1) return max_length Edge Cases Single type: [1, 1, 1] → 3. Two types: [1, 2, 1, 2] → 4. All different: [1, 2, 3, 4] → 2. Single element: [5] → 1. Dry Run Input: fruits = [1, 2, 3, 2, 2] left=0, freq={}, max_fruits=0 right=0: freq={1:1}, distinct=1 ≤ 2 ✅, max_fruits=1 right=1: freq={1:1, 2:1}, distinct=2 ≤ 2 ✅, max_fruits=2 right=2: freq={1:1, 2:1, 3:1}, distinct=3 \u0026gt; 2 ❌ → remove fruits[0]=1: freq={2:1, 3:1}, left=1, distinct=2 ✅ max_fruits = max(2, 2-1+1) = 2 right=3: freq={2:2, 3:1}, distinct=2 ≤ 2 ✅, max_fruits=3 right=4: freq={2:3, 3:1}, distinct=2 ≤ 2 ✅, max_fruits=4 Output: 4 ✅ (subarray [2, 3, 2, 2]) Problem 8: Minimum Window Substring LeetCode Link: https://leetcode.com/problems/minimum-window-substring/\nProblem Statement Given two strings s and t, return the minimum window substring of s such that every character in t (including duplicates) is included in the window. If no such window exists, return \u0026quot;\u0026quot;.\nExample:\nInput: s = \u0026#34;ADOBECODEBANC\u0026#34;, t = \u0026#34;ABC\u0026#34; Output: \u0026#34;BANC\u0026#34; Clarifying Questions \u0026amp; Constraints Characters can appear multiple times in t, and the window must contain at least that many. If multiple answers exist with the same length, any one is acceptable. s and t consist of uppercase and lowercase English letters. Approach Discussion Approach 1: Brute Force\nCheck every substring of s, see if it contains all characters of t. Time: O(n² × m) where n = len(s), m = len(t). ❌ Way too slow. Approach 2: Variable-Size Sliding Window - Shortest (Optimal) ✅\nExpand right to include more characters until the window contains everything in t. Once valid, shrink from left to find the minimum length. Track character frequencies with a counter, and use a formed variable to know when all required characters have been satisfied. Time: O(n + m), Space: O(m + n) in worst case for the frequency maps. Code (Optimal Solution) from collections import Counter def minWindow(s: str, t: str) -\u0026gt; str: \u0026#34;\u0026#34;\u0026#34; Find the shortest substring of s containing all characters of t. Strategy (Variable-size window - finding the SHORTEST valid window): 1. Count the required frequencies from t. 2. Expand right to include characters. 3. Track how many required characters are fully satisfied. 4. Once fully valid, shrink from left while staying valid, recording the min. Key variables: - required: number of distinct characters in t we need to match - formed: number of distinct characters currently satisfied in the window - When formed == required, the window is valid Time Complexity: O(|s| + |t|) - building t\u0026#39;s counter is O(|t|), sliding window is O(|s|) since left and right each move at most |s| times Space Complexity: O(|s| + |t|) - for the frequency maps \u0026#34;\u0026#34;\u0026#34; if not s or not t or len(s) \u0026lt; len(t): return \u0026#34;\u0026#34; # Frequency of each character we need from t t_count = Counter(t) required = len(t_count) # Number of distinct characters to match # Current window state window_count = {} formed = 0 # How many distinct characters in t are fully satisfied # Result tracking: (window_length, left_index, right_index) result = (float(\u0026#39;inf\u0026#39;), 0, 0) left = 0 for right in range(len(s)): # EXPAND: Add s[right] to the window char = s[right] window_count[char] = window_count.get(char, 0) + 1 # Check if this character is now fully satisfied if char in t_count and window_count[char] == t_count[char]: formed += 1 # SHRINK: While the window is valid, try to minimize it while formed == required: # Update result if this window is smaller window_length = right - left + 1 if window_length \u0026lt; result[0]: result = (window_length, left, right) # Remove the leftmost character left_char = s[left] window_count[left_char] -= 1 # Check if removing this character breaks a satisfied condition if left_char in t_count and window_count[left_char] \u0026lt; t_count[left_char]: formed -= 1 left += 1 # Extract the result length, start, end = result return s[start:end + 1] if length != float(\u0026#39;inf\u0026#39;) else \u0026#34;\u0026#34; Edge Cases t longer than s: s=\u0026quot;a\u0026quot;, t=\u0026quot;abc\u0026quot; → \u0026quot;\u0026quot;. Exact match: s=\u0026quot;abc\u0026quot;, t=\u0026quot;abc\u0026quot; → \u0026quot;abc\u0026quot;. No valid window: s=\u0026quot;xyz\u0026quot;, t=\u0026quot;abc\u0026quot; → \u0026quot;\u0026quot;. Duplicate characters in t: s=\u0026quot;aaab\u0026quot;, t=\u0026quot;aa\u0026quot; → \u0026quot;aa\u0026quot;. t is a single character: s=\u0026quot;abcda\u0026quot;, t=\u0026quot;a\u0026quot; → \u0026quot;a\u0026quot;. Dry Run Input: s = \u0026#34;ADOBECODEBANC\u0026#34;, t = \u0026#34;ABC\u0026#34; t_count = {\u0026#39;A\u0026#39;:1, \u0026#39;B\u0026#39;:1, \u0026#39;C\u0026#39;:1}, required = 3 right=0: char=\u0026#39;A\u0026#39;, window={\u0026#39;A\u0026#39;:1}, formed=1 (A satisfied) right=1: char=\u0026#39;D\u0026#39;, window={\u0026#39;A\u0026#39;:1,\u0026#39;D\u0026#39;:1}, formed=1 right=2: char=\u0026#39;O\u0026#39;, window={...,\u0026#39;O\u0026#39;:1}, formed=1 right=3: char=\u0026#39;B\u0026#39;, window={...,\u0026#39;B\u0026#39;:1}, formed=2 (B satisfied) right=4: char=\u0026#39;E\u0026#39;, window={...,\u0026#39;E\u0026#39;:1}, formed=2 right=5: char=\u0026#39;C\u0026#39;, window={...,\u0026#39;C\u0026#39;:1}, formed=3 ✅ (all satisfied!) SHRINK: window \u0026#34;ADOBEC\u0026#34; length=6, result=(6, 0, 5) remove \u0026#39;A\u0026#39;: window_count[\u0026#39;A\u0026#39;]=0 \u0026lt; t_count[\u0026#39;A\u0026#39;]=1 → formed=2, left=1 → stop shrinking (formed \u0026lt; required) right=6: char=\u0026#39;O\u0026#39;, formed still 2 right=7: char=\u0026#39;D\u0026#39;, formed still 2 right=8: char=\u0026#39;E\u0026#39;, formed still 2 right=9: char=\u0026#39;B\u0026#39;, formed still 2 right=10: char=\u0026#39;A\u0026#39;, window_count[\u0026#39;A\u0026#39;]=1 == t_count[\u0026#39;A\u0026#39;]=1 → formed=3 ✅ SHRINK: window \u0026#34;DOBECODEBA\u0026#34; (left=1, right=10) length=10 → not better than 6 remove \u0026#39;D\u0026#39;: not in t_count, formed stays 3 window \u0026#34;OBECODEBA\u0026#34; (left=2, right=10) length=9 → not better remove \u0026#39;O\u0026#39;: not in t_count \u0026#34;BECODEBA\u0026#34; length=8 → not better remove \u0026#39;B\u0026#39;: window_count[\u0026#39;B\u0026#39;]=1, still == t_count[\u0026#39;B\u0026#39;]=1, formed=3 \u0026#34;ECODEBA\u0026#34; length=7 → not better remove \u0026#39;E\u0026#39;: not in t_count \u0026#34;CODEBA\u0026#34; length=6 → ties with current result remove \u0026#39;C\u0026#39;: window_count[\u0026#39;C\u0026#39;]=0 \u0026lt; 1 → formed=2, stop right=11: char=\u0026#39;N\u0026#39;, formed=2 right=12: char=\u0026#39;C\u0026#39;, window_count[\u0026#39;C\u0026#39;]=1 → formed=3 ✅ SHRINK: window \u0026#34;ODEBANC\u0026#34; (left=6, right=12) length=7 → not better than 6 remove \u0026#39;O\u0026#39;: formed=3, left=7 \u0026#34;DEBANC\u0026#34; length=6 → ties remove \u0026#39;D\u0026#39;: formed=3, left=8 \u0026#34;EBANC\u0026#34; length=5 → NEW BEST! result=(5, 8, 12) remove \u0026#39;E\u0026#39;: formed=3, left=9 \u0026#34;BANC\u0026#34; length=4 → NEW BEST! result=(4, 9, 12) remove \u0026#39;B\u0026#39;: window_count[\u0026#39;B\u0026#39;]=0 \u0026lt; 1 → formed=2, stop Output: s[9:13] = \u0026#34;BANC\u0026#34; ✅ Problem 9: Sliding Window Maximum LeetCode Link: https://leetcode.com/problems/sliding-window-maximum/\nProblem Statement Given an array nums and a sliding window of size k, return the maximum value in each window as it slides from left to right.\nExample:\nInput: nums = [1, 3, -1, -3, 5, 3, 6, 7], k = 3 Output: [3, 3, 5, 5, 6, 7] Window [1,3,-1] → max=3 Window [3,-1,-3] → max=3 Window [-1,-3,5] → max=5 Window [-3,5,3] → max=5 Window [5,3,6] → max=6 Window [3,6,7] → max=7 Clarifying Questions \u0026amp; Constraints 1 \u0026lt;= k \u0026lt;= nums.length The output has n - k + 1 elements. This is asking for the max of each window, not just the overall max. Approach Discussion Approach 1: Brute Force\nFor each window, scan all k elements to find the max. Time: O(n × k), Space: O(1) Approach 2: Sorted Container / Heap\nUse a max-heap. Add new element, remove old element. Time: O(n log k) - heap operations are O(log k). Tricky to implement correctly (lazy deletion). Approach 3: Monotonic Deque (Optimal) ✅\nMaintain a deque (double-ended queue) that stores indices of elements. Keep the deque in decreasing order of element values. The front of the deque is always the maximum of the current window. Time: O(n), Space: O(k) Why does the monotonic deque work? The key insight: if nums[i] \u0026gt;= nums[j] and i \u0026gt; j, then nums[j] can never be the maximum of any future window (because nums[i] entered later and is bigger). So we can discard nums[j]. This keeps only \u0026ldquo;useful\u0026rdquo; elements in the deque.\nCode (Both Approaches) # ============================================================ # APPROACH 1: Brute Force - O(n × k) # ============================================================ def maxSlidingWindow_brute(nums: list[int], k: int) -\u0026gt; list[int]: \u0026#34;\u0026#34;\u0026#34; For each window position, find the max by scanning all k elements. Time Complexity: O(n × k) Space Complexity: O(1) extra (not counting the output) \u0026#34;\u0026#34;\u0026#34; result = [] for i in range(len(nums) - k + 1): result.append(max(nums[i:i + k])) return result # ============================================================ # APPROACH 3: Monotonic Deque - O(n) ✅ # ============================================================ from collections import deque def maxSlidingWindow(nums: list[int], k: int) -\u0026gt; list[int]: \u0026#34;\u0026#34;\u0026#34; Find the maximum in each sliding window using a monotonic deque. The deque stores INDICES (not values) and maintains this invariant: - Elements in the deque are in DECREASING order of their values. - The front of the deque is the index of the current window\u0026#39;s maximum. For each new element: 1. Remove indices from the BACK if their values are ≤ current value (they\u0026#39;ll never be the max while current element exists in the window). 2. Add the current index to the back. 3. Remove the FRONT if it\u0026#39;s outside the current window. 4. The front is the max of the current window. Time Complexity: O(n) - each element is added and removed from the deque at most once Space Complexity: O(k) - the deque holds at most k elements \u0026#34;\u0026#34;\u0026#34; dq = deque() # Stores indices, values are in decreasing order result = [] for right in range(len(nums)): # Step 1: Remove elements from the BACK that are smaller than current # They can never be the max while nums[right] is in the window while dq and nums[dq[-1]] \u0026lt;= nums[right]: dq.pop() # Step 2: Add current index to the back dq.append(right) # Step 3: Remove the front if it\u0026#39;s outside the current window # The window is [right - k + 1, right] if dq[0] \u0026lt; right - k + 1: dq.popleft() # Step 4: Once we\u0026#39;ve built the first full window, record the max # (The first full window is complete when right \u0026gt;= k - 1) if right \u0026gt;= k - 1: result.append(nums[dq[0]]) # Front of deque = max of window return result Edge Cases k = 1: Every element is its own window → return the array as-is. k = n: Single window → return [max(nums)]. Ascending array: [1,2,3,4], k=2 → [2,3,4] (front gets popped every time). Descending array: [4,3,2,1], k=2 → [4,3,2] (back never gets popped, front gets popped). Dry Run Input: nums = [1, 3, -1, -3, 5, 3, 6, 7], k = 3 right=0: num=1 deque empty → add 0 dq = [0] (values: [1]) right \u0026lt; 2 → no output yet right=1: num=3 nums[0]=1 ≤ 3 → pop 0 add 1 dq = [1] (values: [3]) right \u0026lt; 2 → no output yet right=2: num=-1 nums[1]=3 \u0026gt; -1 → don\u0026#39;t pop add 2 dq = [1, 2] (values: [3, -1]) right \u0026gt;= 2 → output nums[1] = 3 result = [3] right=3: num=-3 nums[2]=-1 \u0026gt; -3 → don\u0026#39;t pop add 3 dq = [1, 2, 3] (values: [3, -1, -3]) dq[0]=1 \u0026gt;= 3-3+1=1 → front is still in window output nums[1] = 3 result = [3, 3] right=4: num=5 nums[3]=-3 ≤ 5 → pop 3 nums[2]=-1 ≤ 5 → pop 2 nums[1]=3 ≤ 5 → pop 1 add 4 dq = [4] (values: [5]) output nums[4] = 5 result = [3, 3, 5] right=5: num=3 nums[4]=5 \u0026gt; 3 → don\u0026#39;t pop add 5 dq = [4, 5] (values: [5, 3]) output nums[4] = 5 result = [3, 3, 5, 5] right=6: num=6 nums[5]=3 ≤ 6 → pop 5 nums[4]=5 ≤ 6 → pop 4 add 6 dq = [6] (values: [6]) output nums[6] = 6 result = [3, 3, 5, 5, 6] right=7: num=7 nums[6]=6 ≤ 7 → pop 6 add 7 dq = [7] (values: [7]) output nums[7] = 7 result = [3, 3, 5, 5, 6, 7] Output: [3, 3, 5, 5, 6, 7] ✅ Problem 10: Substring with Concatenation of All Words LeetCode Link: https://leetcode.com/problems/substring-with-concatenation-of-all-words/\nProblem Statement Given a string s and an array of strings words (all the same length), find all starting indices of substrings in s that are a concatenation of each word in words exactly once, in any order.\nExample:\nInput: s = \u0026#34;barfoothefoobarman\u0026#34;, words = [\u0026#34;foo\u0026#34;, \u0026#34;bar\u0026#34;] Output: [0, 9] Explanation: s[0:6] = \u0026#34;barfoo\u0026#34; → \u0026#34;bar\u0026#34; + \u0026#34;foo\u0026#34; ✅ s[9:15] = \u0026#34;foobar\u0026#34; → \u0026#34;foo\u0026#34; + \u0026#34;bar\u0026#34; ✅ Clarifying Questions \u0026amp; Constraints All words have the same length. Words can repeat in the words array (e.g., [\u0026quot;foo\u0026quot;, \u0026quot;foo\u0026quot;]). The total window size = word_length × number_of_words. 1 \u0026lt;= words.length \u0026lt;= 5000, 1 \u0026lt;= words[i].length \u0026lt;= 30 Approach Discussion Approach 1: Brute Force\nFor each starting index in s, extract a window of the total size, split it into word-length chunks, and check if the chunks match words. Time: O(n × m × w) where n = len(s), m = number of words, w = word length. Acceptable for moderate inputs but has redundant work. Approach 2: Sliding Window on Word Level (Optimal) ✅\nSince all words have the same length w, we can think of the problem as a sliding window at the word level. We try w different starting offsets (0, 1, \u0026hellip;, w-1) to cover all alignments. For each offset, slide a word-level window, comparing word-sized chunks to the required frequency. Time: O(n × w) - for each of the w offsets, we scan the string once. Space: O(m) for the frequency map. Code (Both Solutions) from collections import Counter # ============================================================ # APPROACH 1: Brute Force - Check each position # ============================================================ def findSubstring_brute(s: str, words: list[str]) -\u0026gt; list[int]: \u0026#34;\u0026#34;\u0026#34; For each starting position, extract words and check if they match. Time Complexity: O(n × m × w) where n=len(s), m=len(words), w=len(words[0]) Space Complexity: O(m) for the word counter \u0026#34;\u0026#34;\u0026#34; if not s or not words: return [] word_len = len(words[0]) num_words = len(words) total_len = word_len * num_words word_count = Counter(words) result = [] for i in range(len(s) - total_len + 1): # Extract the window and split into word-sized chunks seen = {} valid = True for j in range(num_words): word = s[i + j * word_len : i + (j + 1) * word_len] if word not in word_count: valid = False break seen[word] = seen.get(word, 0) + 1 if seen[word] \u0026gt; word_count[word]: valid = False break if valid: result.append(i) return result # ============================================================ # APPROACH 2: Word-Level Sliding Window - O(n × w) ✅ # ============================================================ def findSubstring(s: str, words: list[str]) -\u0026gt; list[int]: \u0026#34;\u0026#34;\u0026#34; Use a sliding window at the word level for each possible starting offset. Key Insight: Since all words have the same length w, any valid concatenation starts at some offset 0, 1, ..., w-1 relative to word boundaries. For each offset, we slide a window of num_words words across s. For offset = 0: check positions 0, w, 2w, 3w, ... For offset = 1: check positions 1, w+1, 2w+1, ... ... For offset = w-1: check positions w-1, 2w-1, ... At each step, we add one word to the right of the window and potentially remove one from the left, maintaining a frequency count. Time Complexity: O(n × w) - for each of w offsets, we scan the string in O(n/w) steps Space Complexity: O(m) where m = number of words \u0026#34;\u0026#34;\u0026#34; if not s or not words: return [] word_len = len(words[0]) num_words = len(words) total_len = word_len * num_words word_count = Counter(words) result = [] # Try each starting offset from 0 to word_len - 1 for offset in range(word_len): # Window state for this offset window = {} formed = 0 # Number of distinct words with matching counts required = len(word_count) left = offset # Left boundary of the window (character index) # Slide through the string in steps of word_len for right_start in range(offset, len(s) - word_len + 1, word_len): # Extract the word entering the window word = s[right_start : right_start + word_len] if word in word_count: # Add the word to the window window[word] = window.get(word, 0) + 1 if window[word] == word_count[word]: formed += 1 elif window[word] == word_count[word] + 1: # We just went over - was matching, now isn\u0026#39;t formed -= 1 # SHRINK: If we have too many of this word, shrink from left while window.get(word, 0) \u0026gt; word_count.get(word, 0): left_word = s[left : left + word_len] if left_word in word_count: if window[left_word] == word_count[left_word]: formed -= 1 window[left_word] -= 1 if window[left_word] == word_count[left_word]: formed += 1 left += word_len # CHECK: If all words are satisfied if formed == required: result.append(left) else: # Word not in our required set - reset the window window.clear() formed = 0 left = right_start + word_len return sorted(result) Edge Cases Single word: s=\u0026quot;foobar\u0026quot;, words=[\u0026quot;foo\u0026quot;] → [0]. No match: s=\u0026quot;abc\u0026quot;, words=[\u0026quot;def\u0026quot;] → []. Duplicate words: s=\u0026quot;aaa\u0026quot;, words=[\u0026quot;a\u0026quot;, \u0026quot;a\u0026quot;] → [0, 1]. s shorter than total_len: s=\u0026quot;ab\u0026quot;, words=[\u0026quot;abc\u0026quot;] → []. Dry Run Input: s = \u0026#34;barfoothefoobarman\u0026#34;, words = [\u0026#34;foo\u0026#34;, \u0026#34;bar\u0026#34;] word_len = 3, num_words = 2, total_len = 6 word_count = {\u0026#34;foo\u0026#34;: 1, \u0026#34;bar\u0026#34;: 1} Offset 0: positions 0, 3, 6, 9, 12, 15 left=0, window={}, formed=0 pos=0: word=\u0026#34;bar\u0026#34; (in word_count) window={\u0026#34;bar\u0026#34;:1}, bar matches → formed=1 pos=3: word=\u0026#34;foo\u0026#34; (in word_count) window={\u0026#34;bar\u0026#34;:1,\u0026#34;foo\u0026#34;:1}, foo matches → formed=2 formed==required ✅ → result.append(0) pos=6: word=\u0026#34;the\u0026#34; (NOT in word_count) → reset: window={}, formed=0, left=9 pos=9: word=\u0026#34;foo\u0026#34; → window={\u0026#34;foo\u0026#34;:1}, formed=1 pos=12: word=\u0026#34;bar\u0026#34; → window={\u0026#34;foo\u0026#34;:1,\u0026#34;bar\u0026#34;:1}, formed=2 formed==required ✅ → result.append(9) pos=15: word=\u0026#34;man\u0026#34; → reset Offsets 1 and 2 produce no additional matches. Output: [0, 9] ✅ Key Takeaways \u0026amp; Summary Quick Reference Table Problem Window Type Time Space Core Trick Max Sum Subarray K Fixed O(n) O(1) Add right, subtract left Longest Substring No Repeat Variable (longest) O(n) O(min(n,m)) HashMap tracks last seen index Max Average Subarray I Fixed O(n) O(1) Same as max sum, divide at end Min Size Subarray Sum Variable (shortest) O(n) O(1) Shrink while valid (positive numbers) Permutation in String Fixed O(n) O(1) Frequency matching with counters Longest Repeating Char Replace Variable (longest) O(n) O(1) window - max_freq ≤ k Fruit Into Baskets Variable (longest) O(n) O(1) At most k=2 distinct Min Window Substring Variable (shortest) O(n) O(m) Frequency match, shrink while valid Sliding Window Maximum Fixed O(n) O(k) Monotonic deque Concat All Words Fixed (word-level) O(n×w) O(m) Word-level window, w offsets Decision Framework Does the problem involve contiguous subarrays/substrings? ├── YES → Sliding Window is likely the right approach │ │ Is the window size given (fixed k)? │ ├── YES → Fixed-Size Window │ │ Build first window, then slide: add right, remove left │ │ │ └── NO → Variable-Size Window │ │ │ Looking for the LONGEST valid window? │ ├── YES → Expand always, shrink when INVALID │ │ │ └── NO (looking for SHORTEST) │ → Expand always, shrink while VALID │ └── NO → Consider other patterns (Two Pointers, Prefix Sum, etc.) Common Mistakes to Avoid Forgetting to handle the first window separately in fixed-size problems. The sliding part starts after the first window is built.\nConfusing when to shrink: For longest → shrink when invalid. For shortest → shrink when valid. Getting this backwards is a very common bug.\nNot cleaning up zero counts in frequency maps. When using Counter comparison (==), a key with count 0 is different from the key being absent. Always del keys that drop to 0 if you\u0026rsquo;re comparing maps.\nOff-by-one errors on window boundaries. The window [left, right] has size right - left + 1. The element leaving a fixed window of size k when right enters is at index right - k.\nAssuming sliding window works with negative numbers for min-size subarray problems. It only works when the window sum is monotonically related to window size (which requires positive numbers).\nWhat\u0026rsquo;s Next? With Two Pointers and Sliding Window covered, the next pattern is Prefix Sum - a technique for answering range-sum queries in O(1) after O(n) preprocessing. It pairs naturally with hashmaps for powerful subarray-sum problems like \u0026ldquo;Subarray Sum Equals K\u0026rdquo;. Stay tuned for Pattern 3!\n💡 Practice Tip: For sliding window problems, always start by asking two questions: (1) Is the window fixed or variable? (2) Am I looking for the longest or shortest? These two answers determine which template to use.\n","permalink":"https://blogs.codeverra.com/blog/dsa-patterns-03-sliding-window/","summary":"\u003ch1 id=\"-pattern-2-sliding-window\"\u003e🔰 Pattern 2: Sliding Window\u003c/h1\u003e\n\u003ch2 id=\"table-of-contents\"\u003eTable of Contents\u003c/h2\u003e\n\u003col\u003e\n\u003cli\u003e\u003ca href=\"#what-is-the-sliding-window-technique\"\u003eWhat is the Sliding Window Technique?\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#when-to-use-sliding-window\"\u003eWhen to Use Sliding Window?\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#types-of-sliding-window\"\u003eTypes of Sliding Window\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#template-code\"\u003eTemplate Code\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#problem-set\"\u003eProblem Set\u003c/a\u003e\n\u003cul\u003e\n\u003cli\u003e\u003ca href=\"#problem-1-maximum-sum-subarray-of-size-k\"\u003eProblem 1: Maximum Sum Subarray of Size K\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#problem-2-longest-substring-without-repeating-characters\"\u003eProblem 2: Longest Substring Without Repeating Characters\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#problem-3-maximum-average-subarray-i\"\u003eProblem 3: Maximum Average Subarray I\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#problem-4-minimum-size-subarray-sum\"\u003eProblem 4: Minimum Size Subarray Sum\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#problem-5-permutation-in-string\"\u003eProblem 5: Permutation in String\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#problem-6-longest-repeating-character-replacement\"\u003eProblem 6: Longest Repeating Character Replacement\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#problem-7-fruit-into-baskets\"\u003eProblem 7: Fruit Into Baskets\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#problem-8-minimum-window-substring\"\u003eProblem 8: Minimum Window Substring\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#problem-9-sliding-window-maximum\"\u003eProblem 9: Sliding Window Maximum\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#problem-10-substring-with-concatenation-of-all-words\"\u003eProblem 10: Substring with Concatenation of All Words\u003c/a\u003e\u003c/li\u003e\n\u003c/ul\u003e\n\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#key-takeaways--summary\"\u003eKey Takeaways \u0026amp; Summary\u003c/a\u003e\u003c/li\u003e\n\u003c/ol\u003e\n\u003chr\u003e\n\u003ch2 id=\"what-is-the-sliding-window-technique\"\u003eWhat is the Sliding Window Technique?\u003c/h2\u003e\n\u003cp\u003eIn the Two Pointers pattern, we learned how two indices can traverse an array efficiently. Sliding Window builds on that idea  -  it maintains a \u003cstrong\u003erange (window) between two pointers\u003c/strong\u003e and cares about the elements \u003cem\u003einside\u003c/em\u003e that range.\u003c/p\u003e","tags":["dsa","dsa-patterns"],"title":"Sliding Window Pattern  -  Complete Guide"},{"content":"Stacks Table of Contents What You Will Learn in This Blog What is a Stack Core Stack Operations Implementing a Stack Implementation 1 \u0026ndash; Using a Python List Implementation 2 \u0026ndash; Using a Linked List Which Implementation to Use The Call Stack How Python Uses a Stack for Function Calls Stack Overflow Monotonic Stack What is a Monotonic Stack Monotonic Increasing Stack Monotonic Decreasing Stack The Template Time and Space Complexity Patterns and Techniques Pattern 1 \u0026ndash; Stack for Matching and Validation Pattern 2 \u0026ndash; Stack for State Tracking Pattern 3 \u0026ndash; Monotonic Stack Summary Key Takeaways Practice Problems Next Steps What You Will Learn in This Blog What a stack is and the LIFO principle The four core operations: push, pop, peek, and isEmpty Two ways to implement a stack from scratch \u0026ndash; using a list and using a linked list How Python uses a stack internally for function calls and recursion What a monotonic stack is and when to reach for it Three problem-solving patterns that use stacks Solved LeetCode problems with full explanation A practice problem table for further study What is a Stack A stack is a linear data structure that follows the Last In, First Out principle \u0026ndash; LIFO. The last element added is the first one to be removed.\nThe classic real world analogy is a stack of plates. You add a new plate on top. When you take a plate, you take it from the top. You cannot take a plate from the middle without removing the ones above it first.\nOther real world examples of LIFO behaviour:\nThe back button in a browser \u0026ndash; each page you visit is pushed onto a stack. Pressing back pops the most recent page. Undo in a text editor \u0026ndash; each action is pushed onto a stack. Ctrl+Z pops and reverses the most recent action. Function calls in any program \u0026ndash; each function call is pushed onto the call stack. When it returns, it is popped. The key constraint of a stack is that you can only interact with the top element. You cannot access elements below the top without first removing everything above them.\nCore Stack Operations A stack has exactly four operations. Everything else is built from these.\nOperation Description Time Complexity push(x) Add element x to the top of the stack O(1) pop() Remove and return the top element O(1) peek() Return the top element without removing it O(1) is_empty() Return True if the stack has no elements O(1) All four operations are O(1). This is what makes a stack efficient \u0026ndash; the constraint of only accessing the top means there is never any searching or shifting.\nImplementing a Stack A stack is an abstract data structure \u0026ndash; it defines behaviour, not implementation. You can build it on top of different underlying structures. We will implement it two ways.\nImplementation 1 \u0026ndash; Using a Python List Python\u0026rsquo;s list is a natural fit for a stack. The end of the list is the top of the stack. append() is push and pop() is pop \u0026ndash; both are O(1) amortised.\nclass Stack: \u0026#34;\u0026#34;\u0026#34; Stack implemented using a Python list. The end of the list represents the top of the stack. \u0026#34;\u0026#34;\u0026#34; def __init__(self): self._data = [] # internal list -- underscore signals it is private def push(self, x): \u0026#34;\u0026#34;\u0026#34; Add element x to the top of the stack. Time complexity: O(1) amortised Space complexity: O(1) \u0026#34;\u0026#34;\u0026#34; self._data.append(x) def pop(self): \u0026#34;\u0026#34;\u0026#34; Remove and return the top element. Raises IndexError if the stack is empty. Time complexity: O(1) Space complexity: O(1) \u0026#34;\u0026#34;\u0026#34; if self.is_empty(): raise IndexError(\u0026#34;Pop from an empty stack\u0026#34;) return self._data.pop() def peek(self): \u0026#34;\u0026#34;\u0026#34; Return the top element without removing it. Raises IndexError if the stack is empty. Time complexity: O(1) Space complexity: O(1) \u0026#34;\u0026#34;\u0026#34; if self.is_empty(): raise IndexError(\u0026#34;Peek at an empty stack\u0026#34;) return self._data[-1] # last element = top of stack def is_empty(self): \u0026#34;\u0026#34;\u0026#34; Return True if the stack is empty. Time complexity: O(1) Space complexity: O(1) \u0026#34;\u0026#34;\u0026#34; return len(self._data) == 0 def __len__(self): return len(self._data) def __str__(self): return str(self._data) + \u0026#34; \u0026lt;- top\u0026#34; # --- Test --- s = Stack() s.push(10) s.push(20) s.push(30) print(s) # [10, 20, 30] \u0026lt;- top print(s.peek()) # 30 (top element, not removed) print(s.pop()) # 30 (removed) print(s.pop()) # 20 (removed) print(s) # [10] \u0026lt;- top print(len(s)) # 1 print(s.is_empty()) # False s.pop() print(s.is_empty()) # True Implementation 2 \u0026ndash; Using a Linked List A linked list also works well for a stack. The head of the linked list is the top of the stack. Push and pop at the head are both O(1) with no amortised cost \u0026ndash; no resizing ever happens.\nclass StackNode: \u0026#34;\u0026#34;\u0026#34;A node in the linked list backing the stack.\u0026#34;\u0026#34;\u0026#34; def __init__(self, data): self.data = data self.next = None class LinkedStack: \u0026#34;\u0026#34;\u0026#34; Stack implemented using a singly linked list. The head of the list is the top of the stack. Push and pop at the head are O(1) -- no resizing, ever. \u0026#34;\u0026#34;\u0026#34; def __init__(self): self._top = None # head of the linked list self._size = 0 def push(self, x): \u0026#34;\u0026#34;\u0026#34; Create a new node and insert it at the head. Time complexity: O(1) Space complexity: O(1) \u0026#34;\u0026#34;\u0026#34; new_node = StackNode(x) new_node.next = self._top # new node points to old top self._top = new_node # update top self._size += 1 def pop(self): \u0026#34;\u0026#34;\u0026#34; Remove and return the head node\u0026#39;s data. Time complexity: O(1) Space complexity: O(1) \u0026#34;\u0026#34;\u0026#34; if self.is_empty(): raise IndexError(\u0026#34;Pop from an empty stack\u0026#34;) value = self._top.data self._top = self._top.next # move top down self._size -= 1 return value def peek(self): \u0026#34;\u0026#34;\u0026#34; Return the head node\u0026#39;s data without removing it. Time complexity: O(1) Space complexity: O(1) \u0026#34;\u0026#34;\u0026#34; if self.is_empty(): raise IndexError(\u0026#34;Peek at an empty stack\u0026#34;) return self._top.data def is_empty(self): return self._top is None def __len__(self): return self._size # --- Test --- ls = LinkedStack() ls.push(10) ls.push(20) ls.push(30) print(ls.peek()) # 30 print(ls.pop()) # 30 print(ls.pop()) # 20 print(len(ls)) # 1 Which Implementation to Use Factor List-based Stack Linked List Stack Push O(1) amortised O(1) always Pop O(1) O(1) Memory Contiguous, cache friendly Scattered, extra pointer per node Simplicity Simpler to write and read Slightly more code Resizing Occasional O(n) resize Never resizes In practice Almost always preferred Useful when resizing overhead matters In Python, the list-based stack is what you will use in virtually every DSA problem. The linked list version is good to know for understanding how stacks work at a lower level.\nIn practice for LeetCode problems: You rarely need to define a Stack class at all. You just use a plain Python list and treat append() as push and pop() as pop. The class is built here so you understand the abstraction fully.\n# What you will actually write in most problems stack = [] stack.append(10) # push stack.append(20) # push top = stack[-1] # peek stack.pop() # pop The Call Stack The call stack is one of the most important applications of the stack data structure. Every time your Python program runs a function, the stack is involved.\nHow Python Uses a Stack for Function Calls When your program calls a function, Python creates a stack frame for that call. This frame holds:\nThe function\u0026rsquo;s local variables The parameters passed to the function The return address \u0026ndash; where to go back to when the function finishes This frame is pushed onto the call stack. When the function returns, its frame is popped off and execution resumes where it left off.\ndef greet(name): message = build_message(name) # push build_message frame print(message) # build_message frame is popped, back here # greet frame is popped when this returns def build_message(name): return \u0026#34;Hello, \u0026#34; + name # this frame is popped when return executes greet(\u0026#34;Alice\u0026#34;) # push greet frame onto the call stack The call stack at the moment build_message is executing:\n| build_message frame | \u0026lt;- top (currently executing) | greet frame | | main / module frame | \u0026lt;- bottom When build_message returns, its frame is popped and greet continues. When greet returns, its frame is popped and the main program continues.\nThis is exactly why recursion works \u0026ndash; each recursive call pushes a new frame. The frames accumulate until a base case is hit, then they are popped one by one as each call returns.\ndef factorial(n): if n == 0: return 1 # base case -- start popping return n * factorial(n-1) # push a new frame # factorial(3) call stack at deepest point: # | factorial(0) | \u0026lt;- top, returns 1 # | factorial(1) | waiting for factorial(0) # | factorial(2) | waiting for factorial(1) # | factorial(3) | waiting for factorial(2) # | main frame | \u0026lt;- bottom Stack Overflow Every program has a maximum call stack size. If recursion goes too deep \u0026ndash; too many frames are pushed without being popped \u0026ndash; Python raises a RecursionError. This is called a stack overflow.\ndef infinite_recursion(n): return infinite_recursion(n + 1) # no base case -- keeps pushing frames # infinite_recursion(0) # RecursionError: maximum recursion depth exceeded # Python\u0026#39;s default recursion limit import sys print(sys.getrecursionlimit()) # typically 1000 # You can increase it, but it is rarely a good idea # sys.setrecursionlimit(10000) This is why every recursive function must have a base case, and why very deep recursion should often be replaced with an iterative approach using an explicit stack.\nMonotonic Stack What is a Monotonic Stack A monotonic stack is a stack where the elements are always kept in a sorted order \u0026ndash; either always increasing from bottom to top, or always decreasing from bottom to top.\nIt is not a separate data structure. It is a technique \u0026ndash; a way of using a regular stack while maintaining an ordering invariant. You enforce the ordering by popping elements that violate it before pushing a new one.\nThis technique efficiently solves a class of problems involving next greater element, next smaller element, previous greater, and previous smaller \u0026ndash; problems that would otherwise require O(n^2) nested loops.\nMonotonic Increasing Stack Elements in the stack are always increasing from bottom to top (smallest at bottom, largest at top). Before pushing a new element, pop all elements that are greater than or equal to the new element. This ensures the stack stays increasing.\nnums = [3, 1, 4, 1, 5, 9, 2, 6] stack = [] for num in nums: # Pop elements that are greater than current -- they violate increasing order while stack and stack[-1] \u0026gt;= num: stack.pop() stack.append(num) print(stack) # [1, 2, 6] -- monotonic increasing from bottom to top Monotonic Decreasing Stack Elements are always decreasing from bottom to top (largest at bottom, smallest at top). Before pushing, pop all elements that are less than or equal to the new element.\nnums = [3, 1, 4, 1, 5, 9, 2, 6] stack = [] for num in nums: # Pop elements that are smaller than current -- they violate decreasing order while stack and stack[-1] \u0026lt;= num: stack.pop() stack.append(num) print(stack) # [9, 6] -- monotonic decreasing from bottom to top The Template The general monotonic stack template. The key insight is that elements are popped at exactly the right moment \u0026ndash; when a new element comes in that breaks the ordering. That moment of popping is when you extract your answer for the popped element.\ndef monotonic_stack_template(nums): stack = [] # stores indices, not values -- indices are more useful result = [0] * len(nums) for i in range(len(nums)): # For a \u0026#34;next greater element\u0026#34; problem: # Pop while current element is greater than the element at stack top while stack and nums[stack[-1]] \u0026lt; nums[i]: index = stack.pop() # nums[i] is the next greater element for nums[index] result[index] = nums[i] stack.append(i) # Elements remaining in stack have no next greater element while stack: result[stack.pop()] = -1 return result Time and Space Complexity Operation Time Complexity Space Complexity push O(1) amortised (list) / O(1) (linked list) O(1) pop O(1) O(1) peek O(1) O(1) is_empty O(1) O(1) Overall space of a stack with n elements \u0026ndash; O(n) Monotonic stack (single pass) O(n) \u0026ndash; each element pushed and popped at most once O(n) The monotonic stack is O(n) even though there is a while loop inside a for loop. The reason: each element is pushed exactly once and popped at most once. The total number of push and pop operations across the entire pass is at most 2n, which is O(n).\nPatterns and Techniques Pattern 1 \u0026ndash; Stack for Matching and Validation The core idea: Use a stack to match opening brackets, tags, or tokens with their corresponding closing counterparts. Push when you see an opener, pop and verify when you see a closer.\nWhen to use it:\nBalanced brackets or parentheses Matching HTML or XML tags Validating any nested structure Solved Problem \u0026ndash; Valid Parentheses LeetCode 20 \u0026ndash; Valid Parentheses\nProblem: Given a string s containing only the characters (, ), {, }, [, and ], determine if the input string is valid. A string is valid if every opening bracket is closed by the same type of bracket in the correct order.\nInput: s = \u0026#34;()\u0026#34; Output: True Input: s = \u0026#34;()[]{}\u0026#34; Output: True Input: s = \u0026#34;(]\u0026#34; Output: False Input: s = \u0026#34;([)]\u0026#34; Output: False Input: s = \u0026#34;{[]}\u0026#34; Output: True Thinking through it:\nWalk through the string character by character. When you see an opening bracket, push it. When you see a closing bracket, check if it matches the most recent opening bracket (the top of the stack). If it does, pop the opening bracket and continue. If it does not, the string is invalid. At the end, the stack should be empty \u0026ndash; every opener was matched.\ndef is_valid(s: str) -\u0026gt; bool: \u0026#34;\u0026#34;\u0026#34; Push opening brackets onto the stack. On each closing bracket, check if it matches the top of the stack. Time complexity: O(n) -- single pass through the string Space complexity: O(n) -- stack can hold at most n/2 opening brackets \u0026#34;\u0026#34;\u0026#34; stack = [] # Map each closing bracket to its corresponding opening bracket matching = { \u0026#39;)\u0026#39;: \u0026#39;(\u0026#39;, \u0026#39;}\u0026#39;: \u0026#39;{\u0026#39;, \u0026#39;]\u0026#39;: \u0026#39;[\u0026#39; } for char in s: if char in \u0026#39;({[\u0026#39;: stack.append(char) # opening bracket -- push it else: # Closing bracket -- check if stack is non-empty # and top of stack matches the expected opener if not stack or stack[-1] != matching[char]: return False stack.pop() # valid match -- pop the opener return len(stack) == 0 # valid only if all openers were matched # Test print(is_valid(\u0026#34;()\u0026#34;)) # True print(is_valid(\u0026#34;()[]{}\u0026#34;)) # True print(is_valid(\u0026#34;(]\u0026#34;)) # False print(is_valid(\u0026#34;([)]\u0026#34;)) # False print(is_valid(\u0026#34;{[]}\u0026#34;)) # True print(is_valid(\u0026#34;\u0026#34;)) # True (empty string is valid) Let us trace through \u0026quot;{[]}\u0026quot;:\nChar Action Stack { Push ['{'] [ Push ['{', '['] ] Top is [, matches ] \u0026ndash; pop ['{'] } Top is {, matches } \u0026ndash; pop [] End Stack is empty Valid Pattern 2 \u0026ndash; Stack for State Tracking The core idea: Use a stack to remember state as you process a sequence. When you need to go back to a previous state, pop the stack.\nWhen to use it:\nProblems where you process a sequence and occasionally need to revert Implementing undo/redo Evaluating expressions Problems with nested structures Solved Problem \u0026ndash; Min Stack LeetCode 155 \u0026ndash; Min Stack\nProblem: Design a stack that supports push, pop, top, and retrieving the minimum element \u0026ndash; all in O(1) time.\nMinStack minStack = new MinStack(); minStack.push(-2); minStack.push(0); minStack.push(-3); minStack.getMin(); --\u0026gt; -3 minStack.pop(); minStack.top(); --\u0026gt; 0 minStack.getMin(); --\u0026gt; -2 Thinking through it:\nThe challenge is getMin() in O(1). A naive approach would scan the entire stack to find the minimum \u0026ndash; that is O(n). The trick is to maintain a second stack that tracks the minimum at each state. Every time you push a value, also push the current minimum onto the min stack. When you pop, also pop the min stack. The top of the min stack is always the current minimum.\nclass MinStack: \u0026#34;\u0026#34;\u0026#34; Uses two stacks: - main_stack: stores all values in push order - min_stack: stores the minimum value at each point in time At any moment, min_stack[-1] is the minimum of all values currently in main_stack. Time complexity: O(1) for all operations Space complexity: O(n) -- two stacks, each holding at most n elements \u0026#34;\u0026#34;\u0026#34; def __init__(self): self.main_stack = [] # stores all pushed values self.min_stack = [] # stores the running minimum def push(self, val: int) -\u0026gt; None: self.main_stack.append(val) # Push the new minimum onto min_stack # If min_stack is empty, the new value is the minimum # Otherwise, the minimum is the smaller of val and current minimum if not self.min_stack: self.min_stack.append(val) else: self.min_stack.append(min(val, self.min_stack[-1])) def pop(self) -\u0026gt; None: self.main_stack.pop() self.min_stack.pop() # keep both stacks in sync def top(self) -\u0026gt; int: return self.main_stack[-1] def getMin(self) -\u0026gt; int: return self.min_stack[-1] # always the current minimum -- O(1) # Test ms = MinStack() ms.push(-2) ms.push(0) ms.push(-3) print(ms.getMin()) # -3 ms.pop() print(ms.top()) # 0 print(ms.getMin()) # -2 Let us trace through the pushes to make the min stack clear:\nOperation main_stack min_stack Explanation push(-2) [-2] [-2] min is -2 push(0) [-2, 0] [-2, -2] min is still -2 push(-3) [-2, 0, -3] [-2, -2, -3] new min is -3 getMin() \u0026ndash; \u0026ndash; top of min_stack = -3 pop() [-2, 0] [-2, -2] both stacks shrink getMin() \u0026ndash; \u0026ndash; top of min_stack = -2 Pattern 3 \u0026ndash; Monotonic Stack The core idea: Maintain a stack in sorted order by popping elements that violate the order before pushing each new element. The moment an element is popped is exactly when you have found the answer for that element.\nWhen to use it:\nNext greater element / next smaller element Previous greater / previous smaller Problems involving temperature, stock prices, or histogram bars Any problem where you need to find the nearest element satisfying a condition Solved Problem \u0026ndash; Daily Temperatures LeetCode 739 \u0026ndash; Daily Temperatures\nProblem: Given an array of integers temperatures where temperatures[i] is the temperature on day i, return an array answer where answer[i] is the number of days you have to wait after day i to get a warmer temperature. If there is no future day with a warmer temperature, answer[i] is 0.\nInput: temperatures = [73, 74, 75, 71, 69, 72, 76, 73] Output: [1, 1, 4, 2, 1, 1, 0, 0] Explanation: Day 0 (73): next warmer is day 1 (74), wait 1 day Day 1 (74): next warmer is day 2 (75), wait 1 day Day 2 (75): next warmer is day 6 (76), wait 4 days Day 3 (71): next warmer is day 5 (72), wait 2 days ... Day 6 (76): no warmer day, answer is 0 Day 7 (73): no warmer day, answer is 0 Thinking through it:\nThe brute force is O(n^2): for each day, scan forward to find the next warmer day. We can do better with a monotonic decreasing stack of indices.\nWe maintain a stack of day indices whose temperatures are waiting for a warmer day. When we process day i, we check the stack. If the temperature today is warmer than the temperature at the index on top of the stack, that index has found its answer. We pop it and record the difference. We keep popping as long as today is warmer than the stack top. Then we push today\u0026rsquo;s index.\ndef daily_temperatures(temperatures: list) -\u0026gt; list: \u0026#34;\u0026#34;\u0026#34; Monotonic decreasing stack of indices. When a warmer temperature is found for a day, pop it and record the gap. Time complexity: O(n) -- each index is pushed and popped at most once Space complexity: O(n) -- stack holds at most n indices \u0026#34;\u0026#34;\u0026#34; n = len(temperatures) answer = [0] * n # default answer is 0 (no warmer day found) stack = [] # stores indices of days waiting for a warmer day for i in range(n): # While today is warmer than the day at the top of the stack while stack and temperatures[i] \u0026gt; temperatures[stack[-1]]: prev_day = stack.pop() answer[prev_day] = i - prev_day # number of days waited stack.append(i) # push today\u0026#39;s index -- waiting for a warmer day # Remaining indices in stack have no warmer day -- answer stays 0 return answer # Test print(daily_temperatures([73, 74, 75, 71, 69, 72, 76, 73])) # [1, 1, 4, 2, 1, 1, 0, 0] Let us trace through the first few steps with [73, 74, 75, 71, 69, 72, 76, 73]:\ni temp Action Stack (indices) answer so far 0 73 Push 0 [0] [0,0,0,0,0,0,0,0] 1 74 74\u0026gt;73: pop 0, answer[0]=1-0=1. Push 1 [1] [1,0,0,0,0,0,0,0] 2 75 75\u0026gt;74: pop 1, answer[1]=2-1=1. Push 2 [2] [1,1,0,0,0,0,0,0] 3 71 71 not \u0026gt; 75. Push 3 [2,3] [1,1,0,0,0,0,0,0] 4 69 69 not \u0026gt; 71. Push 4 [2,3,4] [1,1,0,0,0,0,0,0] 5 72 72\u0026gt;69: pop 4, answer[4]=5-4=1. 72\u0026gt;71: pop 3, answer[3]=5-3=2. 72 not \u0026gt; 75. Push 5 [2,5] [1,1,0,2,1,0,0,0] 6 76 76\u0026gt;72: pop 5, answer[5]=6-5=1. 76\u0026gt;75: pop 2, answer[2]=6-2=4. Push 6 [6] [1,1,4,2,1,1,0,0] 7 73 73 not \u0026gt; 76. Push 7 [6,7] [1,1,4,2,1,1,0,0] Final answer: [1, 1, 4, 2, 1, 1, 0, 0]\nSummary What We Covered Topic Key Point Stack and LIFO Last element pushed is first to be popped List-based stack Use Python list, append is push, pop is pop, -1 index is peek Linked list stack O(1) push and pop without amortised cost, slightly more complex Call stack Python pushes a frame for each function call, pops on return Stack overflow Too many recursive frames cause RecursionError Monotonic stack Keep elements sorted by popping violators \u0026ndash; O(n) for next greater/smaller problems Operation Complexity Operation Time Complexity Space Complexity push O(1) amortised O(1) pop O(1) O(1) peek O(1) O(1) is_empty O(1) O(1) Stack of n elements \u0026ndash; O(n) Monotonic stack pass O(n) O(n) Patterns Introduced Pattern Core Idea Use When Matching and Validation Push openers, pop and verify on closers Balanced brackets, nested structure validation State Tracking Stack remembers past states, pop to revert Min/max tracking, undo, expression evaluation Monotonic Stack Maintain sorted order, pop on violation Next greater/smaller element, temperatures, histogram Key Takeaways A stack is LIFO. You only ever interact with the top. All four operations are O(1). In Python, a plain list is all you need for a stack. append() is push, pop() is pop, [-1] is peek. The call stack is a real stack. Every function call is a push, every return is a pop. Recursion depth is literally stack depth. The monotonic stack is O(n) even though it has a while loop inside a for loop. Each element is pushed once and popped at most once \u0026ndash; total work is 2n. When using a monotonic stack, store indices rather than values. Indices let you compute distances and access the original values. The Min Stack problem teaches a general principle: when you need O(1) access to some aggregate (min, max, sum) of the current stack contents, maintain a parallel stack that tracks that aggregate. Practice Problems Problem Link Difficulty Pattern Valid Parentheses LC 20 Easy Matching and Validation Min Stack LC 155 Medium State Tracking Daily Temperatures LC 739 Medium Monotonic Stack Implement Stack using Queues LC 225 Easy Stack fundamentals Baseball Game LC 682 Easy State Tracking Backspace String Compare LC 844 Easy State Tracking Next Greater Element I LC 496 Easy Monotonic Stack Next Greater Element II LC 503 Medium Monotonic Stack Largest Rectangle in Histogram LC 84 Hard Monotonic Stack Trapping Rain Water LC 42 Hard Monotonic Stack / Two Pointers Next Steps Next blog: [Queues and Deques] \u0026ndash; a data structure where elements are added at one end and removed from the other, following First In First Out order Reference: Codeverra DSA Sheet ","permalink":"https://blogs.codeverra.com/blog/dsa-07-stacks/","summary":"\u003ch1 id=\"stacks\"\u003eStacks\u003c/h1\u003e\n\u003chr\u003e\n\u003ch2 id=\"table-of-contents\"\u003eTable of Contents\u003c/h2\u003e\n\u003cul\u003e\n\u003cli\u003e\u003ca href=\"#what-you-will-learn-in-this-blog\"\u003eWhat You Will Learn in This Blog\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#what-is-a-stack\"\u003eWhat is a Stack\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#core-stack-operations\"\u003eCore Stack Operations\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#implementing-a-stack\"\u003eImplementing a Stack\u003c/a\u003e\n\u003cul\u003e\n\u003cli\u003e\u003ca href=\"#implementation-1----using-a-python-list\"\u003eImplementation 1 \u0026ndash; Using a Python List\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#implementation-2----using-a-linked-list\"\u003eImplementation 2 \u0026ndash; Using a Linked List\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#which-implementation-to-use\"\u003eWhich Implementation to Use\u003c/a\u003e\u003c/li\u003e\n\u003c/ul\u003e\n\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#the-call-stack\"\u003eThe Call Stack\u003c/a\u003e\n\u003cul\u003e\n\u003cli\u003e\u003ca href=\"#how-python-uses-a-stack-for-function-calls\"\u003eHow Python Uses a Stack for Function Calls\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#stack-overflow\"\u003eStack Overflow\u003c/a\u003e\u003c/li\u003e\n\u003c/ul\u003e\n\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#monotonic-stack\"\u003eMonotonic Stack\u003c/a\u003e\n\u003cul\u003e\n\u003cli\u003e\u003ca href=\"#what-is-a-monotonic-stack\"\u003eWhat is a Monotonic Stack\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#monotonic-increasing-stack\"\u003eMonotonic Increasing Stack\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#monotonic-decreasing-stack\"\u003eMonotonic Decreasing Stack\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#the-template\"\u003eThe Template\u003c/a\u003e\u003c/li\u003e\n\u003c/ul\u003e\n\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#time-and-space-complexity\"\u003eTime and Space Complexity\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#patterns-and-techniques\"\u003ePatterns and Techniques\u003c/a\u003e\n\u003cul\u003e\n\u003cli\u003e\u003ca href=\"#pattern-1----stack-for-matching-and-validation\"\u003ePattern 1 \u0026ndash; Stack for Matching and Validation\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#pattern-2----stack-for-state-tracking\"\u003ePattern 2 \u0026ndash; Stack for State Tracking\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#pattern-3----monotonic-stack\"\u003ePattern 3 \u0026ndash; Monotonic Stack\u003c/a\u003e\u003c/li\u003e\n\u003c/ul\u003e\n\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#summary\"\u003eSummary\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#key-takeaways\"\u003eKey Takeaways\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#practice-problems\"\u003ePractice Problems\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#next-steps\"\u003eNext Steps\u003c/a\u003e\u003c/li\u003e\n\u003c/ul\u003e\n\u003chr\u003e\n\u003ch2 id=\"what-you-will-learn-in-this-blog\"\u003eWhat You Will Learn in This Blog\u003c/h2\u003e\n\u003cul\u003e\n\u003cli\u003eWhat a stack is and the LIFO principle\u003c/li\u003e\n\u003cli\u003eThe four core operations: push, pop, peek, and isEmpty\u003c/li\u003e\n\u003cli\u003eTwo ways to implement a stack from scratch \u0026ndash; using a list and using a linked list\u003c/li\u003e\n\u003cli\u003eHow Python uses a stack internally for function calls and recursion\u003c/li\u003e\n\u003cli\u003eWhat a monotonic stack is and when to reach for it\u003c/li\u003e\n\u003cli\u003eThree problem-solving patterns that use stacks\u003c/li\u003e\n\u003cli\u003eSolved LeetCode problems with full explanation\u003c/li\u003e\n\u003cli\u003eA practice problem table for further study\u003c/li\u003e\n\u003c/ul\u003e\n\u003chr\u003e\n\u003ch2 id=\"what-is-a-stack\"\u003eWhat is a Stack\u003c/h2\u003e\n\u003cp\u003eA stack is a linear data structure that follows the \u003cstrong\u003eLast In, First Out\u003c/strong\u003e principle \u0026ndash; LIFO. The last element added is the first one to be removed.\u003c/p\u003e","tags":["dsa"],"title":"Stacks in Python  -  Complete Guide"},{"content":"Strings Table of Contents What You Will Learn in This Blog What is a String ASCII Unicode Strings in Python \u0026ndash; Immutability String Operations and Time Complexity Common String Methods Strings as Arrays Patterns and Techniques Pattern 1 \u0026ndash; Two Pointers on Strings Pattern 2 \u0026ndash; Sliding Window on Strings Pattern 3 \u0026ndash; Hashing for String Problems Summary Key Takeaways Practice Problems Next Steps What You Will Learn in This Blog What a string is at the byte level \u0026ndash; characters, ASCII, and Unicode How Python represents strings in memory Why strings in Python are immutable and what that means in practice All common string operations and their time complexity How to build strings efficiently and avoid a common O(n^2) trap Three patterns applied to strings: Sliding Window, Two Pointers, and Hashing Solved LeetCode problems for each pattern with full explanation A practice problem table for further study What is a String At the most fundamental level, a string is a sequence of characters. But computers do not natively understand characters \u0026ndash; they only understand numbers. So every character is mapped to a number, and that number is stored in memory.\nASCII The original encoding standard is ASCII (American Standard Code for Information Interchange). It maps 128 characters to numbers 0 through 127.\nSome important ASCII values to know \u0026ndash; you will use these in DSA problems:\nCharacter range ASCII values Notes \u0026lsquo;0\u0026rsquo; to \u0026lsquo;9\u0026rsquo; 48 to 57 Digits \u0026lsquo;A\u0026rsquo; to \u0026lsquo;Z\u0026rsquo; 65 to 90 Uppercase letters \u0026lsquo;a\u0026rsquo; to \u0026lsquo;z\u0026rsquo; 97 to 122 Lowercase letters \u0026lsquo;a\u0026rsquo; - \u0026lsquo;A\u0026rsquo; 32 Difference between lowercase and uppercase In Python, you convert between a character and its ASCII value using ord() and chr():\n# ord() gives the ASCII (Unicode) value of a character print(ord(\u0026#39;a\u0026#39;)) # 97 print(ord(\u0026#39;z\u0026#39;)) # 122 print(ord(\u0026#39;A\u0026#39;)) # 65 print(ord(\u0026#39;0\u0026#39;)) # 48 # chr() gives the character for a given value print(chr(97)) # \u0026#39;a\u0026#39; print(chr(65)) # \u0026#39;A\u0026#39; # A very common DSA trick: get the position of a letter in the alphabet # \u0026#39;a\u0026#39; is position 0, \u0026#39;b\u0026#39; is position 1, etc. def letter_position(c): return ord(c) - ord(\u0026#39;a\u0026#39;) print(letter_position(\u0026#39;a\u0026#39;)) # 0 print(letter_position(\u0026#39;e\u0026#39;)) # 4 print(letter_position(\u0026#39;z\u0026#39;)) # 25 This trick \u0026ndash; ord(c) - ord('a') \u0026ndash; appears constantly in character frequency problems. It lets you use an array of size 26 as a frequency table instead of a dictionary.\nUnicode ASCII only covers 128 characters, which is enough for English but not for other languages. Unicode is the modern standard that covers over 140,000 characters across all human languages, symbols, and emoji.\nPython 3 strings are Unicode by default. This means you can work with any language without any special setup.\ns = \u0026#34;hello\u0026#34; # ASCII characters -- each stored in 1 byte internally s = \u0026#34;こんにちは\u0026#34; # Japanese characters -- Unicode s = \u0026#34;مرحبا\u0026#34; # Arabic -- Unicode s = \u0026#34;🐍\u0026#34; # Emoji -- Unicode # len() counts characters, not bytes print(len(\u0026#34;hello\u0026#34;)) # 5 print(len(\u0026#34;こんにちは\u0026#34;)) # 5 For DSA problems, you will almost always be working with lowercase English letters (\u0026lsquo;a\u0026rsquo; to \u0026lsquo;z\u0026rsquo;) or alphanumeric characters. Unicode matters more in production software than in algorithmic problems.\nStrings in Python \u0026ndash; Immutability In Python, strings are immutable. Once a string is created, it cannot be changed. Any operation that appears to modify a string actually creates a new string object in memory.\ns = \u0026#34;hello\u0026#34; print(id(s)) # memory address of s, e.g. 140234567890 s = s + \u0026#34; world\u0026#34; # this does NOT modify \u0026#34;hello\u0026#34; # it creates a NEW string \u0026#34;hello world\u0026#34; and s now points to it print(id(s)) # different memory address -- it is a new object # You cannot modify a character in place -- this raises a TypeError s = \u0026#34;hello\u0026#34; s[0] = \u0026#34;H\u0026#34; # TypeError: \u0026#39;str\u0026#39; object does not support item assignment This is fundamentally different from lists, which are mutable:\n# List -- mutable, can be changed in place nums = [1, 2, 3] nums[0] = 99 # works fine print(nums) # [99, 2, 3] # String -- immutable, cannot be changed in place s = \u0026#34;hello\u0026#34; s[0] = \u0026#34;H\u0026#34; # TypeError Why does immutability matter for DSA?\nIt has a direct impact on performance. Because you cannot modify a string in place, building a string character by character in a loop is expensive if done naively.\n# The wrong way -- O(n^2) # Each += creates a brand new string and copies everything result = \u0026#34;\u0026#34; for char in [\u0026#34;h\u0026#34;, \u0026#34;e\u0026#34;, \u0026#34;l\u0026#34;, \u0026#34;l\u0026#34;, \u0026#34;o\u0026#34;]: result += char # creates a new string each time # The right way -- O(n) # Accumulate parts in a list, join once at the end parts = [] for char in [\u0026#34;h\u0026#34;, \u0026#34;e\u0026#34;, \u0026#34;l\u0026#34;, \u0026#34;l\u0026#34;, \u0026#34;o\u0026#34;]: parts.append(char) # O(1) append result = \u0026#34;\u0026#34;.join(parts) # one O(n) join at the end print(result) # \u0026#34;hello\u0026#34; This is one of the most common performance mistakes beginners make with strings. Always use \u0026quot;\u0026quot;.join(list) when building strings in a loop.\nString Operations and Time Complexity Operation Example Time Complexity Notes Access by index s[i] O(1) Length len(s) O(1) Stored internally Slice s[i:j] O(k) k = j - i, creates a new string Concatenation s1 + s2 O(n + m) Creates a new string Search substring s.find(t) O(n * m) n = len(s), m = len(t) Membership t in s O(n * m) Same as find Split s.split() O(n) Creates a list of substrings Join \u0026quot;\u0026quot;.join(lst) O(n) n = total characters Replace s.replace(a, b) O(n) Scans entire string Strip s.strip() O(n) Scans from both ends Upper / Lower s.upper() O(n) Creates a new string String building in loop with += s += c (in loop) O(n^2) Avoid this String building with join \u0026quot;\u0026quot;.join(parts) O(n) Always prefer this Note on slicing: Every time you slice a string in Python, a new string object is created. This means s[i:j] costs O(k) in both time and space where k is the length of the slice. This is different from some other languages where slices are views into the original string.\nCommon String Methods These are the methods you will use most often in DSA problems.\ns = \u0026#34; Hello, World! \u0026#34; # Case conversion print(s.lower()) # \u0026#34; hello, world! \u0026#34; print(s.upper()) # \u0026#34; HELLO, WORLD! \u0026#34; # Whitespace removal print(s.strip()) # \u0026#34;Hello, World!\u0026#34; print(s.lstrip()) # \u0026#34;Hello, World! \u0026#34; (left only) print(s.rstrip()) # \u0026#34; Hello, World!\u0026#34; (right only) # Search and check print(s.find(\u0026#34;World\u0026#34;)) # 9 (index of first occurrence, -1 if not found) print(s.count(\u0026#34;l\u0026#34;)) # 3 (count of occurrences) print(s.startswith(\u0026#34; H\u0026#34;)) # True print(s.endswith(\u0026#34;! \u0026#34;)) # True # Split and join words = \u0026#34;apple,banana,cherry\u0026#34;.split(\u0026#34;,\u0026#34;) print(words) # [\u0026#39;apple\u0026#39;, \u0026#39;banana\u0026#39;, \u0026#39;cherry\u0026#39;] print(\u0026#34;,\u0026#34;.join(words)) # \u0026#34;apple,banana,cherry\u0026#34; sentence = \u0026#34;hello world python\u0026#34; print(sentence.split()) # [\u0026#39;hello\u0026#39;, \u0026#39;world\u0026#39;, \u0026#39;python\u0026#39;] (splits on whitespace) # Replace print(\u0026#34;hello world\u0026#34;.replace(\u0026#34;world\u0026#34;, \u0026#34;python\u0026#34;)) # \u0026#34;hello python\u0026#34; # Check character types print(\u0026#34;abc\u0026#34;.isalpha()) # True -- all alphabetic print(\u0026#34;123\u0026#34;.isdigit()) # True -- all digits print(\u0026#34;abc123\u0026#34;.isalnum()) # True -- all alphanumeric print(\u0026#34; \u0026#34;.isspace()) # True -- all whitespace print(\u0026#34;Hello\u0026#34;.islower()) # False print(\u0026#34;Hello\u0026#34;.isupper()) # False print(\u0026#34;HELLO\u0026#34;.isupper()) # True Strings as Arrays Even though strings are immutable, you can treat them like arrays for reading purposes \u0026ndash; index into them, slice them, iterate over them.\ns = \u0026#34;hello\u0026#34; # Indexing -- O(1) print(s[0]) # \u0026#39;h\u0026#39; print(s[-1]) # \u0026#39;o\u0026#39; # Slicing -- O(k) print(s[1:4]) # \u0026#39;ell\u0026#39; print(s[::-1]) # \u0026#39;olleh\u0026#39; -- reversed string # Iterating for char in s: print(char) # Enumerate -- gives index and character for i, char in enumerate(s): print(i, char) # 0 h # 1 e # 2 l # 3 l # 4 o # Convert to list if you need to modify chars = list(s) # [\u0026#39;h\u0026#39;, \u0026#39;e\u0026#39;, \u0026#39;l\u0026#39;, \u0026#39;l\u0026#39;, \u0026#39;o\u0026#39;] chars[0] = \u0026#39;H\u0026#39; # now you can modify result = \u0026#34;\u0026#34;.join(chars) # \u0026#39;Hello\u0026#39; Patterns and Techniques The three patterns we introduce here are the same ones introduced in the Arrays blog. If you have not read that blog yet, the links below point to the original explanations.\nTwo Pointers \u0026ndash; Arrays blog Sliding Window \u0026ndash; Arrays blog Hashing / Frequency Counting The goal here is to see the same patterns applied to strings. Seeing a pattern in two different contexts is one of the best ways to truly internalise it.\nPattern 1 \u0026ndash; Two Pointers on Strings The core idea (same as arrays): use two index variables moving through the string, typically from opposite ends toward the middle. This avoids creating a reversed copy and keeps space at O(1).\nWhen to use it on strings:\nPalindrome checking Reversing a string or parts of a string in place (on a character list) Comparing characters from both ends The template:\nleft, right = 0, len(s) - 1 while left \u0026lt; right: # compare s[left] and s[right] # move pointers based on the condition left += 1 right -= 1 Solved Problem \u0026ndash; Valid Palindrome II LeetCode 680 \u0026ndash; Valid Palindrome II\nProblem: Given a string s, return True if it can become a palindrome after deleting at most one character.\nInput: s = \u0026#34;abca\u0026#34; Output: True (remove \u0026#39;b\u0026#39; or \u0026#39;c\u0026#39; to get \u0026#34;aca\u0026#34; or \u0026#34;aba\u0026#34;) Input: s = \u0026#34;abc\u0026#34; Output: False Thinking through it:\nUse two pointers from both ends. As long as characters match, move both pointers inward. When you hit a mismatch, you have one delete to use \u0026ndash; try skipping the left character or the right character and check if either resulting substring is a palindrome.\ndef valid_palindrome(s: str) -\u0026gt; bool: \u0026#34;\u0026#34;\u0026#34; Two pointers from both ends. On first mismatch, try skipping left character or right character. If either skip produces a palindrome, return True. Time complexity: O(n) -- we scan the string at most twice Space complexity: O(1) -- only index variables, no new strings stored (the is_palindrome check uses indices, not slices) \u0026#34;\u0026#34;\u0026#34; def is_palindrome(s, left, right): \u0026#34;\u0026#34;\u0026#34;Check if s[left..right] is a palindrome using two pointers.\u0026#34;\u0026#34;\u0026#34; while left \u0026lt; right: if s[left] != s[right]: return False left += 1 right -= 1 return True left, right = 0, len(s) - 1 while left \u0026lt; right: if s[left] != s[right]: # Mismatch -- try skipping left character or right character # If either works, we can make a palindrome with one deletion return (is_palindrome(s, left + 1, right) or is_palindrome(s, left, right - 1)) left += 1 right -= 1 return True # no mismatch found -- already a palindrome # Test print(valid_palindrome(\u0026#34;abca\u0026#34;)) # True print(valid_palindrome(\u0026#34;abc\u0026#34;)) # False print(valid_palindrome(\u0026#34;racecar\u0026#34;)) # True (already a palindrome) print(valid_palindrome(\u0026#34;deeee\u0026#34;)) # True (remove one \u0026#39;e\u0026#39;) Solved Problem \u0026ndash; Reverse String LeetCode 344 \u0026ndash; Reverse String\nProblem: Write a function that reverses a string. The input is given as an array of characters s. You must do it in-place with O(1) extra memory.\nInput: s = [\u0026#39;h\u0026#39;,\u0026#39;e\u0026#39;,\u0026#39;l\u0026#39;,\u0026#39;l\u0026#39;,\u0026#39;o\u0026#39;] Output: [\u0026#39;o\u0026#39;,\u0026#39;l\u0026#39;,\u0026#39;l\u0026#39;,\u0026#39;e\u0026#39;,\u0026#39;h\u0026#39;] Thinking through it:\nClassic two pointers. Swap the characters at left and right, then move both pointers inward. Stop when they meet in the middle.\ndef reverse_string(s: list) -\u0026gt; None: \u0026#34;\u0026#34;\u0026#34; Swap characters from both ends moving inward. Modifies the list in place -- nothing is returned. Time complexity: O(n) -- each character is visited once Space complexity: O(1) -- swapping in place, no extra memory \u0026#34;\u0026#34;\u0026#34; left, right = 0, len(s) - 1 while left \u0026lt; right: # Swap characters at left and right s[left], s[right] = s[right], s[left] left += 1 right -= 1 # Test s = [\u0026#39;h\u0026#39;,\u0026#39;e\u0026#39;,\u0026#39;l\u0026#39;,\u0026#39;l\u0026#39;,\u0026#39;o\u0026#39;] reverse_string(s) print(s) # [\u0026#39;o\u0026#39;, \u0026#39;l\u0026#39;, \u0026#39;l\u0026#39;, \u0026#39;e\u0026#39;, \u0026#39;h\u0026#39;] Pattern 2 \u0026ndash; Sliding Window on Strings The core idea (same as arrays): maintain a window defined by two pointers left and right that slides across the string. Expand by moving right forward, shrink by moving left forward. This avoids checking every substring from scratch and gives O(n) solutions to problems that would otherwise be O(n^2) or worse.\nWhen to use it on strings:\nLongest substring with some constraint (no repeating characters, at most k distinct characters) Smallest substring containing all required characters Counting substrings that satisfy a condition The variable window template for strings:\nfrom collections import defaultdict def sliding_window_string(s): left = 0 window = defaultdict(int) # track character counts in the window result = 0 for right in range(len(s)): # Expand: add s[right] to the window window[s[right]] += 1 # Shrink: while window violates the condition, move left forward while # window condition is violated: window[s[left]] -= 1 if window[s[left]] == 0: del window[s[left]] left += 1 # Window is now valid -- update result result = max(result, right - left + 1) return result Solved Problem \u0026ndash; Longest Substring Without Repeating Characters LeetCode 3 \u0026ndash; Longest Substring Without Repeating Characters\nProblem: Given a string s, find the length of the longest substring without repeating characters.\nInput: s = \u0026#34;abcabcbb\u0026#34; Output: 3 (substring \u0026#34;abc\u0026#34;) Input: s = \u0026#34;bbbbb\u0026#34; Output: 1 (substring \u0026#34;b\u0026#34;) Input: s = \u0026#34;pwwkew\u0026#34; Output: 3 (substring \u0026#34;wke\u0026#34;) Thinking through it:\nThe brute force checks every substring \u0026ndash; O(n^2) or O(n^3). Using a sliding window, we maintain a window that never contains a duplicate. When we add a character that is already in the window, we shrink from the left until the duplicate is gone. We track the maximum window size seen.\ndef length_of_longest_substring(s: str) -\u0026gt; int: \u0026#34;\u0026#34;\u0026#34; Sliding window with a set tracking characters in the current window. Expand right, shrink left when a duplicate is found. Time complexity: O(n) -- each character is added and removed at most once Space complexity: O(min(n, a)) -- a is the size of the character set (26 for lowercase letters, 128 for ASCII) \u0026#34;\u0026#34;\u0026#34; char_set = set() # characters currently in the window left = 0 max_length = 0 for right in range(len(s)): # Shrink window from left until s[right] is no longer a duplicate while s[right] in char_set: char_set.remove(s[left]) # remove leftmost character left += 1 # shrink window # Now s[right] is not in the window -- safe to add it char_set.add(s[right]) # Update maximum length max_length = max(max_length, right - left + 1) return max_length # Test print(length_of_longest_substring(\u0026#34;abcabcbb\u0026#34;)) # 3 print(length_of_longest_substring(\u0026#34;bbbbb\u0026#34;)) # 1 print(length_of_longest_substring(\u0026#34;pwwkew\u0026#34;)) # 3 print(length_of_longest_substring(\u0026#34;\u0026#34;)) # 0 Let us trace through \u0026quot;abcabcbb\u0026quot; to make sure the logic is clear:\nright s[right] char_set before Action char_set after left max_length 0 \u0026lsquo;a\u0026rsquo; {} add \u0026lsquo;a\u0026rsquo; {\u0026lsquo;a\u0026rsquo;} 0 1 1 \u0026lsquo;b\u0026rsquo; {\u0026lsquo;a\u0026rsquo;} add \u0026lsquo;b\u0026rsquo; {\u0026lsquo;a\u0026rsquo;,\u0026lsquo;b\u0026rsquo;} 0 2 2 \u0026lsquo;c\u0026rsquo; {\u0026lsquo;a\u0026rsquo;,\u0026lsquo;b\u0026rsquo;} add \u0026lsquo;c\u0026rsquo; {\u0026lsquo;a\u0026rsquo;,\u0026lsquo;b\u0026rsquo;,\u0026lsquo;c\u0026rsquo;} 0 3 3 \u0026lsquo;a\u0026rsquo; {\u0026lsquo;a\u0026rsquo;,\u0026lsquo;b\u0026rsquo;,\u0026lsquo;c\u0026rsquo;} remove \u0026lsquo;a\u0026rsquo;, left=1, add \u0026lsquo;a\u0026rsquo; {\u0026lsquo;b\u0026rsquo;,\u0026lsquo;c\u0026rsquo;,\u0026lsquo;a\u0026rsquo;} 1 3 4 \u0026lsquo;b\u0026rsquo; {\u0026lsquo;b\u0026rsquo;,\u0026lsquo;c\u0026rsquo;,\u0026lsquo;a\u0026rsquo;} remove \u0026lsquo;b\u0026rsquo;, left=2, add \u0026lsquo;b\u0026rsquo; {\u0026lsquo;c\u0026rsquo;,\u0026lsquo;a\u0026rsquo;,\u0026lsquo;b\u0026rsquo;} 2 3 5 \u0026lsquo;c\u0026rsquo; {\u0026lsquo;c\u0026rsquo;,\u0026lsquo;a\u0026rsquo;,\u0026lsquo;b\u0026rsquo;} remove \u0026lsquo;c\u0026rsquo;, left=3, add \u0026lsquo;c\u0026rsquo; {\u0026lsquo;a\u0026rsquo;,\u0026lsquo;b\u0026rsquo;,\u0026lsquo;c\u0026rsquo;} 3 3 6 \u0026lsquo;b\u0026rsquo; {\u0026lsquo;a\u0026rsquo;,\u0026lsquo;b\u0026rsquo;,\u0026lsquo;c\u0026rsquo;} remove \u0026lsquo;a\u0026rsquo;, remove \u0026lsquo;b\u0026rsquo;, left=5, add \u0026lsquo;b\u0026rsquo; {\u0026lsquo;c\u0026rsquo;,\u0026lsquo;b\u0026rsquo;} 5 3 7 \u0026lsquo;b\u0026rsquo; {\u0026lsquo;c\u0026rsquo;,\u0026lsquo;b\u0026rsquo;} remove \u0026lsquo;c\u0026rsquo;, remove \u0026lsquo;b\u0026rsquo;, left=7, add \u0026lsquo;b\u0026rsquo; {\u0026lsquo;b\u0026rsquo;} 7 3 Solved Problem \u0026ndash; Minimum Window Substring LeetCode 76 \u0026ndash; Minimum Window Substring\nProblem: Given two strings s and t, return the minimum window substring of s such that every character in t (including duplicates) is included in the window. If no such window exists, return \u0026quot;\u0026quot;.\nInput: s = \u0026#34;ADOBECODEBANC\u0026#34;, t = \u0026#34;ABC\u0026#34; Output: \u0026#34;BANC\u0026#34; Input: s = \u0026#34;a\u0026#34;, t = \u0026#34;a\u0026#34; Output: \u0026#34;a\u0026#34; Input: s = \u0026#34;a\u0026#34;, t = \u0026#34;aa\u0026#34; Output: \u0026#34;\u0026#34; Thinking through it:\nThis is a variable sliding window problem. We need to find the smallest window in s that contains all characters of t. We expand the window by moving right forward. When the window contains all characters of t, we try to shrink it from the left to find the minimum. We use two frequency maps \u0026ndash; one for what we need (t_count) and one for what the current window has (window_count).\nfrom collections import Counter def min_window(s: str, t: str) -\u0026gt; str: \u0026#34;\u0026#34;\u0026#34; Sliding window with two frequency maps. Expand right until window contains all of t. Then shrink left to find the minimum valid window. Repeat until right reaches the end. Time complexity: O(n + m) -- n = len(s), m = len(t) each character in s is added and removed at most once Space complexity: O(m) -- the frequency maps store at most m distinct characters \u0026#34;\u0026#34;\u0026#34; if not t or not s: return \u0026#34;\u0026#34; # Frequency map for characters we need t_count = Counter(t) # Number of unique characters in t that we still need to satisfy required = len(t_count) # Sliding window pointers and state left = 0 formed = 0 # how many unique chars in t are currently satisfied in window window_count = {} # frequency of characters in current window # Result: (window length, left index, right index) result = float(\u0026#39;inf\u0026#39;), 0, 0 for right in range(len(s)): # Add s[right] to the window char = s[right] window_count[char] = window_count.get(char, 0) + 1 # Check if this character\u0026#39;s frequency now satisfies t\u0026#39;s requirement if char in t_count and window_count[char] == t_count[char]: formed += 1 # Try to shrink the window from left while it is still valid while left \u0026lt;= right and formed == required: # This window is valid -- update result if it is smaller window_length = right - left + 1 if window_length \u0026lt; result[0]: result = (window_length, left, right) # Remove s[left] from the window left_char = s[left] window_count[left_char] -= 1 if left_char in t_count and window_count[left_char] \u0026lt; t_count[left_char]: formed -= 1 # window no longer satisfies this character left += 1 if result[0] == float(\u0026#39;inf\u0026#39;): return \u0026#34;\u0026#34; return s[result[1] : result[2] + 1] # Test print(min_window(\u0026#34;ADOBECODEBANC\u0026#34;, \u0026#34;ABC\u0026#34;)) # \u0026#34;BANC\u0026#34; print(min_window(\u0026#34;a\u0026#34;, \u0026#34;a\u0026#34;)) # \u0026#34;a\u0026#34; print(min_window(\u0026#34;a\u0026#34;, \u0026#34;aa\u0026#34;)) # \u0026#34;\u0026#34; Pattern 3 \u0026ndash; Hashing for String Problems The core idea: Use a dictionary or array of size 26 to count character frequencies. Many string problems reduce to comparing or manipulating these frequency counts. This turns O(n * m) brute force comparisons into O(n + m).\nWhen to use it:\nAnagram detection (do two strings have the same characters?) Grouping strings by their character composition Finding strings that are permutations of each other Any problem involving character frequency Two approaches to frequency counting:\nfrom collections import Counter s = \u0026#34;hello\u0026#34; # Approach 1: Counter (cleanest) freq = Counter(s) print(freq) # Counter({\u0026#39;l\u0026#39;: 2, \u0026#39;h\u0026#39;: 1, \u0026#39;e\u0026#39;: 1, \u0026#39;o\u0026#39;: 1}) # Approach 2: Dictionary manually freq = {} for char in s: freq[char] = freq.get(char, 0) + 1 # Approach 3: Fixed array of size 26 (most efficient for lowercase letters) # Index 0 = \u0026#39;a\u0026#39;, index 1 = \u0026#39;b\u0026#39;, ..., index 25 = \u0026#39;z\u0026#39; freq = [0] * 26 for char in s: freq[ord(char) - ord(\u0026#39;a\u0026#39;)] += 1 print(freq) # [0, 0, 0, 0, 1, 0, 1, 0, 0, 0, 0, 2, 0, 0, 1, 0, ...] # e h l l o Solved Problem \u0026ndash; Valid Anagram LeetCode 242 \u0026ndash; Valid Anagram\nProblem: Given two strings s and t, return True if t is an anagram of s, and False otherwise. An anagram uses the same characters the same number of times.\nInput: s = \u0026#34;anagram\u0026#34;, t = \u0026#34;nagaram\u0026#34; Output: True Input: s = \u0026#34;rat\u0026#34;, t = \u0026#34;car\u0026#34; Output: False Thinking through it:\nTwo strings are anagrams if and only if they have exactly the same character frequencies. We have three approaches, each worth knowing.\nfrom collections import Counter def is_anagram_v1(s: str, t: str) -\u0026gt; bool: \u0026#34;\u0026#34;\u0026#34; Approach 1: Sort both strings and compare. Two strings are anagrams if their sorted versions are equal. Time complexity: O(n log n) -- sorting Space complexity: O(n) -- sorted() creates new strings \u0026#34;\u0026#34;\u0026#34; return sorted(s) == sorted(t) def is_anagram_v2(s: str, t: str) -\u0026gt; bool: \u0026#34;\u0026#34;\u0026#34; Approach 2: Use Counter to compare frequency maps. Clean and Pythonic. Time complexity: O(n + m) -- build two counters, compare them Space complexity: O(1) -- at most 26 distinct characters (lowercase letters) \u0026#34;\u0026#34;\u0026#34; return Counter(s) == Counter(t) def is_anagram_v3(s: str, t: str) -\u0026gt; bool: \u0026#34;\u0026#34;\u0026#34; Approach 3: Single frequency array of size 26. Increment for each character in s, decrement for each in t. If all values are 0 at the end, they are anagrams. Time complexity: O(n + m) Space complexity: O(1) -- fixed array of size 26 regardless of input size \u0026#34;\u0026#34;\u0026#34; if len(s) != len(t): return False count = [0] * 26 for i in range(len(s)): count[ord(s[i]) - ord(\u0026#39;a\u0026#39;)] += 1 # increment for s count[ord(t[i]) - ord(\u0026#39;a\u0026#39;)] -= 1 # decrement for t # If all counts are 0, every character balanced out return all(c == 0 for c in count) # Test all three for fn in [is_anagram_v1, is_anagram_v2, is_anagram_v3]: print(fn(\u0026#34;anagram\u0026#34;, \u0026#34;nagaram\u0026#34;)) # True print(fn(\u0026#34;rat\u0026#34;, \u0026#34;car\u0026#34;)) # False Solved Problem \u0026ndash; Group Anagrams LeetCode 49 \u0026ndash; Group Anagrams\nProblem: Given an array of strings strs, group the anagrams together. You can return the answer in any order.\nInput: strs = [\u0026#34;eat\u0026#34;,\u0026#34;tea\u0026#34;,\u0026#34;tan\u0026#34;,\u0026#34;ate\u0026#34;,\u0026#34;nat\u0026#34;,\u0026#34;bat\u0026#34;] Output: [[\u0026#34;bat\u0026#34;],[\u0026#34;nat\u0026#34;,\u0026#34;tan\u0026#34;],[\u0026#34;ate\u0026#34;,\u0026#34;eat\u0026#34;,\u0026#34;tea\u0026#34;]] Thinking through it:\nTwo strings are anagrams if their sorted versions are equal. So we can use the sorted version of each string as a key in a dictionary and group strings with the same key together.\nfrom collections import defaultdict def group_anagrams(strs: list) -\u0026gt; list: \u0026#34;\u0026#34;\u0026#34; Sort each string to get a canonical key. All anagrams will have the same key. Group strings by their key using a defaultdict. Time complexity: O(n * k log k) -- n strings, each of length at most k, sorted Space complexity: O(n * k) -- storing all strings in the groups dictionary \u0026#34;\u0026#34;\u0026#34; groups = defaultdict(list) for s in strs: key = tuple(sorted(s)) # sorted string as the key # use tuple because lists cannot be dict keys groups[key].append(s) return list(groups.values()) # Test print(group_anagrams([\u0026#34;eat\u0026#34;,\u0026#34;tea\u0026#34;,\u0026#34;tan\u0026#34;,\u0026#34;ate\u0026#34;,\u0026#34;nat\u0026#34;,\u0026#34;bat\u0026#34;])) # [[\u0026#39;eat\u0026#39;, \u0026#39;tea\u0026#39;, \u0026#39;ate\u0026#39;], [\u0026#39;tan\u0026#39;, \u0026#39;nat\u0026#39;], [\u0026#39;bat\u0026#39;]] Alternative approach using character count as key:\ndef group_anagrams_v2(strs: list) -\u0026gt; list: \u0026#34;\u0026#34;\u0026#34; Use a frequency array of size 26 as the key instead of sorting. This is slightly faster: O(n * k) instead of O(n * k log k). Time complexity: O(n * k) -- n strings, each of length at most k Space complexity: O(n * k) \u0026#34;\u0026#34;\u0026#34; groups = defaultdict(list) for s in strs: # Build frequency array for this string count = [0] * 26 for char in s: count[ord(char) - ord(\u0026#39;a\u0026#39;)] += 1 key = tuple(count) # convert to tuple so it can be used as a dict key groups[key].append(s) return list(groups.values()) Summary What We Covered Topic Key Point Characters and ASCII Every character maps to a number. ord() and chr() convert between them Unicode Python 3 strings support all languages. For DSA, assume lowercase English unless stated Immutability Strings cannot be changed in place. Every modification creates a new string String building Never use += in a loop. Append to a list and join at the end Strings as arrays You can index, slice, and iterate over strings like arrays, but not modify them Operation Complexity at a Glance Operation Complexity Notes Access by index s[i] O(1) Length len(s) O(1) Slice s[i:j] O(k) Creates a new string Concatenation s1 + s2 O(n + m) Creates a new string String build with += in loop O(n^2) Avoid String build with join O(n) Always prefer Search t in s O(n * m) Split, replace, upper, lower O(n) Patterns Applied to Strings Pattern Core Idea Time Complexity Use When Two Pointers Two indices from both ends moving inward O(n) Palindrome, reverse, pair matching Sliding Window Expand right, shrink left, maintain window state O(n) Longest/shortest substring with a constraint Hashing / Frequency Count Count character frequencies and compare O(n) Anagram, grouping, permutation problems Key Takeaways Strings are immutable in Python. Any change creates a new string. Building strings with += in a loop is O(n^2) \u0026ndash; always use \u0026quot;\u0026quot;.join(list) instead. ord(c) - ord('a') gives the position of a lowercase letter in the alphabet. This lets you use a fixed array of size 26 as a frequency table, which is more efficient than a dictionary for lowercase letter problems. Two Pointers on strings works exactly like on arrays \u0026ndash; just use indices, not the characters directly. This keeps space at O(1). Sliding Window on strings tracks character counts in the window. When the window violates the constraint, shrink from the left. Two strings are anagrams if and only if their character frequency maps are equal. Sort-and-compare is O(n log n). Frequency-count-and-compare is O(n). Practice Problems Work through these on your own. Identify the pattern before writing any code.\nProblem Link Difficulty Pattern Reverse String LC 344 Easy Two Pointers Valid Palindrome LC 125 Easy Two Pointers Valid Anagram LC 242 Easy Hashing Longest Common Prefix LC 14 Easy String traversal First Unique Character LC 387 Easy Hashing Valid Palindrome II LC 680 Easy Two Pointers Longest Substring Without Repeating Characters LC 3 Medium Sliding Window Group Anagrams LC 49 Medium Hashing Longest Repeating Character Replacement LC 424 Medium Sliding Window Minimum Window Substring LC 76 Hard Sliding Window Permutation in String LC 567 Medium Sliding Window + Hashing Find All Anagrams in a String LC 438 Medium Sliding Window + Hashing Next Steps Next blog: [Linked Lists] \u0026ndash; a data structure where elements are not stored contiguously in memory, and how that changes everything about insertion, deletion, and traversal Reference: Codeverra DSA Sheet ","permalink":"https://blogs.codeverra.com/blog/dsa-05-strings/","summary":"\u003ch1 id=\"strings\"\u003eStrings\u003c/h1\u003e\n\u003chr\u003e\n\u003ch2 id=\"table-of-contents\"\u003eTable of Contents\u003c/h2\u003e\n\u003cul\u003e\n\u003cli\u003e\u003ca href=\"#what-you-will-learn-in-this-blog\"\u003eWhat You Will Learn in This Blog\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#what-is-a-string\"\u003eWhat is a String\u003c/a\u003e\n\u003cul\u003e\n\u003cli\u003e\u003ca href=\"#ascii\"\u003eASCII\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#unicode\"\u003eUnicode\u003c/a\u003e\u003c/li\u003e\n\u003c/ul\u003e\n\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#strings-in-python----immutability\"\u003eStrings in Python \u0026ndash; Immutability\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#string-operations-and-time-complexity\"\u003eString Operations and Time Complexity\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#common-string-methods\"\u003eCommon String Methods\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#strings-as-arrays\"\u003eStrings as Arrays\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#patterns-and-techniques\"\u003ePatterns and Techniques\u003c/a\u003e\n\u003cul\u003e\n\u003cli\u003e\u003ca href=\"#pattern-1----two-pointers-on-strings\"\u003ePattern 1 \u0026ndash; Two Pointers on Strings\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#pattern-2----sliding-window-on-strings\"\u003ePattern 2 \u0026ndash; Sliding Window on Strings\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#pattern-3----hashing-for-string-problems\"\u003ePattern 3 \u0026ndash; Hashing for String Problems\u003c/a\u003e\u003c/li\u003e\n\u003c/ul\u003e\n\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#summary\"\u003eSummary\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#key-takeaways\"\u003eKey Takeaways\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#practice-problems\"\u003ePractice Problems\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#next-steps\"\u003eNext Steps\u003c/a\u003e\u003c/li\u003e\n\u003c/ul\u003e\n\u003chr\u003e\n\u003ch2 id=\"what-you-will-learn-in-this-blog\"\u003eWhat You Will Learn in This Blog\u003c/h2\u003e\n\u003cul\u003e\n\u003cli\u003eWhat a string is at the byte level \u0026ndash; characters, ASCII, and Unicode\u003c/li\u003e\n\u003cli\u003eHow Python represents strings in memory\u003c/li\u003e\n\u003cli\u003eWhy strings in Python are immutable and what that means in practice\u003c/li\u003e\n\u003cli\u003eAll common string operations and their time complexity\u003c/li\u003e\n\u003cli\u003eHow to build strings efficiently and avoid a common O(n^2) trap\u003c/li\u003e\n\u003cli\u003eThree patterns applied to strings: Sliding Window, Two Pointers, and Hashing\u003c/li\u003e\n\u003cli\u003eSolved LeetCode problems for each pattern with full explanation\u003c/li\u003e\n\u003cli\u003eA practice problem table for further study\u003c/li\u003e\n\u003c/ul\u003e\n\u003chr\u003e\n\u003ch2 id=\"what-is-a-string\"\u003eWhat is a String\u003c/h2\u003e\n\u003cp\u003eAt the most fundamental level, a string is a sequence of characters. But computers do not natively understand characters \u0026ndash; they only understand numbers. So every character is mapped to a number, and that number is stored in memory.\u003c/p\u003e","tags":["dsa","python"],"title":"Strings in Python  -  Complete Guide"},{"content":"The Complete DSA with Python \u0026ndash; Master Roadmap What This Guide Is A hands-on, no-nonsense guide to learning Data Structures and Algorithms using Python. The goal is to build genuine understanding, not to memorise solutions.\nEvery topic covers:\nThe concept from first principles Python implementation with full code and complexity analysis The common patterns and techniques that come from that data structure Hands-on LeetCode problems tied directly to the topic What You Need Before Starting Basic Python: variables, loops, conditionals, functions, lists, dicts No prior DSA knowledge needed. Everything else is explained from scratch. The Roadmap Part 1 \u0026ndash; Foundation Blog What You Will Learn What is DSA and why do we learn it What data structures and algorithms are, why they matter, how this guide is structured How to use this guide and how to practice Learning philosophy, how to approach problems, how to use LeetCode effectively Python refresher for DSA Lists, dicts, sets, tuples, list comprehensions, functions \u0026ndash; the Python tools you will use constantly Part 2 \u0026ndash; Complexity Analysis Before touching any data structure, you need this vocabulary. Every single blog after this uses it.\nBlog What You Will Learn Time complexity What it is, why it matters, intuition before formulas Big O notation O(1), O(log n), O(n), O(n log n), O(n^2) and beyond \u0026ndash; with examples and intuition Space complexity Memory usage, in-place vs auxiliary space, how to analyse it Analysing code Step-by-step complexity analysis of real code snippets Part 3 \u0026ndash; Data Structures Each blog covers the concept, all operations with complexity, patterns that come from the structure, and LeetCode problems.\nData Structure Concepts Covered Patterns Introduced Arrays Memory layout, Python lists, array module, NumPy intro, dynamic arrays, referential arrays, common operations and complexity Two Pointers, Sliding Window, Prefix Sum, Binary Search on arrays Strings Immutability, string operations and complexity, string building efficiently Sliding Window on strings, Two Pointers, Hashing for string problems Linked Lists Singly and Doubly Linked Lists, pointers, insert/delete/search/reverse, when to use over arrays Fast and Slow Pointers, In-place Reversal Stacks Stack operations, Python list as stack, linked list implementation, the call stack, monotonic stacks Monotonic Stack, state tracking with a stack Queues and Deques Queue and Deque, Python collections.deque, circular queue, when to use each BFS with a queue, Sliding Window Maximum with deque Hash Maps and Hash Sets How hashing works, collision handling, Python dict and set internals, complexity of operations Frequency counting, grouping, seen-so-far tracking Trees Binary Trees, BST, tree traversals (BFS and DFS), pre/in/post order, BST operations and complexity Level Order BFS, DFS path problems Heaps and Priority Queues Min heap, max heap, heapify, Python heapq, heap operations and complexity Top K Elements, Two Heaps Graphs Adjacency list and matrix, BFS and DFS, directed vs undirected, weighted vs unweighted, cyclic vs acyclic Connected components, cycle detection, shortest path Part 4 \u0026ndash; Algorithms Algorithm Concepts Covered Key Problems Sorting Bubble, Selection, Insertion (and why they are slow), Merge Sort, Quick Sort, Python\u0026rsquo;s built-in sort, when to use what Sort Colors, Merge Intervals, Largest Number Binary Search Classic binary search, binary search on the answer, find leftmost/rightmost, rotated arrays Binary Search, Search in Rotated Array, Find Minimum in Rotated Array Recursion How recursion works, the call stack, base cases, thinking recursively Fibonacci, Generate Parentheses, Power function Backtracking Recursive exploration with pruning, decision trees as a mental model Subsets, Permutations, N-Queens, Sudoku Solver Dynamic Programming Overlapping subproblems, optimal substructure, top-down vs bottom-up, 0/1 Knapsack, Unbounded Knapsack, LCS, Palindromes, DP on grids Climbing Stairs, Coin Change, Longest Common Subsequence, Edit Distance Greedy Algorithms The greedy choice property, when greedy works and when it fails Jump Game, Activity Selection Graph Algorithms Topological Sort, Dijkstra, Bellman-Ford, Kruskal, Prim Course Schedule, Shortest Path, Cheapest Flights Within K Stops Part 5 \u0026ndash; Advanced Data Structures Data Structure Concepts Covered Key Problems Tries What a Trie is, insert, search, startsWith, when to use a Trie Implement Trie, Word Search II, Replace Words Union-Find (DSU) Find, Union, path compression, union by rank, cycle detection Number of Connected Components, Redundant Connection, Accounts Merge Segment Trees Range queries, point updates, build, update, query, lazy propagation Range Sum Query, Range Minimum Query Bit Manipulation Bitwise operators, common bit tricks, when to reach for bit manipulation Single Number, Counting Bits, Reverse Bits, Power of Two Part 6 \u0026ndash; Pattern Playbook (Reference) This is a consolidated reference of every pattern covered across the guide. You learn each pattern inside its topic. This section is for review, revision, and recognising patterns when you encounter new problems.\nPattern Where You First Learn It What It Solves Two Pointers Arrays Pair problems, in-place operations, reducing O(n^2) to O(n) Sliding Window Arrays, Strings Subarray and substring problems with a constraint Prefix Sum Arrays Range sum queries, subarray sum problems Binary Search (and variants) Binary Search Search problems, optimisation problems phrased as search Fast and Slow Pointers Linked Lists Cycle detection, finding midpoints In-place Linked List Reversal Linked Lists Reversing parts of a list without extra space Monotonic Stack Stacks Next greater/smaller element problems BFS (Level Order) Trees, Graphs Shortest path in unweighted graphs, level-by-level tree problems DFS Trees, Graphs Path problems, connected components, cycle detection Two Heaps Heaps Problems needing both the max of one half and min of the other Top K Elements Heaps Finding K largest, smallest, or most frequent elements Frequency Counting / Hashing Hash Maps Anagram, duplicate, grouping problems Subsets and Combinations Backtracking Generating all subsets, combinations, permutations Dynamic Programming patterns Dynamic Programming Knapsack, LCS, palindromes, grid DP Topological Sort Graph Algorithms Ordering tasks with dependencies Union-Find applications Union-Find Connected components, dynamic connectivity Bitwise XOR Bit Manipulation Finding missing or non-duplicate numbers efficiently How We Will Build This One blog at a time, in order. Each blog is self-contained and ends with a summary, key takeaways, practice problems, and a pointer to the next topic.\nStart here: What is DSA and why do we learn it\nAdditional Resources Resource Description Link Codeverra DSA Sheet A curated list of 100+ DSA problems organised by topic, with completion tracking. Use this to track your progress as you work through the guide. Open Sheet Solutions Notebook Google Colab notebook with fully explained Python solutions for all problems covered in this guide. Link will be added ","permalink":"https://blogs.codeverra.com/blog/dsa-00-master-roadmap/","summary":"\u003ch1 id=\"the-complete-dsa-with-python--master-roadmap\"\u003eThe Complete DSA with Python \u0026ndash; Master Roadmap\u003c/h1\u003e\n\u003chr\u003e\n\u003ch2 id=\"what-this-guide-is\"\u003eWhat This Guide Is\u003c/h2\u003e\n\u003cp\u003eA hands-on, no-nonsense guide to learning Data Structures and Algorithms using Python. The goal is to build genuine understanding, not to memorise solutions.\u003c/p\u003e\n\u003cp\u003eEvery topic covers:\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003eThe concept from first principles\u003c/li\u003e\n\u003cli\u003ePython implementation with full code and complexity analysis\u003c/li\u003e\n\u003cli\u003eThe common patterns and techniques that come from that data structure\u003c/li\u003e\n\u003cli\u003eHands-on LeetCode problems tied directly to the topic\u003c/li\u003e\n\u003c/ul\u003e\n\u003chr\u003e\n\u003ch2 id=\"what-you-need-before-starting\"\u003eWhat You Need Before Starting\u003c/h2\u003e\n\u003cul\u003e\n\u003cli\u003eBasic Python: variables, loops, conditionals, functions, lists, dicts\u003c/li\u003e\n\u003cli\u003eNo prior DSA knowledge needed. Everything else is explained from scratch.\u003c/li\u003e\n\u003c/ul\u003e\n\u003chr\u003e\n\u003ch2 id=\"the-roadmap\"\u003eThe Roadmap\u003c/h2\u003e\n\u003chr\u003e\n\u003ch3 id=\"part-1--foundation\"\u003ePart 1 \u0026ndash; Foundation\u003c/h3\u003e\n\u003ctable\u003e\n  \u003cthead\u003e\n      \u003ctr\u003e\n          \u003cth\u003eBlog\u003c/th\u003e\n          \u003cth\u003eWhat You Will Learn\u003c/th\u003e\n      \u003c/tr\u003e\n  \u003c/thead\u003e\n  \u003ctbody\u003e\n      \u003ctr\u003e\n          \u003ctd\u003eWhat is DSA and why do we learn it\u003c/td\u003e\n          \u003ctd\u003eWhat data structures and algorithms are, why they matter, how this guide is structured\u003c/td\u003e\n      \u003c/tr\u003e\n      \u003ctr\u003e\n          \u003ctd\u003eHow to use this guide and how to practice\u003c/td\u003e\n          \u003ctd\u003eLearning philosophy, how to approach problems, how to use LeetCode effectively\u003c/td\u003e\n      \u003c/tr\u003e\n      \u003ctr\u003e\n          \u003ctd\u003ePython refresher for DSA\u003c/td\u003e\n          \u003ctd\u003eLists, dicts, sets, tuples, list comprehensions, functions \u0026ndash; the Python tools you will use constantly\u003c/td\u003e\n      \u003c/tr\u003e\n  \u003c/tbody\u003e\n\u003c/table\u003e\n\u003chr\u003e\n\u003ch3 id=\"part-2--complexity-analysis\"\u003ePart 2 \u0026ndash; Complexity Analysis\u003c/h3\u003e\n\u003cp\u003eBefore touching any data structure, you need this vocabulary. Every single blog after this uses it.\u003c/p\u003e","tags":["dsa","learning-roadmap"],"title":"The Complete DSA with Python  -  Master Roadmap"},{"content":"🔰 Pattern 1: Two Pointers Table of Contents What is the Two Pointer Technique? When to Use Two Pointers? Types of Two Pointer Patterns Visual Walkthrough Template Code Problem Set Problem 1: Two Sum II – Input Array Is Sorted Problem 2: Valid Palindrome Problem 3: Remove Duplicates from Sorted Array Problem 4: Container With Most Water Problem 5: 3Sum Problem 6: Trapping Rain Water Problem 7: Sort Colors (Dutch National Flag) Problem 8: Move Zeroes Problem 9: Squares of a Sorted Array Problem 10: Merge Sorted Array Key Takeaways \u0026amp; Summary What is the Two Pointer Technique? Two Pointers is a technique where you use two index variables that move through an array or string - either toward each other from opposite ends, or in the same direction at different speeds - to solve a problem efficiently.\nLet\u0026rsquo;s understand why this is useful with a concrete problem.\nProblem: Given a sorted array [1, 3, 5, 7, 9, 11], find two numbers that add up to 12.\nBrute force approach: Check every pair - (1,3), (1,5), (1,7)... - that\u0026rsquo;s O(n²) comparisons.\nTwo pointer approach: Place one pointer at the start and one at the end:\n[1, 3, 5, 7, 9, 11] target = 12 ^ ^ left right → 1 + 11 = 12 ✅ Found! What if the target were 10?\nStep 1: 1 + 11 = 12 \u0026gt; 10 → sum too big, move right ← Step 2: 1 + 9 = 10 → ✅ Found! What if the target were 8?\nStep 1: 1 + 11 = 12 \u0026gt; 8 → move right ← Step 2: 1 + 9 = 10 \u0026gt; 8 → move right ← Step 3: 1 + 7 = 8 → ✅ Found! Because the array is sorted, we know exactly which pointer to move: if the sum is too large, moving right leftward gives us a smaller value; if too small, moving left rightward gives us a larger value. This gives us an O(n) solution - each pointer moves at most n times.\nWhy is it useful? Reduces time complexity: Often turns an O(n²) brute-force into an O(n) solution. No extra space: Works in-place, so usually O(1) extra space. Simple to implement: Once you spot the pattern, the code is clean and short. When to Use Two Pointers? Look for these signals in a problem:\nSignal Example Array is sorted (or should be sorted) \u0026ldquo;Given a sorted array, find two numbers that\u0026hellip;\u0026rdquo; Need to find a pair/triplet with a certain property \u0026ldquo;Find two elements whose sum is X\u0026rdquo; Need to compare elements from both ends \u0026ldquo;Check if a string is a palindrome\u0026rdquo; Need to remove/rearrange in-place \u0026ldquo;Remove duplicates in-place\u0026rdquo; Need to partition the array \u0026ldquo;Move all zeroes to the end\u0026rdquo; Involves merging two sorted structures \u0026ldquo;Merge two sorted arrays\u0026rdquo; Types of Two Pointer Patterns 1. Opposite Direction (Converging) Pointers Array: [1, 3, 5, 7, 9, 11] ^ ^ left right left moves → ← right moves They move toward each other until they meet. Use when:\nArray is sorted and you need to find pairs Checking palindromes Problems involving \u0026ldquo;two ends\u0026rdquo; of the array 2. Same Direction (Fast \u0026amp; Slow) Pointers Array: [0, 0, 1, 1, 2, 3] ^ ^ slow fast Both move → slow tracks the \u0026#34;valid\u0026#34; position, fast scans ahead. Use when:\nRemoving duplicates in-place Partitioning an array (e.g., move zeroes) Finding a specific element arrangement 3. Three Pointers (Extension) Array: [2, 0, 1, 2, 0, 1] ^ ^ ^ low mid high Used for 3-way partitioning (like Dutch National Flag). Visual Walkthrough Example: Find two numbers in a sorted array that sum to 10 Array: [1, 3, 4, 6, 8, 11] Target: 10 Step 1: left=0, right=5 → 1 + 11 = 12 \u0026gt; 10 → move right ← [1, 3, 4, 6, 8, 11] ^ ^ Step 2: left=0, right=4 → 1 + 8 = 9 \u0026lt; 10 → move left → [1, 3, 4, 6, 8, 11] ^ ^ Step 3: left=1, right=4 → 3 + 8 = 11 \u0026gt; 10 → move right ← [1, 3, 4, 6, 8, 11] ^ ^ Step 4: left=1, right=3 → 3 + 6 = 9 \u0026lt; 10 → move left → [1, 3, 4, 6, 8, 11] ^ ^ Step 5: left=2, right=3 → 4 + 6 = 10 = target ✅ FOUND! [1, 3, 4, 6, 8, 11] ^ ^ Key Insight: Because the array is sorted:\nIf the sum is too big → we need a smaller number → move right left If the sum is too small → we need a bigger number → move left right Template Code Template 1: Opposite Direction Pointers def two_pointer_opposite(arr): \u0026#34;\u0026#34;\u0026#34; Template for converging (opposite direction) two pointers. Useful for: pair finding, palindrome checks, container problems. Time Complexity: O(n) - each pointer moves at most n times Space Complexity: O(1) - only two variables \u0026#34;\u0026#34;\u0026#34; left = 0 right = len(arr) - 1 while left \u0026lt; right: # Process current pair (arr[left], arr[right]) if some_condition: left += 1 # Move left pointer forward elif other_condition: right -= 1 # Move right pointer backward else: # Found the answer or process result break # or continue based on problem return result Template 2: Same Direction Pointers (Slow \u0026amp; Fast) def two_pointer_same_direction(arr): \u0026#34;\u0026#34;\u0026#34; Template for same-direction (slow/fast) two pointers. Useful for: removing duplicates, partitioning, rearranging. Time Complexity: O(n) - fast pointer traverses the array once Space Complexity: O(1) - in-place modification \u0026#34;\u0026#34;\u0026#34; slow = 0 # Tracks the position for next valid element for fast in range(len(arr)): if some_condition(arr[fast]): arr[slow] = arr[fast] slow += 1 return slow # Usually returns the new length Problem Set Difficulty Progression # Problem Difficulty Key Concept 1 Two Sum II Easy Opposite direction basics 2 Valid Palindrome Easy Opposite direction on strings 3 Remove Duplicates from Sorted Array Easy Same direction (slow/fast) 4 Container With Most Water Medium Opposite direction with greedy 5 3Sum Medium Sort + two pointers 6 Trapping Rain Water Hard Opposite direction with tracking 7 Sort Colors Medium Three pointers 8 Move Zeroes Easy Same direction partitioning 9 Squares of a Sorted Array Easy Opposite direction with merge 10 Merge Sorted Array Easy Reverse two pointers Problem 1: Two Sum II – Input Array Is Sorted LeetCode Link: https://leetcode.com/problems/two-sum-ii-input-array-is-sorted/\nProblem Statement Given a 1-indexed sorted array numbers, find two numbers that add up to a specific target. Return their indices (1-indexed).\nExample:\nInput: numbers = [2, 7, 11, 15], target = 9 Output: [1, 2] (because numbers[0] + numbers[1] = 2 + 7 = 9) Clarifying Questions \u0026amp; Constraints The array is already sorted in non-decreasing order. There is exactly one solution. You may not use the same element twice. The answer must use constant extra space (O(1)). Approach Discussion Approach 1: Brute Force\nTry every pair (i, j) where i \u0026lt; j. Check if numbers[i] + numbers[j] == target. Time Complexity: O(n²) - two nested loops Space Complexity: O(1) ❌ Too slow for large inputs. Doesn\u0026rsquo;t use the \u0026ldquo;sorted\u0026rdquo; property at all. Approach 2: HashMap\nFor each number, check if target - number exists in a hashmap. Time Complexity: O(n) Space Complexity: O(n) - need extra space for the hashmap ❌ Works, but the problem says \u0026ldquo;constant extra space\u0026rdquo;. Doesn\u0026rsquo;t use the \u0026ldquo;sorted\u0026rdquo; property. Approach 3: Two Pointers (Optimal) ✅\nPlace left at index 0 and right at the last index. If sum \u0026lt; target, move left right (we need a bigger number). If sum \u0026gt; target, move right left (we need a smaller number). If sum == target, we found our answer! Time Complexity: O(n) - each pointer moves at most n steps Space Complexity: O(1) - only two variables Why does this work? Because the array is sorted! When the sum is too small, moving left forward guarantees a larger value. When the sum is too large, moving right backward guarantees a smaller value. We never skip a valid pair.\nCode (Optimal Solution) def twoSum(numbers: list[int], target: int) -\u0026gt; list[int]: \u0026#34;\u0026#34;\u0026#34; Find two numbers in a sorted array that sum to target. Uses the two-pointer (opposite direction) technique. Args: numbers: A sorted (non-decreasing) list of integers (1-indexed in problem) target: The target sum we\u0026#39;re looking for Returns: A list of two 1-indexed positions [i, j] where numbers[i-1] + numbers[j-1] = target Time Complexity: O(n) - each pointer moves at most n times total Space Complexity: O(1) - only two integer variables used \u0026#34;\u0026#34;\u0026#34; # Initialize two pointers at opposite ends left = 0 right = len(numbers) - 1 while left \u0026lt; right: # Calculate the current sum current_sum = numbers[left] + numbers[right] if current_sum == target: # Found the pair! Return 1-indexed positions return [left + 1, right + 1] elif current_sum \u0026lt; target: # Sum is too small → we need a bigger number → move left forward left += 1 else: # Sum is too big → we need a smaller number → move right backward right -= 1 # Problem guarantees a solution exists, so we\u0026#39;ll never reach here return [] Edge Cases Array of length 2: [1, 3], target=4 → Only one pair to check, works fine. Negative numbers: [-3, -1, 0, 2, 4], target=1 → Works because sorting handles negatives. Duplicates: [1, 1, 3, 5], target=2 → Both pointers will find [1, 2]. Dry Run Input: numbers = [2, 7, 11, 15], target = 9 Step 1: left=0, right=3 sum = numbers[0] + numbers[3] = 2 + 15 = 17 17 \u0026gt; 9 → move right ← (right = 2) Step 2: left=0, right=2 sum = numbers[0] + numbers[2] = 2 + 11 = 13 13 \u0026gt; 9 → move right ← (right = 1) Step 3: left=0, right=1 sum = numbers[0] + numbers[1] = 2 + 7 = 9 9 == 9 → ✅ Found! Return [1, 2] Output: [1, 2] Problem 2: Valid Palindrome LeetCode Link: https://leetcode.com/problems/valid-palindrome/\nProblem Statement Given a string s, determine if it is a palindrome, considering only alphanumeric characters and ignoring cases.\nExample:\nInput: s = \u0026#34;A man, a plan, a canal: Panama\u0026#34; Output: True (after cleanup: \u0026#34;amanaplanacanalpanama\u0026#34; is a palindrome) Clarifying Questions \u0026amp; Constraints We ignore all non-alphanumeric characters (spaces, punctuation, etc.). Comparison is case-insensitive. An empty string is considered a valid palindrome. Approach Discussion Approach 1: Clean + Reverse\nRemove all non-alphanumeric characters, convert to lowercase. Check if the cleaned string equals its reverse. Time Complexity: O(n) Space Complexity: O(n) - creating the cleaned string ✅ Simple but uses extra space. Approach 2: Two Pointers (Optimal) ✅\nUse left and right pointers starting from both ends. Skip non-alphanumeric characters. Compare characters (case-insensitive) at each step. Time Complexity: O(n) Space Complexity: O(1) - no extra string created Code (Both Solutions) # ============================================================ # APPROACH 1: Clean + Reverse (Simpler but O(n) space) # ============================================================ def isPalindrome_clean(s: str) -\u0026gt; bool: \u0026#34;\u0026#34;\u0026#34; Check palindrome by cleaning the string first, then comparing with reverse. Time Complexity: O(n) - one pass to clean, one pass to reverse and compare Space Complexity: O(n) - storing the cleaned string \u0026#34;\u0026#34;\u0026#34; # Step 1: Keep only alphanumeric characters and convert to lowercase cleaned = \u0026#34;\u0026#34; for char in s: if char.isalnum(): cleaned += char.lower() # Step 2: Check if cleaned string equals its reverse return cleaned == cleaned[::-1] # ============================================================ # APPROACH 2: Two Pointers (Optimal - O(1) space) # ============================================================ def isPalindrome(s: str) -\u0026gt; bool: \u0026#34;\u0026#34;\u0026#34; Check palindrome using two pointers without creating a new string. The idea: place one pointer at the start and one at the end. Skip non-alphanumeric characters. Compare characters at both pointers. If all pairs match → palindrome. Time Complexity: O(n) - each pointer traverses the string at most once Space Complexity: O(1) - only two integer pointers used \u0026#34;\u0026#34;\u0026#34; left = 0 right = len(s) - 1 while left \u0026lt; right: # Skip non-alphanumeric characters from the left while left \u0026lt; right and not s[left].isalnum(): left += 1 # Skip non-alphanumeric characters from the right while left \u0026lt; right and not s[right].isalnum(): right -= 1 # Compare characters (case-insensitive) if s[left].lower() != s[right].lower(): return False # Mismatch found → not a palindrome # Move both pointers inward left += 1 right -= 1 # All pairs matched → it\u0026#39;s a palindrome return True Edge Cases Empty string: \u0026quot;\u0026quot; → True (nothing to mismatch). Single character: \u0026quot;a\u0026quot; → True. Only non-alphanumeric: \u0026quot;,.!?\u0026quot; → True (no characters to compare). Mixed case: \u0026quot;Aa\u0026quot; → True (case-insensitive comparison). Dry Run Input: s = \u0026#34;A man, a plan, a canal: Panama\u0026#34; Pointers start: left=0 (\u0026#39;A\u0026#39;), right=29 (\u0026#39;a\u0026#39;) Step 1: left=0 (\u0026#39;A\u0026#39;), right=29 (\u0026#39;a\u0026#39;) \u0026#39;a\u0026#39; == \u0026#39;a\u0026#39; ✅ → left=1, right=28 Step 2: left=1 (\u0026#39; \u0026#39;) → skip → left=2 (\u0026#39;m\u0026#39;) right=28 (\u0026#39;m\u0026#39;) \u0026#39;m\u0026#39; == \u0026#39;m\u0026#39; ✅ → left=3, right=27 Step 3: left=3 (\u0026#39;a\u0026#39;), right=27 (\u0026#39;a\u0026#39;) \u0026#39;a\u0026#39; == \u0026#39;a\u0026#39; ✅ → left=4, right=26 ... (continues, all characters match) Final: left \u0026gt;= right → return True ✅ Problem 3: Remove Duplicates from Sorted Array LeetCode Link: https://leetcode.com/problems/remove-duplicates-from-sorted-array/\nProblem Statement Given a sorted array nums, remove duplicates in-place such that each element appears only once. Return the number of unique elements.\nExample:\nInput: nums = [1, 1, 2] Output: 2 (nums becomes [1, 2, _]) Clarifying Questions \u0026amp; Constraints Must modify the array in-place with O(1) extra space. The relative order of elements must be maintained. Return k = number of unique elements. First k elements of nums should hold the result. Elements beyond position k don\u0026rsquo;t matter. Approach Discussion Approach 1: Using a Set (Not In-Place)\nInsert all elements into a set, copy back. Time: O(n), Space: O(n) ❌ Violates O(1) space constraint. Approach 2: Two Pointers - Slow \u0026amp; Fast (Optimal) ✅\nslow marks where the next unique element should go. fast scans through the array looking for new unique values. Whenever nums[fast] != nums[slow], we found a new unique element → copy it to slow + 1. Time Complexity: O(n), Space Complexity: O(1) Code (Optimal Solution) def removeDuplicates(nums: list[int]) -\u0026gt; int: \u0026#34;\u0026#34;\u0026#34; Remove duplicates from a sorted array in-place using slow/fast pointers. How it works: - \u0026#39;slow\u0026#39; always points to the last confirmed unique element. - \u0026#39;fast\u0026#39; scans ahead looking for the next different element. - When fast finds something new, we place it right after slow. Time Complexity: O(n) - single pass through the array Space Complexity: O(1) - only two pointers, modification is in-place \u0026#34;\u0026#34;\u0026#34; # Edge case: empty array if not nums: return 0 # \u0026#39;slow\u0026#39; starts at index 0 - the first element is always unique slow = 0 # \u0026#39;fast\u0026#39; starts at index 1 and scans the rest of the array for fast in range(1, len(nums)): # If we found a new unique element (different from the last unique one) if nums[fast] != nums[slow]: slow += 1 # Move slow to the next position nums[slow] = nums[fast] # Place the new unique element there # \u0026#39;slow\u0026#39; is the index of the last unique element # Number of unique elements = slow + 1 return slow + 1 Edge Cases Empty array: [] → return 0. All same: [7, 7, 7, 7] → return 1, nums = [7, ...]. Already unique: [1, 2, 3] → return 3, no changes needed. Length 1: [5] → return 1. Dry Run Input: nums = [0, 0, 1, 1, 1, 2, 2, 3, 3, 4] Initial: slow = 0 fast=1: nums[1]=0 == nums[0]=0 → skip fast=2: nums[2]=1 != nums[0]=0 → slow=1, nums[1]=1 → [0,1,1,1,1,2,2,3,3,4] fast=3: nums[3]=1 == nums[1]=1 → skip fast=4: nums[4]=1 == nums[1]=1 → skip fast=5: nums[5]=2 != nums[1]=1 → slow=2, nums[2]=2 → [0,1,2,1,1,2,2,3,3,4] fast=6: nums[6]=2 == nums[2]=2 → skip fast=7: nums[7]=3 != nums[2]=2 → slow=3, nums[3]=3 → [0,1,2,3,1,2,2,3,3,4] fast=8: nums[8]=3 == nums[3]=3 → skip fast=9: nums[9]=4 != nums[3]=3 → slow=4, nums[4]=4 → [0,1,2,3,4,2,2,3,3,4] Return: slow + 1 = 5 First 5 elements: [0, 1, 2, 3, 4] ✅ Problem 4: Container With Most Water LeetCode Link: https://leetcode.com/problems/container-with-most-water/\nProblem Statement Given n non-negative integers height[0], height[1], ..., height[n-1] where each represents a vertical line at position i, find two lines that together with the x-axis forms a container that holds the most water.\nExample:\nInput: height = [1, 8, 6, 2, 5, 4, 8, 3, 7] Output: 49 (between index 1 and index 8: min(8,7) * (8-1) = 49) Clarifying Questions \u0026amp; Constraints n \u0026gt;= 2 (at least two lines). You cannot tilt the container. Water = min(height[left], height[right]) * (right - left). Approach Discussion Approach 1: Brute Force\nTry every pair (i, j) and compute the area. Time Complexity: O(n²) - two nested loops Space Complexity: O(1) ❌ Too slow for large inputs. Approach 2: Two Pointers (Optimal) ✅\nStart with the widest container (left=0, right=n-1). Calculate the area. Update the maximum. Key insight: Move the pointer with the shorter height, because: The width is going to decrease no matter what. So our only hope of finding a bigger area is to find a taller line. Moving the taller line inward can never increase the area (the bottleneck is the shorter line). Time Complexity: O(n), Space Complexity: O(1) Code (Both Solutions) # ============================================================ # APPROACH 1: Brute Force - O(n²) # ============================================================ def maxArea_brute(height: list[int]) -\u0026gt; int: \u0026#34;\u0026#34;\u0026#34; Try every pair of lines and compute the area. Time Complexity: O(n²) - checking all pairs Space Complexity: O(1) \u0026#34;\u0026#34;\u0026#34; max_water = 0 n = len(height) for i in range(n): for j in range(i + 1, n): # Water is limited by the shorter line water = min(height[i], height[j]) * (j - i) max_water = max(max_water, water) return max_water # ============================================================ # APPROACH 2: Two Pointers - O(n) ✅ # ============================================================ def maxArea(height: list[int]) -\u0026gt; int: \u0026#34;\u0026#34;\u0026#34; Find the maximum water container using two pointers. Strategy: - Start with the widest container (pointers at both ends). - Always move the pointer pointing to the SHORTER line. - Why? The area is limited by the shorter line. Moving the taller line inward will only decrease width without helping the height bottleneck. Moving the shorter line gives us a chance at finding a taller line. Time Complexity: O(n) - each pointer moves at most n times total Space Complexity: O(1) - only three variables \u0026#34;\u0026#34;\u0026#34; left = 0 right = len(height) - 1 max_water = 0 while left \u0026lt; right: # Calculate the area formed by the two current lines # Height = the shorter of the two lines # Width = distance between the two lines width = right - left current_height = min(height[left], height[right]) current_water = current_height * width # Update the maximum area found so far max_water = max(max_water, current_water) # Move the pointer with the shorter line inward # (If equal, moving either is fine - both are equally limiting) if height[left] \u0026lt; height[right]: left += 1 else: right -= 1 return max_water Edge Cases Two elements: [1, 1] → area = min(1,1) * 1 = 1. Descending order: [5, 4, 3, 2, 1] → max area at widest possible with good heights. One very tall line: [1, 1, 1, 100, 1, 1] → the tall line doesn\u0026rsquo;t help alone. Dry Run Input: height = [1, 8, 6, 2, 5, 4, 8, 3, 7] Step 1: left=0(h=1), right=8(h=7) area = min(1,7) * 8 = 8, max_water = 8 height[left]=1 \u0026lt; height[right]=7 → left++ Step 2: left=1(h=8), right=8(h=7) area = min(8,7) * 7 = 49, max_water = 49 height[left]=8 \u0026gt; height[right]=7 → right-- Step 3: left=1(h=8), right=7(h=3) area = min(8,3) * 6 = 18, max_water = 49 height[left]=8 \u0026gt; height[right]=3 → right-- Step 4: left=1(h=8), right=6(h=8) area = min(8,8) * 5 = 40, max_water = 49 height[left]=8 == height[right]=8 → right-- Step 5: left=1(h=8), right=5(h=4) area = min(8,4) * 4 = 16, max_water = 49 height[left]=8 \u0026gt; height[right]=4 → right-- Step 6: left=1(h=8), right=4(h=5) area = min(8,5) * 3 = 15, max_water = 49 height[left]=8 \u0026gt; height[right]=5 → right-- Step 7: left=1(h=8), right=3(h=2) area = min(8,2) * 2 = 4, max_water = 49 height[left]=8 \u0026gt; height[right]=2 → right-- Step 8: left=1(h=8), right=2(h=6) area = min(8,6) * 1 = 6, max_water = 49 height[left]=8 \u0026gt; height[right]=6 → right-- left \u0026gt;= right → STOP Output: 49 ✅ Problem 5: 3Sum LeetCode Link: https://leetcode.com/problems/3sum/\nProblem Statement Given an integer array nums, return all the triplets [nums[i], nums[j], nums[k]] such that i != j != k and nums[i] + nums[j] + nums[k] == 0. The result must not contain duplicate triplets.\nExample:\nInput: nums = [-1, 0, 1, 2, -1, -4] Output: [[-1, -1, 2], [-1, 0, 1]] Clarifying Questions \u0026amp; Constraints Result set must not contain duplicate triplets (order doesn\u0026rsquo;t matter). 3 \u0026lt;= nums.length \u0026lt;= 3000 Array is not sorted initially. Elements can be negative, zero, or positive. Approach Discussion Approach 1: Brute Force (Three Nested Loops)\nCheck every combination of three elements. Use a set to avoid duplicates. Time: O(n³), Space: O(n) for the set ❌ Way too slow. Approach 2: Sort + Two Pointers (Optimal) ✅\nSort the array first. Fix one element (nums[i]) and use Two Pointers to find two elements in the remaining part that sum to -nums[i]. Skip duplicates at each level to avoid duplicate triplets. Time Complexity: O(n²) - O(n log n) for sort + O(n) × O(n) for nested two-pointer Space Complexity: O(1) extra (ignoring the output and sort space) Why sort? Sorting lets us:\nUse the two-pointer technique (needs sorted data). Easily skip duplicates by checking nums[i] == nums[i-1]. Code (Optimal Solution) def threeSum(nums: list[int]) -\u0026gt; list[list[int]]: \u0026#34;\u0026#34;\u0026#34; Find all unique triplets that sum to zero using sort + two pointers. Strategy: 1. Sort the array. 2. For each element nums[i], use two pointers on the remaining subarray to find pairs that sum to -nums[i]. 3. Skip duplicates at every level to ensure unique triplets. Time Complexity: O(n²) - Sorting: O(n log n) - Outer loop: O(n), inner two-pointer: O(n) → total O(n²) - Overall: O(n²) dominates Space Complexity: O(1) extra (not counting the output list) - Sorting may use O(log n) depending on implementation \u0026#34;\u0026#34;\u0026#34; nums.sort() # Step 1: Sort the array result = [] n = len(nums) for i in range(n - 2): # Fix the first element # OPTIMIZATION: If nums[i] \u0026gt; 0, no triplet can sum to 0 # (all remaining elements are \u0026gt;= nums[i] \u0026gt; 0) if nums[i] \u0026gt; 0: break # SKIP DUPLICATES for the first element # If nums[i] == nums[i-1], we\u0026#39;ve already processed this value if i \u0026gt; 0 and nums[i] == nums[i - 1]: continue # Two pointers for the remaining subarray left = i + 1 right = n - 1 target = -nums[i] # We need nums[left] + nums[right] = -nums[i] while left \u0026lt; right: current_sum = nums[left] + nums[right] if current_sum \u0026lt; target: # Sum too small → need a bigger number → move left forward left += 1 elif current_sum \u0026gt; target: # Sum too big → need a smaller number → move right backward right -= 1 else: # Found a valid triplet! result.append([nums[i], nums[left], nums[right]]) # SKIP DUPLICATES for the second element while left \u0026lt; right and nums[left] == nums[left + 1]: left += 1 # SKIP DUPLICATES for the third element while left \u0026lt; right and nums[right] == nums[right - 1]: right -= 1 # Move both pointers inward to look for more triplets left += 1 right -= 1 return result Edge Cases All zeros: [0, 0, 0, 0] → [[0, 0, 0]] (only one unique triplet). No valid triplets: [1, 2, 3] → []. All positive: [1, 2, 3, 4] → [] (can\u0026rsquo;t sum to 0). Heavy duplicates: [-1, -1, -1, 2, 2] → [[-1, -1, 2]]. Dry Run Input: nums = [-1, 0, 1, 2, -1, -4] After sort: [-4, -1, -1, 0, 1, 2] i=0: nums[0] = -4, target = 4 left=1(-1), right=5(2): sum = -1+2 = 1 \u0026lt; 4 → left++ left=2(-1), right=5(2): sum = -1+2 = 1 \u0026lt; 4 → left++ left=3(0), right=5(2): sum = 0+2 = 2 \u0026lt; 4 → left++ left=4(1), right=5(2): sum = 1+2 = 3 \u0026lt; 4 → left++ left=5 \u0026gt;= right=5 → STOP i=1: nums[1] = -1, target = 1 left=2(-1), right=5(2): sum = -1+2 = 1 == 1 ✅ → add [-1, -1, 2] → skip duplicate left: nums[2]==nums[3]? -1==0? No → skip duplicate right: nums[5]==nums[4]? 2==1? No → left=3, right=4 left=3(0), right=4(1): sum = 0+1 = 1 == 1 ✅ → add [-1, 0, 1] → left=4, right=3 → STOP i=2: nums[2] = -1, but nums[2]==nums[1] → SKIP (duplicate) i=3: nums[3] = 0, target = 0 left=4(1), right=5(2): sum = 1+2 = 3 \u0026gt; 0 → right-- left=4 \u0026gt;= right=4 → STOP Output: [[-1, -1, 2], [-1, 0, 1]] ✅ Problem 6: Trapping Rain Water LeetCode Link: https://leetcode.com/problems/trapping-rain-water/\nProblem Statement Given n non-negative integers representing an elevation map where the width of each bar is 1, compute how much water it can trap after raining.\nExample:\nInput: height = [0, 1, 0, 2, 1, 0, 1, 3, 2, 1, 2, 1] Output: 6 Clarifying Questions \u0026amp; Constraints n \u0026gt;= 1 height[i] \u0026gt;= 0 Water at position i = min(max_left, max_right) - height[i] (if positive) Approach Discussion Approach 1: Brute Force\nFor each bar, find the tallest bar to its left and right. Water at position i = min(left_max, right_max) - height[i]. Time: O(n²), Space: O(1) ❌ Too slow for large inputs. Approach 2: Prefix/Suffix Arrays\nPre-compute left_max[i] and right_max[i] arrays. Time: O(n), Space: O(n) - two extra arrays Approach 3: Two Pointers (Optimal) ✅\nUse left and right pointers from both ends. Track left_max and right_max as we go. Key insight: If left_max \u0026lt; right_max, the water at left is determined by left_max (regardless of the actual right_max, it\u0026rsquo;s at least as high). So we can safely process the left side. Time Complexity: O(n), Space Complexity: O(1) Code (Optimal Solution) def trap(height: list[int]) -\u0026gt; int: \u0026#34;\u0026#34;\u0026#34; Calculate trapped rainwater using two pointers. Core Insight: Water at any position depends on min(max_left, max_right) - height[i]. If left_max \u0026lt; right_max: → The water at \u0026#39;left\u0026#39; is bounded by left_max (since right_max is even bigger, it won\u0026#39;t be the bottleneck). So we can safely calculate water at \u0026#39;left\u0026#39;. If right_max \u0026lt;= left_max: → Same logic applies to the \u0026#39;right\u0026#39; side. Time Complexity: O(n) - single pass with two pointers Space Complexity: O(1) - only a few variables \u0026#34;\u0026#34;\u0026#34; if not height: return 0 left = 0 right = len(height) - 1 left_max = 0 # Tallest bar seen from the left so far right_max = 0 # Tallest bar seen from the right so far water = 0 while left \u0026lt; right: if height[left] \u0026lt; height[right]: # Process the left side if height[left] \u0026gt;= left_max: # Current bar is taller than anything seen from left → no water here # Update left_max left_max = height[left] else: # Water can be trapped: left_max - height[left] water += left_max - height[left] left += 1 else: # Process the right side if height[right] \u0026gt;= right_max: # Current bar is taller than anything seen from right → no water here right_max = height[right] else: # Water can be trapped: right_max - height[right] water += right_max - height[right] right -= 1 return water Edge Cases Flat: [3, 3, 3] → 0 (no dips to trap water). Single bar: [5] → 0. Ascending: [1, 2, 3, 4] → 0 (water runs off the left). Descending: [4, 3, 2, 1] → 0 (water runs off the right). V-shape: [3, 0, 3] → 3. Dry Run Input: height = [0, 1, 0, 2, 1, 0, 1, 3, 2, 1, 2, 1] Initial: left=0, right=11, left_max=0, right_max=0, water=0 Step 1: h[0]=0 \u0026lt; h[11]=1 → process left h[0]=0 \u0026gt;= left_max=0 → left_max=0, left=1 Step 2: h[1]=1 \u0026gt;= h[11]=1 → process right h[11]=1 \u0026gt;= right_max=0 → right_max=1, right=10 Step 3: h[1]=1 \u0026lt; h[10]=2 → process left h[1]=1 \u0026gt;= left_max=0 → left_max=1, left=2 Step 4: h[2]=0 \u0026lt; h[10]=2 → process left h[2]=0 \u0026lt; left_max=1 → water += 1-0 = 1, left=3 water=1 Step 5: h[3]=2 \u0026gt;= h[10]=2 → process right h[10]=2 \u0026gt;= right_max=1 → right_max=2, right=9 Step 6: h[3]=2 \u0026gt;= h[9]=1 → process right h[9]=1 \u0026lt; right_max=2 → water += 2-1 = 1, right=8 water=2 Step 7: h[3]=2 \u0026gt;= h[8]=2 → process right h[8]=2 \u0026gt;= right_max=2 → right_max=2, right=7 Step 8: h[3]=2 \u0026lt; h[7]=3 → process left h[3]=2 \u0026gt;= left_max=1 → left_max=2, left=4 Step 9: h[4]=1 \u0026lt; h[7]=3 → process left h[4]=1 \u0026lt; left_max=2 → water += 2-1 = 1, left=5 water=3 Step 10: h[5]=0 \u0026lt; h[7]=3 → process left h[5]=0 \u0026lt; left_max=2 → water += 2-0 = 2, left=6 water=5 Step 11: h[6]=1 \u0026lt; h[7]=3 → process left h[6]=1 \u0026lt; left_max=2 → water += 2-1 = 1, left=7 water=6 left=7 \u0026gt;= right=7 → STOP Output: 6 ✅ Problem 7: Sort Colors (Dutch National Flag) LeetCode Link: https://leetcode.com/problems/sort-colors/\nProblem Statement Given an array nums with n objects colored red (0), white (1), and blue (2), sort them in-place so that objects of the same color are adjacent, in the order red, white, blue.\nExample:\nInput: nums = [2, 0, 2, 1, 1, 0] Output: [0, 0, 1, 1, 2, 2] Clarifying Questions \u0026amp; Constraints Must solve in-place without using the library sort function. One-pass algorithm is preferred. Only contains values 0, 1, and 2. Approach Discussion Approach 1: Counting Sort\nCount the number of 0s, 1s, and 2s, then overwrite the array. Time: O(n), Space: O(1) ✅ Works but requires two passes (one to count, one to write). Approach 2: Dutch National Flag - Three Pointers (Optimal) ✅\nUse three pointers: low, mid, high. low = boundary for 0s (everything before low is 0) mid = current element being examined high = boundary for 2s (everything after high is 2) Time: O(n), Space: O(1), single pass Code (Both Solutions) # ============================================================ # APPROACH 1: Counting Sort - Two passes # ============================================================ def sortColors_count(nums: list[int]) -\u0026gt; None: \u0026#34;\u0026#34;\u0026#34; Count occurrences of 0, 1, 2, then overwrite the array. Time Complexity: O(n) - two passes through the array Space Complexity: O(1) - only three counters \u0026#34;\u0026#34;\u0026#34; count = [0, 0, 0] # Pass 1: Count each color for num in nums: count[num] += 1 # Pass 2: Overwrite the array idx = 0 for color in range(3): for _ in range(count[color]): nums[idx] = color idx += 1 # ============================================================ # APPROACH 2: Dutch National Flag - Single pass ✅ # ============================================================ def sortColors(nums: list[int]) -\u0026gt; None: \u0026#34;\u0026#34;\u0026#34; Sort colors in a single pass using three pointers (Dutch National Flag). Pointer roles: - low: Everything BEFORE low is guaranteed to be 0 - mid: Current element being examined - high: Everything AFTER high is guaranteed to be 2 - Between low and mid: all 1s Array zones: [0...0 | 1...1 | unsorted | 2...2] 0 low mid high n-1 Rules: - If nums[mid] == 0: swap with low, move both forward - If nums[mid] == 1: it\u0026#39;s in the right place, just move mid - If nums[mid] == 2: swap with high, move high back (DON\u0026#39;T move mid!) Why don\u0026#39;t we move mid when swapping with high? Because the element swapped FROM high is unexamined - it could be 0, 1, or 2. Time Complexity: O(n) - single pass, mid moves from 0 to high Space Complexity: O(1) - only three pointers \u0026#34;\u0026#34;\u0026#34; low = 0 # Next position for 0 mid = 0 # Current element high = len(nums) - 1 # Next position for 2 while mid \u0026lt;= high: if nums[mid] == 0: # Swap current element with the low boundary nums[low], nums[mid] = nums[mid], nums[low] low += 1 # Expand the 0-zone mid += 1 # Safe to move forward (swapped element is 0 or 1) elif nums[mid] == 1: # 1 is already in the middle section - just skip it mid += 1 else: # nums[mid] == 2 # Swap current element with the high boundary nums[mid], nums[high] = nums[high], nums[mid] high -= 1 # Expand the 2-zone # DON\u0026#39;T increment mid! The swapped element needs to be examined Edge Cases Already sorted: [0, 0, 1, 2, 2] → no swaps needed. Reverse sorted: [2, 2, 1, 0, 0] → fully rearranged. All same: [1, 1, 1] → no swaps. Single element: [0] → already sorted. Dry Run Input: nums = [2, 0, 2, 1, 1, 0] Initial: low=0, mid=0, high=5 mid=0: nums[0]=2 → swap(nums[0], nums[5]) → [0,0,2,1,1,2], high=4 DON\u0026#39;T move mid (need to check swapped value) mid=0: nums[0]=0 → swap(nums[0], nums[0]) → [0,0,2,1,1,2], low=1, mid=1 mid=1: nums[1]=0 → swap(nums[1], nums[1]) → [0,0,2,1,1,2], low=2, mid=2 mid=2: nums[2]=2 → swap(nums[2], nums[4]) → [0,0,1,1,2,2], high=3 mid=2: nums[2]=1 → mid=3 mid=3: nums[3]=1 → mid=4 mid=4 \u0026gt; high=3 → STOP Result: [0, 0, 1, 1, 2, 2] ✅ Problem 8: Move Zeroes LeetCode Link: https://leetcode.com/problems/move-zeroes/\nProblem Statement Given an array nums, move all 0s to the end while maintaining the relative order of the non-zero elements. Must be done in-place.\nExample:\nInput: nums = [0, 1, 0, 3, 12] Output: [1, 3, 12, 0, 0] Clarifying Questions \u0026amp; Constraints Must be in-place (no extra array). Maintain relative order of non-zero elements. Minimize the total number of operations. Approach Discussion Approach 1: Using Extra Array\nCollect all non-zero elements, then append zeros. Time: O(n), Space: O(n) ❌ Not in-place. Approach 2: Two Pointers - Slow \u0026amp; Fast (Optimal) ✅\nslow tracks where the next non-zero element should go. fast scans through the array. When fast finds a non-zero, swap it with slow position. Time: O(n), Space: O(1) Code (Optimal Solution) def moveZeroes(nums: list[int]) -\u0026gt; None: \u0026#34;\u0026#34;\u0026#34; Move all zeroes to the end using the slow/fast pointer technique. How it works: - \u0026#39;slow\u0026#39; points to the position where the next non-zero should go. - \u0026#39;fast\u0026#39; scans the entire array. - When fast finds a non-zero, we swap it into the slow position. - This naturally pushes all zeros toward the end. Why swap instead of overwrite? Swapping preserves all elements. Overwriting would lose zeros and require a separate step to fill them in. Time Complexity: O(n) - single pass through the array Space Complexity: O(1) - only two pointers \u0026#34;\u0026#34;\u0026#34; slow = 0 # Position for next non-zero element for fast in range(len(nums)): if nums[fast] != 0: # Swap the non-zero element into the \u0026#39;slow\u0026#39; position nums[slow], nums[fast] = nums[fast], nums[slow] slow += 1 # After the loop: # - Elements before \u0026#39;slow\u0026#39; are all non-zero (in original order) # - Elements from \u0026#39;slow\u0026#39; onward are all zeros Edge Cases No zeros: [1, 2, 3] → unchanged. All zeros: [0, 0, 0] → unchanged. Zeros at start: [0, 0, 1] → [1, 0, 0]. Single element: [0] or [5] → unchanged. Dry Run Input: nums = [0, 1, 0, 3, 12] Initial: slow = 0 fast=0: nums[0]=0 → skip (it\u0026#39;s zero) fast=1: nums[1]=1 → swap(nums[0], nums[1]) → [1,0,0,3,12], slow=1 fast=2: nums[2]=0 → skip fast=3: nums[3]=3 → swap(nums[1], nums[3]) → [1,3,0,0,12], slow=2 fast=4: nums[4]=12 → swap(nums[2], nums[4]) → [1,3,12,0,0], slow=3 Output: [1, 3, 12, 0, 0] ✅ Problem 9: Squares of a Sorted Array LeetCode Link: https://leetcode.com/problems/squares-of-a-sorted-array/\nProblem Statement Given an integer array nums sorted in non-decreasing order, return an array of the squares of each number sorted in non-decreasing order.\nExample:\nInput: nums = [-4, -1, 0, 3, 10] Output: [0, 1, 9, 16, 100] Clarifying Questions \u0026amp; Constraints Array is sorted (can have negative numbers). Need to return a new sorted array of squares. Must be O(n) - can\u0026rsquo;t just square and sort (that\u0026rsquo;s O(n log n)). Approach Discussion Approach 1: Square + Sort\nSquare every element, then sort. Time: O(n log n), Space: O(n) ❌ Not O(n). Approach 2: Two Pointers from Both Ends (Optimal) ✅\nKey insight: The largest squares are at the ends of the array (since negatives with large absolute value and large positives both produce large squares). Use two pointers from both ends, compare absolute values, and fill the result array from right to left (largest to smallest). Time: O(n), Space: O(n) (for the result array - required) Code (Both Solutions) # ============================================================ # APPROACH 1: Square + Sort - O(n log n) # ============================================================ def sortedSquares_sort(nums: list[int]) -\u0026gt; list[int]: \u0026#34;\u0026#34;\u0026#34; Square every element and sort the result. Time Complexity: O(n log n) - dominated by sorting Space Complexity: O(n) - for the result array \u0026#34;\u0026#34;\u0026#34; return sorted(x * x for x in nums) # ============================================================ # APPROACH 2: Two Pointers - O(n) ✅ # ============================================================ def sortedSquares(nums: list[int]) -\u0026gt; list[int]: \u0026#34;\u0026#34;\u0026#34; Produce sorted squares using two pointers from both ends. Key insight: In a sorted array with negatives, the LARGEST squares are at the ends (e.g., [-4, -1, 0, 3, 10] → squares: [16, 1, 0, 9, 100]). The largest is either at the far left or far right. Strategy: Compare absolute values at both ends, place the larger square at the END of the result array, and work backwards. Time Complexity: O(n) - single pass through the array Space Complexity: O(n) - for the result array (can\u0026#39;t avoid this) \u0026#34;\u0026#34;\u0026#34; n = len(nums) result = [0] * n # Pre-allocate result array left = 0 right = n - 1 pos = n - 1 # Fill from the end (largest first) while left \u0026lt;= right: left_sq = nums[left] ** 2 right_sq = nums[right] ** 2 if left_sq \u0026gt; right_sq: # Left end has the larger square result[pos] = left_sq left += 1 else: # Right end has the larger square (or they\u0026#39;re equal) result[pos] = right_sq right -= 1 pos -= 1 # Move to the next position (going backwards) return result Edge Cases All positive: [1, 2, 3] → [1, 4, 9]. All negative: [-3, -2, -1] → [1, 4, 9]. Single element: [5] → [25]. Zeros: [-2, 0, 0, 3] → [0, 0, 4, 9]. Dry Run Input: nums = [-4, -1, 0, 3, 10] Initial: left=0, right=4, pos=4, result=[0,0,0,0,0] Step 1: left_sq = (-4)²=16, right_sq = 10²=100 100 \u0026gt; 16 → result[4]=100, right=3 result = [0, 0, 0, 0, 100] Step 2: left_sq = (-4)²=16, right_sq = 3²=9 16 \u0026gt; 9 → result[3]=16, left=1 result = [0, 0, 0, 16, 100] Step 3: left_sq = (-1)²=1, right_sq = 3²=9 9 \u0026gt; 1 → result[2]=9, right=2 result = [0, 0, 9, 16, 100] Step 4: left_sq = (-1)²=1, right_sq = 0²=0 1 \u0026gt; 0 → result[1]=1, left=2 result = [0, 1, 9, 16, 100] Step 5: left=2, right=2 (left \u0026lt;= right) left_sq = 0²=0, right_sq = 0²=0 0 == 0 → result[0]=0, right=1 result = [0, 1, 9, 16, 100] left=2 \u0026gt; right=1 → STOP Output: [0, 1, 9, 16, 100] ✅ Problem 10: Merge Sorted Array LeetCode Link: https://leetcode.com/problems/merge-sorted-array/\nProblem Statement Given two sorted arrays nums1 and nums2, merge nums2 into nums1 as one sorted array in-place. nums1 has a size of m + n, where the first m elements are the actual values and the rest are 0s (placeholders).\nExample:\nInput: nums1 = [1, 2, 3, 0, 0, 0], m = 3, nums2 = [2, 5, 6], n = 3 Output: [1, 2, 2, 3, 5, 6] Clarifying Questions \u0026amp; Constraints nums1 has enough space (size m + n) to hold all elements. Must be done in-place in nums1. Both arrays are sorted in non-decreasing order. Approach Discussion Approach 1: Merge + Sort\nCopy nums2 into the extra space in nums1, then sort. Time: O((m+n) log(m+n)), Space: O(1) ❌ Doesn\u0026rsquo;t leverage the sorted property. Approach 2: Two Pointers from Front (with extra space)\nStandard merge like in merge sort. Time: O(m+n), Space: O(m) - need temp array ❌ Uses extra space. Approach 3: Two Pointers from Back (Optimal) ✅\nKey insight: Start filling nums1 from the end (rightmost position). Compare the largest remaining elements from both arrays. This way, we never overwrite elements in nums1 that we haven\u0026rsquo;t processed yet! Time: O(m+n), Space: O(1) Code (Optimal Solution) def merge(nums1: list[int], m: int, nums2: list[int], n: int) -\u0026gt; None: \u0026#34;\u0026#34;\u0026#34; Merge nums2 into nums1 in-place using two pointers from the back. Why from the back? If we start from the front, we\u0026#39;d overwrite elements in nums1 that we still need. But the END of nums1 is just zeros (placeholders), so it\u0026#39;s safe to write there! Strategy: - p1 points to the last real element in nums1 (index m-1) - p2 points to the last element in nums2 (index n-1) - pos points to the last position in nums1 (index m+n-1) - Compare nums1[p1] and nums2[p2], place the larger one at pos Time Complexity: O(m + n) - each element is placed exactly once Space Complexity: O(1) - everything is done in-place \u0026#34;\u0026#34;\u0026#34; # Pointers starting from the end p1 = m - 1 # Last real element in nums1 p2 = n - 1 # Last element in nums2 pos = m + n - 1 # Last position in nums1 (where we\u0026#39;ll place elements) # Merge from back to front while p1 \u0026gt;= 0 and p2 \u0026gt;= 0: if nums1[p1] \u0026gt; nums2[p2]: # nums1\u0026#39;s element is larger → place it at the end nums1[pos] = nums1[p1] p1 -= 1 else: # nums2\u0026#39;s element is larger (or equal) → place it at the end nums1[pos] = nums2[p2] p2 -= 1 pos -= 1 # If there are remaining elements in nums2, copy them # (If p1 \u0026gt;= 0, those elements are already in place in nums1!) while p2 \u0026gt;= 0: nums1[pos] = nums2[p2] p2 -= 1 pos -= 1 Edge Cases nums2 is empty: nums1=[1], m=1, nums2=[], n=0 → no changes. nums1 is empty: nums1=[0,0], m=0, nums2=[1,2], n=2 → just copy nums2. All nums2 smaller: nums1=[3,4,5,0,0], m=3, nums2=[1,2], n=2 → [1,2,3,4,5]. All nums2 larger: nums1=[1,2,0,0], m=2, nums2=[3,4], n=2 → [1,2,3,4]. Dry Run Input: nums1 = [1, 2, 3, 0, 0, 0], m=3, nums2 = [2, 5, 6], n=3 Initial: p1=2, p2=2, pos=5 Step 1: nums1[2]=3 vs nums2[2]=6 → 6 is bigger nums1[5]=6, p2=1, pos=4 nums1 = [1, 2, 3, 0, 0, 6] Step 2: nums1[2]=3 vs nums2[1]=5 → 5 is bigger nums1[4]=5, p2=0, pos=3 nums1 = [1, 2, 3, 0, 5, 6] Step 3: nums1[2]=3 vs nums2[0]=2 → 3 is bigger nums1[3]=3, p1=1, pos=2 nums1 = [1, 2, 3, 3, 5, 6] Step 4: nums1[1]=2 vs nums2[0]=2 → equal, place nums2 nums1[2]=2, p2=-1, pos=1 nums1 = [1, 2, 2, 3, 5, 6] p2 \u0026lt; 0 → STOP (remaining nums1 elements are already in place) Output: [1, 2, 2, 3, 5, 6] ✅ Key Takeaways \u0026amp; Summary Quick Reference Table Problem Pattern Type Time Space Core Trick Two Sum II Opposite direction O(n) O(1) Sorted → shrink search space from both ends Valid Palindrome Opposite direction O(n) O(1) Skip non-alphanumeric, compare ends Remove Duplicates Same direction O(n) O(1) Slow tracks unique, fast scans Container With Most Water Opposite direction O(n) O(1) Move the shorter line inward 3Sum Sort + opposite O(n²) O(1) Fix one, two-pointer for rest, skip duplicates Trapping Rain Water Opposite direction O(n) O(1) Process the side with smaller max Sort Colors Three pointers O(n) O(1) Dutch National Flag partitioning Move Zeroes Same direction O(n) O(1) Swap non-zeros to the front Squares of Sorted Array Opposite direction O(n) O(n) Largest squares at ends, fill backwards Merge Sorted Array Reverse two pointers O(m+n) O(1) Fill from the back to avoid overwriting Pattern Recognition Cheat Sheet Is the array sorted (or should you sort it)? ├── YES → Can you use OPPOSITE direction pointers? │ ├── Finding a pair with target sum? → Two Sum II pattern │ ├── Finding triplets? → 3Sum pattern (fix one + two pointers) │ ├── Comparing from both ends? → Palindrome / Container pattern │ └── Merging two sorted things? → Merge pattern (consider reverse!) │ └── NO → Can you use SAME direction (slow/fast) pointers? ├── Removing elements in-place? → Remove Duplicates / Move Zeroes ├── Partitioning by categories? → Dutch National Flag └── Need to track running max from both sides? → Trapping Rain Water Top 5 Things to Remember Sorted array = Think two pointers first. The sorted property is what makes moving pointers inward safe - you know which direction gives you a larger or smaller value.\nOpposite direction for pair/comparison problems (shrink the search space), same direction for in-place modification problems (slow writes, fast reads).\nAlways ask: \u0026ldquo;Why is it safe to move this pointer?\u0026rdquo; The proof of correctness for two-pointer problems comes from understanding why you won\u0026rsquo;t miss the optimal answer.\nDuplicate handling is crucial in problems like 3Sum. Always think about what happens with repeated values.\n\u0026ldquo;From the back\u0026rdquo; trick (as in Merge Sorted Array and Squares of Sorted Array) is powerful when you\u0026rsquo;d otherwise overwrite needed data.\nWhat\u0026rsquo;s Next? Now that you\u0026rsquo;ve mastered Two Pointers, the natural next pattern is Sliding Window - it builds on the same-direction pointer concept but adds a \u0026ldquo;window\u0026rdquo; of elements between the two pointers. Stay tuned for Pattern 2!\n💡 Practice Tip: Don\u0026rsquo;t just read the solutions - try solving each problem yourself first for 15-20 minutes. If you\u0026rsquo;re stuck, read only the approach section (not the code) and try again. The struggle is where the learning happens!\n","permalink":"https://blogs.codeverra.com/blog/dsa-patterns-01-two-pointers/","summary":"\u003ch1 id=\"-pattern-1-two-pointers\"\u003e🔰 Pattern 1: Two Pointers\u003c/h1\u003e\n\u003ch2 id=\"table-of-contents\"\u003eTable of Contents\u003c/h2\u003e\n\u003col\u003e\n\u003cli\u003e\u003ca href=\"#what-is-the-two-pointer-technique\"\u003eWhat is the Two Pointer Technique?\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#when-to-use-two-pointers\"\u003eWhen to Use Two Pointers?\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#types-of-two-pointer-patterns\"\u003eTypes of Two Pointer Patterns\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#visual-walkthrough\"\u003eVisual Walkthrough\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#template-code\"\u003eTemplate Code\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#problem-set\"\u003eProblem Set\u003c/a\u003e\n\u003cul\u003e\n\u003cli\u003e\u003ca href=\"#problem-1-two-sum-ii--input-array-is-sorted\"\u003eProblem 1: Two Sum II – Input Array Is Sorted\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#problem-2-valid-palindrome\"\u003eProblem 2: Valid Palindrome\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#problem-3-remove-duplicates-from-sorted-array\"\u003eProblem 3: Remove Duplicates from Sorted Array\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#problem-4-container-with-most-water\"\u003eProblem 4: Container With Most Water\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#problem-5-3sum\"\u003eProblem 5: 3Sum\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#problem-6-trapping-rain-water\"\u003eProblem 6: Trapping Rain Water\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#problem-7-sort-colors-dutch-national-flag\"\u003eProblem 7: Sort Colors (Dutch National Flag)\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#problem-8-move-zeroes\"\u003eProblem 8: Move Zeroes\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#problem-9-squares-of-a-sorted-array\"\u003eProblem 9: Squares of a Sorted Array\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#problem-10-merge-sorted-array\"\u003eProblem 10: Merge Sorted Array\u003c/a\u003e\u003c/li\u003e\n\u003c/ul\u003e\n\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#key-takeaways--summary\"\u003eKey Takeaways \u0026amp; Summary\u003c/a\u003e\u003c/li\u003e\n\u003c/ol\u003e\n\u003chr\u003e\n\u003ch2 id=\"what-is-the-two-pointer-technique\"\u003eWhat is the Two Pointer Technique?\u003c/h2\u003e\n\u003cblockquote\u003e\n\u003cp\u003e\u003cstrong\u003eTwo Pointers\u003c/strong\u003e is a technique where you use two index variables that move through an array or string  -  either toward each other from opposite ends, or in the same direction at different speeds  -  to solve a problem efficiently.\u003c/p\u003e","tags":["dsa","dsa-patterns"],"title":"Two Pointers Pattern  -  Complete Guide"},{"content":"Type Hints in Python Writing Code That Explains Itself \u0026ndash; The typing Module and Beyond Before We Begin \u0026ndash; Why Type Hints Matter Here is a function:\ndef calculate_discount(price, discount, is_member): ... What type is price? A float? An int? A string like \u0026quot;1299\u0026quot;? What is discount? A percentage like 18.5? Or a multiplier like 0.185? What does is_member do to the result?\nYou cannot tell without reading the implementation. In a codebase with hundreds of functions, this adds up to an enormous cognitive load.\nNow look at this:\ndef calculate_discount(price: float, discount: float, is_member: bool) -\u0026gt; float: ... You know exactly what each parameter expects and what the function returns. Before reading a single line of the body.\nType hints are optional annotations that describe the expected types of variables, function parameters, and return values. Python does not enforce them at runtime \u0026ndash; your code runs exactly the same with or without them. But they provide three major benefits:\n1. Documentation that cannot go stale. Comments lie. Type hints are part of the code and tools can check them.\n2. IDE intelligence. VS Code, PyCharm, and other editors use type hints to give you accurate autocomplete, catch typos, and show warnings before you run anything.\n3. Static analysis with mypy. A tool called mypy reads your type hints and reports type errors before your code runs. It catches entire categories of bugs \u0026ndash; passing a string where a list is expected, calling a method on None, returning the wrong type \u0026ndash; all without executing the code.\nType hints become more valuable as your codebase grows. In a 50-line script they are optional convenience. In a 50-file project with multiple developers they are essential infrastructure.\nTable of Contents Basic Type Annotations Function Annotations \u0026ndash; Parameters and Return Types Built-in Collection Types Optional and Union Any \u0026ndash; The Escape Hatch Callable \u0026ndash; Annotating Functions as Arguments Type Aliases TypeVar \u0026ndash; Generic Functions Annotating Classes TypedDict \u0026ndash; Typing Dictionaries Protocols \u0026ndash; Structural Typing Literal \u0026ndash; Specific Allowed Values Final \u0026ndash; Constants Type Guards and Narrowing mypy \u0026ndash; Static Type Checking Practical Patterns Summary and Key Takeaways 1. Basic Type Annotations Variable annotations # Syntax: variable_name: type = value name: str = \u0026#34;Aarav Sharma\u0026#34; age: int = 21 cgpa: float = 8.9 active: bool = True # Annotation without assignment (declares intent) student_id: int # student_id is not yet assigned -- using it raises NameError # but the annotation is recorded in __annotations__ The annotation is a hint, not enforcement # Python does NOT enforce type hints at runtime name: str = 42 # no error -- runs fine name: str = [1,2,3] # still no error # The type checker (mypy) would flag these, not Python itself 2. Function Annotations \u0026ndash; Parameters and Return Types # Syntax: def func(param: type, param2: type) -\u0026gt; return_type: def greet(name: str) -\u0026gt; str: return f\u0026#34;Namaste, {name}!\u0026#34; def add(a: int, b: int) -\u0026gt; int: return a + b def divide(a: float, b: float) -\u0026gt; float: if b == 0: raise ValueError(\u0026#34;Cannot divide by zero\u0026#34;) return a / b def send_notification(user_id: int, message: str) -\u0026gt; None: \u0026#34;\u0026#34;\u0026#34;None means this function returns nothing (no return value).\u0026#34;\u0026#34;\u0026#34; print(f\u0026#34;Sending to {user_id}: {message}\u0026#34;) Inspecting annotations at runtime def calculate_gst(price: float, rate: float = 0.18) -\u0026gt; float: return round(price * (1 + rate), 2) print(calculate_gst.__annotations__) # {\u0026#39;price\u0026#39;: \u0026lt;class \u0026#39;float\u0026#39;\u0026gt;, \u0026#39;rate\u0026#39;: \u0026lt;class \u0026#39;float\u0026#39;\u0026gt;, \u0026#39;return\u0026#39;: \u0026lt;class \u0026#39;float\u0026#39;\u0026gt;} Default values with type hints def create_profile( name: str, age: int, city: str = \u0026#34;Bangalore\u0026#34;, is_premium: bool = False, ) -\u0026gt; dict: return { \u0026#34;name\u0026#34;: name, \u0026#34;age\u0026#34;: age, \u0026#34;city\u0026#34;: city, \u0026#34;is_premium\u0026#34;: is_premium, } 3. Built-in Collection Types Python 3.9+ \u0026ndash; use built-in types directly From Python 3.9 onwards, you can use the built-in collection types directly as generic types without importing from typing.\n# Python 3.9+ def process_marks(marks: list[int]) -\u0026gt; float: return sum(marks) / len(marks) def get_city_populations() -\u0026gt; dict[str, int]: return {\u0026#34;Delhi\u0026#34;: 32941000, \u0026#34;Mumbai\u0026#34;: 20667656} def unique_tags(items: list[str]) -\u0026gt; set[str]: return set(items) def get_coordinates() -\u0026gt; tuple[float, float]: return (28.6139, 77.2090) # Nested def group_by_dept(employees: list[str]) -\u0026gt; dict[str, list[str]]: ... Python 3.8 and below \u0026ndash; import from typing # Python 3.8 and below from typing import List, Dict, Set, Tuple, FrozenSet def process_marks(marks: List[int]) -\u0026gt; float: return sum(marks) / len(marks) def get_city_populations() -\u0026gt; Dict[str, int]: return {\u0026#34;Delhi\u0026#34;: 32941000} def unique_tags(items: List[str]) -\u0026gt; Set[str]: return set(items) def get_coordinates() -\u0026gt; Tuple[float, float]: return (28.6139, 77.2090) Tuple \u0026ndash; fixed length vs variable length # Fixed-length tuple: Tuple[type1, type2, type3] def get_student_record() -\u0026gt; tuple[str, int, float]: return (\u0026#34;Aarav\u0026#34;, 101, 8.9) # Variable-length tuple of a single type: tuple[int, ...] # The ... (Ellipsis) means \u0026#34;zero or more of this type\u0026#34; def get_scores() -\u0026gt; tuple[int, ...]: return (88, 92, 75, 90, 85) 4. Optional and Union Optional \u0026ndash; a value that might be None Optional[X] means the value is either type X or None. It is equivalent to Union[X, None].\nfrom typing import Optional # This function returns a string or None def find_student(student_id: int, students: list[dict]) -\u0026gt; Optional[str]: for student in students: if student[\u0026#34;id\u0026#34;] == student_id: return student[\u0026#34;name\u0026#34;] return None # not found # Parameter that can be None def greet(name: Optional[str] = None) -\u0026gt; str: if name is None: return \u0026#34;Hello, stranger!\u0026#34; return f\u0026#34;Namaste, {name}!\u0026#34; # Python 3.10+ shorthand using | (pipe) def find_student_v2(student_id: int) -\u0026gt; str | None: ... def greet_v2(name: str | None = None) -\u0026gt; str: ... Always narrow Optional before using from typing import Optional def get_discount(user: Optional[dict]) -\u0026gt; float: # mypy will warn if you use user without checking None first if user is None: return 0.0 # After the None check, mypy knows user is dict here return 0.10 if user.get(\u0026#34;is_premium\u0026#34;) else 0.0 Union \u0026ndash; one of several types from typing import Union # A value that can be int OR float def format_price(price: Union[int, float]) -\u0026gt; str: return f\u0026#34;Rs.{price:,.2f}\u0026#34; # Python 3.10+ shorthand def format_price_v2(price: int | float) -\u0026gt; str: return f\u0026#34;Rs.{price:,.2f}\u0026#34; # Union with more types def process_input(value: Union[str, int, list]) -\u0026gt; str: if isinstance(value, list): return \u0026#34;, \u0026#34;.join(str(v) for v in value) return str(value) 5. Any \u0026ndash; The Escape Hatch Any is compatible with every type. Use it sparingly when you genuinely cannot or do not want to type a value.\nfrom typing import Any # A function that accepts anything def log(value: Any) -\u0026gt; None: print(f\u0026#34;[LOG] {value}\u0026#34;) # A container of mixed types def process_raw_data(data: list[Any]) -\u0026gt; list[str]: return [str(item) for item in data] Any disables type checking for that variable. The type checker will not warn you about anything you do with an Any value. Overusing it defeats the purpose of type hints. Use it as a last resort.\n6. Callable \u0026ndash; Annotating Functions as Arguments When a function takes another function as an argument, use Callable.\nfrom typing import Callable # Callable[[arg_types], return_type] # Callable[[int, int], int] means: takes two ints, returns an int def apply(func: Callable[[int, int], int], a: int, b: int) -\u0026gt; int: return func(a, b) def add(x: int, y: int) -\u0026gt; int: return x + y result = apply(add, 3, 4) # 7 # A function that takes no arguments and returns a string def get_greeting_fn() -\u0026gt; Callable[[], str]: def greet(): return \u0026#34;Namaste!\u0026#34; return greet # A callback that accepts anything and returns None def run_with_callback( data: list, callback: Callable[[list], None] ) -\u0026gt; None: callback(data) 7. Type Aliases Give a complex type a short, readable name.\nfrom typing import TypeAlias # Python 3.10+ # Simple alias StudentName: TypeAlias = str Marks: TypeAlias = list[int] StudentID: TypeAlias = int # Complex alias StudentRecord: TypeAlias = dict[str, str | int | list[int]] CityPopMap: TypeAlias = dict[str, int] Matrix: TypeAlias = list[list[float]] # Use the aliases in function signatures def get_average(marks: Marks) -\u0026gt; float: return sum(marks) / len(marks) def find_student( student_id: StudentID, students: list[StudentRecord] ) -\u0026gt; StudentRecord | None: ... # For Python 3.8/3.9 -- just assign directly StudentName = str Marks = list[int] # works but less explicit 8. TypeVar \u0026ndash; Generic Functions TypeVar lets you write functions that work with any type while preserving the type relationship between input and output.\nfrom typing import TypeVar T = TypeVar(\u0026#34;T\u0026#34;) # T can be any type # A function where the return type is the same as the input type def first_element(items: list[T]) -\u0026gt; T: return items[0] # Type checker knows the return type based on input result1 = first_element([1, 2, 3]) # result1 is int result2 = first_element([\u0026#34;a\u0026#34;, \u0026#34;b\u0026#34;]) # result2 is str result3 = first_element([1.0, 2.0]) # result3 is float # Constrained TypeVar -- only specific types allowed Number = TypeVar(\u0026#34;Number\u0026#34;, int, float) def double(value: Number) -\u0026gt; Number: return value * 2 double(5) # valid -- int double(3.14) # valid -- float double(\u0026#34;hi\u0026#34;) # type error -- str not allowed 9. Annotating Classes from typing import Optional, ClassVar class Student: # ClassVar -- belongs to the class, not instances institution: ClassVar[str] = \u0026#34;Codeverra Institute\u0026#34; _count: ClassVar[int] = 0 # Instance variables annotated in __init__ def __init__( self, name: str, roll: int, cgpa: float, courses: list[str] | None = None, ) -\u0026gt; None: self.name: str = name self.roll: int = roll self.cgpa: float = cgpa self.courses: list[str] = courses or [] Student._count += 1 def average_marks(self, marks: list[int]) -\u0026gt; float: return sum(marks) / len(marks) def add_course(self, course: str) -\u0026gt; None: self.courses.append(course) def get_summary(self) -\u0026gt; dict[str, str | int | float]: return { \u0026#34;name\u0026#34;: self.name, \u0026#34;roll\u0026#34;: self.roll, \u0026#34;cgpa\u0026#34;: self.cgpa, } @classmethod def get_count(cls) -\u0026gt; int: return cls._count @staticmethod def is_valid_cgpa(cgpa: float) -\u0026gt; bool: return 0.0 \u0026lt;= cgpa \u0026lt;= 10.0 init with no return type __init__ always returns None. Annotate it as -\u0026gt; None or omit the return annotation \u0026ndash; both are acceptable.\n10. TypedDict \u0026ndash; Typing Dictionaries When a dictionary always has specific keys with specific types, use TypedDict to document and check its structure.\nfrom typing import TypedDict, NotRequired # NotRequired: Python 3.11+ class StudentRecord(TypedDict): name: str roll: int cgpa: float city: str class OrderItem(TypedDict): product: str quantity: int price: float discount: NotRequired[float] # this key is optional # Usage student: StudentRecord = { \u0026#34;name\u0026#34;: \u0026#34;Aarav Sharma\u0026#34;, \u0026#34;roll\u0026#34;: 101, \u0026#34;cgpa\u0026#34;: 8.9, \u0026#34;city\u0026#34;: \u0026#34;Bangalore\u0026#34;, } # Type checker will flag missing keys or wrong types bad_student: StudentRecord = { \u0026#34;name\u0026#34;: \u0026#34;Priya\u0026#34;, \u0026#34;roll\u0026#34;: \u0026#34;102\u0026#34;, # type error -- should be int \u0026#34;cgpa\u0026#34;: 9.1, # city is missing -- type error } # Functions with TypedDict def get_student_name(student: StudentRecord) -\u0026gt; str: return student[\u0026#34;name\u0026#34;] def process_order_items(items: list[OrderItem]) -\u0026gt; float: return sum(item[\u0026#34;quantity\u0026#34;] * item[\u0026#34;price\u0026#34;] for item in items) 11. Protocols \u0026ndash; Structural Typing A Protocol defines an interface \u0026ndash; a set of methods an object must have. Any class that has those methods satisfies the Protocol, regardless of whether it explicitly inherits from it. This is called structural subtyping or duck typing with type safety.\nfrom typing import Protocol, runtime_checkable # Define the interface class Printable(Protocol): def print_summary(self) -\u0026gt; str: ... # the ... means \u0026#34;required but not implemented here\u0026#34; class Serializable(Protocol): def to_dict(self) -\u0026gt; dict: ... def to_json(self) -\u0026gt; str: ... # These classes do NOT inherit from Printable or Serializable # but they satisfy the protocol by having the required methods class Student: def __init__(self, name: str, cgpa: float): self.name = name self.cgpa = cgpa def print_summary(self) -\u0026gt; str: # satisfies Printable return f\u0026#34;Student: {self.name} ({self.cgpa})\u0026#34; class Product: def __init__(self, name: str, price: float): self.name = name self.price = price def print_summary(self) -\u0026gt; str: # also satisfies Printable return f\u0026#34;Product: {self.name} @ Rs.{self.price}\u0026#34; # This function works with ANY Printable -- structural duck typing def display(item: Printable) -\u0026gt; None: print(item.print_summary()) display(Student(\u0026#34;Aarav\u0026#34;, 8.9)) # valid display(Product(\u0026#34;Laptop\u0026#34;, 65000)) # also valid # @runtime_checkable lets you use isinstance() with Protocol @runtime_checkable class HasName(Protocol): name: str class City: def __init__(self, name: str): self.name = name print(isinstance(City(\u0026#34;Delhi\u0026#34;), HasName)) # True 12. Literal \u0026ndash; Specific Allowed Values Literal constrains a value to a specific set of allowed values.\nfrom typing import Literal # Only \u0026#34;asc\u0026#34; or \u0026#34;desc\u0026#34; are valid def sort_students( students: list[dict], order: Literal[\u0026#34;asc\u0026#34;, \u0026#34;desc\u0026#34;] = \u0026#34;asc\u0026#34; ) -\u0026gt; list[dict]: reverse = (order == \u0026#34;desc\u0026#34;) return sorted(students, key=lambda s: s[\u0026#34;name\u0026#34;], reverse=reverse) # Only these specific integers def set_log_level(level: Literal[10, 20, 30, 40, 50]) -\u0026gt; None: import logging logging.getLogger().setLevel(level) # Status strings OrderStatus = Literal[\u0026#34;Placed\u0026#34;, \u0026#34;Confirmed\u0026#34;, \u0026#34;Shipped\u0026#34;, \u0026#34;Delivered\u0026#34;, \u0026#34;Cancelled\u0026#34;] def update_status(order_id: int, status: OrderStatus) -\u0026gt; None: print(f\u0026#34;Order {order_id} status: {status}\u0026#34;) update_status(1, \u0026#34;Shipped\u0026#34;) # valid update_status(1, \u0026#34;Processing\u0026#34;) # type error -- not in Literal 13. Final \u0026ndash; Constants Final marks a variable as a constant that should not be reassigned.\nfrom typing import Final # Module-level constants MAX_STUDENTS: Final = 500 DEFAULT_CITY: Final[str] = \u0026#34;Bangalore\u0026#34; GST_RATE: Final[float] = 0.18 API_VERSION: Final = \u0026#34;v2\u0026#34; # In a class class Config: MAX_RETRIES: Final = 3 BASE_URL: Final[str] = \u0026#34;https://api.codeverra.com\u0026#34; DEBUG: Final[bool] = False # Type checker will warn if you try to reassign MAX_STUDENTS = 600 # type error -- cannot reassign Final 14. Type Guards and Narrowing Type narrowing is when the type checker understands that within a specific code block, a variable has a more specific type.\nfrom typing import Union def process(value: Union[str, int, list]) -\u0026gt; str: # After isinstance check, type is narrowed if isinstance(value, str): return value.upper() # type checker knows value is str here elif isinstance(value, int): return str(value * 2) # type checker knows value is int here else: return \u0026#34;, \u0026#34;.join(str(v) for v in value) # type checker knows: list # None narrowing from typing import Optional def get_name_length(name: Optional[str]) -\u0026gt; int: if name is None: return 0 # After the None check, type checker knows name is str here return len(name) # .upper() and len() now work without warnings TypeGuard \u0026ndash; custom type narrowing functions from typing import TypeGuard def is_string_list(val: list) -\u0026gt; TypeGuard[list[str]]: \u0026#34;\u0026#34;\u0026#34;Returns True if val is a list where all items are strings.\u0026#34;\u0026#34;\u0026#34; return all(isinstance(x, str) for x in val) def process_names(data: list) -\u0026gt; None: if is_string_list(data): # type checker knows data is list[str] here for name in data: print(name.upper()) # .upper() is valid -- name is str 15. mypy \u0026ndash; Static Type Checking mypy is the standard static type checker for Python. It reads your type annotations and reports errors without running your code.\nInstallation and basic use pip install mypy # Check a single file mypy main.py # Check an entire directory mypy my_project/ # More strict checking mypy --strict main.py Understanding mypy output # example.py def greet(name: str) -\u0026gt; str: return f\u0026#34;Namaste, {name}!\u0026#34; greet(42) # passing int where str expected $ mypy example.py example.py:4: error: Argument 1 to \u0026#34;greet\u0026#34; has incompatible type \u0026#34;int\u0026#34;; expected \u0026#34;str\u0026#34; Found 1 error in 1 file (checked 1 source file) Common mypy errors and fixes # Error: Item \u0026#34;None\u0026#34; of \u0026#34;str | None\u0026#34; has no attribute \u0026#34;upper\u0026#34; def process(name: str | None) -\u0026gt; str: return name.upper() # mypy error -- name might be None # Fix: narrow the type first def process(name: str | None) -\u0026gt; str: if name is None: return \u0026#34;\u0026#34; return name.upper() # safe -- name is str here # Error: Incompatible return value type def get_count() -\u0026gt; int: return \u0026#34;five\u0026#34; # mypy error -- returning str, not int # Fix: return correct type def get_count() -\u0026gt; int: return 5 # Error: Cannot infer type of \u0026#34;result\u0026#34; result = [] # mypy: need type annotation result.append(1) # Fix: annotate result: list[int] = [] result.append(1) mypy configuration \u0026ndash; mypy.ini # mypy.ini (in project root) [mypy] python_version = 3.11 warn_return_any = True warn_unused_configs = True disallow_untyped_defs = True ignore_missing_imports = True # Per-module overrides [mypy-pandas.*] ignore_missing_imports = True [mypy-numpy.*] ignore_missing_imports = True Gradual typing \u0026ndash; adding types incrementally You do not need to type everything at once. mypy works with partially typed codebases. Start with the most important functions and add more annotations over time.\n# Ignore a specific line result = get_data() # type: ignore # Ignore a specific error result = get_data() # type: ignore[assignment] 16. Practical Patterns Pattern 1 \u0026ndash; Type a data pipeline from typing import Optional # Define types for your data domain StudentID = int CourseName = str MarksRecord = dict[CourseName, float] def load_students(filepath: str) -\u0026gt; list[dict[str, str | int]]: \u0026#34;\u0026#34;\u0026#34;Load student records from CSV.\u0026#34;\u0026#34;\u0026#34; ... def calculate_cgpa(marks: MarksRecord) -\u0026gt; float: \u0026#34;\u0026#34;\u0026#34;Calculate CGPA from subject marks.\u0026#34;\u0026#34;\u0026#34; if not marks: return 0.0 return round(sum(marks.values()) / len(marks) / 10, 2) def get_toppers( students: list[dict], threshold: float = 8.5, top_n: Optional[int] = None, ) -\u0026gt; list[dict]: \u0026#34;\u0026#34;\u0026#34;Return students above CGPA threshold, optionally limited to top N.\u0026#34;\u0026#34;\u0026#34; qualifying = [s for s in students if s.get(\u0026#34;cgpa\u0026#34;, 0) \u0026gt;= threshold] qualifying.sort(key=lambda s: s[\u0026#34;cgpa\u0026#34;], reverse=True) return qualifying[:top_n] if top_n else qualifying Pattern 2 \u0026ndash; Type a class hierarchy from typing import Protocol from abc import ABC, abstractmethod class PaymentProcessor(Protocol): def process(self, amount: float) -\u0026gt; bool: ... def refund(self, amount: float, reason: str) -\u0026gt; bool: ... class UPIPayment: def __init__(self, upi_id: str) -\u0026gt; None: self.upi_id = upi_id def process(self, amount: float) -\u0026gt; bool: print(f\u0026#34;Processing Rs.{amount:.2f} via UPI {self.upi_id}\u0026#34;) return True def refund(self, amount: float, reason: str) -\u0026gt; bool: print(f\u0026#34;Refunding Rs.{amount:.2f}: {reason}\u0026#34;) return True # Works with any PaymentProcessor without explicit inheritance def checkout(processor: PaymentProcessor, total: float) -\u0026gt; None: success = processor.process(total) if not success: print(\u0026#34;Payment failed\u0026#34;) Pattern 3 \u0026ndash; Type hints in real Pandas code import pandas as pd from typing import Optional def load_ipl_data(filepath: str) -\u0026gt; pd.DataFrame: return pd.read_csv(filepath) def filter_by_team( df: pd.DataFrame, team: str, ) -\u0026gt; pd.DataFrame: return df[df[\u0026#34;batting_team\u0026#34;] == team] def top_scorers( df: pd.DataFrame, n: int = 10, min_runs: Optional[int] = None, ) -\u0026gt; pd.DataFrame: grouped = df.groupby(\u0026#34;batsman\u0026#34;)[\u0026#34;batsman_runs\u0026#34;].sum().reset_index() grouped.columns = [\u0026#34;batsman\u0026#34;, \u0026#34;total_runs\u0026#34;] if min_runs: grouped = grouped[grouped[\u0026#34;total_runs\u0026#34;] \u0026gt;= min_runs] return grouped.nlargest(n, \u0026#34;total_runs\u0026#34;) Pattern 4 \u0026ndash; Overload for multiple signatures When a function behaves differently based on input type:\nfrom typing import overload @overload def format_value(value: int) -\u0026gt; str: ... @overload def format_value(value: float) -\u0026gt; str: ... @overload def format_value(value: list[int]) -\u0026gt; list[str]: ... def format_value(value): if isinstance(value, list): return [str(v) for v in value] return f\u0026#34;{value:.2f}\u0026#34; if isinstance(value, float) else str(value) 17. Summary and Key Takeaways What type hints give you Type hints do not change how your code runs. They give you documentation that tools can check, IDE intelligence that makes you faster, and static analysis that catches bugs before they reach production.\nThe essential types to know # Primitives x: int = 42 y: float = 3.14 z: str = \u0026#34;hello\u0026#34; b: bool = True n: None = None # Collections (Python 3.9+) lst: list[int] = [1, 2, 3] dct: dict[str, int] = {\u0026#34;a\u0026#34;: 1} tup: tuple[str, int] = (\u0026#34;Aarav\u0026#34;, 21) st: set[str] = {\u0026#34;python\u0026#34;, \u0026#34;data\u0026#34;} # Optional and Union maybe: str | None = None # Python 3.10+ either: int | str = 42 # Python 3.10+ from typing import Optional, Union maybe2: Optional[str] = None # older syntax either2: Union[int, str] = 42 # older syntax # Any (avoid overuse) from typing import Any anything: Any = ... # Callable from typing import Callable fn: Callable[[int, int], int] # takes two ints, returns int # Return nothing def side_effect() -\u0026gt; None: ... The modern shorthand (Python 3.10+) # Old # New (3.10+) Optional[str] --\u0026gt; str | None Union[int, str] --\u0026gt; int | str Union[int, str, None] --\u0026gt; int | str | None Adoption strategy Start here: 1. Annotate all function signatures (parameters and return types) 2. Annotate class __init__ parameters These two alone catch 80% of type-related bugs. Then add: 3. Annotate module-level variables and constants 4. Use TypedDict for dictionary schemas 5. Use Literal for constrained string/int values Advanced: 6. Add Protocol for structural typing 7. Add TypeVar for generic functions 8. Run mypy in CI/CD to prevent type regressions Common mistakes # 1. Using Optional when you mean Union # Wrong: def process(x: Optional[Union[int, str]]) -\u0026gt; None: ... # Right: just use | None def process(x: int | str | None) -\u0026gt; None: ... # 2. Forgetting to narrow Optional before use def get_length(s: Optional[str]) -\u0026gt; int: return len(s) # error: s might be None # Fix: def get_length(s: Optional[str]) -\u0026gt; int: return len(s) if s is not None else 0 # 3. Using List (capital L) in Python 3.9+ from typing import List def process(items: List[int]) -\u0026gt; None: ... # works but outdated def process(items: list[int]) -\u0026gt; None: ... # preferred in 3.9+ # 4. Overusing Any def process(data: Any) -\u0026gt; Any: ... # defeats the purpose Made with care for Codeverra learners | codeverra.com\n","permalink":"https://blogs.codeverra.com/blog/python-18-type-hints/","summary":"\u003ch1 id=\"type-hints-in-python\"\u003eType Hints in Python\u003c/h1\u003e\n\u003ch3 id=\"writing-code-that-explains-itself--the-typing-module-and-beyond\"\u003eWriting Code That Explains Itself \u0026ndash; The typing Module and Beyond\u003c/h3\u003e\n\u003chr\u003e\n\u003ch2 id=\"before-we-begin--why-type-hints-matter\"\u003eBefore We Begin \u0026ndash; Why Type Hints Matter\u003c/h2\u003e\n\u003cp\u003eHere is a function:\u003c/p\u003e\n\u003cdiv class=\"highlight\"\u003e\u003cpre tabindex=\"0\" class=\"chroma\"\u003e\u003ccode class=\"language-python\" data-lang=\"python\"\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"k\"\u003edef\u003c/span\u003e \u003cspan class=\"nf\"\u003ecalculate_discount\u003c/span\u003e\u003cspan class=\"p\"\u003e(\u003c/span\u003e\u003cspan class=\"n\"\u003eprice\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e \u003cspan class=\"n\"\u003ediscount\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e \u003cspan class=\"n\"\u003eis_member\u003c/span\u003e\u003cspan class=\"p\"\u003e):\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e    \u003cspan class=\"o\"\u003e...\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/code\u003e\u003c/pre\u003e\u003c/div\u003e\u003cp\u003eWhat type is \u003ccode\u003eprice\u003c/code\u003e? A float? An int? A string like \u003ccode\u003e\u0026quot;1299\u0026quot;\u003c/code\u003e?\nWhat is \u003ccode\u003ediscount\u003c/code\u003e? A percentage like \u003ccode\u003e18.5\u003c/code\u003e? Or a multiplier like \u003ccode\u003e0.185\u003c/code\u003e?\nWhat does \u003ccode\u003eis_member\u003c/code\u003e do to the result?\u003c/p\u003e\n\u003cp\u003eYou cannot tell without reading the implementation. In a codebase with\nhundreds of functions, this adds up to an enormous cognitive load.\u003c/p\u003e","tags":["python"],"title":"Type Hints in Python"},{"content":"What is DSA and Why Do We Learn It Table of Contents What You Will Learn in This Blog What is a Data Structure What is an Algorithm Why Do We Learn DSA How This Guide is Structured How to Practice \u0026ndash; The Right Way The Problem-Solving Framework How to Use LeetCode Common Mistakes to Avoid What You Need Before Continuing Summary Key Takeaways Next Steps What You Will Learn in This Blog What data structures are and why they exist What algorithms are and how they relate to data structures Why learning DSA makes you a better problem solver How this guide is structured and how to navigate it How to practice effectively \u0026ndash; LeetCode strategy, problem-solving framework, and common mistakes to avoid What is a Data Structure When you write a program, you are almost always working with data. You store it, retrieve it, update it, and delete it. A data structure is simply a way of organising that data so that your program can work with it efficiently.\nThink of it like this. Imagine you have a pile of books on the floor. You can find a specific book, but it takes time because there is no order. Now imagine those same books arranged alphabetically on a shelf. Finding a book is now much faster. The books are the data. The shelf with alphabetical ordering is the data structure.\nDifferent situations call for different data structures. Sometimes you need fast lookup. Sometimes you need to process things in the order they arrived. Sometimes you need to find the shortest path between two points. The data structure you choose directly affects how efficiently your program runs.\nHere are a few examples to make this concrete:\nSituation A good data structure to use Why You want to check if a username already exists Hash Set Lookup is O(1) on average You are processing customer support tickets in order of arrival Queue First in, first out You are implementing the back button in a browser Stack Last in, first out You want to find the shortest route between two cities Graph Models connections between points You want autocomplete suggestions as someone types Trie Efficient prefix search You do not need to understand all of these yet. The point is that each data structure is a tool, and knowing which tool to use for which job is exactly what this guide teaches you.\nWhat is an Algorithm An algorithm is a step-by-step set of instructions to solve a problem. Every time you write code that does something, you are implementing an algorithm \u0026ndash; whether you realise it or not.\nA simple example: searching for a name in a list.\nOne approach is to go through every name one by one until you find it. That works. But if the list is sorted, you can do something smarter \u0026ndash; start in the middle, check if the name you are looking for comes before or after, and eliminate half the list with each step. That is binary search, and it is dramatically faster.\nBoth approaches are algorithms. The difference is how efficiently they solve the problem. And that difference matters enormously at scale.\nThe relationship between data structures and algorithms is tight. Algorithms operate on data structures. The same algorithm can perform very differently depending on the data structure it is working with. This is why DSA is always taught together \u0026ndash; they are two sides of the same coin.\nWhy Do We Learn DSA This is the most important question to answer honestly, because the wrong answer leads to the wrong mindset.\nThe wrong reason: to pass a coding test.\nThe right reason: to become a better problem solver and write code that actually works well.\nHere is what DSA actually gives you:\n1. The ability to think about efficiency\nWithout DSA, you might write code that works correctly but is far too slow for real use. DSA gives you the vocabulary and the tools to ask: \u0026ldquo;Is there a better way to do this?\u0026rdquo; and actually answer that question.\n2. A toolkit of reusable patterns\nMost problems, however they are phrased, reduce to a small set of patterns. Recognising those patterns is a skill. DSA is how you build that skill.\n3. Confidence with hard problems\nHard problems feel hard because you do not yet have the building blocks to break them down. Once you understand trees, graphs, dynamic programming, and the other structures in this guide, problems that used to feel impossible start to feel approachable.\n4. Better code in everyday work\nKnowing the difference between a list and a hash map, understanding when to use a stack, knowing why sorting before searching is often worth it \u0026ndash; these are not just academic concepts. They show up in real code every day.\nHow This Guide is Structured This guide is divided into six parts. Each part builds on the previous one.\nPart What it covers Foundation What DSA is, how to practice, Python tools you need Complexity Analysis How to measure and compare the efficiency of code Data Structures Arrays, Strings, Linked Lists, Stacks, Queues, Hash Maps, Trees, Heaps, Graphs Algorithms Sorting, Binary Search, Recursion, Backtracking, Dynamic Programming, Greedy, Graph Algorithms Advanced Data Structures Tries, Union-Find, Segment Trees, Bit Manipulation Pattern Playbook A consolidated reference of every pattern you have learned, for review and revision Every data structure blog follows this structure:\nThe concept explained from first principles Python implementation with documented code Time and space complexity of all operations Patterns and techniques that come from this data structure LeetCode problems to practice, ordered from easy to hard Do not skip ahead. The guide is sequential by design. Complexity analysis, for example, is covered before any data structure because every data structure blog uses that vocabulary.\nHow to Practice \u0026ndash; The Right Way This is where most people go wrong. They read a solution, think they understand it, move on, and then cannot solve a similar problem a week later. Here is a framework that actually works.\nThe Problem-Solving Framework When you sit down with a new problem, follow these steps in order. Do not jump to code immediately.\nStep 1: Understand the problem fully\nRead the problem twice. Identify:\nWhat is the input? What is the output? What are the constraints? (size of input, value ranges) What are the edge cases? (empty input, single element, negative numbers) Write out a small example by hand before touching code.\nStep 2: Think out loud before coding\nAsk yourself:\nWhat data structure fits this problem? Have I seen a similar pattern before? What is the brute force solution? (Always start here \u0026ndash; a slow correct solution is better than no solution) Can I improve on brute force? Step 3: Write the brute force first\nCode the simplest solution that works. Do not optimise yet. Getting a working solution on paper gives you something to improve, and sometimes the brute force is good enough.\nStep 4: Analyse and optimise\nNow ask: what is the time and space complexity of my solution? Can I do better? This is where your knowledge of data structures and algorithms directly applies.\nStep 5: Code the optimised solution cleanly\nWrite it with proper variable names and comments. Sloppy code that works is still sloppy.\nStep 6: Test with edge cases\nRun through your edge cases manually before submitting. Empty input, single element, maximum size input, repeated elements.\nHow to Use LeetCode LeetCode is a tool, not a goal. Here is how to use it well.\nSpend at least 20-30 minutes on a problem before looking at a hint\nThe struggle is where the learning happens. If you look at the solution immediately, you are not learning to solve problems \u0026ndash; you are learning to read solutions. Those are very different skills.\nWhen you are truly stuck, look at the hint, not the full solution\nA small nudge in the right direction is much more valuable than reading the full answer. Try to implement after just the hint.\nWhen you read a solution, do not just read it \u0026ndash; reproduce it\nClose the solution. Open a blank editor. Write it from memory. If you cannot, read it again and try once more. Repeat until you can write it without looking.\nCome back to the problem 2-3 days later\nSpaced repetition is one of the most well-researched learning techniques. Solving a problem once and never revisiting it is a weak way to learn. Schedule a review.\nTrack what you have solved and what pattern it used\nUse the Codeverra DSA sheet (linked in the roadmap) to track your progress. Note the pattern, not just the problem name. \u0026ldquo;This was a sliding window problem\u0026rdquo; is more useful than \u0026ldquo;I solved LC 3.\u0026rdquo;\nCommon Mistakes to Avoid Mistake Why it hurts What to do instead Reading the solution immediately You learn recognition, not problem solving Spend at least 20 minutes struggling first Solving a problem once and moving on You forget it within a week Revisit problems after 2-3 days Skipping complexity analysis You do not know if your solution is actually good Always state time and space complexity after solving Jumping to the optimised solution You skip the thinking process Always write brute force first Solving 200 problems without understanding patterns Volume without insight Group problems by pattern, understand the why Giving up on hard problems too quickly Hard problems build the most skill Sit with discomfort longer before seeking help What You Need Before Continuing This guide assumes you are comfortable with basic Python. Specifically:\nVariables, data types, conditionals, loops Functions and return values Python lists and dictionaries at a basic level Basic understanding of classes (helpful but not required immediately) If you need a refresher on any of these, the next blog covers exactly the Python you need for DSA \u0026ndash; no more, no less.\nSummary A data structure is a way of organising data so it can be used efficiently. Different problems call for different data structures. An algorithm is a step-by-step procedure to solve a problem. Algorithms operate on data structures and their efficiency depends heavily on which data structure they use. We learn DSA to become better problem solvers, to write efficient code, and to build a toolkit of patterns that apply across thousands of problems. This guide is structured in six parts, from foundations through to advanced structures, with LeetCode problems integrated into every topic. Effective practice means struggling before looking at solutions, reproducing solutions from memory, revisiting problems after a few days, and always thinking in terms of patterns and complexity. Key Takeaways Data structures and algorithms are not separate topics \u0026ndash; they are two sides of the same coin. The goal is pattern recognition and problem-solving ability, not memorisation of solutions. The problem-solving framework is: understand fully, think before coding, brute force first, then optimise. Spaced repetition and active recall are the most effective ways to retain what you learn. LeetCode is a practice tool. Use it to test your understanding, not to chase a number. Next Steps Next blog: [Python Refresher for DSA] \u0026ndash; the Python tools you will use throughout this guide After that: [Complexity Analysis] \u0026ndash; how to measure and compare the efficiency of any piece of code Reference: Codeverra DSA Sheet \u0026ndash; use this to track the problems you solve as you work through the guide ","permalink":"https://blogs.codeverra.com/blog/dsa-01-what-is-dsa/","summary":"\u003ch1 id=\"what-is-dsa-and-why-do-we-learn-it\"\u003eWhat is DSA and Why Do We Learn It\u003c/h1\u003e\n\u003chr\u003e\n\u003ch2 id=\"table-of-contents\"\u003eTable of Contents\u003c/h2\u003e\n\u003cul\u003e\n\u003cli\u003e\u003ca href=\"#what-you-will-learn-in-this-blog\"\u003eWhat You Will Learn in This Blog\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#what-is-a-data-structure\"\u003eWhat is a Data Structure\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#what-is-an-algorithm\"\u003eWhat is an Algorithm\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#why-do-we-learn-dsa\"\u003eWhy Do We Learn DSA\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#how-this-guide-is-structured\"\u003eHow This Guide is Structured\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#how-to-practice----the-right-way\"\u003eHow to Practice \u0026ndash; The Right Way\u003c/a\u003e\n\u003cul\u003e\n\u003cli\u003e\u003ca href=\"#the-problem-solving-framework\"\u003eThe Problem-Solving Framework\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#how-to-use-leetcode\"\u003eHow to Use LeetCode\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#common-mistakes-to-avoid\"\u003eCommon Mistakes to Avoid\u003c/a\u003e\u003c/li\u003e\n\u003c/ul\u003e\n\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#what-you-need-before-continuing\"\u003eWhat You Need Before Continuing\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#summary\"\u003eSummary\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#key-takeaways\"\u003eKey Takeaways\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#next-steps\"\u003eNext Steps\u003c/a\u003e\u003c/li\u003e\n\u003c/ul\u003e\n\u003chr\u003e\n\u003ch2 id=\"what-you-will-learn-in-this-blog\"\u003eWhat You Will Learn in This Blog\u003c/h2\u003e\n\u003cul\u003e\n\u003cli\u003eWhat data structures are and why they exist\u003c/li\u003e\n\u003cli\u003eWhat algorithms are and how they relate to data structures\u003c/li\u003e\n\u003cli\u003eWhy learning DSA makes you a better problem solver\u003c/li\u003e\n\u003cli\u003eHow this guide is structured and how to navigate it\u003c/li\u003e\n\u003cli\u003eHow to practice effectively \u0026ndash; LeetCode strategy, problem-solving framework, and common mistakes to avoid\u003c/li\u003e\n\u003c/ul\u003e\n\u003chr\u003e\n\u003ch2 id=\"what-is-a-data-structure\"\u003eWhat is a Data Structure\u003c/h2\u003e\n\u003cp\u003eWhen you write a program, you are almost always working with data. You store it, retrieve it, update it, and delete it. A data structure is simply a way of organising that data so that your program can work with it efficiently.\u003c/p\u003e","tags":["dsa"],"title":"What is DSA and Why Do We Learn It"},{"content":"Working with the Real World APIs, JSON, Dates, and HTTP Requests in Python Before We Begin \u0026ndash; What Changes After This Guide Everything you have learned so far has worked entirely inside your own machine. You write code, it runs, it produces output, it reads files you already have.\nThis guide is where that changes.\nAfter this guide your Python programs will be able to:\nAsk a server in another country for the current weather in Mumbai Get the latest currency exchange rate between INR and USD Pull real-time cricket scores from a sports API Send data to a web service and get a response back Work with dates and times the way real applications do This is not tutorial-level Python anymore. This is the Python that powers every data pipeline, every web application, every automation script, and every data science project that talks to the outside world.\nThe three tools you will learn:\njson \u0026ndash; the language that APIs speak. Every web service sends and receives data in JSON format. You already saw the basics in the modules guide \u0026ndash; here you learn everything.\ndatetime \u0026ndash; the module for working with dates and times. API responses always include timestamps. Data always has dates. Understanding datetime properly separates code that handles time correctly from code that silently produces wrong answers.\nrequests \u0026ndash; the most downloaded Python library in the world. The tool that lets your Python code speak HTTP \u0026ndash; the same protocol your browser uses to load websites.\nBy the end, you will build a complete project that calls real public APIs, processes the responses, handles errors, and presents the output cleanly.\nTable of Contents What is an API and Why Does It Matter? How HTTP Works \u0026ndash; The Request-Response Cycle JSON \u0026ndash; The Language of APIs The datetime Module \u0026ndash; Complete Guide The requests Library Query Parameters and Headers Handling Errors Properly POST Requests and Sending Data Sessions and Authentication Basics Complete Project \u0026ndash; India Weather and Currency Dashboard Summary and Key Takeaways Practice Questions Solutions 1. What is an API and Why Does It Matter? The restaurant analogy Imagine you walk into a restaurant. You cannot walk into the kitchen and make your own food. Instead, there is a menu \u0026ndash; a defined list of things you can ask for. You place an order with the waiter. The kitchen prepares it. The waiter brings it back.\nAn API (Application Programming Interface) works exactly like this.\nThe menu is the API documentation \u0026ndash; the list of things you can ask for You are the client (your Python code) The waiter is the HTTP protocol The kitchen is the server The food is the data you get back You never see how the kitchen works. You do not need to. You just follow the menu and get your order back.\nWhy APIs exist Consider these scenarios:\nWeather data: The Indian Meteorological Department has sensors across the country collecting real-time weather data. They expose an API so that apps like weather.com, Zomato (for restaurant delivery ETAs), and farmers' dashboards can all access that data without each needing to build their own sensor network.\nPayment processing: Razorpay and PayU have built complex, secure payment infrastructure. They expose an API so that every e-commerce site in India does not have to rebuild that infrastructure. Your food delivery app calls Razorpay\u0026rsquo;s API when you pay for your order.\nCurrency rates: Banks and financial data providers collect exchange rates. They expose APIs so that fintech apps, import/export software, and travel platforms can show accurate conversion rates.\nAPIs let services share data and functionality without sharing their internal code or infrastructure.\nREST APIs \u0026ndash; the standard Most modern APIs are REST APIs (Representational State Transfer). REST is not a protocol or a technology \u0026ndash; it is a set of conventions for how to structure an API over HTTP. The conventions are:\nGET /students -- retrieve a list of students GET /students/101 -- retrieve student with ID 101 POST /students -- create a new student PUT /students/101 -- update student 101 (full update) PATCH /students/101 -- update student 101 (partial update) DELETE /students/101 -- delete student 101 Each URL is a resource. The HTTP method tells you what to do with it.\n2. How HTTP Works \u0026ndash; The Request-Response Cycle When your browser loads a webpage, or when your Python code calls an API, the same thing happens:\nYour Python Code (Client) | | HTTP Request | ┌─────────────────────────────────┐ | │ Method: GET │ | │ URL: api.example.com/data │ | │ Headers: Accept: application/json│ | │ Body: (empty for GET) │ | └─────────────────────────────────┘ | ▼ (travels over the internet) | Remote Server | | HTTP Response | ┌─────────────────────────────────┐ | │ Status: 200 OK │ | │ Headers: Content-Type: app/json │ | │ Body: {\u0026#34;temperature\u0026#34;: 32, │ | │ \u0026#34;city\u0026#34;: \u0026#34;Mumbai\u0026#34;} │ | └─────────────────────────────────┘ | ▼ Your Python Code (receives and processes response) HTTP Status Codes Status codes tell you what happened. You must always check these in your code.\n2xx -- Success 200 OK -- request succeeded, data in response body 201 Created -- resource was created successfully (POST) 204 No Content -- success but no data to return (DELETE) 3xx -- Redirection 301 Moved Permanently -- URL has changed permanently 302 Found -- temporary redirect 4xx -- Client Error (you made a mistake) 400 Bad Request -- your request is malformed 401 Unauthorized -- authentication required 403 Forbidden -- authenticated but not allowed 404 Not Found -- the resource does not exist 422 Unprocessable -- request understood but invalid data 429 Too Many Requests -- you are being rate-limited 5xx -- Server Error (their problem) 500 Internal Server Error -- something crashed on the server 502 Bad Gateway -- server received an invalid response 503 Service Unavailable -- server is down or overloaded # In code, always check status codes response = requests.get(\u0026#34;https://api.example.com/data\u0026#34;) if response.status_code == 200: data = response.json() elif response.status_code == 404: print(\u0026#34;Resource not found\u0026#34;) elif response.status_code == 429: print(\u0026#34;Rate limited -- wait before retrying\u0026#34;) else: print(f\u0026#34;Unexpected status: {response.status_code}\u0026#34;) 3. JSON \u0026ndash; The Language of APIs JSON (JavaScript Object Notation) is the data format that virtually every modern API uses to send and receive data. Despite the name, it has nothing to do with JavaScript in practice \u0026ndash; it is a language-independent text format that maps cleanly to Python\u0026rsquo;s own data structures.\nJSON to Python mapping JSON Type Python Type Example ───────────────────────────────────────────────────── object dict {\u0026#34;name\u0026#34;: \u0026#34;Aarav\u0026#34;} array list [1, 2, 3] string str \u0026#34;Namaste\u0026#34; number (int) int 42 number (float) float 3.14 true True true → True false False false → False null None null → None The four core json functions import json # json.loads() -- parse a JSON STRING into a Python object json_string = \u0026#39;{\u0026#34;name\u0026#34;: \u0026#34;Aarav\u0026#34;, \u0026#34;city\u0026#34;: \u0026#34;Bangalore\u0026#34;, \u0026#34;score\u0026#34;: 88}\u0026#39; data = json.loads(json_string) print(data) # {\u0026#39;name\u0026#39;: \u0026#39;Aarav\u0026#39;, \u0026#39;city\u0026#39;: \u0026#39;Bangalore\u0026#39;, \u0026#39;score\u0026#39;: 88} print(type(data)) # \u0026lt;class \u0026#39;dict\u0026#39;\u0026gt; print(data[\u0026#34;name\u0026#34;]) # Aarav # json.dumps() -- convert a Python object TO a JSON string student = { \u0026#34;name\u0026#34;: \u0026#34;Priya Patel\u0026#34;, \u0026#34;roll\u0026#34;: 102, \u0026#34;marks\u0026#34;: [88, 92, 75, 90, 85], \u0026#34;active\u0026#34;: True, \u0026#34;notes\u0026#34;: None } json_out = json.dumps(student) print(json_out) # {\u0026#34;name\u0026#34;: \u0026#34;Priya Patel\u0026#34;, \u0026#34;roll\u0026#34;: 102, \u0026#34;marks\u0026#34;: [88, 92, 75, 90, 85], \u0026#34;active\u0026#34;: true, \u0026#34;notes\u0026#34;: null} # json.dumps() with formatting options pretty = json.dumps(student, indent=2, sort_keys=True) print(pretty) # { # \u0026#34;active\u0026#34;: true, # \u0026#34;marks\u0026#34;: [88, 92, 75, 90, 85], # \u0026#34;name\u0026#34;: \u0026#34;Priya Patel\u0026#34;, # ... # } # json.load() -- read JSON FROM a file with open(\u0026#34;students.json\u0026#34;, \u0026#34;r\u0026#34;, encoding=\u0026#34;utf-8\u0026#34;) as f: students = json.load(f) # json.dump() -- write Python object TO a file as JSON with open(\u0026#34;output.json\u0026#34;, \u0026#34;w\u0026#34;, encoding=\u0026#34;utf-8\u0026#34;) as f: json.dump(student, f, indent=2, ensure_ascii=False) # ensure_ascii=False preserves non-ASCII characters (Hindi, Tamil, etc.) Working with nested JSON Real API responses are deeply nested. You need to navigate them confidently.\n# A typical API response from a cricket scores API api_response = \u0026#34;\u0026#34;\u0026#34; { \u0026#34;status\u0026#34;: \u0026#34;success\u0026#34;, \u0026#34;match\u0026#34;: { \u0026#34;teams\u0026#34;: { \u0026#34;home\u0026#34;: {\u0026#34;name\u0026#34;: \u0026#34;India\u0026#34;, \u0026#34;short\u0026#34;: \u0026#34;IND\u0026#34;, \u0026#34;score\u0026#34;: 287}, \u0026#34;away\u0026#34;: {\u0026#34;name\u0026#34;: \u0026#34;Australia\u0026#34;, \u0026#34;short\u0026#34;: \u0026#34;AUS\u0026#34;, \u0026#34;score\u0026#34;: 245} }, \u0026#34;venue\u0026#34;: \u0026#34;Wankhede Stadium, Mumbai\u0026#34;, \u0026#34;format\u0026#34;: \u0026#34;ODI\u0026#34;, \u0026#34;status\u0026#34;: \u0026#34;India won by 42 runs\u0026#34; }, \u0026#34;top_performers\u0026#34;: [ {\u0026#34;name\u0026#34;: \u0026#34;Rohit Sharma\u0026#34;, \u0026#34;runs\u0026#34;: 87, \u0026#34;balls\u0026#34;: 76}, {\u0026#34;name\u0026#34;: \u0026#34;Virat Kohli\u0026#34;, \u0026#34;runs\u0026#34;: 102, \u0026#34;balls\u0026#34;: 95}, {\u0026#34;name\u0026#34;: \u0026#34;Jasprit Bumrah\u0026#34;, \u0026#34;wickets\u0026#34;: 3, \u0026#34;economy\u0026#34;: 4.2} ], \u0026#34;timestamp\u0026#34;: \u0026#34;2024-03-15T19:30:00Z\u0026#34; } \u0026#34;\u0026#34;\u0026#34; data = json.loads(api_response) # Navigate nested structure with chained key access home_team = data[\u0026#34;match\u0026#34;][\u0026#34;teams\u0026#34;][\u0026#34;home\u0026#34;][\u0026#34;name\u0026#34;] home_score = data[\u0026#34;match\u0026#34;][\u0026#34;teams\u0026#34;][\u0026#34;home\u0026#34;][\u0026#34;score\u0026#34;] match_status = data[\u0026#34;match\u0026#34;][\u0026#34;status\u0026#34;] venue = data[\u0026#34;match\u0026#34;][\u0026#34;venue\u0026#34;] print(f\u0026#34;{home_team}: {home_score}\u0026#34;) # India: 287 print(f\u0026#34;Status: {match_status}\u0026#34;) # India won by 42 runs print(f\u0026#34;Venue: {venue}\u0026#34;) # Wankhede Stadium, Mumbai # Access list items top_scorer = data[\u0026#34;top_performers\u0026#34;][1] # Virat Kohli (index 1) print(f\u0026#34;Top scorer: {top_scorer[\u0026#39;name\u0026#39;]} -- {top_scorer[\u0026#39;runs\u0026#39;]} runs\u0026#34;) # Iterate over the list print(\u0026#34;\\nTop Performers:\u0026#34;) for performer in data[\u0026#34;top_performers\u0026#34;]: if \u0026#34;runs\u0026#34; in performer: print(f\u0026#34; {performer[\u0026#39;name\u0026#39;]}: {performer[\u0026#39;runs\u0026#39;]} runs\u0026#34;) elif \u0026#34;wickets\u0026#34; in performer: print(f\u0026#34; {performer[\u0026#39;name\u0026#39;]}: {performer[\u0026#39;wickets\u0026#39;]} wickets\u0026#34;) Safe access with .get() Real API responses are not always complete. Fields can be missing. Always use .get() with a default when accessing keys that might not exist.\n# Risky -- KeyError if field is missing user_name = data[\u0026#34;user\u0026#34;][\u0026#34;profile\u0026#34;][\u0026#34;display_name\u0026#34;] # Safe -- returns None (or your default) if any key is missing user_name = (data .get(\u0026#34;user\u0026#34;, {}) .get(\u0026#34;profile\u0026#34;, {}) .get(\u0026#34;display_name\u0026#34;, \u0026#34;Anonymous\u0026#34;)) Handling non-serialisable types Some Python objects cannot be converted to JSON directly. The most common ones you will encounter:\nimport json import datetime from decimal import Decimal data = { \u0026#34;date\u0026#34;: datetime.date.today(), # not JSON serialisable \u0026#34;price\u0026#34;: Decimal(\u0026#34;1234.56\u0026#34;), # not JSON serialisable \u0026#34;name\u0026#34;: \u0026#34;Aarav\u0026#34;, # fine } # This raises TypeError # json.dumps(data) # Solution 1: Convert manually before serialising data_clean = { \u0026#34;date\u0026#34;: data[\u0026#34;date\u0026#34;].isoformat(), # \u0026#34;2024-03-15\u0026#34; \u0026#34;price\u0026#34;: float(data[\u0026#34;price\u0026#34;]), # 1234.56 \u0026#34;name\u0026#34;: data[\u0026#34;name\u0026#34;], } print(json.dumps(data_clean, indent=2)) # Solution 2: Custom encoder (for when you do this frequently) class CustomEncoder(json.JSONEncoder): def default(self, obj): if isinstance(obj, (datetime.date, datetime.datetime)): return obj.isoformat() if isinstance(obj, Decimal): return float(obj) return super().default(obj) print(json.dumps(data, indent=2, cls=CustomEncoder)) 4. The datetime Module \u0026ndash; Complete Guide Dates and times are everywhere in real-world data. Every API response has a timestamp. Every database record has a created_at field. Every report covers a date range.\nGetting date handling right is surprisingly tricky \u0026ndash; timezones, formats, arithmetic, and parsing all have subtle traps. This section gives you a complete, practical reference.\nThe three main classes from datetime import date, time, datetime, timedelta, timezone # date -- year, month, day only today = date.today() print(today) # 2024-03-15 print(type(today)) # \u0026lt;class \u0026#39;datetime.date\u0026#39;\u0026gt; # time -- hour, minute, second, microsecond only (no date) t = time(14, 30, 45) print(t) # 14:30:45 # datetime -- date AND time combined (most commonly used) now = datetime.now() # local time print(now) # 2024-03-15 14:30:45.123456 Creating specific dates and datetimes from datetime import date, datetime # Specific date independence_day = date(1947, 8, 15) print(independence_day) # 1947-08-15 # Specific datetime ipl_match = datetime(2024, 3, 22, 19, 30, 0) # 7:30 PM on 22 March 2024 print(ipl_match) # 2024-03-22 19:30:00 # Today and now print(date.today()) # current date print(datetime.now()) # current date and time (local timezone) print(datetime.utcnow()) # current UTC time (timezone-naive) Formatting \u0026ndash; datetime to string (strftime) strftime = \u0026ldquo;string format time\u0026rdquo; \u0026ndash; converts a datetime TO a string.\nfrom datetime import datetime now = datetime(2024, 3, 15, 14, 30, 45) # Common format codes print(now.strftime(\u0026#34;%Y-%m-%d\u0026#34;)) # 2024-03-15 (ISO format) print(now.strftime(\u0026#34;%d/%m/%Y\u0026#34;)) # 15/03/2024 (Indian format) print(now.strftime(\u0026#34;%d %B %Y\u0026#34;)) # 15 March 2024 print(now.strftime(\u0026#34;%A, %d %B %Y\u0026#34;)) # Friday, 15 March 2024 print(now.strftime(\u0026#34;%d %b %Y %H:%M\u0026#34;)) # 15 Mar 2024 14:30 print(now.strftime(\u0026#34;%I:%M %p\u0026#34;)) # 02:30 PM print(now.strftime(\u0026#34;%Y-%m-%dT%H:%M:%SZ\u0026#34;)) # 2024-03-15T14:30:45Z (ISO 8601) Complete format code reference:\nDate codes: %Y -- 4-digit year (2024) %y -- 2-digit year (24) %m -- Month as zero-padded int (03) %B -- Full month name (March) %b -- Abbreviated month (Mar) %d -- Day as zero-padded int (15) %A -- Full weekday name (Friday) %a -- Abbreviated weekday (Fri) %j -- Day of the year (075) %W -- Week number (11) Time codes: %H -- 24-hour hour (00-23) (14) %I -- 12-hour hour (01-12) (02) %M -- Minute (00-59) (30) %S -- Second (00-59) (45) %f -- Microsecond (000000-999999) %p -- AM or PM (PM) Timezone: %Z -- Timezone name (IST) %z -- UTC offset (+0530) Parsing \u0026ndash; string to datetime (strptime) strptime = \u0026ldquo;string parse time\u0026rdquo; \u0026ndash; converts a string TO a datetime. This is what you use when an API gives you a date as a string.\nfrom datetime import datetime # Parse various date string formats date1 = datetime.strptime(\u0026#34;15-03-2024\u0026#34;, \u0026#34;%d-%m-%Y\u0026#34;) date2 = datetime.strptime(\u0026#34;March 15, 2024\u0026#34;, \u0026#34;%B %d, %Y\u0026#34;) date3 = datetime.strptime(\u0026#34;15/03/24 14:30\u0026#34;, \u0026#34;%d/%m/%y %H:%M\u0026#34;) date4 = datetime.strptime(\u0026#34;2024-03-15T14:30:00Z\u0026#34;, \u0026#34;%Y-%m-%dT%H:%M:%SZ\u0026#34;) date5 = datetime.strptime(\u0026#34;Fri, 15 Mar 2024\u0026#34;, \u0026#34;%a, %d %b %Y\u0026#34;) print(date1) # 2024-03-15 00:00:00 print(date4) # 2024-03-15 14:30:00 # Python 3.7+ -- use fromisoformat() for standard ISO 8601 strings date6 = datetime.fromisoformat(\u0026#34;2024-03-15T14:30:00\u0026#34;) print(date6) # 2024-03-15 14:30:00 # Python 3.11+ -- fromisoformat() handles the trailing Z # date7 = datetime.fromisoformat(\u0026#34;2024-03-15T14:30:00Z\u0026#34;) # works in 3.11+ # For older Python -- strip the Z manually api_timestamp = \u0026#34;2024-03-15T14:30:00Z\u0026#34; date8 = datetime.strptime(api_timestamp, \u0026#34;%Y-%m-%dT%H:%M:%SZ\u0026#34;) Date arithmetic with timedelta from datetime import date, datetime, timedelta today = date.today() # Add or subtract days tomorrow = today + timedelta(days=1) last_week = today - timedelta(weeks=1) thirty_days = today + timedelta(days=30) two_hours = datetime.now() + timedelta(hours=2) ninety_mins = datetime.now() + timedelta(minutes=90) print(f\u0026#34;Tomorrow : {tomorrow}\u0026#34;) print(f\u0026#34;Last week : {last_week}\u0026#34;) print(f\u0026#34;30 days out : {thirty_days}\u0026#34;) # Difference between two dates independence_day = date(1947, 8, 15) diff = today - independence_day print(f\u0026#34;India has been independent for {diff.days:,} days\u0026#34;) print(f\u0026#34;That is approximately {diff.days // 365} years\u0026#34;) # Comparing dates deadline = date(2024, 12, 31) if today \u0026gt; deadline: print(\u0026#34;Deadline has passed\u0026#34;) elif today == deadline: print(\u0026#34;Today is the deadline!\u0026#34;) else: days_left = (deadline - today).days print(f\u0026#34;Deadline in {days_left} days\u0026#34;) Replacing parts of a datetime from datetime import datetime now = datetime.now() # Replace creates a new datetime with specific fields changed start_of_day = now.replace(hour=0, minute=0, second=0, microsecond=0) end_of_day = now.replace(hour=23, minute=59, second=59, microsecond=999999) first_of_month = now.replace(day=1) print(f\u0026#34;Start of today : {start_of_day}\u0026#34;) print(f\u0026#34;End of today : {end_of_day}\u0026#34;) Working with timezones Timezone handling is one of the trickiest parts of datetime. A common mistake is mixing timezone-aware and timezone-naive datetime objects.\nfrom datetime import datetime, timezone, timedelta # UTC -- the universal reference timezone utc_now = datetime.now(timezone.utc) print(utc_now) # 2024-03-15 09:00:00+00:00 # IST (Indian Standard Time) = UTC + 5:30 IST = timezone(timedelta(hours=5, minutes=30)) ist_now = datetime.now(IST) print(ist_now) # 2024-03-15 14:30:00+05:30 # Convert UTC to IST utc_time = datetime(2024, 3, 15, 9, 0, 0, tzinfo=timezone.utc) ist_time = utc_time.astimezone(IST) print(ist_time) # 2024-03-15 14:30:00+05:30 # Working with API timestamps # APIs typically return UTC timestamps api_ts = \u0026#34;2024-03-15T09:00:00Z\u0026#34; utc_dt = datetime.strptime(api_ts, \u0026#34;%Y-%m-%dT%H:%M:%SZ\u0026#34;).replace(tzinfo=timezone.utc) ist_dt = utc_dt.astimezone(IST) print(f\u0026#34;UTC: {utc_dt.strftime(\u0026#39;%H:%M\u0026#39;)} IST: {ist_dt.strftime(\u0026#39;%H:%M\u0026#39;)}\u0026#34;) # UTC: 09:00 IST: 14:30 # For serious timezone work, use the \u0026#39;zoneinfo\u0026#39; module (Python 3.9+) from zoneinfo import ZoneInfo india_tz = ZoneInfo(\u0026#34;Asia/Kolkata\u0026#34;) ist_now_v2 = datetime.now(india_tz) print(ist_now_v2) # 2024-03-15 14:30:00+05:30 Unix timestamps APIs sometimes return timestamps as Unix timestamps \u0026ndash; seconds since January 1, 1970 UTC.\nimport time from datetime import datetime, timezone # Current Unix timestamp ts = time.time() print(ts) # 1710500400.123 # Convert Unix timestamp to datetime dt = datetime.fromtimestamp(ts) # local time dt_utc = datetime.fromtimestamp(ts, tz=timezone.utc) # UTC print(dt) # 2024-03-15 14:30:00.123000 print(dt_utc) # 2024-03-15 09:00:00.123000+00:00 # Convert datetime to Unix timestamp dt = datetime(2024, 3, 15, 14, 30, 0) ts = dt.timestamp() print(ts) # 1710500400.0 5. The requests Library requests is a third-party library (not in the standard library) that makes HTTP requests simple and human-friendly. It was created because Python\u0026rsquo;s built-in urllib is verbose and awkward.\npip install requests Making your first GET request import requests # The simplest possible request response = requests.get(\u0026#34;https://api.restcountries.com/v3.1/name/india\u0026#34;) # What you get back is a Response object print(type(response)) # \u0026lt;class \u0026#39;requests.models.Response\u0026#39;\u0026gt; print(response.status_code) # 200 print(response.headers[\u0026#34;content-type\u0026#34;]) # application/json;charset=utf-8 The Response object \u0026ndash; everything inside it import requests response = requests.get(\u0026#34;https://api.restcountries.com/v3.1/name/india\u0026#34;) # Status print(response.status_code) # 200 print(response.ok) # True if 200-299, False otherwise # Content -- three ways to access it response.text # response body as a string response.content # response body as raw bytes response.json() # parse body as JSON -- returns Python dict/list # Headers print(response.headers) # dict of all headers print(response.headers.get(\u0026#34;content-type\u0026#34;)) # specific header # URL (after any redirects) print(response.url) # the final URL # Response time print(response.elapsed) # timedelta -- how long the request took A complete example \u0026ndash; RestCountries API RestCountries (https://restcountries.com) is a free, open API with no authentication required. It returns data about countries worldwide.\nimport requests import json def get_country_info(country_name): \u0026#34;\u0026#34;\u0026#34; Fetch information about a country from the RestCountries API. Args: country_name (str): Country name to look up Returns: dict: Country data, or None if not found \u0026#34;\u0026#34;\u0026#34; url = f\u0026#34;https://api.restcountries.com/v3.1/name/{country_name}\u0026#34; response = requests.get(url) if response.status_code == 200: # API returns a list -- take the first result countries = response.json() return countries[0] elif response.status_code == 404: print(f\u0026#34;Country \u0026#39;{country_name}\u0026#39; not found.\u0026#34;) return None else: print(f\u0026#34;Error: {response.status_code}\u0026#34;) return None # Fetch India\u0026#39;s data india = get_country_info(\u0026#34;India\u0026#34;) if india: name = india[\u0026#34;name\u0026#34;][\u0026#34;common\u0026#34;] capital = india[\u0026#34;capital\u0026#34;][0] population = india[\u0026#34;population\u0026#34;] area = india[\u0026#34;area\u0026#34;] currencies = india[\u0026#34;currencies\u0026#34;] languages = india[\u0026#34;languages\u0026#34;] region = india[\u0026#34;region\u0026#34;] flag = india[\u0026#34;flag\u0026#34;] print(f\u0026#34;\\n{flag} {name}\u0026#34;) print(f\u0026#34;Capital : {capital}\u0026#34;) print(f\u0026#34;Population : {population:,}\u0026#34;) print(f\u0026#34;Area : {area:,} km²\u0026#34;) print(f\u0026#34;Region : {region}\u0026#34;) # Currencies is a nested dict for code, info in currencies.items(): print(f\u0026#34;Currency : {info[\u0026#39;name\u0026#39;]} ({code}, {info[\u0026#39;symbol\u0026#39;]})\u0026#34;) # Languages is a dict of code: name lang_list = list(languages.values()) print(f\u0026#34;Languages : {\u0026#39;, \u0026#39;.join(lang_list[:5])}\u0026#34;) 🇮🇳 India Capital : New Delhi Population : 1,428,627,663 Area : 3,287,590 km² Region : Asia Currency : Indian rupee (INR, ₹) Languages : Hindi, English, Bengali, Telugu, Marathi 6. Query Parameters and Headers Query parameters Query parameters are the ?key=value\u0026amp;key2=value2 part of a URL. They filter, sort, or configure the data you are requesting.\nimport requests # Wrong way -- building URL strings manually (error-prone) url = \u0026#34;https://api.example.com/matches?sport=cricket\u0026amp;format=T20\u0026amp;season=2024\u0026#34; # Right way -- pass params as a dict, requests handles encoding params = { \u0026#34;sport\u0026#34;: \u0026#34;cricket\u0026#34;, \u0026#34;format\u0026#34;: \u0026#34;T20\u0026#34;, \u0026#34;season\u0026#34;: 2024, } response = requests.get(\u0026#34;https://api.example.com/matches\u0026#34;, params=params) print(response.url) # https://api.example.com/matches?sport=cricket\u0026amp;format=T20\u0026amp;season=2024 # requests handles URL encoding automatically (spaces -\u0026gt; %20, etc.) Real example \u0026ndash; Open-Meteo Weather API Open-Meteo (https://open-meteo.com) is a completely free, open weather API. No account, no API key, no rate limits for reasonable use.\nimport requests def get_weather(city_name, latitude, longitude): \u0026#34;\u0026#34;\u0026#34; Fetch current weather for a location using Open-Meteo API. No API key required. Args: city_name (str): Display name of the city latitude (float): City latitude longitude (float): City longitude Returns: dict: Weather data \u0026#34;\u0026#34;\u0026#34; url = \u0026#34;https://api.open-meteo.com/v1/forecast\u0026#34; params = { \u0026#34;latitude\u0026#34;: latitude, \u0026#34;longitude\u0026#34;: longitude, \u0026#34;current\u0026#34;: \u0026#34;temperature_2m,relative_humidity_2m,wind_speed_10m,weather_code\u0026#34;, \u0026#34;timezone\u0026#34;: \u0026#34;Asia/Kolkata\u0026#34;, \u0026#34;forecast_days\u0026#34;: 1, } response = requests.get(url, params=params, timeout=10) response.raise_for_status() # raises exception for 4xx/5xx data = response.json() current = data[\u0026#34;current\u0026#34;] return { \u0026#34;city\u0026#34;: city_name, \u0026#34;temperature\u0026#34;: current[\u0026#34;temperature_2m\u0026#34;], \u0026#34;humidity\u0026#34;: current[\u0026#34;relative_humidity_2m\u0026#34;], \u0026#34;wind_speed\u0026#34;: current[\u0026#34;wind_speed_10m\u0026#34;], \u0026#34;time\u0026#34;: current[\u0026#34;time\u0026#34;], } # Coordinates for major Indian cities cities = [ (\u0026#34;Mumbai\u0026#34;, 19.0760, 72.8777), (\u0026#34;Delhi\u0026#34;, 28.6139, 77.2090), (\u0026#34;Bangalore\u0026#34;, 12.9716, 77.5946), (\u0026#34;Chennai\u0026#34;, 13.0827, 80.2707), (\u0026#34;Kolkata\u0026#34;, 22.5726, 88.3639), ] print(\u0026#34;Current Weather in Indian Cities\u0026#34;) print(\u0026#34;=\u0026#34; * 50) for city, lat, lon in cities: weather = get_weather(city, lat, lon) print(f\u0026#34;\\n{weather[\u0026#39;city\u0026#39;]}\u0026#34;) print(f\u0026#34; Temperature : {weather[\u0026#39;temperature\u0026#39;]}°C\u0026#34;) print(f\u0026#34; Humidity : {weather[\u0026#39;humidity\u0026#39;]}%\u0026#34;) print(f\u0026#34; Wind Speed : {weather[\u0026#39;wind_speed\u0026#39;]} km/h\u0026#34;) Headers Headers carry metadata about your request. The most common ones you will set or read:\nimport requests # Setting request headers headers = { \u0026#34;Accept\u0026#34;: \u0026#34;application/json\u0026#34;, # tell server you want JSON back \u0026#34;Content-Type\u0026#34;: \u0026#34;application/json\u0026#34;, # tell server you are sending JSON \u0026#34;Authorization\u0026#34;: \u0026#34;Bearer your_token\u0026#34;, # authentication token \u0026#34;User-Agent\u0026#34;: \u0026#34;MyApp/1.0\u0026#34;, # identify your application \u0026#34;Accept-Language\u0026#34;: \u0026#34;en-IN\u0026#34;, # language preference } response = requests.get( \u0026#34;https://api.example.com/data\u0026#34;, headers=headers ) # Reading response headers print(response.headers[\u0026#34;content-type\u0026#34;]) print(response.headers.get(\u0026#34;x-rate-limit-remaining\u0026#34;, \u0026#34;N/A\u0026#34;)) print(response.headers.get(\u0026#34;x-request-id\u0026#34;)) 7. Handling Errors Properly Bad error handling is one of the most common mistakes in code that talks to APIs. Networks fail. Servers go down. Rate limits are hit. Your code must handle all of this gracefully.\nThe two types of failures 1. Network/connection failure -- The request never reached the server -- Python raises a requests.exceptions.* exception -- No status code because there was no response 2. HTTP error response -- The server responded, but with an error status code -- Python does NOT automatically raise an exception for 4xx/5xx -- You must check response.status_code or call response.raise_for_status() raise_for_status() \u0026ndash; the one-liner check import requests response = requests.get(\u0026#34;https://api.example.com/data\u0026#34;) # Raises requests.exceptions.HTTPError for 4xx and 5xx responses # Does nothing for 2xx responses response.raise_for_status() # After this line, you know the request succeeded data = response.json() Complete error handling pattern import requests from requests.exceptions import ( ConnectionError, # network unreachable Timeout, # request took too long HTTPError, # 4xx or 5xx response RequestException, # base class for all requests errors ) def fetch_data(url, params=None, timeout=10): \u0026#34;\u0026#34;\u0026#34; Fetch data from an API with comprehensive error handling. Returns: dict: Parsed JSON data on success None: On any failure (with error printed) \u0026#34;\u0026#34;\u0026#34; try: response = requests.get(url, params=params, timeout=timeout) response.raise_for_status() # raises HTTPError for 4xx/5xx return response.json() except Timeout: print(f\u0026#34;Request timed out after {timeout}s: {url}\u0026#34;) return None except ConnectionError: print(f\u0026#34;Network error -- cannot connect to: {url}\u0026#34;) print(\u0026#34;Check your internet connection.\u0026#34;) return None except HTTPError as e: status = e.response.status_code if status == 404: print(f\u0026#34;Resource not found (404): {url}\u0026#34;) elif status == 401: print(f\u0026#34;Authentication required (401): {url}\u0026#34;) elif status == 403: print(f\u0026#34;Access forbidden (403): {url}\u0026#34;) elif status == 429: print(f\u0026#34;Rate limit exceeded (429). Wait before retrying.\u0026#34;) elif status \u0026gt;= 500: print(f\u0026#34;Server error ({status}). Try again later.\u0026#34;) else: print(f\u0026#34;HTTP error {status}: {e}\u0026#34;) return None except ValueError as e: # response.json() failed -- response was not valid JSON print(f\u0026#34;Invalid JSON in response: {e}\u0026#34;) return None except RequestException as e: # Catch-all for any other requests error print(f\u0026#34;Request failed: {e}\u0026#34;) return None # Usage data = fetch_data( \u0026#34;https://api.open-meteo.com/v1/forecast\u0026#34;, params={\u0026#34;latitude\u0026#34;: 28.6, \u0026#34;longitude\u0026#34;: 77.2, \u0026#34;current\u0026#34;: \u0026#34;temperature_2m\u0026#34;} ) if data: print(f\u0026#34;Temperature in Delhi: {data[\u0026#39;current\u0026#39;][\u0026#39;temperature_2m\u0026#39;]}°C\u0026#34;) Timeouts \u0026ndash; always set them import requests # NEVER do this -- will hang forever if server is slow or unresponsive response = requests.get(\u0026#34;https://api.example.com/data\u0026#34;) # ALWAYS set a timeout # timeout=(connect_timeout, read_timeout) response = requests.get( \u0026#34;https://api.example.com/data\u0026#34;, timeout=(5, 30) # 5s to connect, 30s to read response ) # Or a single value applies to both response = requests.get(\u0026#34;https://api.example.com/data\u0026#34;, timeout=10) Retrying failed requests import requests import time def fetch_with_retry(url, params=None, max_retries=3, backoff=2): \u0026#34;\u0026#34;\u0026#34; Fetch a URL with exponential backoff retry logic. backoff=2 means: wait 2s, then 4s, then 8s between retries. \u0026#34;\u0026#34;\u0026#34; for attempt in range(1, max_retries + 1): try: response = requests.get(url, params=params, timeout=10) response.raise_for_status() return response.json() except requests.exceptions.Timeout: wait = backoff ** attempt print(f\u0026#34;Timeout on attempt {attempt}/{max_retries}. Retrying in {wait}s...\u0026#34;) time.sleep(wait) except requests.exceptions.HTTPError as e: # Do not retry client errors (4xx) -- they will not resolve if e.response.status_code \u0026lt; 500: print(f\u0026#34;Client error {e.response.status_code} -- not retrying.\u0026#34;) return None wait = backoff ** attempt print(f\u0026#34;Server error on attempt {attempt}/{max_retries}. Retrying in {wait}s...\u0026#34;) time.sleep(wait) except requests.exceptions.ConnectionError: wait = backoff ** attempt print(f\u0026#34;Connection error on attempt {attempt}/{max_retries}. Retrying in {wait}s...\u0026#34;) time.sleep(wait) print(f\u0026#34;All {max_retries} attempts failed.\u0026#34;) return None 8. POST Requests and Sending Data GET requests retrieve data. POST requests send data to create or update something on the server.\nimport requests import json # Sending JSON data in the request body url = \u0026#34;https://api.example.com/students\u0026#34; data = { \u0026#34;name\u0026#34;: \u0026#34;Aarav Sharma\u0026#34;, \u0026#34;roll\u0026#34;: 201, \u0026#34;course\u0026#34;: \u0026#34;Data Science\u0026#34;, \u0026#34;email\u0026#34;: \u0026#34;aarav@example.com\u0026#34; } # Method 1: Use json= parameter -- requests sets Content-Type automatically response = requests.post(url, json=data, timeout=10) # Method 2: Manually serialise and set headers headers = {\u0026#34;Content-Type\u0026#34;: \u0026#34;application/json\u0026#34;} response = requests.post(url, data=json.dumps(data), headers=headers, timeout=10) # Sending form data (like an HTML form submission) form_data = { \u0026#34;username\u0026#34;: \u0026#34;aarav_sharma\u0026#34;, \u0026#34;password\u0026#34;: \u0026#34;secure_password_123\u0026#34; } response = requests.post(url, data=form_data, timeout=10) # not json=, but data= print(response.status_code) # 201 if successfully created if response.status_code == 201: created = response.json() print(f\u0026#34;Created: {created}\u0026#34;) PUT and PATCH \u0026ndash; updating data import requests student_id = 201 # PUT -- full update (replace the entire resource) full_update = { \u0026#34;name\u0026#34;: \u0026#34;Aarav Sharma\u0026#34;, \u0026#34;roll\u0026#34;: 201, \u0026#34;course\u0026#34;: \u0026#34;Machine Learning\u0026#34;, # changed \u0026#34;email\u0026#34;: \u0026#34;aarav@example.com\u0026#34; } response = requests.put( f\u0026#34;https://api.example.com/students/{student_id}\u0026#34;, json=full_update, timeout=10 ) # PATCH -- partial update (only the fields you want to change) partial_update = { \u0026#34;course\u0026#34;: \u0026#34;Machine Learning\u0026#34; # only updating the course } response = requests.patch( f\u0026#34;https://api.example.com/students/{student_id}\u0026#34;, json=partial_update, timeout=10 ) # DELETE -- remove a resource response = requests.delete( f\u0026#34;https://api.example.com/students/{student_id}\u0026#34;, timeout=10 ) print(response.status_code) # 204 No Content if successfully deleted 9. Sessions and Authentication Basics Sessions \u0026ndash; reusing connections and settings A Session object persists settings across multiple requests to the same server. Useful when you make many requests and want to share headers, authentication, or cookies.\nimport requests # Without session -- you repeat headers every time headers = {\u0026#34;Authorization\u0026#34;: \u0026#34;Bearer my_token\u0026#34;, \u0026#34;User-Agent\u0026#34;: \u0026#34;MyApp/1.0\u0026#34;} r1 = requests.get(\u0026#34;https://api.example.com/students\u0026#34;, headers=headers) r2 = requests.get(\u0026#34;https://api.example.com/courses\u0026#34;, headers=headers) r3 = requests.get(\u0026#34;https://api.example.com/grades\u0026#34;, headers=headers) # With session -- set headers once, reused for all requests session = requests.Session() session.headers.update({ \u0026#34;Authorization\u0026#34;: \u0026#34;Bearer my_token\u0026#34;, \u0026#34;User-Agent\u0026#34;: \u0026#34;MyApp/1.0\u0026#34;, \u0026#34;Accept\u0026#34;: \u0026#34;application/json\u0026#34;, }) r1 = session.get(\u0026#34;https://api.example.com/students\u0026#34;) r2 = session.get(\u0026#34;https://api.example.com/courses\u0026#34;) r3 = session.get(\u0026#34;https://api.example.com/grades\u0026#34;) # Sessions also reuse TCP connections -- faster for multiple requests # Always close the session when done session.close() # Or use as a context manager with requests.Session() as session: session.headers.update({\u0026#34;Authorization\u0026#34;: \u0026#34;Bearer my_token\u0026#34;}) r1 = session.get(\u0026#34;https://api.example.com/students\u0026#34;) r2 = session.get(\u0026#34;https://api.example.com/courses\u0026#34;) # Session automatically closed here Authentication \u0026ndash; the common patterns import requests from requests.auth import HTTPBasicAuth # 1. API Key in header (most common for public APIs) response = requests.get( \u0026#34;https://api.example.com/data\u0026#34;, headers={\u0026#34;X-API-Key\u0026#34;: \u0026#34;your_api_key_here\u0026#34;}, timeout=10 ) # 2. API Key as query parameter response = requests.get( \u0026#34;https://api.example.com/data\u0026#34;, params={\u0026#34;api_key\u0026#34;: \u0026#34;your_api_key_here\u0026#34;}, timeout=10 ) # 3. Bearer token (JWT -- common in modern APIs) token = \u0026#34;eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9...\u0026#34; response = requests.get( \u0026#34;https://api.example.com/data\u0026#34;, headers={\u0026#34;Authorization\u0026#34;: f\u0026#34;Bearer {token}\u0026#34;}, timeout=10 ) # 4. Basic authentication (username + password) response = requests.get( \u0026#34;https://api.example.com/data\u0026#34;, auth=HTTPBasicAuth(\u0026#34;username\u0026#34;, \u0026#34;password\u0026#34;), timeout=10 ) # Shorthand: response = requests.get( \u0026#34;https://api.example.com/data\u0026#34;, auth=(\u0026#34;username\u0026#34;, \u0026#34;password\u0026#34;), timeout=10 ) Storing API keys safely Never hardcode API keys in your source code. They end up in git history and can be exposed publicly.\nimport os import requests # WRONG -- never do this API_KEY = \u0026#34;sk-abc123xyz\u0026#34; # RIGHT -- read from environment variable API_KEY = os.environ.get(\u0026#34;MY_API_KEY\u0026#34;) if not API_KEY: raise ValueError(\u0026#34;MY_API_KEY environment variable not set.\u0026#34;) response = requests.get( \u0026#34;https://api.example.com/data\u0026#34;, headers={\u0026#34;Authorization\u0026#34;: f\u0026#34;Bearer {API_KEY}\u0026#34;}, timeout=10 ) # Set the environment variable in your terminal before running export MY_API_KEY=\u0026#34;sk-abc123xyz\u0026#34; python my_script.py # Or use a .env file with python-dotenv # pip install python-dotenv # Using python-dotenv (for .env files) from dotenv import load_dotenv import os load_dotenv() # reads .env file and sets environment variables API_KEY = os.environ.get(\u0026#34;MY_API_KEY\u0026#34;) # .env file (add this to .gitignore!) MY_API_KEY=sk-abc123xyz DATABASE_URL=postgresql://user:pass@localhost/mydb 10. Complete Project \u0026ndash; India Weather and Currency Dashboard Now we put everything together. This project:\nCalls the Open-Meteo API (free, no auth) for real-time weather data for major Indian cities Calls the ExchangeRate API (free tier, no auth needed for basic use) for INR currency conversion Parses and formats all timestamps properly using datetime Handles all errors gracefully Presents a clean dashboard in the terminal Both APIs are completely free and require no sign-up for basic use.\n\u0026#34;\u0026#34;\u0026#34; india_dashboard.py A real-time dashboard showing: - Current weather for major Indian cities - INR exchange rates against major currencies Uses: - Open-Meteo API: https://open-meteo.com (free, no auth) - Open Exchange Rates (simplified): https://open.er-api.com (free tier) - requests, datetime, json \u0026#34;\u0026#34;\u0026#34; import requests import json from datetime import datetime, timezone, timedelta from requests.exceptions import RequestException, Timeout, ConnectionError, HTTPError # ─── Configuration ──────────────────────────────────────────────────────────── # Indian Standard Time = UTC + 5:30 IST = timezone(timedelta(hours=5, minutes=30)) # Major Indian cities with coordinates CITIES = [ {\u0026#34;name\u0026#34;: \u0026#34;Mumbai\u0026#34;, \u0026#34;lat\u0026#34;: 19.0760, \u0026#34;lon\u0026#34;: 72.8777}, {\u0026#34;name\u0026#34;: \u0026#34;Delhi\u0026#34;, \u0026#34;lat\u0026#34;: 28.6139, \u0026#34;lon\u0026#34;: 77.2090}, {\u0026#34;name\u0026#34;: \u0026#34;Bangalore\u0026#34;, \u0026#34;lat\u0026#34;: 12.9716, \u0026#34;lon\u0026#34;: 77.5946}, {\u0026#34;name\u0026#34;: \u0026#34;Chennai\u0026#34;, \u0026#34;lat\u0026#34;: 13.0827, \u0026#34;lon\u0026#34;: 80.2707}, {\u0026#34;name\u0026#34;: \u0026#34;Kolkata\u0026#34;, \u0026#34;lat\u0026#34;: 22.5726, \u0026#34;lon\u0026#34;: 88.3639}, {\u0026#34;name\u0026#34;: \u0026#34;Hyderabad\u0026#34;, \u0026#34;lat\u0026#34;: 17.3850, \u0026#34;lon\u0026#34;: 78.4867}, ] # Currencies to compare against INR TARGET_CURRENCIES = [\u0026#34;USD\u0026#34;, \u0026#34;EUR\u0026#34;, \u0026#34;GBP\u0026#34;, \u0026#34;JPY\u0026#34;, \u0026#34;AED\u0026#34;, \u0026#34;SGD\u0026#34;] # WMO Weather interpretation codes (simplified) WEATHER_CODES = { 0: \u0026#34;Clear sky\u0026#34;, 1: \u0026#34;Mainly clear\u0026#34;, 2: \u0026#34;Partly cloudy\u0026#34;, 3: \u0026#34;Overcast\u0026#34;, 45: \u0026#34;Foggy\u0026#34;, 48: \u0026#34;Icy fog\u0026#34;, 51: \u0026#34;Light drizzle\u0026#34;, 53: \u0026#34;Moderate drizzle\u0026#34;, 61: \u0026#34;Slight rain\u0026#34;, 63: \u0026#34;Moderate rain\u0026#34;, 65: \u0026#34;Heavy rain\u0026#34;, 71: \u0026#34;Slight snowfall\u0026#34;, 80: \u0026#34;Slight showers\u0026#34;, 95: \u0026#34;Thunderstorm\u0026#34;, } # ─── API Functions ──────────────────────────────────────────────────────────── def fetch_weather(city): \u0026#34;\u0026#34;\u0026#34; Fetch current weather for one city using Open-Meteo API. Args: city (dict): {\u0026#34;name\u0026#34;: str, \u0026#34;lat\u0026#34;: float, \u0026#34;lon\u0026#34;: float} Returns: dict: Parsed weather data, or None on failure \u0026#34;\u0026#34;\u0026#34; url = \u0026#34;https://api.open-meteo.com/v1/forecast\u0026#34; params = { \u0026#34;latitude\u0026#34;: city[\u0026#34;lat\u0026#34;], \u0026#34;longitude\u0026#34;: city[\u0026#34;lon\u0026#34;], \u0026#34;current\u0026#34;: \u0026#34;,\u0026#34;.join([ \u0026#34;temperature_2m\u0026#34;, \u0026#34;apparent_temperature\u0026#34;, \u0026#34;relative_humidity_2m\u0026#34;, \u0026#34;wind_speed_10m\u0026#34;, \u0026#34;weather_code\u0026#34;, ]), \u0026#34;timezone\u0026#34;: \u0026#34;Asia/Kolkata\u0026#34;, \u0026#34;forecast_days\u0026#34;: 1, } try: response = requests.get(url, params=params, timeout=10) response.raise_for_status() data = response.json() current = data[\u0026#34;current\u0026#34;] # Parse the timestamp the API returns # Format: \u0026#34;2024-03-15T14:30\u0026#34; observed_at = datetime.strptime(current[\u0026#34;time\u0026#34;], \u0026#34;%Y-%m-%dT%H:%M\u0026#34;) observed_at = observed_at.replace(tzinfo=IST) # Look up weather description weather_code = current.get(\u0026#34;weather_code\u0026#34;, 0) description = WEATHER_CODES.get(weather_code, f\u0026#34;Code {weather_code}\u0026#34;) return { \u0026#34;city\u0026#34;: city[\u0026#34;name\u0026#34;], \u0026#34;temperature\u0026#34;: current[\u0026#34;temperature_2m\u0026#34;], \u0026#34;feels_like\u0026#34;: current[\u0026#34;apparent_temperature\u0026#34;], \u0026#34;humidity\u0026#34;: current[\u0026#34;relative_humidity_2m\u0026#34;], \u0026#34;wind_speed\u0026#34;: current[\u0026#34;wind_speed_10m\u0026#34;], \u0026#34;description\u0026#34;: description, \u0026#34;observed_at\u0026#34;: observed_at, } except Timeout: print(f\u0026#34; [Weather] Timeout fetching {city[\u0026#39;name\u0026#39;]}\u0026#34;) return None except ConnectionError: print(f\u0026#34; [Weather] No internet connection\u0026#34;) return None except HTTPError as e: print(f\u0026#34; [Weather] HTTP {e.response.status_code} for {city[\u0026#39;name\u0026#39;]}\u0026#34;) return None except (KeyError, ValueError) as e: print(f\u0026#34; [Weather] Unexpected response format: {e}\u0026#34;) return None def fetch_exchange_rates(): \u0026#34;\u0026#34;\u0026#34; Fetch INR exchange rates using the free Open Exchange Rates API. Returns: dict: {currency_code: rate_from_1_INR} or None on failure \u0026#34;\u0026#34;\u0026#34; # This free endpoint gives rates relative to USD -- we convert to INR url = \u0026#34;https://open.er-api.com/v6/latest/INR\u0026#34; try: response = requests.get(url, timeout=10) response.raise_for_status() data = response.json() if data.get(\u0026#34;result\u0026#34;) != \u0026#34;success\u0026#34;: print(f\u0026#34; [Rates] API returned non-success: {data.get(\u0026#39;result\u0026#39;)}\u0026#34;) return None # Parse the update timestamp # The API returns Unix timestamp updated_ts = data.get(\u0026#34;time_last_update_unix\u0026#34;, 0) updated_at = datetime.fromtimestamp(updated_ts, tz=timezone.utc) updated_ist = updated_at.astimezone(IST) # Extract rates for our target currencies all_rates = data.get(\u0026#34;rates\u0026#34;, {}) rates = { currency: rate for currency, rate in all_rates.items() if currency in TARGET_CURRENCIES } return { \u0026#34;base\u0026#34;: \u0026#34;INR\u0026#34;, \u0026#34;rates\u0026#34;: rates, \u0026#34;updated_at\u0026#34;: updated_ist, } except Timeout: print(\u0026#34; [Rates] Timeout fetching exchange rates\u0026#34;) return None except ConnectionError: print(\u0026#34; [Rates] No internet connection\u0026#34;) return None except HTTPError as e: print(f\u0026#34; [Rates] HTTP {e.response.status_code}\u0026#34;) return None except (KeyError, ValueError) as e: print(f\u0026#34; [Rates] Unexpected response format: {e}\u0026#34;) return None # ─── Display Functions ──────────────────────────────────────────────────────── def print_header(): now_ist = datetime.now(IST) print(\u0026#34;\\n\u0026#34; + \u0026#34;=\u0026#34; * 60) print(\u0026#34; INDIA DASHBOARD -- LIVE DATA\u0026#34;) print(f\u0026#34; {now_ist.strftime(\u0026#39;%A, %d %B %Y %I:%M %p IST\u0026#39;)}\u0026#34;) print(\u0026#34;=\u0026#34; * 60) def print_weather_section(weather_data): print(\u0026#34;\\n CURRENT WEATHER -- MAJOR INDIAN CITIES\u0026#34;) print(\u0026#34; \u0026#34; + \u0026#34;-\u0026#34; * 56) print(f\u0026#34; {\u0026#39;City\u0026#39;:\u0026lt;12} {\u0026#39;Temp\u0026#39;:\u0026gt;6} {\u0026#39;Feels\u0026#39;:\u0026gt;6} {\u0026#39;Humidity\u0026#39;:\u0026gt;9} {\u0026#39;Wind\u0026#39;:\u0026gt;7} Condition\u0026#34;) print(\u0026#34; \u0026#34; + \u0026#34;-\u0026#34; * 56) for w in weather_data: if w is None: continue print( f\u0026#34; {w[\u0026#39;city\u0026#39;]:\u0026lt;12}\u0026#34; f\u0026#34; {w[\u0026#39;temperature\u0026#39;]:\u0026gt;5.1f}C\u0026#34; f\u0026#34; {w[\u0026#39;feels_like\u0026#39;]:\u0026gt;5.1f}C\u0026#34; f\u0026#34; {w[\u0026#39;humidity\u0026#39;]:\u0026gt;7}%\u0026#34; f\u0026#34; {w[\u0026#39;wind_speed\u0026#39;]:\u0026gt;5.1f}km/h\u0026#34; f\u0026#34; {w[\u0026#39;description\u0026#39;]}\u0026#34; ) # Show data freshness valid = [w for w in weather_data if w is not None] if valid: observed = valid[0][\u0026#34;observed_at\u0026#34;] print(f\u0026#34;\\n Data observed at: {observed.strftime(\u0026#39;%I:%M %p IST\u0026#39;)}\u0026#34;) def print_currency_section(rate_data): if rate_data is None: print(\u0026#34;\\n EXCHANGE RATES -- Unavailable\u0026#34;) return print(\u0026#34;\\n INR EXCHANGE RATES\u0026#34;) print(\u0026#34; \u0026#34; + \u0026#34;-\u0026#34; * 40) print(f\u0026#34; {\u0026#39;Currency\u0026#39;:\u0026lt;10} {\u0026#39;1 INR =\u0026#39;:\u0026gt;15} {\u0026#39;100 INR =\u0026#39;:\u0026gt;12}\u0026#34;) print(\u0026#34; \u0026#34; + \u0026#34;-\u0026#34; * 40) currency_names = { \u0026#34;USD\u0026#34;: \u0026#34;US Dollar\u0026#34;, \u0026#34;EUR\u0026#34;: \u0026#34;Euro\u0026#34;, \u0026#34;GBP\u0026#34;: \u0026#34;British Pound\u0026#34;, \u0026#34;JPY\u0026#34;: \u0026#34;Japanese Yen\u0026#34;, \u0026#34;AED\u0026#34;: \u0026#34;UAE Dirham\u0026#34;, \u0026#34;SGD\u0026#34;: \u0026#34;Singapore Dollar\u0026#34;, } for currency, rate in rate_data[\u0026#34;rates\u0026#34;].items(): name = currency_names.get(currency, currency) per_100 = rate * 100 print( f\u0026#34; {currency} ({name[:12]:\u0026lt;12})\u0026#34; f\u0026#34; {rate:\u0026gt;10.4f}\u0026#34; f\u0026#34; {per_100:\u0026gt;10.4f}\u0026#34; ) updated = rate_data[\u0026#34;updated_at\u0026#34;] print(f\u0026#34;\\n Rates updated: {updated.strftime(\u0026#39;%d %b %Y, %I:%M %p IST\u0026#39;)}\u0026#34;) def print_footer(): print(\u0026#34;\\n\u0026#34; + \u0026#34;=\u0026#34; * 60) print(\u0026#34; Data sources:\u0026#34;) print(\u0026#34; Weather : Open-Meteo (open-meteo.com)\u0026#34;) print(\u0026#34; Rates : Open Exchange Rates (open.er-api.com)\u0026#34;) print(\u0026#34;=\u0026#34; * 60 + \u0026#34;\\n\u0026#34;) # ─── Main ───────────────────────────────────────────────────────────────────── def main(): print_header() # Fetch weather for all cities print(\u0026#34;\\n Fetching weather data...\u0026#34;) weather_data = [] for city in CITIES: result = fetch_weather(city) weather_data.append(result) # Fetch exchange rates print(\u0026#34; Fetching exchange rates...\u0026#34;) rate_data = fetch_exchange_rates() # Display everything print_weather_section(weather_data) print_currency_section(rate_data) print_footer() if __name__ == \u0026#34;__main__\u0026#34;: main() Sample output (actual values will reflect real current data):\n============================================================ INDIA DASHBOARD -- LIVE DATA Friday, 15 March 2024 02:30 PM IST ============================================================ Fetching weather data... Fetching exchange rates... CURRENT WEATHER -- MAJOR INDIAN CITIES -------------------------------------------------------- City Temp Feels Humidity Wind Condition -------------------------------------------------------- Mumbai 32.4C 36.1C 78% 18.5km/h Partly cloudy Delhi 28.1C 26.8C 45% 12.3km/h Clear sky Bangalore 26.3C 25.9C 62% 8.4km/h Mainly clear Chennai 34.8C 39.2C 81% 22.1km/h Partly cloudy Kolkata 33.2C 37.5C 74% 15.7km/h Slight rain Hyderabad 31.7C 33.4C 55% 10.2km/h Clear sky Data observed at: 02:15 PM IST INR EXCHANGE RATES ---------------------------------------- Currency 1 INR = 100 INR = ---------------------------------------- USD (US Dollar ) 0.0120 1.2000 EUR (Euro ) 0.0111 1.1100 GBP (British Pound) 0.0095 0.9500 JPY (Japanese Yen ) 1.8200 182.0000 AED (UAE Dirham ) 0.0441 4.4100 SGD (Singapore Dol) 0.0162 1.6200 Rates updated: 15 Mar 2024, 09:00 AM IST ============================================================ Data sources: Weather : Open-Meteo (open-meteo.com) Rates : Open Exchange Rates (open.er-api.com) ============================================================ 11. Summary and Key Takeaways What you learned in this guide JSON\nJSON is the universal data format for APIs. It maps directly to Python dicts, lists, strings, numbers, booleans, and None. json.loads() parses a JSON string into Python. json.dumps() converts Python to a JSON string. json.load() / json.dump() work with files. Always use .get() with a default when navigating API responses \u0026ndash; fields can be missing in unexpected ways. Dates, Decimals, and custom objects are not JSON-serialisable by default. Convert them manually or write a custom encoder. datetime\nUse datetime.now() for local time, datetime.now(timezone.utc) for UTC. strftime converts datetime TO a string (format for output). strptime converts a string TO a datetime (parse from input). timedelta is how you do date arithmetic. API timestamps are almost always UTC. Convert to IST using astimezone(IST) where IST = timezone(timedelta(hours=5, minutes=30)). For Python 3.9+ use zoneinfo.ZoneInfo(\u0026quot;Asia/Kolkata\u0026quot;) for cleaner timezone handling. Timezone-aware and timezone-naive datetimes cannot be mixed. Pick one and stay consistent throughout your code. requests\nrequests.get(url, params={}, headers={}, timeout=10) is the core pattern. Always set a timeout. Code without timeouts will hang indefinitely. Always handle errors. Networks fail, servers crash, rate limits are hit. Use response.raise_for_status() as a one-liner to catch 4xx and 5xx errors. Use response.json() to parse the response body directly into Python. Use a Session when making multiple requests to the same server. Never hardcode API keys. Use environment variables. The three mental models to keep 1. The request-response cycle\nEvery API call is a conversation. You send a request with a method, URL, headers, and body. You get back a response with a status code, headers, and body. The status code tells you what happened before you look at the body.\n2. JSON is just text\nAn API response is just a string of text that follows the JSON format. response.json() converts that text into Python objects you can work with. json.dumps() converts Python objects back to text to send in a request body.\n3. Time is always somewhere\nEvery datetime exists in a timezone. When you read a timestamp from an API, always ask: is this UTC or local time? Convert to IST for display in Indian applications. Store and compare in UTC for consistency.\nCommon mistakes to avoid # 1. No timeout -- never do this requests.get(\u0026#34;https://api.example.com/data\u0026#34;) # 2. No error handling -- never do this response = requests.get(url) data = response.json() # crashes if request failed # 3. Hardcoded API key -- never do this API_KEY = \u0026#34;sk-abc123\u0026#34; # 4. Ignoring status code -- never do this response = requests.get(url) data = response.json() # might be an error response # 5. Mixing timezone-aware and naive datetimes now = datetime.now() # naive (no timezone) utc = datetime.now(timezone.utc) # aware (UTC) diff = utc - now # TypeError! # 6. Using string concatenation for URLs url = \u0026#34;https://api.example.com/data?city=\u0026#34; + city_name + \u0026#34;\u0026amp;format=json\u0026#34; # Use params= instead -- requests handles encoding for you Where these skills lead Every project from here builds on what you learned in this guide.\nData Science: Pulling datasets from APIs (financial data, sports stats, census data, weather) for analysis in Pandas.\nMachine Learning: Calling inference APIs, sending data to model endpoints, receiving predictions back.\nBackend Engineering: Building your own APIs with FastAPI or Django REST Framework \u0026ndash; which means understanding the request-response cycle from the server side.\nData Engineering: Calling third-party data sources, ingesting API data into databases, scheduling regular data pulls with timestamps.\nAutomation: Sending WhatsApp messages via Twilio API, posting to Slack, triggering webhooks, interacting with GitHub API.\nThe pattern is always the same: make a request, check the status, parse the response, handle the errors. Once this pattern is natural to you, every API in the world becomes accessible.\n12. Practice Questions All questions are self-contained. Try each one before reading the solution. Questions are ordered: Easy (1-5), Medium (6-10), Hard (11-15).\nEasy Q1 \u0026ndash; JSON Parsing Parse the JSON string below and print: the student\u0026rsquo;s name, their highest mark, and whether they passed (average \u0026gt;= 60).\njson_str = \u0026#39;\u0026#39;\u0026#39; { \u0026#34;student\u0026#34;: { \u0026#34;name\u0026#34;: \u0026#34;Arjun Mehta\u0026#34;, \u0026#34;roll\u0026#34;: 305, \u0026#34;city\u0026#34;: \u0026#34;Pune\u0026#34; }, \u0026#34;marks\u0026#34;: { \u0026#34;maths\u0026#34;: 78, \u0026#34;science\u0026#34;: 55, \u0026#34;english\u0026#34;: 62, \u0026#34;history\u0026#34;: 48, \u0026#34;cs\u0026#34;: 91 }, \u0026#34;active\u0026#34;: true } \u0026#39;\u0026#39;\u0026#39; Q2 \u0026ndash; datetime Formatting Given the datetime below, print it in five different formats:\nISO format: 2024-03-15 Indian format: 15/03/2024 Long format: Friday, 15 March 2024 Time only: 02:30 PM Full: Fri 15 Mar 2024, 14:30:45 from datetime import datetime dt = datetime(2024, 3, 15, 14, 30, 45) Q3 \u0026ndash; datetime Parsing Parse each of these date strings into a datetime object and print them all in the same format: DD-MM-YYYY HH:MM.\ndates = [ \u0026#34;2024-03-15T09:30:00Z\u0026#34;, \u0026#34;15 March 2024 09:30\u0026#34;, \u0026#34;15/03/24 09:30\u0026#34;, \u0026#34;March 15, 2024 09:30 AM\u0026#34;, ] Q4 \u0026ndash; First GET Request Use the RestCountries API to fetch information about Japan and print:\nCommon name Capital city Population (formatted with commas) Currency name and symbol Whether it is a UN member URL: https://api.restcountries.com/v3.1/name/japan\nQ5 \u0026ndash; datetime Arithmetic Write a function age_calculator(dob_str) that takes a date of birth as \u0026quot;DD-MM-YYYY\u0026quot; and returns a tuple of (years, months, days) representing the person\u0026rsquo;s exact age today.\nprint(age_calculator(\u0026#34;15-08-1947\u0026#34;)) # India\u0026#39;s independence day # Expected: something like (76, 7, 0) -- varies by when you run it Medium Q6 \u0026ndash; Safe JSON Navigation The following API response sometimes has missing fields. Write a function extract_profile(response_json) that safely extracts: name, email, city, and subscription tier. Use .get() with sensible defaults for all fields.\n# Sometimes complete response1 = \u0026#39;{\u0026#34;user\u0026#34;: {\u0026#34;name\u0026#34;: \u0026#34;Priya\u0026#34;, \u0026#34;email\u0026#34;: \u0026#34;p@x.com\u0026#34;, \u0026#34;location\u0026#34;: {\u0026#34;city\u0026#34;: \u0026#34;Mumbai\u0026#34;}, \u0026#34;subscription\u0026#34;: {\u0026#34;tier\u0026#34;: \u0026#34;pro\u0026#34;}}}\u0026#39; # Sometimes incomplete response2 = \u0026#39;{\u0026#34;user\u0026#34;: {\u0026#34;name\u0026#34;: \u0026#34;Rohan\u0026#34;}}\u0026#39; response3 = \u0026#39;{\u0026#34;error\u0026#34;: \u0026#34;user not found\u0026#34;}\u0026#39; Q7 \u0026ndash; Working with Timezones An international tech company schedules a meeting at 10:00 AM IST. Write a function meeting_times(ist_hour, ist_minute) that prints the meeting time in IST, UTC, New York (UTC-5), London (UTC+0 in winter), Dubai (UTC+4), and Singapore (UTC+8).\nQ8 \u0026ndash; Error Handling Write a robust function get_country_population(country_name) that:\nCalls the RestCountries API Returns the population as an integer if successful Returns None for 404 (country not found) Returns None for any network error Raises ValueError if the country name is empty Never crashes regardless of what the API returns Test it with: \u0026ldquo;India\u0026rdquo;, \u0026ldquo;Brazil\u0026rdquo;, \u0026ldquo;notacountry\u0026rdquo;, and \u0026quot;\u0026quot;\nQ9 \u0026ndash; Multiple API Calls Fetch data for five South Asian countries (India, Pakistan, Bangladesh, Sri Lanka, Nepal) from the RestCountries API and build a comparison table showing: name, population, area, and capital. Sort by population descending. Print as a formatted table.\nQ10 \u0026ndash; JSON to CSV Fetch the top 10 countries by population from RestCountries (https://api.restcountries.com/v3.1/all) and save them to a CSV file with columns: name, population, area, region, capital. Use only the standard library (json, csv, requests is fine to use).\nHard Q11 \u0026ndash; Weather Comparison Using the Open-Meteo API, fetch the weather forecast for the next 7 days for Mumbai and Delhi. For each day show the max and min temperature for both cities. Find the day with the greatest temperature difference between the two cities and print it.\nHint: Add daily=temperature_2m_max,temperature_2m_min and forecast_days=7 to your params.\nQ12 \u0026ndash; Currency Trend Simulator Using the exchange rate API from the project, write a function that:\nTakes an amount in INR and a target currency Converts it to the target currency Simulates what the value would have been if the rate were 5% better or worse Prints a comparison showing current value, optimistic value, and pessimistic value Formats all monetary values appropriately for the currency (USD with 2 decimals, JPY with 0 decimals, etc.) Q13 \u0026ndash; API Rate Limiter Build a RateLimitedSession class that wraps requests.Session and:\nAccepts a requests_per_minute limit Tracks when each request was made Automatically waits (sleeps) before making a request if the rate limit would be exceeded Logs every request and wait with a timestamp session = RateLimitedSession(requests_per_minute=10) # These 15 calls should automatically pace themselves to stay under 10/min for city in cities: response = session.get(f\u0026#34;https://api.example.com/weather?city={city}\u0026#34;) Q14 \u0026ndash; Response Caching Build a simple file-based cache for API responses that:\nStores responses as JSON files named by URL hash Accepts a max_age_seconds parameter (default 3600 \u0026ndash; one hour) Returns cached response if it exists and is not expired Fetches fresh data and updates cache if expired or missing Has a clear_cache() method that removes all cached files cache = APICache(cache_dir=\u0026#34;api_cache\u0026#34;, max_age_seconds=3600) # First call -- fetches from API and caches data = cache.get(\u0026#34;https://api.open-meteo.com/v1/forecast\u0026#34;, params={...}) # Second call (within an hour) -- returns from cache, no network request data = cache.get(\u0026#34;https://api.open-meteo.com/v1/forecast\u0026#34;, params={...}) Q15 \u0026ndash; Full Pipeline Build a complete data pipeline that:\nFetches the list of all countries from RestCountries API Filters to Asian countries only For each Asian country, extracts: name, capital, population, area, currencies Saves the raw API response to raw_countries.json Saves the processed data to asian_countries.csv Prints a summary report: Total countries in Asia Most populated country Largest country by area Number of unique currencies used Countries that use INR (Indian Rupee) All API calls must have error handling and timeouts The entire pipeline must complete within 30 seconds or raise TimeoutError 13. Solutions Q1 \u0026ndash; JSON Parsing import json json_str = \u0026#39;\u0026#39;\u0026#39; { \u0026#34;student\u0026#34;: {\u0026#34;name\u0026#34;: \u0026#34;Arjun Mehta\u0026#34;, \u0026#34;roll\u0026#34;: 305, \u0026#34;city\u0026#34;: \u0026#34;Pune\u0026#34;}, \u0026#34;marks\u0026#34;: {\u0026#34;maths\u0026#34;: 78, \u0026#34;science\u0026#34;: 55, \u0026#34;english\u0026#34;: 62, \u0026#34;history\u0026#34;: 48, \u0026#34;cs\u0026#34;: 91}, \u0026#34;active\u0026#34;: true } \u0026#39;\u0026#39;\u0026#39; data = json.loads(json_str) name = data[\u0026#34;student\u0026#34;][\u0026#34;name\u0026#34;] marks = list(data[\u0026#34;marks\u0026#34;].values()) avg = sum(marks) / len(marks) passed = avg \u0026gt;= 60 print(f\u0026#34;Name : {name}\u0026#34;) print(f\u0026#34;Highest : {max(marks)}\u0026#34;) print(f\u0026#34;Average : {avg:.1f}\u0026#34;) print(f\u0026#34;Result : {\u0026#39;Pass\u0026#39; if passed else \u0026#39;Fail\u0026#39;}\u0026#34;) Q2 \u0026ndash; datetime Formatting from datetime import datetime dt = datetime(2024, 3, 15, 14, 30, 45) print(dt.strftime(\u0026#34;%Y-%m-%d\u0026#34;)) # 2024-03-15 print(dt.strftime(\u0026#34;%d/%m/%Y\u0026#34;)) # 15/03/2024 print(dt.strftime(\u0026#34;%A, %d %B %Y\u0026#34;)) # Friday, 15 March 2024 print(dt.strftime(\u0026#34;%I:%M %p\u0026#34;)) # 02:30 PM print(dt.strftime(\u0026#34;%a %d %b %Y, %H:%M:%S\u0026#34;)) # Fri 15 Mar 2024, 14:30:45 Q3 \u0026ndash; datetime Parsing from datetime import datetime dates = [ (\u0026#34;2024-03-15T09:30:00Z\u0026#34;, \u0026#34;%Y-%m-%dT%H:%M:%SZ\u0026#34;), (\u0026#34;15 March 2024 09:30\u0026#34;, \u0026#34;%d %B %Y %H:%M\u0026#34;), (\u0026#34;15/03/24 09:30\u0026#34;, \u0026#34;%d/%m/%y %H:%M\u0026#34;), (\u0026#34;March 15, 2024 09:30 AM\u0026#34;, \u0026#34;%B %d, %Y %I:%M %p\u0026#34;), ] for date_str, fmt in dates: dt = datetime.strptime(date_str, fmt) print(dt.strftime(\u0026#34;%d-%m-%Y %H:%M\u0026#34;)) Q4 \u0026ndash; RestCountries Japan import requests response = requests.get( \u0026#34;https://api.restcountries.com/v3.1/name/japan\u0026#34;, timeout=10 ) response.raise_for_status() japan = response.json()[0] name = japan[\u0026#34;name\u0026#34;][\u0026#34;common\u0026#34;] capital = japan[\u0026#34;capital\u0026#34;][0] pop = japan[\u0026#34;population\u0026#34;] un = japan.get(\u0026#34;unMember\u0026#34;, False) currencies = japan.get(\u0026#34;currencies\u0026#34;, {}) for code, info in currencies.items(): currency = f\u0026#34;{info[\u0026#39;name\u0026#39;]} ({info.get(\u0026#39;symbol\u0026#39;, code)})\u0026#34; print(f\u0026#34;Name : {name}\u0026#34;) print(f\u0026#34;Capital : {capital}\u0026#34;) print(f\u0026#34;Population: {pop:,}\u0026#34;) print(f\u0026#34;Currency : {currency}\u0026#34;) print(f\u0026#34;UN Member : {\u0026#39;Yes\u0026#39; if un else \u0026#39;No\u0026#39;}\u0026#34;) Q5 \u0026ndash; Age Calculator from datetime import date def age_calculator(dob_str): \u0026#34;\u0026#34;\u0026#34;Return (years, months, days) of age from \u0026#39;DD-MM-YYYY\u0026#39; string.\u0026#34;\u0026#34;\u0026#34; dob = date(*reversed([int(x) for x in dob_str.split(\u0026#34;-\u0026#34;)])) today = date.today() years = today.year - dob.year months = today.month - dob.month days = today.day - dob.day if days \u0026lt; 0: months -= 1 # Days in previous month prev_month = today.month - 1 if today.month \u0026gt; 1 else 12 prev_year = today.year if today.month \u0026gt; 1 else today.year - 1 import calendar days += calendar.monthrange(prev_year, prev_month)[1] if months \u0026lt; 0: years -= 1 months += 12 return years, months, days years, months, days = age_calculator(\u0026#34;15-08-1947\u0026#34;) print(f\u0026#34;India\u0026#39;s age: {years} years, {months} months, {days} days\u0026#34;) Q6 \u0026ndash; Safe JSON Navigation import json def extract_profile(response_json): try: data = json.loads(response_json) except json.JSONDecodeError: return {\u0026#34;name\u0026#34;: \u0026#34;Unknown\u0026#34;, \u0026#34;email\u0026#34;: \u0026#34;N/A\u0026#34;, \u0026#34;city\u0026#34;: \u0026#34;N/A\u0026#34;, \u0026#34;tier\u0026#34;: \u0026#34;free\u0026#34;} user = data.get(\u0026#34;user\u0026#34;, {}) return { \u0026#34;name\u0026#34;: user.get(\u0026#34;name\u0026#34;, \u0026#34;Unknown\u0026#34;), \u0026#34;email\u0026#34;: user.get(\u0026#34;email\u0026#34;, \u0026#34;N/A\u0026#34;), \u0026#34;city\u0026#34;: user.get(\u0026#34;location\u0026#34;, {}).get(\u0026#34;city\u0026#34;, \u0026#34;N/A\u0026#34;), \u0026#34;tier\u0026#34;: user.get(\u0026#34;subscription\u0026#34;, {}).get(\u0026#34;tier\u0026#34;, \u0026#34;free\u0026#34;), } response1 = \u0026#39;{\u0026#34;user\u0026#34;: {\u0026#34;name\u0026#34;: \u0026#34;Priya\u0026#34;, \u0026#34;email\u0026#34;: \u0026#34;p@x.com\u0026#34;, \u0026#34;location\u0026#34;: {\u0026#34;city\u0026#34;: \u0026#34;Mumbai\u0026#34;}, \u0026#34;subscription\u0026#34;: {\u0026#34;tier\u0026#34;: \u0026#34;pro\u0026#34;}}}\u0026#39; response2 = \u0026#39;{\u0026#34;user\u0026#34;: {\u0026#34;name\u0026#34;: \u0026#34;Rohan\u0026#34;}}\u0026#39; response3 = \u0026#39;{\u0026#34;error\u0026#34;: \u0026#34;user not found\u0026#34;}\u0026#39; for r in [response1, response2, response3]: print(extract_profile(r)) Q7 \u0026ndash; Timezone Meeting Times from datetime import datetime, timezone, timedelta def meeting_times(ist_hour, ist_minute): IST = timezone(timedelta(hours=5, minutes=30)) UTC = timezone.utc NEW_YORK = timezone(timedelta(hours=-5)) LONDON = timezone(timedelta(hours=0)) DUBAI = timezone(timedelta(hours=4)) SINGAPORE = timezone(timedelta(hours=8)) ist_time = datetime.now(IST).replace( hour=ist_hour, minute=ist_minute, second=0, microsecond=0 ) zones = [ (\u0026#34;IST (India)\u0026#34;, IST), (\u0026#34;UTC\u0026#34;, UTC), (\u0026#34;New York\u0026#34;, NEW_YORK), (\u0026#34;London\u0026#34;, LONDON), (\u0026#34;Dubai\u0026#34;, DUBAI), (\u0026#34;Singapore\u0026#34;, SINGAPORE), ] print(f\u0026#34;Meeting scheduled for {ist_time.strftime(\u0026#39;%I:%M %p\u0026#39;)} IST:\u0026#34;) for label, tz in zones: local_time = ist_time.astimezone(tz) print(f\u0026#34; {label:\u0026lt;18}: {local_time.strftime(\u0026#39;%I:%M %p %Z\u0026#39;)}\u0026#34;) meeting_times(10, 0) Q8 \u0026ndash; Robust Country Population import requests from requests.exceptions import RequestException def get_country_population(country_name): if not country_name or not country_name.strip(): raise ValueError(\u0026#34;Country name cannot be empty.\u0026#34;) try: response = requests.get( f\u0026#34;https://api.restcountries.com/v3.1/name/{country_name.strip()}\u0026#34;, timeout=10 ) if response.status_code == 404: print(f\u0026#34;Country not found: {country_name}\u0026#34;) return None response.raise_for_status() data = response.json() return data[0][\u0026#34;population\u0026#34;] except RequestException as e: print(f\u0026#34;Network error: {e}\u0026#34;) return None except (KeyError, IndexError, ValueError): print(f\u0026#34;Unexpected response format for: {country_name}\u0026#34;) return None for country in [\u0026#34;India\u0026#34;, \u0026#34;Brazil\u0026#34;, \u0026#34;notacountry\u0026#34;]: pop = get_country_population(country) if pop: print(f\u0026#34;{country}: {pop:,}\u0026#34;) try: get_country_population(\u0026#34;\u0026#34;) except ValueError as e: print(f\u0026#34;Error: {e}\u0026#34;) Q9 \u0026ndash; South Asian Countries Table import requests countries = [\u0026#34;India\u0026#34;, \u0026#34;Pakistan\u0026#34;, \u0026#34;Bangladesh\u0026#34;, \u0026#34;Sri Lanka\u0026#34;, \u0026#34;Nepal\u0026#34;] data = [] for country in countries: try: resp = requests.get( f\u0026#34;https://api.restcountries.com/v3.1/name/{country}\u0026#34;, timeout=10 ) resp.raise_for_status() info = resp.json()[0] capital = info.get(\u0026#34;capital\u0026#34;, [\u0026#34;N/A\u0026#34;])[0] data.append({ \u0026#34;name\u0026#34;: info[\u0026#34;name\u0026#34;][\u0026#34;common\u0026#34;], \u0026#34;population\u0026#34;: info[\u0026#34;population\u0026#34;], \u0026#34;area\u0026#34;: info.get(\u0026#34;area\u0026#34;, 0), \u0026#34;capital\u0026#34;: capital, }) except Exception as e: print(f\u0026#34;Failed for {country}: {e}\u0026#34;) data.sort(key=lambda x: x[\u0026#34;population\u0026#34;], reverse=True) print(f\u0026#34;\\n{\u0026#39;Country\u0026#39;:\u0026lt;15} {\u0026#39;Population\u0026#39;:\u0026gt;14} {\u0026#39;Area (km²)\u0026#39;:\u0026gt;12} {\u0026#39;Capital\u0026#39;}\u0026#34;) print(\u0026#34;-\u0026#34; * 55) for d in data: print(f\u0026#34;{d[\u0026#39;name\u0026#39;]:\u0026lt;15} {d[\u0026#39;population\u0026#39;]:\u0026gt;14,} {d[\u0026#39;area\u0026#39;]:\u0026gt;12,.0f} {d[\u0026#39;capital\u0026#39;]}\u0026#34;) Q10 \u0026ndash; Q15 Questions 10 through 15 are extended project-level exercises. Full solutions are available on learn.codeverra.com.\nGuidance:\nQ10: Use response.json() to get all countries, sort by population, slice top 10, use csv.DictWriter Q11: Add daily=temperature_2m_max,temperature_2m_min to params, parse data[\u0026quot;daily\u0026quot;] Q12: Fetch rate once, compute amount * rate, then amount * rate * 1.05 and * 0.95 Q13: Use collections.deque to track request timestamps, time.sleep() to throttle Q14: Use hashlib.md5(url.encode()).hexdigest() for filename, json for storage, os.path.getmtime() for age Q15: Chain all steps in a main() function, wrap each step in try/except 14. What Comes Next With this guide complete, here is where you stand in the curriculum:\nPhase Topic Status Phase 1 Python Basics Complete Phase 1 Control Flow Coming soon Phase 1 Loops Complete Phase 1 Collections Complete Phase 2 Functions Complete Phase 2 Exception Handling Complete Phase 2 File Handling Complete Phase 2 Modules and Packages Complete Phase 2 OOP Complete Phase 3 NumPy Complete Phase 3 Pandas Complete Phase 3 Matplotlib Complete Phase 3 Seaborn Complete Phase 3 Working with the Real World Complete \u0026ndash; this guide Next in the series: Git Basics \u0026ndash; version control for every Python developer.\nMade with care for Codeverra learners | codeverra.com\n","permalink":"https://blogs.codeverra.com/blog/python-19-working-with-real-world/","summary":"\u003ch1 id=\"working-with-the-real-world\"\u003eWorking with the Real World\u003c/h1\u003e\n\u003ch3 id=\"apis-json-dates-and-http-requests-in-python\"\u003eAPIs, JSON, Dates, and HTTP Requests in Python\u003c/h3\u003e\n\u003chr\u003e\n\u003ch2 id=\"before-we-begin--what-changes-after-this-guide\"\u003eBefore We Begin \u0026ndash; What Changes After This Guide\u003c/h2\u003e\n\u003cp\u003eEverything you have learned so far has worked entirely inside your own machine.\nYou write code, it runs, it produces output, it reads files you already have.\u003c/p\u003e\n\u003cp\u003eThis guide is where that changes.\u003c/p\u003e\n\u003cp\u003eAfter this guide your Python programs will be able to:\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003eAsk a server in another country for the current weather in Mumbai\u003c/li\u003e\n\u003cli\u003eGet the latest currency exchange rate between INR and USD\u003c/li\u003e\n\u003cli\u003ePull real-time cricket scores from a sports API\u003c/li\u003e\n\u003cli\u003eSend data to a web service and get a response back\u003c/li\u003e\n\u003cli\u003eWork with dates and times the way real applications do\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003eThis is not tutorial-level Python anymore. This is the Python that powers\nevery data pipeline, every web application, every automation script, and every\ndata science project that talks to the outside world.\u003c/p\u003e","tags":["python"],"title":"Working with the Real World in Python"},{"content":"🔢 NumPy Masterclass and Practice Sheet From Zero to Confident - A Beginner\u0026rsquo;s Guide to Numerical Computing in Python Who is this for? You know Python basics - lists, loops, functions - and you want to level up into data science. NumPy is your first real step. Everything in data science (Pandas, Scikit-learn, TensorFlow) is built on top of it.\n📌 Table of Contents What is NumPy and Why Should You Care? Installing and Importing NumPy Creating Arrays Array Attributes Indexing and Slicing Array Operations - Math Made Easy Universal Functions (ufuncs) Aggregation Functions Reshaping and Transposing Stacking and Splitting Arrays Boolean Masking and Fancy Indexing Broadcasting Working with Random Numbers Linear Algebra with NumPy Sorting and Searching Copies vs Views Saving and Loading Arrays Practice Questions What We Covered + What\u0026rsquo;s Next 1. What is NumPy and Why Should You Care? Python lists are flexible and easy to use - but they are slow when it comes to heavy number crunching. If you have a million exam scores and want to find the average, a Python loop will take noticeably longer than it should.\nNumPy (Numerical Python) solves this by storing data in tightly packed arrays in memory - similar to how C and Fortran work internally. The result? Operations on NumPy arrays can be 10x to 100x faster than equivalent Python loops.\nNumPy gives you:\nA powerful ndarray object (N-dimensional array) Fast mathematical operations on entire arrays at once Tools for linear algebra, random number generation, and statistics The foundation that Pandas, Matplotlib, and Scikit-learn are all built on Think of it this way: If Python lists are a notebook, NumPy arrays are a spreadsheet column - structured, typed, and ready for bulk operations.\n2. Installing and Importing NumPy pip install numpy import numpy as np # \u0026#39;np\u0026#39; is the universal alias - use this everywhere 3. Creating Arrays This is where everything starts. NumPy gives you many ways to create arrays depending on what you need.\n3.1 From a Python List import numpy as np # Cricket scores of Virat Kohli in a 5-match series scores = np.array([82, 45, 110, 67, 93]) print(scores) # Output: [ 82 45 110 67 93] 3.2 2D Array (Matrix) from Nested Lists # Marks of 3 students (Aarav, Priya, Rohan) in 4 subjects marks = np.array([ [78, 85, 90, 88], # Aarav [92, 76, 88, 95], # Priya [65, 70, 80, 75] # Rohan ]) print(marks) 3.3 Built-in Array Creators These are shortcuts to generate common arrays quickly.\n# All zeros - placeholder when you don\u0026#39;t have data yet np.zeros((3, 4)) # 3 rows, 4 columns of 0.0 # All ones - useful for initializations np.ones((2, 3)) # Identity matrix - used in linear algebra np.eye(4) # Fill with a specific value np.full((2, 3), 7) # 2x3 matrix filled with 7 # Range of values (like Python\u0026#39;s range, but returns an array) np.arange(1, 11) # [1, 2, 3, ..., 10] np.arange(0, 100, 10) # [0, 10, 20, ..., 90] # Evenly spaced values between two points np.linspace(0, 1, 5) # [0.0, 0.25, 0.5, 0.75, 1.0] arange vs linspace: Use arange when you know the step size. Use linspace when you know how many points you want.\n3.4 Array Data Types NumPy arrays are typed - every element is the same data type. This is what makes them fast.\n# Default type is inferred a = np.array([1, 2, 3]) print(a.dtype) # int64 # Specify type explicitly b = np.array([1.5, 2.5, 3.5], dtype=np.float32) c = np.array([True, False, True], dtype=bool) # Convert type d = a.astype(float) Common dtypes: int32, int64, float32, float64, bool, str\n4. Array Attributes Before doing anything with an array, it helps to inspect it. These attributes tell you everything about the array\u0026rsquo;s structure.\n# Monthly rainfall (mm) in major Indian cities # Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec rainfall = np.array([ [5, 8, 10, 15, 40, 200, 350, 320, 200, 80, 20, 8], # Mumbai [2, 3, 5, 12, 30, 180, 280, 260, 170, 60, 15, 5], # Pune [10, 12, 20, 25, 60, 100, 120, 110, 100, 50, 30, 15], # Hyderabad ]) print(rainfall.ndim) # 2 → number of dimensions print(rainfall.shape) # (3, 12) → 3 rows, 12 columns print(rainfall.size) # 36 → total number of elements print(rainfall.dtype) # int64 print(rainfall.itemsize) # 8 → bytes per element shape is your most-used attribute. Always check it before any operation to avoid dimension mismatches.\n5. Indexing and Slicing Accessing specific elements or sections of an array.\n5.1 1D Indexing players = np.array([\u0026#34;Rohit\u0026#34;, \u0026#34;Virat\u0026#34;, \u0026#34;Shubman\u0026#34;, \u0026#34;KL Rahul\u0026#34;, \u0026#34;Hardik\u0026#34;]) print(players[0]) # Rohit (first element) print(players[-1]) # Hardik (last element) print(players[1:4]) # [\u0026#39;Virat\u0026#39;, \u0026#39;Shubman\u0026#39;, \u0026#39;KL Rahul\u0026#39;] print(players[::2]) # Every 2nd: [\u0026#39;Rohit\u0026#39;, \u0026#39;Shubman\u0026#39;, \u0026#39;Hardik\u0026#39;] 5.2 2D Indexing For 2D arrays, think of it as [row, column].\nmarks = np.array([ [78, 85, 90, 88], # Aarav [92, 76, 88, 95], # Priya [65, 70, 80, 75] # Rohan ]) print(marks[0, 0]) # 78 - Aarav\u0026#39;s first subject print(marks[1, 3]) # 95 - Priya\u0026#39;s fourth subject print(marks[2, :]) # [65 70 80 75] - Rohan\u0026#39;s all marks (entire row) print(marks[:, 1]) # [85 76 70] - All students\u0026#39; second subject (entire column) print(marks[0:2, 1:3]) # Rows 0-1, Columns 1-2 (submatrix) 5.3 Modifying Elements marks[0, 0] = 80 # Update Aarav\u0026#39;s first mark to 80 marks[2, :] = [70, 75, 85, 80] # Replace Rohan\u0026#39;s entire row 6. Array Operations - Math Made Easy One of NumPy\u0026rsquo;s biggest strengths: operations apply to every element at once, no loops needed.\n6.1 Element-wise Arithmetic # Temperature in Celsius for Delhi, Mumbai, Bangalore, Chennai, Kolkata temp_c = np.array([32.5, 28.0, 22.0, 35.0, 30.0]) # Convert all to Fahrenheit in one line temp_f = (temp_c * 9/5) + 32 print(temp_f) # [90.5 82.4 71.6 95. 86. ] # IPL ticket prices before and after 10% GST base_prices = np.array([500, 1000, 1500, 3000, 5000]) after_gst = base_prices * 1.10 print(after_gst) # [ 550. 1100. 1650. 3300. 5500.] 6.2 Array-to-Array Operations # Test 1 and Test 2 scores for 5 students test1 = np.array([45, 60, 55, 70, 80]) test2 = np.array([50, 58, 62, 65, 75]) total = test1 + test2 diff = test2 - test1 avg = (test1 + test2) / 2 print(\u0026#34;Total:\u0026#34;, total) print(\u0026#34;Difference:\u0026#34;, diff) print(\u0026#34;Average:\u0026#34;, avg) Key rule: Operations between two arrays work element-by-element. Both arrays must have compatible shapes.\n6.3 Comparison Operators scores = np.array([45, 60, 72, 55, 88, 91, 38]) print(scores \u0026gt; 60) # [False False True False True True False] print(scores == 55) # [False False False True False False False] 7. Universal Functions (ufuncs) NumPy\u0026rsquo;s built-in mathematical functions that operate element-wise on arrays. Much faster than applying math module functions in a loop.\ndistances_km = np.array([120.5, 345.0, 78.3, 560.0]) # Square root of each element print(np.sqrt(distances_km)) # Absolute value errors = np.array([-2.5, 3.1, -0.8, 4.4]) print(np.abs(errors)) # Power np.power(np.array([2, 3, 4]), 2) # [4, 9, 16] # Rounding np.round(np.array([2.456, 3.891, 1.234]), 2) # [2.46 3.89 1.23] # Floor and ceiling np.floor(np.array([2.7, 3.1, 4.9])) # [2. 3. 4.] np.ceil(np.array([2.1, 3.0, 4.2])) # [3. 3. 5.] Trigonometric and Logarithmic angles = np.array([0, 30, 45, 60, 90]) radians = np.deg2rad(angles) np.sin(radians) np.cos(radians) # Logarithms revenues = np.array([1000, 5000, 25000, 100000]) np.log(revenues) # Natural log np.log10(revenues) # Base-10 log np.log2(revenues) # Base-2 log # Exponential np.exp(np.array([1, 2, 3])) # [e^1, e^2, e^3] 8. Aggregation Functions Aggregation functions summarize an entire array (or along an axis) into a single value.\n# Monthly sales (in ₹ lakhs) for a shop in Bengaluru monthly_sales = np.array([12.5, 15.0, 9.8, 18.2, 22.0, 16.5, 20.1, 19.4, 14.3, 17.8, 25.0, 30.0]) print(\u0026#34;Total Sales: ₹\u0026#34;, np.sum(monthly_sales), \u0026#34;L\u0026#34;) print(\u0026#34;Average: ₹\u0026#34;, np.mean(monthly_sales), \u0026#34;L\u0026#34;) print(\u0026#34;Median: ₹\u0026#34;, np.median(monthly_sales), \u0026#34;L\u0026#34;) print(\u0026#34;Std Dev: \u0026#34;, np.std(monthly_sales)) print(\u0026#34;Variance: \u0026#34;, np.var(monthly_sales)) print(\u0026#34;Max Month: ₹\u0026#34;, np.max(monthly_sales), \u0026#34;L\u0026#34;) print(\u0026#34;Min Month: ₹\u0026#34;, np.min(monthly_sales), \u0026#34;L\u0026#34;) print(\u0026#34;Range: ₹\u0026#34;, np.ptp(monthly_sales), \u0026#34;L\u0026#34;) # peak-to-peak Aggregation Along Axes (2D) When you have a 2D array, you can aggregate across rows or down columns using the axis parameter.\n# Quarterly scores of 4 sales reps in Jaipur office # Q1 Q2 Q3 Q4 sales = np.array([ [120, 135, 110, 145], # Anjali [98, 105, 120, 130], # Deepak [140, 150, 138, 160], # Meera [88, 92, 95, 105], # Suresh ]) # axis=0 → collapse rows → result has one value per COLUMN print(\u0026#34;Best per quarter:\u0026#34;, np.max(sales, axis=0)) # [140 150 138 160] # axis=1 → collapse columns → result has one value per ROW print(\u0026#34;Each rep\u0026#39;s total:\u0026#34;, np.sum(sales, axis=1)) # [510 453 588 380] print(\u0026#34;Each rep\u0026#39;s avg: \u0026#34;, np.mean(sales, axis=1)) Axis rule: axis=0 works down the rows (per column). axis=1 works across columns (per row). A good trick: axis=0 reduces the array by its first dimension.\n9. Reshaping and Transposing Sometimes you need to change the shape of an array without changing its data.\n9.1 reshape() # Sensor readings from 12 checkpoints, taken in sequence readings = np.arange(1, 13) print(readings) # [ 1 2 3 4 5 6 7 8 9 10 11 12] # Reshape into 3 rows x 4 columns grid = readings.reshape(3, 4) print(grid) # [[ 1 2 3 4] # [ 5 6 7 8] # [ 9 10 11 12]] # Use -1 to let NumPy infer one dimension automatically readings.reshape(4, -1) # 4 rows, NumPy figures out columns = 3 readings.reshape(-1, 6) # NumPy figures out rows = 2, 6 columns 9.2 flatten() and ravel() Both convert a multi-dimensional array into 1D.\ngrid = np.array([[1, 2, 3], [4, 5, 6]]) flat = grid.flatten() # Returns a copy rav = grid.ravel() # Returns a view (faster, be careful with modification) 9.3 Transpose Flips rows and columns. Common in matrix math and data prep.\n# 3 students x 4 subjects → transpose to 4 subjects x 3 students marks = np.array([[78, 85, 90, 88], [92, 76, 88, 95], [65, 70, 80, 75]]) print(marks.shape) # (3, 4) print(marks.T.shape) # (4, 3) print(marks.T) 9.4 expand_dims() and squeeze() a = np.array([1, 2, 3]) # shape: (3,) b = np.expand_dims(a, axis=0) # shape: (1, 3) - adds a row dimension c = np.expand_dims(a, axis=1) # shape: (3, 1) - adds a column dimension d = np.squeeze(b) # shape: (3,) - removes size-1 dimensions 10. Stacking and Splitting Arrays 10.1 Stacking - Combining Arrays # Marks from two class sections section_a = np.array([[78, 85], [92, 76]]) section_b = np.array([[88, 91], [70, 80]]) # Vertical stack - add more rows combined_v = np.vstack([section_a, section_b]) print(combined_v.shape) # (4, 2) # Horizontal stack - add more columns combined_h = np.hstack([section_a, section_b]) print(combined_h.shape) # (2, 4) # np.concatenate - explicit and flexible np.concatenate([section_a, section_b], axis=0) # same as vstack np.concatenate([section_a, section_b], axis=1) # same as hstack 10.2 Splitting - Breaking Arrays Apart data = np.arange(1, 13).reshape(4, 3) # Split into equal parts along rows parts = np.vsplit(data, 2) # 2 parts of 2 rows each print(parts[0]) print(parts[1]) # Split at specific indices top, bottom = np.split(data, [2], axis=0) # first 2 rows vs last 2 rows 11. Boolean Masking and Fancy Indexing These are powerful techniques to filter and select data without writing loops.\n11.1 Boolean Masking # AQI readings across 10 Indian cities cities = np.array([\u0026#34;Delhi\u0026#34;, \u0026#34;Mumbai\u0026#34;, \u0026#34;Lucknow\u0026#34;, \u0026#34;Chennai\u0026#34;, \u0026#34;Kanpur\u0026#34;, \u0026#34;Agra\u0026#34;, \u0026#34;Bangalore\u0026#34;, \u0026#34;Hyderabad\u0026#34;, \u0026#34;Patna\u0026#34;, \u0026#34;Ahmedabad\u0026#34;]) aqi = np.array([312, 98, 280, 75, 320, 290, 65, 88, 305, 210]) # Create a boolean mask poor_air = aqi \u0026gt; 200 print(poor_air) # [ True False True False True True False False True True] # Use mask to filter print(cities[poor_air]) # [\u0026#39;Delhi\u0026#39; \u0026#39;Lucknow\u0026#39; \u0026#39;Kanpur\u0026#39; \u0026#39;Agra\u0026#39; \u0026#39;Patna\u0026#39; \u0026#39;Ahmedabad\u0026#39;] print(aqi[poor_air]) # [312 280 320 290 305 210] # Combine conditions moderate = (aqi \u0026gt;= 100) \u0026amp; (aqi \u0026lt;= 200) print(cities[moderate]) # [\u0026#39;Ahmedabad\u0026#39;] # NOT operator good_air_cities = cities[~poor_air] 11.2 np.where() np.where is like a vectorized if-else - apply a label or value based on a condition.\n# Grade students based on marks marks = np.array([45, 72, 88, 55, 91, 38, 65]) grade = np.where(marks \u0026gt;= 60, \u0026#34;Pass\u0026#34;, \u0026#34;Fail\u0026#34;) print(grade) # [\u0026#39;Fail\u0026#39; \u0026#39;Pass\u0026#39; \u0026#39;Pass\u0026#39; \u0026#39;Fail\u0026#39; \u0026#39;Pass\u0026#39; \u0026#39;Fail\u0026#39; \u0026#39;Pass\u0026#39;] # Nested where for multiple categories category = np.where(marks \u0026gt;= 80, \u0026#34;Distinction\u0026#34;, np.where(marks \u0026gt;= 60, \u0026#34;Pass\u0026#34;, \u0026#34;Fail\u0026#34;)) print(category) # [\u0026#39;Fail\u0026#39; \u0026#39;Pass\u0026#39; \u0026#39;Distinction\u0026#39; \u0026#39;Fail\u0026#39; \u0026#39;Distinction\u0026#39; \u0026#39;Fail\u0026#39; \u0026#39;Pass\u0026#39;] 11.3 Fancy Indexing Select multiple specific elements using an index array.\nipl_teams = np.array([\u0026#34;CSK\u0026#34;, \u0026#34;MI\u0026#34;, \u0026#34;RCB\u0026#34;, \u0026#34;KKR\u0026#34;, \u0026#34;DC\u0026#34;, \u0026#34;SRH\u0026#34;, \u0026#34;PBKS\u0026#34;, \u0026#34;RR\u0026#34;]) # Select specific teams by index selected = ipl_teams[[0, 2, 5]] # CSK, RCB, SRH print(selected) # Select elements from 2D array marks = np.array([[78, 85, 90], [92, 76, 88], [65, 70, 80]]) # Pick (row 0, col 2), (row 1, col 0), (row 2, col 1) print(marks[[0, 1, 2], [2, 0, 1]]) # [90, 92, 70] 12. Broadcasting Broadcasting is how NumPy handles operations between arrays of different shapes. Instead of raising an error, NumPy \u0026ldquo;stretches\u0026rdquo; the smaller array to match the larger one - without actually copying data.\nSimple Example prices = np.array([100, 200, 300, 400]) # Add ₹50 to all prices - the scalar 50 is broadcast across the array discounted = prices - 50 print(discounted) # [ 50 150 250 350] 2D Example # Base salaries of 4 employees in Pune base_salary = np.array([40000, 55000, 70000, 90000]) # shape: (4,) # Monthly bonuses for 3 months monthly_bonus = np.array([5000, 7000, 10000]).reshape(3, 1) # shape: (3, 1) # Broadcasting: (3, 1) broadcasts with (4,) → result is (3, 4) total = base_salary + monthly_bonus print(total) # Each row = that month\u0026#39;s total (base + bonus) for all 4 employees Broadcasting Rules Two dimensions are compatible if:\nThey are equal, OR One of them is 1 NumPy compares shapes from the trailing (rightmost) dimension backwards.\nArray A: (3, 4) Array B: (4,) → treated as (1, 4) → broadcast to (3, 4) ✅ Array A: (3, 4) Array B: (3, 1) → broadcast to (3, 4) ✅ Array A: (3, 4) Array B: (3, 3) → 4 ≠ 3 and neither is 1 → ❌ Error 13. Working with Random Numbers np.random is used for simulations, generating test data, and probabilistic modeling.\nrng = np.random.default_rng(seed=42) # Seed for reproducibility Generating Random Arrays # Random floats between 0 and 1 rng.random(5) # Random integers (e.g., simulate dice rolls) rng.integers(1, 7, size=10) # 10 rolls of a dice # Random floats in a range - simulate stock prices rng.uniform(low=100, high=500, size=8) Sampling from Distributions # Normal distribution - e.g., heights of students in Mumbai heights = rng.normal(loc=165, scale=10, size=1000) # mean=165cm, std=10cm print(f\u0026#34;Mean: {heights.mean():.1f}, Std: {heights.std():.1f}\u0026#34;) # Binomial - simulate 50 coin flips (10 times) rng.binomial(n=50, p=0.5, size=10) # Poisson - number of customers arriving at a Hyderabad store per hour rng.poisson(lam=15, size=24) # 24 hours # Exponential rng.exponential(scale=2.0, size=100) Random Sampling and Shuffling players = np.array([\u0026#34;Rohit\u0026#34;, \u0026#34;Virat\u0026#34;, \u0026#34;Dhoni\u0026#34;, \u0026#34;Bumrah\u0026#34;, \u0026#34;Jadeja\u0026#34;, \u0026#34;Ashwin\u0026#34;, \u0026#34;Pant\u0026#34;, \u0026#34;Shubman\u0026#34;, \u0026#34;Surya\u0026#34;, \u0026#34;Hardik\u0026#34;, \u0026#34;Siraj\u0026#34;]) # Pick 4 players randomly rng.choice(players, size=4, replace=False) # Shuffle in-place rng.shuffle(players) print(players) 14. Linear Algebra with NumPy np.linalg provides core linear algebra operations. This section is especially important if you\u0026rsquo;re heading toward machine learning.\n14.1 Matrix Multiplication # Marks matrix: 3 students x 3 subjects marks = np.array([[80, 75, 90], [70, 85, 95], [60, 65, 70]]) # Weight matrix: importance of each subject weights = np.array([[0.3], [0.3], [0.4]]) # Weighted scores for each student weighted = marks @ weights # @ is the matrix multiplication operator print(weighted) # np.dot - same as @ for 2D arrays result = np.dot(marks, weights) 14.2 Determinant and Inverse A = np.array([[2, 1], [5, 3]]) det = np.linalg.det(A) print(\u0026#34;Determinant:\u0026#34;, det) # 1.0 inv = np.linalg.inv(A) print(\u0026#34;Inverse:\\n\u0026#34;, inv) # Verify: A @ A_inv should be the identity matrix print(np.round(A @ inv)) 14.3 Solving Linear Equations # System of equations: # 2x + y = 8 # 5x + 3y = 21 A = np.array([[2, 1], [5, 3]]) b = np.array([8, 21]) solution = np.linalg.solve(A, b) print(\u0026#34;x =\u0026#34;, solution[0], \u0026#34;, y =\u0026#34;, solution[1]) # x = 3.0, y = 2.0 14.4 Eigenvalues and Norms M = np.array([[4, 2], [1, 3]]) eigenvalues, eigenvectors = np.linalg.eig(M) print(\u0026#34;Eigenvalues:\u0026#34;, eigenvalues) # Vector norm (magnitude) v = np.array([3, 4]) print(\u0026#34;L2 norm:\u0026#34;, np.linalg.norm(v)) # 5.0 # Matrix norm print(\u0026#34;Matrix norm:\u0026#34;, np.linalg.norm(M)) 15. Sorting and Searching 15.1 Sorting # District-wise rainfall (mm) during monsoon rainfall = np.array([320, 185, 450, 275, 510, 390, 220]) # Returns a sorted copy (original unchanged) sorted_rain = np.sort(rainfall) print(sorted_rain) # [185 220 275 320 390 450 510] # Sort in descending order print(np.sort(rainfall)[::-1]) # Sort in-place (modifies original) rainfall.sort() 15.2 argsort() - Indices of Sorted Order Very useful when you want to rank items.\nplayers = np.array([\u0026#34;Rohit\u0026#34;, \u0026#34;Virat\u0026#34;, \u0026#34;Shubman\u0026#34;, \u0026#34;Dhoni\u0026#34;, \u0026#34;Jadeja\u0026#34;]) centuries = np.array([45, 80, 12, 70, 3]) # Get indices that would sort the array order = np.argsort(centuries)[::-1] # descending print(players[order]) # [\u0026#39;Virat\u0026#39; \u0026#39;Dhoni\u0026#39; \u0026#39;Rohit\u0026#39; \u0026#39;Shubman\u0026#39; \u0026#39;Jadeja\u0026#39;] print(centuries[order]) # [80 70 45 12 3] 15.3 Searching aqi = np.array([312, 98, 280, 75, 320, 290, 65, 88]) # Index of the minimum/maximum element print(np.argmin(aqi)) # 6 (AQI = 65) print(np.argmax(aqi)) # 4 (AQI = 320) # All indices where condition is True print(np.where(aqi \u0026gt; 200)) # (array([0, 2, 4, 5]),) # Check if any/all elements satisfy condition print(np.any(aqi \u0026gt; 300)) # True print(np.all(aqi \u0026gt; 50)) # True 16. Copies vs Views This is one of the most common sources of bugs for beginners. Understanding this will save you a lot of headaches.\nThe Problem original = np.array([10, 20, 30, 40, 50]) # SLICE = VIEW (points to same memory) view = original[1:4] view[0] = 999 print(original) # [10 999 30 40 50] ← original was CHANGED! # COPY = independent (different memory) original = np.array([10, 20, 30, 40, 50]) copy = original[1:4].copy() copy[0] = 999 print(original) # [10 20 30 40 50] ← original is SAFE How to Check a = np.array([1, 2, 3]) b = a[:] # view c = a.copy() # copy print(b.base is a) # True → b is a view of a print(c.base is a) # False → c is independent Rule of thumb: If you plan to modify a slice and don\u0026rsquo;t want to affect the original, always call .copy().\n17. Saving and Loading Arrays scores = np.array([82, 45, 110, 67, 93]) # Save single array to .npy file (binary format) np.save(\u0026#34;kohli_scores.npy\u0026#34;, scores) # Load it back loaded = np.load(\u0026#34;kohli_scores.npy\u0026#34;) print(loaded) # Save multiple arrays to a .npz file marks_2023 = np.array([78, 85, 90]) marks_2024 = np.array([82, 79, 95]) np.savez(\u0026#34;student_marks.npz\u0026#34;, year2023=marks_2023, year2024=marks_2024) # Load npz data = np.load(\u0026#34;student_marks.npz\u0026#34;) print(data[\u0026#34;year2023\u0026#34;]) print(data[\u0026#34;year2024\u0026#34;]) # Save as plain text (CSV-compatible) matrix = np.array([[1, 2, 3], [4, 5, 6]]) np.savetxt(\u0026#34;matrix.csv\u0026#34;, matrix, delimiter=\u0026#34;,\u0026#34;, fmt=\u0026#34;%d\u0026#34;) loaded_matrix = np.loadtxt(\u0026#34;matrix.csv\u0026#34;, delimiter=\u0026#34;,\u0026#34;) 18. Practice Questions Test yourself. Try to solve these without looking at the solutions first.\nQ1 - Array Creation Create a NumPy array of the first 20 odd numbers. Then reshape it into a 4×5 matrix.\nQ2 - Slicing Given the array below (representing daily temperatures in Delhi for a week), extract temperatures from Tuesday to Friday.\ntemps = np.array([22, 25, 27, 24, 26, 23, 28]) # Mon Tue Wed Thu Fri Sat Sun Q3 - Broadcasting You have the base price of 5 products on Flipkart. Apply a 12% GST to all prices using a single operation (no loops).\nbase_prices = np.array([299, 499, 1299, 2499, 9999]) Q4 - Boolean Masking Given a list of student names and their marks, print the names of students who scored above 70.\nnames = np.array([\u0026#34;Aarav\u0026#34;, \u0026#34;Priya\u0026#34;, \u0026#34;Rohan\u0026#34;, \u0026#34;Sneha\u0026#34;, \u0026#34;Karan\u0026#34;]) marks = np.array([65, 82, 58, 91, 74]) Q5 - Aggregation with Axis You have quarterly revenue (₹ crore) for 3 branches (Delhi, Mumbai, Hyderabad) over 4 quarters. Find:\nTotal annual revenue per branch Best performing quarter across all branches revenue = np.array([ [45, 52, 48, 60], # Delhi [38, 41, 55, 62], # Mumbai [30, 35, 40, 45], # Hyderabad ]) Q6 - argsort Ranking Given IPL teams and their win counts this season, print team names ranked from most wins to fewest.\nteams = np.array([\u0026#34;CSK\u0026#34;, \u0026#34;MI\u0026#34;, \u0026#34;RCB\u0026#34;, \u0026#34;KKR\u0026#34;, \u0026#34;DC\u0026#34;]) wins = np.array([9, 7, 5, 10, 6]) Q7 - np.where Given AQI readings for 8 cities, create a label array: \u0026ldquo;Good\u0026rdquo; (AQI \u0026lt; 100), \u0026ldquo;Moderate\u0026rdquo; (100–200), \u0026ldquo;Poor\u0026rdquo; (\u0026gt; 200).\naqi = np.array([65, 145, 312, 88, 220, 175, 55, 280]) Q8 - Linear Algebra Solve the following system of equations using np.linalg.solve:\n3x + 2y = 16 x + 4y = 14 Q9 - Random Simulation Simulate the scores of 500 students in an exam where scores follow a normal distribution with mean = 68 and standard deviation = 12. Then find:\nHow many students scored above 80 The percentage of students who failed (scored below 40) Q10 - Copies vs Views Predict the output of the following code before running it:\na = np.array([1, 2, 3, 4, 5]) b = a[2:] b[0] = 99 print(a) Then fix it so that modifying b does not affect a.\nQ11 - Stacking You have exam marks from two different test dates for the same 4 students. Stack them vertically and horizontally and explain the shape of each result.\ntest1 = np.array([[55, 70], [80, 65]]) test2 = np.array([[60, 72], [85, 68]]) Q12 - Rainfall Analysis Use the rainfall data below to answer:\nWhich month had the highest average rainfall across all cities? Which city had the most total annual rainfall? # Months: Jan–Dec | Cities: Delhi, Mumbai, Kolkata rainfall = np.array([ [20, 25, 18, 10, 5, 180, 220, 200, 120, 30, 5, 10], [ 5, 8, 10, 15, 40, 200, 350, 320, 200, 80, 20, 8], [10, 15, 20, 40, 80, 250, 380, 360, 250, 120, 30, 15], ]) Q13 - Normalization Normalize the following array of house prices (in ₹ lakhs) to a 0–1 scale using min-max normalization: normalized = (x - min) / (max - min)\nprices = np.array([25, 45, 60, 80, 35, 120, 55]) Q14 - Matrix Operations Create a 4×4 matrix with values from 1 to 16. Then:\nExtract the diagonal Find the trace (sum of diagonal) Replace all even numbers with 0 Q15 - Full Pipeline You are given marks of 6 students across 5 subjects. Write NumPy code to:\nCalculate each student\u0026rsquo;s total and average Find the class topper (highest average) Find subjects where the class average is below 65 Give \u0026ldquo;Scholarship\u0026rdquo; to students with average ≥ 80 and \u0026ldquo;Regular\u0026rdquo; to others student_names = np.array([\u0026#34;Aarav\u0026#34;, \u0026#34;Priya\u0026#34;, \u0026#34;Rohan\u0026#34;, \u0026#34;Sneha\u0026#34;, \u0026#34;Karan\u0026#34;, \u0026#34;Meera\u0026#34;]) marks = np.array([ [78, 85, 90, 88, 92], [92, 76, 88, 95, 80], [65, 70, 60, 75, 68], [88, 91, 85, 90, 94], [55, 60, 58, 62, 70], [82, 88, 79, 85, 91], ]) 19. What We Covered + What\u0026rsquo;s Next ✅ What This Masterclass Covered Section Topics Array Basics Creating arrays, data types, built-in creators Array Properties shape, ndim, size, dtype, itemsize Indexing \u0026amp; Slicing 1D/2D indexing, modifying elements Math Operations Element-wise, array-to-array, comparisons Universal Functions sqrt, abs, log, exp, trig functions Aggregation sum, mean, median, std, min, max with axis Reshaping reshape, flatten, ravel, transpose, expand_dims Stacking \u0026amp; Splitting vstack, hstack, concatenate, split Boolean Masking Filtering arrays, np.where for conditional values Fancy Indexing Index arrays, multi-dim selection Broadcasting Shape compatibility rules, real-world use cases Random Numbers Distributions, sampling, shuffling, seeding Linear Algebra Matrix multiply, inverse, solve, eigenvalues, norms Sorting \u0026amp; Searching sort, argsort, argmin/max, any/all Copies vs Views Memory behavior, when to use .copy() File I/O save, load, savez, savetxt, loadtxt 🚀 What to Explore Next If this masterclass felt comfortable, here\u0026rsquo;s where to go next:\n1. Pandas The next layer on top of NumPy. Pandas adds labels (column names, row indices), handles missing data, and makes tabular data analysis much more convenient. Every data analyst uses it daily.\n2. Matplotlib \u0026amp; Seaborn Visualize your NumPy and Pandas data. Histograms, scatter plots, heatmaps, line charts - all powered by arrays underneath.\n3. Structured Arrays NumPy supports arrays where each column can have a different dtype (like a lightweight table). Useful for low-memory work.\n4. NumPy Memory Layout (Advanced) Understanding C-order vs F-order, memory contiguity, and how strides work. Important for performance-critical code.\n5. Scikit-learn Machine learning library that expects NumPy arrays as input. Once you\u0026rsquo;re comfortable with array math, the transition to ML is much smoother.\n6. SciPy Built on top of NumPy. Adds advanced scientific computing - integration, optimization, signal processing, statistics.\nFinal thought: NumPy\u0026rsquo;s real power isn\u0026rsquo;t any single function - it\u0026rsquo;s the mindset shift from \u0026ldquo;loop over every element\u0026rdquo; to \u0026ldquo;operate on the whole array at once.\u0026rdquo; The more naturally you think in arrays, the faster and cleaner your data code becomes.\nKeep practicing. The practice questions above are your gym. 💪\n20. Practice Question Solutions Q1 \u0026ndash; Array Creation import numpy as np # First 20 odd numbers: 1, 3, 5, ..., 39 odd_numbers = np.arange(1, 40, 2) print(odd_numbers) # [ 1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39] matrix = odd_numbers.reshape(4, 5) print(matrix) # [[ 1 3 5 7 9] # [11 13 15 17 19] # [21 23 25 27 29] # [31 33 35 37 39]] Q2 \u0026ndash; Slicing temps = np.array([22, 25, 27, 24, 26, 23, 28]) # Mon Tue Wed Thu Fri Sat Sun # idx: 0 1 2 3 4 5 6 # Tuesday is index 1, Friday is index 4 (inclusive), so slice [1:5] tue_to_fri = temps[1:5] print(tue_to_fri) # [25 27 24 26] Q3 \u0026ndash; Broadcasting base_prices = np.array([299, 499, 1299, 2499, 9999]) # Multiply entire array by 1.12 in one operation -- no loop needed after_gst = np.round(base_prices * 1.12, 2) print(after_gst) # [ 334.88 558.88 1454.88 2798.88 11198.88] Q4 \u0026ndash; Boolean Masking names = np.array([\u0026#34;Aarav\u0026#34;, \u0026#34;Priya\u0026#34;, \u0026#34;Rohan\u0026#34;, \u0026#34;Sneha\u0026#34;, \u0026#34;Karan\u0026#34;]) marks = np.array([65, 82, 58, 91, 74]) mask = marks \u0026gt; 70 print(names[mask]) # [\u0026#39;Priya\u0026#39; \u0026#39;Sneha\u0026#39; \u0026#39;Karan\u0026#39;] Q5 \u0026ndash; Aggregation with Axis revenue = np.array([ [45, 52, 48, 60], # Delhi [38, 41, 55, 62], # Mumbai [30, 35, 40, 45], # Hyderabad ]) # axis=1 collapses columns --\u0026gt; one total per row (per branch) branch_totals = np.sum(revenue, axis=1) print(\u0026#34;Annual revenue per branch:\u0026#34;, branch_totals) # [205 196 150] # Delhi: 205, Mumbai: 196, Hyderabad: 150 # axis=0 collapses rows --\u0026gt; one value per column (per quarter) best_quarter_idx = np.argmax(np.sum(revenue, axis=0)) print(\u0026#34;Best quarter:\u0026#34;, best_quarter_idx + 1) # Quarter 4 (index 3) had the highest combined revenue Q6 \u0026ndash; argsort Ranking teams = np.array([\u0026#34;CSK\u0026#34;, \u0026#34;MI\u0026#34;, \u0026#34;RCB\u0026#34;, \u0026#34;KKR\u0026#34;, \u0026#34;DC\u0026#34;]) wins = np.array([9, 7, 5, 10, 6]) # argsort gives indices that would sort the array ascending # [::-1] reverses to descending order ranked_idx = np.argsort(wins)[::-1] print(\u0026#34;Teams ranked by wins:\u0026#34;) for i, idx in enumerate(ranked_idx, start=1): print(f\u0026#34; {i}. {teams[idx]} -- {wins[idx]} wins\u0026#34;) # 1. KKR -- 10 wins # 2. CSK -- 9 wins # 3. MI -- 7 wins # 4. DC -- 6 wins # 5. RCB -- 5 wins Q7 \u0026ndash; np.where aqi = np.array([65, 145, 312, 88, 220, 175, 55, 280]) labels = np.where(aqi \u0026lt; 100, \u0026#34;Good\u0026#34;, np.where(aqi \u0026lt;= 200, \u0026#34;Moderate\u0026#34;, \u0026#34;Poor\u0026#34;)) print(labels) # [\u0026#39;Good\u0026#39; \u0026#39;Moderate\u0026#39; \u0026#39;Poor\u0026#39; \u0026#39;Good\u0026#39; \u0026#39;Poor\u0026#39; \u0026#39;Moderate\u0026#39; \u0026#39;Good\u0026#39; \u0026#39;Poor\u0026#39;] Q8 \u0026ndash; Linear Algebra # 3x + 2y = 16 # x + 4y = 14 A = np.array([[3, 2], [1, 4]]) b = np.array([16, 14]) solution = np.linalg.solve(A, b) print(f\u0026#34;x = {solution[0]}, y = {solution[1]}\u0026#34;) # x = 3.0, y = 2.75 # Verify print(np.allclose(A @ solution, b)) # True Q9 \u0026ndash; Random Simulation rng = np.random.default_rng(seed=42) scores = rng.normal(loc=68, scale=12, size=500) above_80 = np.sum(scores \u0026gt; 80) print(f\u0026#34;Students who scored above 80: {above_80}\u0026#34;) failed_pct = np.sum(scores \u0026lt; 40) / 500 * 100 print(f\u0026#34;Percentage who failed: {failed_pct:.1f}%\u0026#34;) # Note: exact numbers vary slightly with seed, but expected values are: # above 80 --\u0026gt; roughly 13-15% of 500 (~65-75 students) # below 40 --\u0026gt; roughly 1-2% of 500 (~5-10 students) Q10 \u0026ndash; Copies vs Views a = np.array([1, 2, 3, 4, 5]) b = a[2:] b[0] = 99 print(a) # Output: [ 1 2 99 4 5] # b is a VIEW of a -- modifying b modifies the original array too Fix by using .copy():\na = np.array([1, 2, 3, 4, 5]) b = a[2:].copy() # now b is an independent copy b[0] = 99 print(a) # [ 1 2 3 4 5] -- original is unchanged print(b) # [99 4 5] Q11 \u0026ndash; Stacking test1 = np.array([[55, 70], [80, 65]]) test2 = np.array([[60, 72], [85, 68]]) # Vertical stack -- adds more rows v = np.vstack([test1, test2]) print(\u0026#34;vstack shape:\u0026#34;, v.shape) # (4, 2) -- 4 rows, 2 columns print(v) # [[55 70] # [80 65] # [60 72] # [85 68]] # Horizontal stack -- adds more columns h = np.hstack([test1, test2]) print(\u0026#34;hstack shape:\u0026#34;, h.shape) # (2, 4) -- 2 rows, 4 columns print(h) # [[55 70 60 72] # [80 65 85 68]] Q12 \u0026ndash; Rainfall Analysis # Cities: Delhi (row 0), Mumbai (row 1), Kolkata (row 2) # Months: Jan (col 0) to Dec (col 11) rainfall = np.array([ [20, 25, 18, 10, 5, 180, 220, 200, 120, 30, 5, 10], [ 5, 8, 10, 15, 40, 200, 350, 320, 200, 80, 20, 8], [10, 15, 20, 40, 80, 250, 380, 360, 250, 120, 30, 15], ]) months = [\u0026#34;Jan\u0026#34;,\u0026#34;Feb\u0026#34;,\u0026#34;Mar\u0026#34;,\u0026#34;Apr\u0026#34;,\u0026#34;May\u0026#34;,\u0026#34;Jun\u0026#34;,\u0026#34;Jul\u0026#34;,\u0026#34;Aug\u0026#34;,\u0026#34;Sep\u0026#34;,\u0026#34;Oct\u0026#34;,\u0026#34;Nov\u0026#34;,\u0026#34;Dec\u0026#34;] cities = [\u0026#34;Delhi\u0026#34;, \u0026#34;Mumbai\u0026#34;, \u0026#34;Kolkata\u0026#34;] # Monthly average across all cities (axis=0 averages across rows) monthly_avg = np.mean(rainfall, axis=0) wettest_month = np.argmax(monthly_avg) print(f\u0026#34;Wettest month: {months[wettest_month]}\u0026#34;) # July (index 6) # Total annual rainfall per city (axis=1 sums across columns) annual_totals = np.sum(rainfall, axis=1) wettest_city = np.argmax(annual_totals) print(f\u0026#34;Most rainfall: {cities[wettest_city]} ({annual_totals[wettest_city]} mm)\u0026#34;) # Kolkata Q13 \u0026ndash; Normalization prices = np.array([25, 45, 60, 80, 35, 120, 55]) min_price = np.min(prices) max_price = np.max(prices) normalized = (prices - min_price) / (max_price - min_price) print(np.round(normalized, 3)) # [0. 0.211 0.368 0.579 0.105 1. 0.316] # 25 maps to 0.0 (minimum), 120 maps to 1.0 (maximum) Q14 \u0026ndash; Matrix Operations matrix = np.arange(1, 17).reshape(4, 4) print(matrix) # [[ 1 2 3 4] # [ 5 6 7 8] # [ 9 10 11 12] # [13 14 15 16]] # Extract diagonal diag = np.diag(matrix) print(\u0026#34;Diagonal:\u0026#34;, diag) # [ 1 6 11 16] # Trace (sum of diagonal) trace = np.trace(matrix) print(\u0026#34;Trace:\u0026#34;, trace) # 34 # Replace all even numbers with 0 matrix_copy = matrix.copy() matrix_copy[matrix_copy % 2 == 0] = 0 print(matrix_copy) # [[ 1 0 3 0] # [ 5 0 7 0] # [ 9 0 11 0] # [13 0 15 0]] Q15 \u0026ndash; Full Pipeline student_names = np.array([\u0026#34;Aarav\u0026#34;, \u0026#34;Priya\u0026#34;, \u0026#34;Rohan\u0026#34;, \u0026#34;Sneha\u0026#34;, \u0026#34;Karan\u0026#34;, \u0026#34;Meera\u0026#34;]) marks = np.array([ [78, 85, 90, 88, 92], [92, 76, 88, 95, 80], [65, 70, 60, 75, 68], [88, 91, 85, 90, 94], [55, 60, 58, 62, 70], [82, 88, 79, 85, 91], ]) # 1. Total and average per student totals = np.sum(marks, axis=1) averages = np.mean(marks, axis=1) print(\u0026#34;Totals :\u0026#34;, totals) # [433 431 338 448 305 425] print(\u0026#34;Averages:\u0026#34;, np.round(averages, 1)) # [86.6 86.2 67.6 89.6 61.0 85.0] # 2. Class topper topper_idx = np.argmax(averages) print(f\u0026#34;Topper: {student_names[topper_idx]} with avg {averages[topper_idx]:.1f}\u0026#34;) # Sneha with avg 89.6 # 3. Subjects where class average is below 65 subject_avgs = np.mean(marks, axis=0) subject_names = np.array([\u0026#34;Maths\u0026#34;, \u0026#34;Science\u0026#34;, \u0026#34;English\u0026#34;, \u0026#34;History\u0026#34;, \u0026#34;CS\u0026#34;]) weak_subjects = subject_names[subject_avgs \u0026lt; 65] print(\u0026#34;Subjects below class avg 65:\u0026#34;, weak_subjects) # No subject is below 65 in this dataset -- print will show empty array # (All subject averages: [76.7 78.3 76.7 82.5 82.5]) # 4. Scholarship vs Regular labels = np.where(averages \u0026gt;= 80, \u0026#34;Scholarship\u0026#34;, \u0026#34;Regular\u0026#34;) for name, label in zip(student_names, labels): print(f\u0026#34; {name}: {label}\u0026#34;) # Aarav : Scholarship # Priya : Scholarship # Rohan : Regular # Sneha : Scholarship # Karan : Regular # Meera : Scholarship Made with care for Codeverra learners | codeverra.com\n","permalink":"https://blogs.codeverra.com/blog/numpy-practice/","summary":"\u003ch1 id=\"-numpy-masterclass-and-practice-sheet\"\u003e🔢 NumPy Masterclass and Practice Sheet\u003c/h1\u003e\n\u003ch3 id=\"from-zero-to-confident---a-beginner\"\u003e\u003cem\u003eFrom Zero to Confident - A Beginner\u0026rsquo;s Guide to Numerical Computing in Python\u003c/em\u003e\u003c/h3\u003e\n\u003chr\u003e\n\u003cblockquote\u003e\n\u003cp\u003e\u003cstrong\u003eWho is this for?\u003c/strong\u003e\nYou know Python basics  -  lists, loops, functions  -  and you want to level up into data science.\nNumPy is your first real step. Everything in data science (Pandas, Scikit-learn, TensorFlow) is built on top of it.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003chr\u003e\n\u003ch2 id=\"-table-of-contents\"\u003e📌 Table of Contents\u003c/h2\u003e\n\u003col\u003e\n\u003cli\u003e\u003ca href=\"#1-what-is-numpy-and-why-should-you-care\"\u003eWhat is NumPy and Why Should You Care?\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#2-installing-and-importing-numpy\"\u003eInstalling and Importing NumPy\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#3-creating-arrays\"\u003eCreating Arrays\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#4-array-attributes\"\u003eArray Attributes\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#5-indexing-and-slicing\"\u003eIndexing and Slicing\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#6-array-operations--math-made-easy\"\u003eArray Operations  -  Math Made Easy\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#7-universal-functions-ufuncs\"\u003eUniversal Functions (ufuncs)\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#8-aggregation-functions\"\u003eAggregation Functions\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#9-reshaping-and-transposing\"\u003eReshaping and Transposing\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#10-stacking-and-splitting-arrays\"\u003eStacking and Splitting Arrays\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#11-boolean-masking-and-fancy-indexing\"\u003eBoolean Masking and Fancy Indexing\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#12-broadcasting\"\u003eBroadcasting\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#13-working-with-random-numbers\"\u003eWorking with Random Numbers\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#14-linear-algebra-with-numpy\"\u003eLinear Algebra with NumPy\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#15-sorting-and-searching\"\u003eSorting and Searching\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#16-copies-vs-views\"\u003eCopies vs Views\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#17-saving-and-loading-arrays\"\u003eSaving and Loading Arrays\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#18-practice-questions\"\u003ePractice Questions\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#19-what-we-covered--whats-next\"\u003eWhat We Covered + What\u0026rsquo;s Next\u003c/a\u003e\u003c/li\u003e\n\u003c/ol\u003e\n\u003chr\u003e\n\u003ch2 id=\"1-what-is-numpy-and-why-should-you-care\"\u003e1. What is NumPy and Why Should You Care?\u003c/h2\u003e\n\u003cp\u003ePython lists are flexible and easy to use  -  but they are \u003cstrong\u003eslow\u003c/strong\u003e when it comes to heavy number crunching. If you have a million exam scores and want to find the average, a Python loop will take noticeably longer than it should.\u003c/p\u003e","tags":["python","numpy","data-analysis"],"title":"Complete Guide to Numpy Library in Python"},{"content":"🐼 Pandas Masterclass From Raw Data to Real Insights - A Beginner\u0026rsquo;s Guide to Data Analysis in Python Who is this for? You\u0026rsquo;ve done the NumPy Masterclass (or know the basics of Python and arrays). Now you want to work with real tabular data - rows, columns, missing values, merges, and group-level summaries. That\u0026rsquo;s exactly what Pandas is for.\n📌 Table of Contents What is Pandas and Why Do You Need It? Installing and Importing Pandas The Two Core Data Structures Creating DataFrames Reading and Writing Data Exploring a DataFrame Selecting Data - loc, iloc, and Column Access Filtering Rows Adding, Modifying, and Dropping Columns Sorting Data Handling Missing Data String Operations Working with Dates and Times apply(), map(), and applymap() GroupBy - Aggregation and Split-Apply-Combine Merging and Joining DataFrames Pivot Tables and Crosstabs Reshaping - melt() and stack()/unstack() MultiIndex DataFrames Performance Tips Practice Questions What We Covered + What\u0026rsquo;s Next 1. What is Pandas and Why Do You Need It? NumPy arrays are powerful, but they have one limitation: every column must be the same data type, and there are no column names or row labels.\nReal-world data doesn\u0026rsquo;t look like that. A sales table has strings (city names), integers (quantities), floats (prices), and dates - all in the same table.\nPandas gives you:\nA DataFrame - a 2D table with named columns and labeled rows, where each column can be a different type A Series - a single labeled column Built-in tools for reading CSVs, Excel files, SQL, and JSON Powerful data cleaning (missing values, duplicates, type conversions) Grouping, aggregating, merging, pivoting - the full analytics toolkit Think of it this way: NumPy is a calculator. Pandas is a spreadsheet that also knows how to do the math.\n2. Installing and Importing Pandas pip install pandas import pandas as pd import numpy as np # often used alongside pandas 3. The Two Core Data Structures 3.1 Series - A Labeled 1D Array A Series is like a single column from a spreadsheet. It has values and an index (labels for each row).\n# Monthly revenue (₹ lakhs) for a chai shop in Nagpur revenue = pd.Series( [12.5, 15.0, 9.8, 18.2, 22.0, 16.5], index=[\u0026#34;Jan\u0026#34;, \u0026#34;Feb\u0026#34;, \u0026#34;Mar\u0026#34;, \u0026#34;Apr\u0026#34;, \u0026#34;May\u0026#34;, \u0026#34;Jun\u0026#34;], name=\u0026#34;Revenue (₹L)\u0026#34; ) print(revenue) # Jan 12.5 # Feb 15.0 # Mar 9.8 # Apr 18.2 # May 22.0 # Jun 16.5 # Name: Revenue (₹L), dtype: float64 print(revenue[\u0026#34;Apr\u0026#34;]) # 18.2 print(revenue[[\u0026#34;Jan\u0026#34;, \u0026#34;Mar\u0026#34;, \u0026#34;May\u0026#34;]]) # Multiple labels print(revenue.mean()) # 15.67 A Series always has an index. By default it\u0026rsquo;s 0, 1, 2\u0026hellip; but you can make it anything - dates, city names, IDs.\n3.2 DataFrame - A Labeled 2D Table A DataFrame is a collection of Series sharing the same index. Think of it as a table where each column is a Series.\ndata = { \u0026#34;Name\u0026#34;: [\u0026#34;Aarav\u0026#34;, \u0026#34;Priya\u0026#34;, \u0026#34;Rohan\u0026#34;, \u0026#34;Sneha\u0026#34;, \u0026#34;Karan\u0026#34;], \u0026#34;City\u0026#34;: [\u0026#34;Delhi\u0026#34;, \u0026#34;Mumbai\u0026#34;, \u0026#34;Bangalore\u0026#34;, \u0026#34;Hyderabad\u0026#34;, \u0026#34;Pune\u0026#34;], \u0026#34;Score\u0026#34;: [85, 92, 78, 95, 70], \u0026#34;Grade\u0026#34;: [\u0026#34;B\u0026#34;, \u0026#34;A\u0026#34;, \u0026#34;B\u0026#34;, \u0026#34;A\u0026#34;, \u0026#34;C\u0026#34;] } df = pd.DataFrame(data) print(df) # Name City Score Grade # 0 Aarav Delhi 85 B # 1 Priya Mumbai 92 A # 2 Rohan Bangalore 78 B # 3 Sneha Hyderabad 95 A # 4 Karan Pune 70 C 4. Creating DataFrames There are several ways to create a DataFrame depending on what data you start with.\nFrom a Dictionary employees = pd.DataFrame({ \u0026#34;emp_id\u0026#34;: [101, 102, 103, 104, 105], \u0026#34;name\u0026#34;: [\u0026#34;Vikram\u0026#34;, \u0026#34;Ananya\u0026#34;, \u0026#34;Suresh\u0026#34;, \u0026#34;Divya\u0026#34;, \u0026#34;Rahul\u0026#34;], \u0026#34;department\u0026#34;: [\u0026#34;Engineering\u0026#34;, \u0026#34;HR\u0026#34;, \u0026#34;Sales\u0026#34;, \u0026#34;Engineering\u0026#34;, \u0026#34;Marketing\u0026#34;], \u0026#34;salary\u0026#34;: [75000, 55000, 48000, 82000, 60000], \u0026#34;city\u0026#34;: [\u0026#34;Bangalore\u0026#34;, \u0026#34;Mumbai\u0026#34;, \u0026#34;Delhi\u0026#34;, \u0026#34;Hyderabad\u0026#34;, \u0026#34;Chennai\u0026#34;] }) From a List of Dictionaries Each dictionary = one row. Very common when reading JSON APIs.\norders = pd.DataFrame([ {\u0026#34;order_id\u0026#34;: 1001, \u0026#34;product\u0026#34;: \u0026#34;Laptop\u0026#34;, \u0026#34;qty\u0026#34;: 1, \u0026#34;price\u0026#34;: 65000}, {\u0026#34;order_id\u0026#34;: 1002, \u0026#34;product\u0026#34;: \u0026#34;Mouse\u0026#34;, \u0026#34;qty\u0026#34;: 3, \u0026#34;price\u0026#34;: 850}, {\u0026#34;order_id\u0026#34;: 1003, \u0026#34;product\u0026#34;: \u0026#34;Monitor\u0026#34;, \u0026#34;qty\u0026#34;: 2, \u0026#34;price\u0026#34;: 18000}, ]) From NumPy Array marks_array = np.array([[78, 85, 90], [92, 76, 88], [65, 70, 80]]) df_marks = pd.DataFrame( marks_array, index=[\u0026#34;Aarav\u0026#34;, \u0026#34;Priya\u0026#34;, \u0026#34;Rohan\u0026#34;], columns=[\u0026#34;Maths\u0026#34;, \u0026#34;Science\u0026#34;, \u0026#34;English\u0026#34;] ) Setting a Custom Index employees.set_index(\u0026#34;emp_id\u0026#34;, inplace=True) # Now rows are labeled by emp_id instead of 0, 1, 2... 5. Reading and Writing Data In practice, you almost never create DataFrames manually - you read them from files.\nReading CSV # Read a CSV file df = pd.read_csv(\u0026#34;zomato_bangalore.csv\u0026#34;) # Useful parameters df = pd.read_csv( \u0026#34;zomato_bangalore.csv\u0026#34;, usecols=[\u0026#34;name\u0026#34;, \u0026#34;rate\u0026#34;, \u0026#34;location\u0026#34;, \u0026#34;cuisines\u0026#34;], # only load these columns nrows=500, # load only first 500 rows (great for large files) skiprows=1, # skip first row if it\u0026#39;s not a header na_values=[\u0026#34;--\u0026#34;, \u0026#34;N/A\u0026#34;, \u0026#34;none\u0026#34;], # treat these as NaN encoding=\u0026#34;utf-8\u0026#34; ) Writing CSV df.to_csv(\u0026#34;cleaned_data.csv\u0026#34;, index=False) # index=False avoids writing row numbers Reading Excel df = pd.read_excel(\u0026#34;ipl_stats.xlsx\u0026#34;, sheet_name=\u0026#34;Season_2023\u0026#34;) df.to_excel(\u0026#34;output.xlsx\u0026#34;, sheet_name=\u0026#34;Results\u0026#34;, index=False) Reading JSON df = pd.read_json(\u0026#34;api_response.json\u0026#34;) df.to_json(\u0026#34;output.json\u0026#34;, orient=\u0026#34;records\u0026#34;, indent=2) Reading from SQL import sqlite3 conn = sqlite3.connect(\u0026#34;sales.db\u0026#34;) df = pd.read_sql(\u0026#34;SELECT * FROM orders WHERE city = \u0026#39;Mumbai\u0026#39;\u0026#34;, conn) Pro tip: For large CSV files, use nrows first to preview the data before loading everything.\n6. Exploring a DataFrame The first thing to do when you load any dataset is to understand its shape and content. These are your go-to commands.\n# Using a sample employee dataset print(df.shape) # (105, 5) - 105 rows, 5 columns print(df.head()) # First 5 rows print(df.head(10)) # First 10 rows print(df.tail()) # Last 5 rows print(df.sample(5)) # 5 random rows print(df.columns) # Column names print(df.index) # Row index info print(df.dtypes) # Data type of each column print(df.info()) # Summary: dtypes + non-null counts + memory usage print(df.describe()) # Statistical summary of numeric columns print(df.describe(include=\u0026#34;all\u0026#34;)) # Include object/string columns too print(df.nunique()) # Number of unique values per column print(df[\u0026#34;city\u0026#34;].value_counts()) # Frequency count of each city print(df[\u0026#34;city\u0026#34;].unique()) # Distinct city values print(df.isnull().sum()) # Count of missing values per column df.info() is your best friend on a new dataset. It tells you data types AND which columns have nulls - both critical for cleaning.\n7. Selecting Data - loc, iloc, and Column Access Pandas gives you multiple ways to select rows and columns. The two main methods are loc (label-based) and iloc (position-based).\nColumn Selection # Single column → returns a Series df[\u0026#34;name\u0026#34;] # Multiple columns → returns a DataFrame df[[\u0026#34;name\u0026#34;, \u0026#34;city\u0026#34;, \u0026#34;salary\u0026#34;]] Row Selection with loc (label-based) loc uses index labels and is inclusive on both ends.\n# Single row by label df.loc[101] # Row with index label 101 # Range of labels df.loc[101:104] # Rows 101, 102, 103, 104 (all inclusive) # Row + column selection df.loc[101, \u0026#34;name\u0026#34;] # One cell df.loc[101:103, [\u0026#34;name\u0026#34;, \u0026#34;city\u0026#34;]] # Rows 101-103, specific columns # All rows, specific column df.loc[:, \u0026#34;salary\u0026#34;] Row Selection with iloc (position-based) iloc uses integer positions (0-based) and is exclusive on the end, like Python slicing.\ndf.iloc[0] # First row df.iloc[-1] # Last row df.iloc[0:5] # Rows 0 to 4 (position 5 excluded) df.iloc[0, 2] # Row 0, Column 2 (single cell) df.iloc[0:3, 1:4] # Rows 0-2, Columns 1-3 df.iloc[[0, 2, 4]] # Specific rows by position Key Difference loc iloc Rows Index labels (e.g. 101) Integer positions (0, 1, 2...) End point Inclusive Exclusive (like Python slice) Best for Named/custom indexes Default integer indexes 8. Filtering Rows Filtering is how you answer questions like \u0026ldquo;show me all orders from Hyderabad above ₹10,000.\u0026rdquo;\nSingle Condition employees = pd.DataFrame({ \u0026#34;name\u0026#34;: [\u0026#34;Vikram\u0026#34;, \u0026#34;Ananya\u0026#34;, \u0026#34;Suresh\u0026#34;, \u0026#34;Divya\u0026#34;, \u0026#34;Rahul\u0026#34;, \u0026#34;Pooja\u0026#34;, \u0026#34;Amit\u0026#34;], \u0026#34;department\u0026#34;: [\u0026#34;Engineering\u0026#34;, \u0026#34;HR\u0026#34;, \u0026#34;Sales\u0026#34;, \u0026#34;Engineering\u0026#34;, \u0026#34;Marketing\u0026#34;, \u0026#34;HR\u0026#34;, \u0026#34;Sales\u0026#34;], \u0026#34;salary\u0026#34;: [75000, 55000, 48000, 82000, 60000, 52000, 45000], \u0026#34;city\u0026#34;: [\u0026#34;Bangalore\u0026#34;, \u0026#34;Mumbai\u0026#34;, \u0026#34;Delhi\u0026#34;, \u0026#34;Hyderabad\u0026#34;, \u0026#34;Chennai\u0026#34;, \u0026#34;Pune\u0026#34;, \u0026#34;Delhi\u0026#34;], \u0026#34;experience\u0026#34;: [4, 2, 6, 5, 3, 1, 7] }) # Engineers engineers = employees[employees[\u0026#34;department\u0026#34;] == \u0026#34;Engineering\u0026#34;] # Salary above 60k high_earners = employees[employees[\u0026#34;salary\u0026#34;] \u0026gt; 60000] # City is Delhi delhi_team = employees[employees[\u0026#34;city\u0026#34;] == \u0026#34;Delhi\u0026#34;] Multiple Conditions # Use \u0026amp; for AND, | for OR - always wrap each condition in parentheses # Engineers in Bangalore eng_blr = employees[ (employees[\u0026#34;department\u0026#34;] == \u0026#34;Engineering\u0026#34;) \u0026amp; (employees[\u0026#34;city\u0026#34;] == \u0026#34;Bangalore\u0026#34;) ] # Sales OR HR department support = employees[ (employees[\u0026#34;department\u0026#34;] == \u0026#34;Sales\u0026#34;) | (employees[\u0026#34;department\u0026#34;] == \u0026#34;HR\u0026#34;) ] # Experience between 3 and 6 years mid_level = employees[ (employees[\u0026#34;experience\u0026#34;] \u0026gt;= 3) \u0026amp; (employees[\u0026#34;experience\u0026#34;] \u0026lt;= 6) ] isin() - Filter by List of Values # Employees in specific cities metro_employees = employees[employees[\u0026#34;city\u0026#34;].isin([\u0026#34;Mumbai\u0026#34;, \u0026#34;Delhi\u0026#34;, \u0026#34;Bangalore\u0026#34;])] # Exclude a list of departments non_tech = employees[~employees[\u0026#34;department\u0026#34;].isin([\u0026#34;Engineering\u0026#34;, \u0026#34;Data Science\u0026#34;])] between() - Numeric Range Filter mid_salary = employees[employees[\u0026#34;salary\u0026#34;].between(50000, 70000)] query() - SQL-style Filtering # Cleaner syntax for complex filters result = employees.query(\u0026#34;salary \u0026gt; 60000 and city == \u0026#39;Bangalore\u0026#39;\u0026#34;) result = employees.query(\u0026#34;department in [\u0026#39;HR\u0026#39;, \u0026#39;Sales\u0026#39;] and experience \u0026gt;= 3\u0026#34;) 9. Adding, Modifying, and Dropping Columns Adding New Columns # Direct assignment employees[\u0026#34;annual_salary\u0026#34;] = employees[\u0026#34;salary\u0026#34;] * 12 # Conditional column employees[\u0026#34;level\u0026#34;] = np.where(employees[\u0026#34;experience\u0026#34;] \u0026gt;= 5, \u0026#34;Senior\u0026#34;, \u0026#34;Junior\u0026#34;) # Calculated from multiple columns employees[\u0026#34;salary_per_year_exp\u0026#34;] = employees[\u0026#34;salary\u0026#34;] / employees[\u0026#34;experience\u0026#34;] Modifying Existing Columns # Apply a transformation to a whole column employees[\u0026#34;salary\u0026#34;] = employees[\u0026#34;salary\u0026#34;] * 1.10 # 10% hike for everyone # Conditional update using loc employees.loc[employees[\u0026#34;experience\u0026#34;] \u0026gt;= 5, \u0026#34;salary\u0026#34;] *= 1.15 # extra 15% for seniors Renaming Columns employees.rename(columns={ \u0026#34;name\u0026#34;: \u0026#34;employee_name\u0026#34;, \u0026#34;city\u0026#34;: \u0026#34;work_city\u0026#34; }, inplace=True) # Rename all columns at once employees.columns = [\u0026#34;emp_name\u0026#34;, \u0026#34;dept\u0026#34;, \u0026#34;sal\u0026#34;, \u0026#34;city\u0026#34;, \u0026#34;exp\u0026#34;, \u0026#34;annual_sal\u0026#34;, \u0026#34;level\u0026#34;, \u0026#34;sal_per_exp\u0026#34;] Dropping Columns and Rows # Drop column employees.drop(columns=[\u0026#34;salary_per_year_exp\u0026#34;], inplace=True) # Drop multiple columns employees.drop(columns=[\u0026#34;level\u0026#34;, \u0026#34;annual_salary\u0026#34;], inplace=True) # Drop rows by index label employees.drop(index=[2, 4], inplace=True) # Drop rows matching a condition employees = employees[employees[\u0026#34;salary\u0026#34;] \u0026gt;= 40000] # keep only salary \u0026gt;= 40k Reordering Columns # Specify the exact column order you want employees = employees[[\u0026#34;name\u0026#34;, \u0026#34;city\u0026#34;, \u0026#34;department\u0026#34;, \u0026#34;experience\u0026#34;, \u0026#34;salary\u0026#34;]] 10. Sorting Data students = pd.DataFrame({ \u0026#34;name\u0026#34;: [\u0026#34;Aarav\u0026#34;, \u0026#34;Priya\u0026#34;, \u0026#34;Rohan\u0026#34;, \u0026#34;Sneha\u0026#34;, \u0026#34;Karan\u0026#34;, \u0026#34;Meera\u0026#34;], \u0026#34;city\u0026#34;: [\u0026#34;Delhi\u0026#34;, \u0026#34;Mumbai\u0026#34;, \u0026#34;Delhi\u0026#34;, \u0026#34;Hyderabad\u0026#34;, \u0026#34;Mumbai\u0026#34;, \u0026#34;Bangalore\u0026#34;], \u0026#34;marks\u0026#34;: [85, 92, 78, 95, 70, 88], \u0026#34;subject\u0026#34;: [\u0026#34;Maths\u0026#34;, \u0026#34;Science\u0026#34;, \u0026#34;Maths\u0026#34;, \u0026#34;English\u0026#34;, \u0026#34;Science\u0026#34;, \u0026#34;Maths\u0026#34;] }) # Sort by single column (ascending by default) students.sort_values(\u0026#34;marks\u0026#34;) # Sort descending students.sort_values(\u0026#34;marks\u0026#34;, ascending=False) # Sort by multiple columns students.sort_values([\u0026#34;city\u0026#34;, \u0026#34;marks\u0026#34;], ascending=[True, False]) # → Cities sorted A-Z, within each city marks sorted high to low # Sort by index students.sort_index() # Reset index after sorting students = students.sort_values(\u0026#34;marks\u0026#34;, ascending=False).reset_index(drop=True) # drop=True prevents old index from becoming a column 11. Handling Missing Data Real datasets always have missing values. Pandas represents them as NaN (Not a Number) for numeric columns and None/NaN for object columns.\nimport numpy as np df = pd.DataFrame({ \u0026#34;name\u0026#34;: [\u0026#34;Anjali\u0026#34;, \u0026#34;Deepak\u0026#34;, \u0026#34;Ritu\u0026#34;, \u0026#34;Mohan\u0026#34;, \u0026#34;Kavya\u0026#34;], \u0026#34;age\u0026#34;: [25, np.nan, 30, 28, np.nan], \u0026#34;city\u0026#34;: [\u0026#34;Jaipur\u0026#34;, \u0026#34;Delhi\u0026#34;, None, \u0026#34;Mumbai\u0026#34;, \u0026#34;Pune\u0026#34;], \u0026#34;income\u0026#34;: [45000, 60000, np.nan, np.nan, 52000] }) Detecting Missing Values df.isnull() # True/False for each cell df.isnull().sum() # Count of nulls per column df.isnull().sum() / len(df) * 100 # Percentage missing df.notnull() # Inverse - True where NOT null Dropping Missing Values # Drop rows with ANY null value df.dropna() # Drop rows only if ALL values are null df.dropna(how=\u0026#34;all\u0026#34;) # Drop rows only if specific columns have nulls df.dropna(subset=[\u0026#34;income\u0026#34;, \u0026#34;age\u0026#34;]) # Drop columns with more than 50% missing threshold = len(df) * 0.5 df.dropna(thresh=threshold, axis=1) Filling Missing Values # Fill with a fixed value df[\u0026#34;city\u0026#34;].fillna(\u0026#34;Unknown\u0026#34;, inplace=True) # Fill numeric columns with mean/median df[\u0026#34;age\u0026#34;].fillna(df[\u0026#34;age\u0026#34;].mean(), inplace=True) df[\u0026#34;income\u0026#34;].fillna(df[\u0026#34;income\u0026#34;].median(), inplace=True) # Forward fill - use the previous row\u0026#39;s value df[\u0026#34;income\u0026#34;].fillna(method=\u0026#34;ffill\u0026#34;) # Backward fill - use the next row\u0026#39;s value df[\u0026#34;income\u0026#34;].fillna(method=\u0026#34;bfill\u0026#34;) # Fill different columns with different values df.fillna({\u0026#34;age\u0026#34;: df[\u0026#34;age\u0026#34;].mean(), \u0026#34;city\u0026#34;: \u0026#34;Unknown\u0026#34;, \u0026#34;income\u0026#34;: 0}) Interpolation Useful for time series data where filling with mean doesn\u0026rsquo;t make sense.\n# Monthly temperatures in Shimla (some readings missing) temps = pd.Series([15.0, np.nan, 17.5, np.nan, np.nan, 20.0]) temps.interpolate() # Fills gaps linearly between known values # [15.0, 16.25, 17.5, 18.33, 19.17, 20.0] 12. String Operations Pandas provides a .str accessor to apply string methods to an entire Series at once - no loops needed.\ndf = pd.DataFrame({ \u0026#34;name\u0026#34;: [\u0026#34; aarav sharma \u0026#34;, \u0026#34;PRIYA SINGH\u0026#34;, \u0026#34;rohan Mehta\u0026#34;, \u0026#34;sneha PATEL\u0026#34;], \u0026#34;email\u0026#34;: [\u0026#34;aarav@gmail.com\u0026#34;, \u0026#34;priya@yahoo.com\u0026#34;, \u0026#34;rohan@outlook.com\u0026#34;, \u0026#34;sneha@gmail.com\u0026#34;], \u0026#34;city\u0026#34;: [\u0026#34;New Delhi\u0026#34;, \u0026#34;Mumbai\u0026#34;, \u0026#34;Bengaluru\u0026#34;, \u0026#34;Surat\u0026#34;], \u0026#34;pincode\u0026#34;: [\u0026#34;110001\u0026#34;, \u0026#34;400001\u0026#34;, \u0026#34;560001\u0026#34;, \u0026#34;395001\u0026#34;] }) # Case df[\u0026#34;name\u0026#34;].str.upper() df[\u0026#34;name\u0026#34;].str.lower() df[\u0026#34;name\u0026#34;].str.title() df[\u0026#34;name\u0026#34;].str.strip() # Remove leading/trailing spaces # Checking content df[\u0026#34;email\u0026#34;].str.contains(\u0026#34;gmail\u0026#34;) # [True, False, False, True] df[\u0026#34;email\u0026#34;].str.endswith(\u0026#34;.com\u0026#34;) df[\u0026#34;name\u0026#34;].str.startswith(\u0026#34;A\u0026#34;) # Extracting df[\u0026#34;email\u0026#34;].str.split(\u0026#34;@\u0026#34;).str[0] # Get username part df[\u0026#34;email\u0026#34;].str.split(\u0026#34;@\u0026#34;).str[1] # Get domain part df[\u0026#34;name\u0026#34;].str.len() # Length of each string df[\u0026#34;city\u0026#34;].str[:3] # First 3 characters # Replacing df[\u0026#34;name\u0026#34;].str.replace(\u0026#34; \u0026#34;, \u0026#34; \u0026#34;) df[\u0026#34;pincode\u0026#34;].str.replace(\u0026#34;^1\u0026#34;, \u0026#34;2\u0026#34;, regex=True) # Regex replace # Counting and finding df[\u0026#34;name\u0026#34;].str.count(\u0026#34;a\u0026#34;) # Count \u0026#39;a\u0026#39; in each name df[\u0026#34;city\u0026#34;].str.find(\u0026#34;u\u0026#34;) # Position of \u0026#39;u\u0026#39; 13. Working with Dates and Times Date handling is one of Pandas\u0026rsquo; killer features. Once a column is a proper datetime, you unlock a whole set of operations.\n# Sample sales data df = pd.DataFrame({ \u0026#34;order_date\u0026#34;: [\u0026#34;2024-01-15\u0026#34;, \u0026#34;2024-03-22\u0026#34;, \u0026#34;2024-07-04\u0026#34;, \u0026#34;2024-11-12\u0026#34;, \u0026#34;2024-12-25\u0026#34;], \u0026#34;customer\u0026#34;: [\u0026#34;Arjun\u0026#34;, \u0026#34;Meera\u0026#34;, \u0026#34;Sameer\u0026#34;, \u0026#34;Lata\u0026#34;, \u0026#34;Vijay\u0026#34;], \u0026#34;amount\u0026#34;: [1200, 3500, 800, 4200, 2100] }) # Convert string to datetime df[\u0026#34;order_date\u0026#34;] = pd.to_datetime(df[\u0026#34;order_date\u0026#34;]) print(df.dtypes) # order_date → datetime64[ns] Extracting Date Components df[\u0026#34;year\u0026#34;] = df[\u0026#34;order_date\u0026#34;].dt.year df[\u0026#34;month\u0026#34;] = df[\u0026#34;order_date\u0026#34;].dt.month df[\u0026#34;day\u0026#34;] = df[\u0026#34;order_date\u0026#34;].dt.day df[\u0026#34;weekday\u0026#34;] = df[\u0026#34;order_date\u0026#34;].dt.day_name() # \u0026#39;Monday\u0026#39;, \u0026#39;Tuesday\u0026#39;... df[\u0026#34;quarter\u0026#34;] = df[\u0026#34;order_date\u0026#34;].dt.quarter df[\u0026#34;week\u0026#34;] = df[\u0026#34;order_date\u0026#34;].dt.isocalendar().week Date Arithmetic # Time since order df[\u0026#34;days_since_order\u0026#34;] = (pd.Timestamp.today() - df[\u0026#34;order_date\u0026#34;]).dt.days # Add/subtract time df[\u0026#34;delivery_date\u0026#34;] = df[\u0026#34;order_date\u0026#34;] + pd.Timedelta(days=5) df[\u0026#34;return_window\u0026#34;] = df[\u0026#34;order_date\u0026#34;] + pd.DateOffset(months=1) Filtering by Date # Orders after July 2024 df[df[\u0026#34;order_date\u0026#34;] \u0026gt; \u0026#34;2024-07-01\u0026#34;] # Orders in a specific range df[df[\u0026#34;order_date\u0026#34;].between(\u0026#34;2024-03-01\u0026#34;, \u0026#34;2024-09-30\u0026#34;)] # Orders in Q4 df[df[\u0026#34;order_date\u0026#34;].dt.quarter == 4] Resampling Time Series # Monthly sales total (requires datetime index) df.set_index(\u0026#34;order_date\u0026#34;, inplace=True) monthly = df[\u0026#34;amount\u0026#34;].resample(\u0026#34;ME\u0026#34;).sum() # Month-end frequency quarterly = df[\u0026#34;amount\u0026#34;].resample(\u0026#34;QE\u0026#34;).mean() # Available frequencies: \u0026#39;D\u0026#39; (day), \u0026#39;W\u0026#39; (week), \u0026#39;ME\u0026#39; (month-end), # \u0026#39;QE\u0026#39; (quarter-end), \u0026#39;YE\u0026#39; (year-end) 14. apply(), map(), and applymap() These three methods let you apply custom functions to your data in different ways.\nmap() - Element-wise on a Series Apply a function or mapping to each element of a Series.\nstudents = pd.DataFrame({ \u0026#34;name\u0026#34;: [\u0026#34;Aarav\u0026#34;, \u0026#34;Priya\u0026#34;, \u0026#34;Rohan\u0026#34;, \u0026#34;Sneha\u0026#34;, \u0026#34;Karan\u0026#34;], \u0026#34;marks\u0026#34;: [85, 92, 78, 95, 60] }) # Using a dictionary mapping grade_map = {85: \u0026#34;B\u0026#34;, 92: \u0026#34;A+\u0026#34;, 78: \u0026#34;B+\u0026#34;, 95: \u0026#34;A+\u0026#34;, 60: \u0026#34;D\u0026#34;} students[\u0026#34;grade_dict\u0026#34;] = students[\u0026#34;marks\u0026#34;].map(grade_map) # Using a lambda function students[\u0026#34;marks_normalized\u0026#34;] = students[\u0026#34;marks\u0026#34;].map(lambda x: round(x / 100, 2)) apply() - Row-wise or Column-wise on a DataFrame The most powerful and flexible method. Apply a function along rows (axis=1) or columns (axis=0).\n# apply() on a single column (same as map) students[\u0026#34;grade\u0026#34;] = students[\u0026#34;marks\u0026#34;].apply(lambda x: \u0026#34;A+\u0026#34; if x \u0026gt;= 90 else \u0026#34;A\u0026#34; if x \u0026gt;= 80 else \u0026#34;B\u0026#34; if x \u0026gt;= 70 else \u0026#34;C\u0026#34; ) # apply() across rows (axis=1) - access multiple columns at once students[\u0026#34;result_summary\u0026#34;] = students.apply( lambda row: f\u0026#34;{row[\u0026#39;name\u0026#39;]} scored {row[\u0026#39;marks\u0026#39;]} and got {row[\u0026#39;grade\u0026#39;]}\u0026#34;, axis=1 ) # apply() with a named function def classify_performance(marks): if marks \u0026gt;= 90: return \u0026#34;Distinction\u0026#34; elif marks \u0026gt;= 75: return \u0026#34;First Class\u0026#34; elif marks \u0026gt;= 60: return \u0026#34;Second Class\u0026#34; else: return \u0026#34;Pass\u0026#34; students[\u0026#34;performance\u0026#34;] = students[\u0026#34;marks\u0026#34;].apply(classify_performance) # apply() across columns (axis=0) - summarize each column df_scores = pd.DataFrame({ \u0026#34;Maths\u0026#34;: [78, 92, 65, 88], \u0026#34;Science\u0026#34;: [85, 76, 70, 91], \u0026#34;English\u0026#34;: [90, 88, 80, 75] }) # Range (max - min) for each subject df_scores.apply(lambda col: col.max() - col.min(), axis=0) # Total marks for each student df_scores.apply(lambda row: row.sum(), axis=1) DataFrame.map() (formerly applymap) - Element-wise on Entire DataFrame Apply a function to every single cell.\n# Round all values to 1 decimal place df_scores_float = df_scores / 3.0 df_scores_float.map(lambda x: round(x, 1)) Quick guide:\nSeries.map() → every element of one column DataFrame.apply(axis=1) → every row (can use multiple columns) DataFrame.apply(axis=0) → every column (summarize each column) DataFrame.map() → every single cell in the whole DataFrame 15. GroupBy - Aggregation and Split-Apply-Combine GroupBy is one of the most important Pandas features. It splits your data into groups, applies a function to each group, and combines the results.\n# E-commerce order data - a Flipkart-style dataset orders = pd.DataFrame({ \u0026#34;order_id\u0026#34;: [1001, 1002, 1003, 1004, 1005, 1006, 1007, 1008, 1009, 1010], \u0026#34;customer\u0026#34;: [\u0026#34;Aarav\u0026#34;, \u0026#34;Priya\u0026#34;, \u0026#34;Aarav\u0026#34;, \u0026#34;Rohan\u0026#34;, \u0026#34;Priya\u0026#34;, \u0026#34;Sneha\u0026#34;, \u0026#34;Rohan\u0026#34;, \u0026#34;Aarav\u0026#34;, \u0026#34;Sneha\u0026#34;, \u0026#34;Priya\u0026#34;], \u0026#34;city\u0026#34;: [\u0026#34;Delhi\u0026#34;, \u0026#34;Mumbai\u0026#34;, \u0026#34;Delhi\u0026#34;, \u0026#34;Bangalore\u0026#34;, \u0026#34;Mumbai\u0026#34;, \u0026#34;Hyderabad\u0026#34;, \u0026#34;Bangalore\u0026#34;, \u0026#34;Delhi\u0026#34;, \u0026#34;Hyderabad\u0026#34;, \u0026#34;Mumbai\u0026#34;], \u0026#34;category\u0026#34;: [\u0026#34;Electronics\u0026#34;, \u0026#34;Clothing\u0026#34;, \u0026#34;Books\u0026#34;, \u0026#34;Electronics\u0026#34;, \u0026#34;Books\u0026#34;, \u0026#34;Clothing\u0026#34;, \u0026#34;Electronics\u0026#34;, \u0026#34;Books\u0026#34;, \u0026#34;Electronics\u0026#34;, \u0026#34;Clothing\u0026#34;], \u0026#34;amount\u0026#34;: [15000, 2500, 450, 32000, 350, 1800, 28000, 600, 22000, 3200], \u0026#34;status\u0026#34;: [\u0026#34;Delivered\u0026#34;, \u0026#34;Delivered\u0026#34;, \u0026#34;Delivered\u0026#34;, \u0026#34;Pending\u0026#34;, \u0026#34;Delivered\u0026#34;, \u0026#34;Delivered\u0026#34;, \u0026#34;Delivered\u0026#34;, \u0026#34;Cancelled\u0026#34;, \u0026#34;Delivered\u0026#34;, \u0026#34;Delivered\u0026#34;] }) Basic GroupBy # Total amount per customer orders.groupby(\u0026#34;customer\u0026#34;)[\u0026#34;amount\u0026#34;].sum() # Average order value per city orders.groupby(\u0026#34;city\u0026#34;)[\u0026#34;amount\u0026#34;].mean() # Count of orders per category orders.groupby(\u0026#34;category\u0026#34;)[\u0026#34;order_id\u0026#34;].count() Multiple Aggregations with agg() # Multiple stats per category orders.groupby(\u0026#34;category\u0026#34;)[\u0026#34;amount\u0026#34;].agg([\u0026#34;sum\u0026#34;, \u0026#34;mean\u0026#34;, \u0026#34;count\u0026#34;, \u0026#34;max\u0026#34;, \u0026#34;min\u0026#34;]) # Custom names for aggregations orders.groupby(\u0026#34;category\u0026#34;)[\u0026#34;amount\u0026#34;].agg( total_revenue=(\u0026#34;sum\u0026#34;), avg_order=(\u0026#34;mean\u0026#34;), order_count=(\u0026#34;count\u0026#34;) ) # Different aggregations for different columns orders.groupby(\u0026#34;city\u0026#34;).agg({ \u0026#34;amount\u0026#34;: [\u0026#34;sum\u0026#34;, \u0026#34;mean\u0026#34;], \u0026#34;order_id\u0026#34;: \u0026#34;count\u0026#34; }) GroupBy on Multiple Columns # Revenue by city AND category orders.groupby([\u0026#34;city\u0026#34;, \u0026#34;category\u0026#34;])[\u0026#34;amount\u0026#34;].sum() # Orders by city and status orders.groupby([\u0026#34;city\u0026#34;, \u0026#34;status\u0026#34;])[\u0026#34;order_id\u0026#34;].count() transform() - GroupBy Without Reducing transform applies a function to each group but returns a result the same size as the original DataFrame - very useful for adding group-level stats back to the original rows.\n# Add \u0026#34;city_avg_order\u0026#34; as a new column (same value for all rows in the same city) orders[\u0026#34;city_avg_order\u0026#34;] = orders.groupby(\u0026#34;city\u0026#34;)[\u0026#34;amount\u0026#34;].transform(\u0026#34;mean\u0026#34;) # Flag orders that are above their city\u0026#39;s average orders[\u0026#34;above_city_avg\u0026#34;] = orders[\u0026#34;amount\u0026#34;] \u0026gt; orders[\u0026#34;city_avg_order\u0026#34;] filter() - Keep Only Groups That Meet a Condition # Keep only cities where total revenue \u0026gt; 30000 high_rev_cities = orders.groupby(\u0026#34;city\u0026#34;).filter(lambda g: g[\u0026#34;amount\u0026#34;].sum() \u0026gt; 30000) 16. Merging and Joining DataFrames Combining data from multiple tables is a daily task in data analysis. Pandas merge() works like SQL JOINs.\n# Customer details customers = pd.DataFrame({ \u0026#34;cust_id\u0026#34;: [1, 2, 3, 4, 5], \u0026#34;name\u0026#34;: [\u0026#34;Aarav\u0026#34;, \u0026#34;Priya\u0026#34;, \u0026#34;Rohan\u0026#34;, \u0026#34;Sneha\u0026#34;, \u0026#34;Karan\u0026#34;], \u0026#34;city\u0026#34;: [\u0026#34;Delhi\u0026#34;, \u0026#34;Mumbai\u0026#34;, \u0026#34;Bangalore\u0026#34;, \u0026#34;Hyderabad\u0026#34;, \u0026#34;Pune\u0026#34;], \u0026#34;tier\u0026#34;: [\u0026#34;Gold\u0026#34;, \u0026#34;Silver\u0026#34;, \u0026#34;Gold\u0026#34;, \u0026#34;Platinum\u0026#34;, \u0026#34;Silver\u0026#34;] }) # Orders placed orders = pd.DataFrame({ \u0026#34;order_id\u0026#34;: [101, 102, 103, 104, 105, 106], \u0026#34;cust_id\u0026#34;: [1, 2, 1, 3, 6, 4], # cust_id 6 doesn\u0026#39;t exist in customers \u0026#34;product\u0026#34;: [\u0026#34;Laptop\u0026#34;, \u0026#34;Phone\u0026#34;, \u0026#34;Tablet\u0026#34;, \u0026#34;TV\u0026#34;, \u0026#34;Watch\u0026#34;, \u0026#34;Headphones\u0026#34;], \u0026#34;amount\u0026#34;: [65000, 18000, 32000, 45000, 9500, 4200] }) Inner Join (default) - Only matching rows merged = pd.merge(orders, customers, on=\u0026#34;cust_id\u0026#34;) # Only orders where cust_id exists in both tables (cust_id 6 dropped) Left Join - All rows from left, match from right merged = pd.merge(orders, customers, on=\u0026#34;cust_id\u0026#34;, how=\u0026#34;left\u0026#34;) # All 6 orders kept; cust_id=6 gets NaN for customer columns Right Join - All rows from right, match from left merged = pd.merge(orders, customers, on=\u0026#34;cust_id\u0026#34;, how=\u0026#34;right\u0026#34;) # All 5 customers kept; Karan (no orders) gets NaN for order columns Outer Join - All rows from both merged = pd.merge(orders, customers, on=\u0026#34;cust_id\u0026#34;, how=\u0026#34;outer\u0026#34;) # All customers + all orders; NaN where no match Merging on Different Column Names # If the join key has different names in each table pd.merge(orders, customers, left_on=\u0026#34;cust_id\u0026#34;, right_on=\u0026#34;customer_id\u0026#34;) concat() - Stack DataFrames Use concat when tables have the same structure (same columns) and you want to stack them.\n# Q1 and Q2 sales data q1_sales = pd.DataFrame({\u0026#34;month\u0026#34;: [\u0026#34;Jan\u0026#34;,\u0026#34;Feb\u0026#34;,\u0026#34;Mar\u0026#34;], \u0026#34;revenue\u0026#34;: [120, 135, 118]}) q2_sales = pd.DataFrame({\u0026#34;month\u0026#34;: [\u0026#34;Apr\u0026#34;,\u0026#34;May\u0026#34;,\u0026#34;Jun\u0026#34;], \u0026#34;revenue\u0026#34;: [145, 160, 152]}) # Stack vertically (add more rows) full_h1 = pd.concat([q1_sales, q2_sales], ignore_index=True) # Stack horizontally (add more columns) pd.concat([df1, df2], axis=1) Join Types Summary INNER → Only rows with a match in BOTH tables LEFT → All rows from the LEFT table; NaN if no match on right RIGHT → All rows from the RIGHT table; NaN if no match on left OUTER → All rows from BOTH tables; NaN where no match 17. Pivot Tables and Crosstabs pivot_table() - Spreadsheet-style Summarization If you\u0026rsquo;ve used pivot tables in Excel, this is the same idea - but in code.\nipl = pd.DataFrame({ \u0026#34;player\u0026#34;: [\u0026#34;Kohli\u0026#34;, \u0026#34;Rohit\u0026#34;, \u0026#34;Dhoni\u0026#34;, \u0026#34;Kohli\u0026#34;, \u0026#34;Rohit\u0026#34;, \u0026#34;Bumrah\u0026#34;, \u0026#34;Dhoni\u0026#34;, \u0026#34;Kohli\u0026#34;], \u0026#34;team\u0026#34;: [\u0026#34;RCB\u0026#34;, \u0026#34;MI\u0026#34;, \u0026#34;CSK\u0026#34;, \u0026#34;RCB\u0026#34;, \u0026#34;MI\u0026#34;, \u0026#34;MI\u0026#34;, \u0026#34;CSK\u0026#34;, \u0026#34;RCB\u0026#34;], \u0026#34;season\u0026#34;: [2022, 2022, 2022, 2023, 2023, 2023, 2023, 2023], \u0026#34;runs\u0026#34;: [405, 380, 220, 639, 442, 5, 250, 741], \u0026#34;matches\u0026#34;: [16, 14, 14, 17, 16, 14, 15, 17] }) # Pivot: Average runs per player per season pivot = ipl.pivot_table( values=\u0026#34;runs\u0026#34;, index=\u0026#34;player\u0026#34;, columns=\u0026#34;season\u0026#34;, aggfunc=\u0026#34;sum\u0026#34;, fill_value=0 # Replace NaN with 0 ) print(pivot) # Multiple values and aggregation functions ipl.pivot_table( values=[\u0026#34;runs\u0026#34;, \u0026#34;matches\u0026#34;], index=\u0026#34;team\u0026#34;, aggfunc={\u0026#34;runs\u0026#34;: \u0026#34;sum\u0026#34;, \u0026#34;matches\u0026#34;: \u0026#34;mean\u0026#34;} ) # Add row/column totals with margins ipl.pivot_table(values=\u0026#34;runs\u0026#34;, index=\u0026#34;team\u0026#34;, columns=\u0026#34;season\u0026#34;, aggfunc=\u0026#34;sum\u0026#34;, margins=True, margins_name=\u0026#34;Total\u0026#34;) crosstab() - Frequency Tables Crosstab is a shortcut for counting combinations of categorical variables.\nemployees = pd.DataFrame({ \u0026#34;department\u0026#34;: [\u0026#34;Engineering\u0026#34;, \u0026#34;HR\u0026#34;, \u0026#34;Sales\u0026#34;, \u0026#34;Engineering\u0026#34;, \u0026#34;HR\u0026#34;, \u0026#34;Sales\u0026#34;, \u0026#34;Engineering\u0026#34;], \u0026#34;city\u0026#34;: [\u0026#34;Bangalore\u0026#34;, \u0026#34;Mumbai\u0026#34;, \u0026#34;Delhi\u0026#34;, \u0026#34;Hyderabad\u0026#34;, \u0026#34;Delhi\u0026#34;, \u0026#34;Mumbai\u0026#34;, \u0026#34;Bangalore\u0026#34;], \u0026#34;level\u0026#34;: [\u0026#34;Senior\u0026#34;, \u0026#34;Junior\u0026#34;, \u0026#34;Senior\u0026#34;, \u0026#34;Junior\u0026#34;, \u0026#34;Senior\u0026#34;, \u0026#34;Junior\u0026#34;, \u0026#34;Senior\u0026#34;] }) # Count of employees per department per city pd.crosstab(employees[\u0026#34;department\u0026#34;], employees[\u0026#34;city\u0026#34;]) # With percentages pd.crosstab(employees[\u0026#34;department\u0026#34;], employees[\u0026#34;city\u0026#34;], normalize=\u0026#34;index\u0026#34;) # Row % pd.crosstab(employees[\u0026#34;department\u0026#34;], employees[\u0026#34;city\u0026#34;], normalize=\u0026#34;columns\u0026#34;) # Column % # With a values column (not just counts) pd.crosstab(employees[\u0026#34;department\u0026#34;], employees[\u0026#34;level\u0026#34;], values=np.random.randint(40000, 90000, 7), aggfunc=\u0026#34;mean\u0026#34;) 18. Reshaping - melt() and stack()/unstack() Sometimes your data is in the wrong shape for analysis. These tools reshape it.\nmelt() - Wide to Long Format \u0026ldquo;Wide\u0026rdquo; data has one column per category. \u0026ldquo;Long\u0026rdquo; data has a category column and a value column. Long format is required by most plotting libraries.\n# Wide format: one column per subject wide = pd.DataFrame({ \u0026#34;student\u0026#34;: [\u0026#34;Aarav\u0026#34;, \u0026#34;Priya\u0026#34;, \u0026#34;Rohan\u0026#34;], \u0026#34;Maths\u0026#34;: [78, 92, 65], \u0026#34;Science\u0026#34;: [85, 76, 70], \u0026#34;English\u0026#34;: [90, 88, 80] }) # Melt to long format long = pd.melt( wide, id_vars=[\u0026#34;student\u0026#34;], # columns to keep as-is value_vars=[\u0026#34;Maths\u0026#34;, \u0026#34;Science\u0026#34;, \u0026#34;English\u0026#34;], # columns to unpivot var_name=\u0026#34;subject\u0026#34;, # name for the new category column value_name=\u0026#34;marks\u0026#34; # name for the new value column ) print(long) # student subject marks # 0 Aarav Maths 78 # 1 Priya Maths 92 # 2 Rohan Maths 65 # 3 Aarav Science 85 # ... and so on pivot() - Long to Wide Format (inverse of melt) # Convert long back to wide wide_again = long.pivot(index=\u0026#34;student\u0026#34;, columns=\u0026#34;subject\u0026#34;, values=\u0026#34;marks\u0026#34;) wide_again.columns.name = None # Clean up the column axis name wide_again.reset_index(inplace=True) stack() and unstack() # stack() moves column labels → row labels (makes DataFrame taller) # unstack() moves row labels → column labels (makes DataFrame wider) df = pd.DataFrame({ \u0026#34;Q1\u0026#34;: [100, 200], \u0026#34;Q2\u0026#34;: [150, 250] }, index=[\u0026#34;Delhi\u0026#34;, \u0026#34;Mumbai\u0026#34;]) stacked = df.stack() # Delhi Q1 100 # Q2 150 # Mumbai Q1 200 # Q2 250 unstacked = stacked.unstack() # back to original shape 19. MultiIndex DataFrames Sometimes one level of indexing isn\u0026rsquo;t enough. MultiIndex lets you have hierarchical row or column labels.\n# Create MultiIndex from tuples index = pd.MultiIndex.from_tuples([ (\u0026#34;Delhi\u0026#34;, \u0026#34;Q1\u0026#34;), (\u0026#34;Delhi\u0026#34;, \u0026#34;Q2\u0026#34;), (\u0026#34;Mumbai\u0026#34;, \u0026#34;Q1\u0026#34;), (\u0026#34;Mumbai\u0026#34;, \u0026#34;Q2\u0026#34;), (\u0026#34;Chennai\u0026#34;, \u0026#34;Q1\u0026#34;), (\u0026#34;Chennai\u0026#34;, \u0026#34;Q2\u0026#34;), ], names=[\u0026#34;City\u0026#34;, \u0026#34;Quarter\u0026#34;]) revenue = pd.Series([120, 145, 180, 210, 95, 115], index=index) print(revenue) # Selecting with MultiIndex revenue[\u0026#34;Delhi\u0026#34;] # All Delhi quarters revenue[\u0026#34;Delhi\u0026#34;, \u0026#34;Q1\u0026#34;] # Specific cell revenue.loc[\u0026#34;Mumbai\u0026#34;:\u0026#34;Chennai\u0026#34;] # Range of cities MultiIndex on DataFrames arrays = [ [\u0026#34;North\u0026#34;, \u0026#34;North\u0026#34;, \u0026#34;South\u0026#34;, \u0026#34;South\u0026#34;], [\u0026#34;Delhi\u0026#34;, \u0026#34;Chandigarh\u0026#34;, \u0026#34;Chennai\u0026#34;, \u0026#34;Bangalore\u0026#34;] ] multi_idx = pd.MultiIndex.from_arrays(arrays, names=[\u0026#34;Zone\u0026#34;, \u0026#34;City\u0026#34;]) df_multi = pd.DataFrame({ \u0026#34;Sales\u0026#34;: [450, 120, 380, 290], \u0026#34;Expenses\u0026#34;: [200, 80, 170, 140] }, index=multi_idx) # Select entire zone df_multi.loc[\u0026#34;North\u0026#34;] # Select specific city df_multi.loc[(\u0026#34;South\u0026#34;, \u0026#34;Chennai\u0026#34;)] # Cross-section df_multi.xs(\u0026#34;Delhi\u0026#34;, level=\u0026#34;City\u0026#34;) Flattening MultiIndex # After groupby with multiple aggregations, flatten column MultiIndex df.columns = [\u0026#34;_\u0026#34;.join(col).strip() for col in df.columns.values] 20. Performance Tips As your datasets grow, these habits will save you real time.\nUse Appropriate Data Types # Check memory usage df.info(memory_usage=\u0026#34;deep\u0026#34;) df.memory_usage(deep=True) # Downcast integers df[\u0026#34;age\u0026#34;] = pd.to_numeric(df[\u0026#34;age\u0026#34;], downcast=\u0026#34;integer\u0026#34;) # int64 → int8 if possible # Convert low-cardinality string columns to \u0026#39;category\u0026#39; # (e.g. a \u0026#39;city\u0026#39; column with 10 unique cities repeated 1 million times) df[\u0026#34;city\u0026#34;] = df[\u0026#34;city\u0026#34;].astype(\u0026#34;category\u0026#34;) df[\u0026#34;department\u0026#34;] = df[\u0026#34;department\u0026#34;].astype(\u0026#34;category\u0026#34;) # Can cut memory by 10-50x for string columns! Avoid Loops - Use Vectorized Operations # SLOW - Python loop for i in range(len(df)): df.loc[i, \u0026#34;tax\u0026#34;] = df.loc[i, \u0026#34;salary\u0026#34;] * 0.30 # FAST - vectorized df[\u0026#34;tax\u0026#34;] = df[\u0026#34;salary\u0026#34;] * 0.30 Use query() for Readable Filters # Avoids creating temporary boolean Series result = df.query(\u0026#34;salary \u0026gt; 60000 and city == \u0026#39;Bangalore\u0026#39; and experience \u0026gt;= 3\u0026#34;) Read Only What You Need # Only load required columns df = pd.read_csv(\u0026#34;large_file.csv\u0026#34;, usecols=[\u0026#34;name\u0026#34;, \u0026#34;city\u0026#34;, \u0026#34;amount\u0026#34;]) # Load in chunks for very large files for chunk in pd.read_csv(\u0026#34;huge_file.csv\u0026#34;, chunksize=10000): process(chunk) copy() to Avoid SettingWithCopyWarning # When slicing a DataFrame and modifying it, always copy first subset = df[df[\u0026#34;city\u0026#34;] == \u0026#34;Delhi\u0026#34;].copy() subset[\u0026#34;new_col\u0026#34;] = \u0026#34;value\u0026#34; # Safe - no warning 21. Practice Questions Work through these on your own. They cover the full range of what you\u0026rsquo;ve learned.\nQ1 - DataFrame Creation and Exploration Create a DataFrame for 8 students with columns: name, city, marks_theory, marks_practical. Add a total column and display basic statistics using describe().\nQ2 - loc and iloc Using the DataFrame below, extract:\nRows where emp_id is between 102 and 104 using loc The last 3 rows using iloc The salary and city columns for the first 2 rows df = pd.DataFrame({ \u0026#34;emp_id\u0026#34;: [101, 102, 103, 104, 105], \u0026#34;name\u0026#34;: [\u0026#34;Vikram\u0026#34;, \u0026#34;Ananya\u0026#34;, \u0026#34;Suresh\u0026#34;, \u0026#34;Divya\u0026#34;, \u0026#34;Rahul\u0026#34;], \u0026#34;salary\u0026#34;: [75000, 55000, 48000, 82000, 60000], \u0026#34;city\u0026#34;: [\u0026#34;Bangalore\u0026#34;, \u0026#34;Mumbai\u0026#34;, \u0026#34;Delhi\u0026#34;, \u0026#34;Hyderabad\u0026#34;, \u0026#34;Chennai\u0026#34;] }).set_index(\u0026#34;emp_id\u0026#34;) Q3 - Filtering Using the Zomato-style dataset below, find:\nAll restaurants in Koramangala with rating above 4.0 Restaurants that serve \u0026ldquo;North Indian\u0026rdquo; OR \u0026ldquo;Chinese\u0026rdquo; cuisine Restaurants with votes between 500 and 2000 zomato = pd.DataFrame({ \u0026#34;name\u0026#34;: [\u0026#34;The Fatty Bao\u0026#34;, \u0026#34;Truffles\u0026#34;, \u0026#34;Meghana Foods\u0026#34;, \u0026#34;Smoke House Deli\u0026#34;, \u0026#34;Vidyarthi Bhavan\u0026#34;], \u0026#34;location\u0026#34;: [\u0026#34;Koramangala\u0026#34;, \u0026#34;Koramangala\u0026#34;, \u0026#34;Koramangala\u0026#34;, \u0026#34;Indiranagar\u0026#34;, \u0026#34;Gandhi Bazaar\u0026#34;], \u0026#34;cuisine\u0026#34;: [\u0026#34;Asian\u0026#34;, \u0026#34;American\u0026#34;, \u0026#34;North Indian\u0026#34;, \u0026#34;American\u0026#34;, \u0026#34;South Indian\u0026#34;], \u0026#34;rating\u0026#34;: [4.1, 4.4, 4.2, 4.0, 4.5], \u0026#34;votes\u0026#34;: [4200, 9800, 7600, 3200, 12000], \u0026#34;cost_for2\u0026#34;:[800, 600, 500, 1100, 150] }) Q4 - Missing Data Load the dataset below and:\nCount nulls in each column Fill age with the median, city with \u0026ldquo;Unknown\u0026rdquo;, salary with column mean Drop rows where both age and salary are null df = pd.DataFrame({ \u0026#34;name\u0026#34;: [\u0026#34;Arjun\u0026#34;, \u0026#34;Bhavna\u0026#34;, \u0026#34;Chirag\u0026#34;, \u0026#34;Divya\u0026#34;, \u0026#34;Esha\u0026#34;], \u0026#34;age\u0026#34;: [28, None, 32, None, 25], \u0026#34;city\u0026#34;: [\u0026#34;Pune\u0026#34;, None, \u0026#34;Delhi\u0026#34;, \u0026#34;Surat\u0026#34;, None], \u0026#34;salary\u0026#34;: [50000, 65000, None, None, 48000] }) Q5 - String Operations Given the dataset below:\nStandardize name to title case and strip whitespace Extract the domain from each email (e.g. \u0026ldquo;gmail.com\u0026rdquo;) Find all customers from Gmail df = pd.DataFrame({ \u0026#34;name\u0026#34;: [\u0026#34; arjun SHARMA\u0026#34;, \u0026#34;PRIYA patel \u0026#34;, \u0026#34;rohan VERMA\u0026#34;], \u0026#34;email\u0026#34;: [\u0026#34;arjun@gmail.com\u0026#34;, \u0026#34;priya@yahoo.in\u0026#34;, \u0026#34;rohan@gmail.com\u0026#34;] }) Q6 - Date Operations Given a dataset of Diwali sale orders:\nConvert order_date to datetime Extract month name and day of week Find all orders placed in October Calculate how many days ago each order was placed df = pd.DataFrame({ \u0026#34;order_date\u0026#34;: [\u0026#34;2024-10-15\u0026#34;, \u0026#34;2024-10-22\u0026#34;, \u0026#34;2024-11-01\u0026#34;, \u0026#34;2024-10-28\u0026#34;, \u0026#34;2024-11-05\u0026#34;], \u0026#34;customer\u0026#34;: [\u0026#34;Ravi\u0026#34;, \u0026#34;Sita\u0026#34;, \u0026#34;Arjun\u0026#34;, \u0026#34;Deepa\u0026#34;, \u0026#34;Mohan\u0026#34;], \u0026#34;amount\u0026#34;: [5000, 3200, 8900, 4100, 2200] }) Q7 - apply() and map() Using the employee dataset:\nUse apply() to create an experience_band column: \u0026ldquo;Fresher\u0026rdquo; (\u0026lt;2yr), \u0026ldquo;Mid-level\u0026rdquo; (2–5yr), \u0026ldquo;Senior\u0026rdquo; (\u0026gt;5yr) Use map() to create a city_tier column (Delhi/Mumbai/Bangalore = Tier 1, others = Tier 2) employees = pd.DataFrame({ \u0026#34;name\u0026#34;: [\u0026#34;Aarav\u0026#34;, \u0026#34;Priya\u0026#34;, \u0026#34;Rohan\u0026#34;, \u0026#34;Sneha\u0026#34;, \u0026#34;Karan\u0026#34;], \u0026#34;city\u0026#34;: [\u0026#34;Delhi\u0026#34;, \u0026#34;Pune\u0026#34;, \u0026#34;Mumbai\u0026#34;, \u0026#34;Jaipur\u0026#34;, \u0026#34;Bangalore\u0026#34;], \u0026#34;experience\u0026#34;: [1, 3, 7, 4, 2] }) Q8 - GroupBy Using the IPL dataset:\nFind total runs per team Find average strike rate per player Find the team with the most boundaries (fours + sixes) Use transform to add a column showing each player\u0026rsquo;s team\u0026rsquo;s total runs alongside each row ipl = pd.DataFrame({ \u0026#34;player\u0026#34;: [\u0026#34;Rohit\u0026#34;, \u0026#34;Kohli\u0026#34;, \u0026#34;Dhoni\u0026#34;, \u0026#34;Jadeja\u0026#34;, \u0026#34;Bumrah\u0026#34;, \u0026#34;Surya\u0026#34;, \u0026#34;Shubman\u0026#34;], \u0026#34;team\u0026#34;: [\u0026#34;MI\u0026#34;, \u0026#34;RCB\u0026#34;, \u0026#34;CSK\u0026#34;, \u0026#34;CSK\u0026#34;, \u0026#34;MI\u0026#34;, \u0026#34;MI\u0026#34;, \u0026#34;GT\u0026#34;], \u0026#34;runs\u0026#34;: [440, 639, 210, 180, 15, 380, 520], \u0026#34;fours\u0026#34;: [42, 55, 20, 18, 1, 35, 48], \u0026#34;sixes\u0026#34;: [20, 30, 10, 8, 0, 22, 25], \u0026#34;sr\u0026#34;: [148, 145, 125, 120, 100, 165, 140] }) Q9 - Merging You have two tables: students and exam_results. Merge them to:\nShow all students even if they didn\u0026rsquo;t appear in the exam Show only students who appeared in the exam Find students with no exam record students = pd.DataFrame({ \u0026#34;student_id\u0026#34;: [1, 2, 3, 4, 5], \u0026#34;name\u0026#34;: [\u0026#34;Aarav\u0026#34;, \u0026#34;Priya\u0026#34;, \u0026#34;Rohan\u0026#34;, \u0026#34;Sneha\u0026#34;, \u0026#34;Karan\u0026#34;], \u0026#34;class\u0026#34;: [\u0026#34;10A\u0026#34;, \u0026#34;10B\u0026#34;, \u0026#34;10A\u0026#34;, \u0026#34;10B\u0026#34;, \u0026#34;10A\u0026#34;] }) exam_results = pd.DataFrame({ \u0026#34;student_id\u0026#34;: [1, 2, 3, 5], \u0026#34;score\u0026#34;: [88, 91, 74, 65], \u0026#34;grade\u0026#34;: [\u0026#34;A\u0026#34;, \u0026#34;A+\u0026#34;, \u0026#34;B\u0026#34;, \u0026#34;C\u0026#34;] }) Q10 - Pivot Table Using the sales data below, create a pivot table showing total revenue for each product across each city. Add row and column totals.\nsales = pd.DataFrame({ \u0026#34;city\u0026#34;: [\u0026#34;Delhi\u0026#34;,\u0026#34;Delhi\u0026#34;,\u0026#34;Mumbai\u0026#34;,\u0026#34;Mumbai\u0026#34;,\u0026#34;Bangalore\u0026#34;,\u0026#34;Bangalore\u0026#34;,\u0026#34;Delhi\u0026#34;,\u0026#34;Mumbai\u0026#34;], \u0026#34;product\u0026#34;: [\u0026#34;Laptop\u0026#34;,\u0026#34;Phone\u0026#34;,\u0026#34;Laptop\u0026#34;,\u0026#34;Tablet\u0026#34;,\u0026#34;Phone\u0026#34;,\u0026#34;Laptop\u0026#34;,\u0026#34;Tablet\u0026#34;,\u0026#34;Phone\u0026#34;], \u0026#34;revenue\u0026#34;: [65000, 18000, 62000, 32000, 16000, 70000, 28000, 20000] }) Q11 - melt() and Reshape The table below is in wide format. Melt it into long format with columns student, subject, and marks. Then find the subject-wise average.\nwide = pd.DataFrame({ \u0026#34;student\u0026#34;: [\u0026#34;Aarav\u0026#34;, \u0026#34;Priya\u0026#34;, \u0026#34;Rohan\u0026#34;, \u0026#34;Sneha\u0026#34;], \u0026#34;Maths\u0026#34;: [78, 92, 65, 88], \u0026#34;Science\u0026#34;: [85, 76, 70, 91], \u0026#34;English\u0026#34;: [90, 88, 80, 85], \u0026#34;History\u0026#34;: [72, 80, 68, 79] }) Q12 - Performance Optimization Take the dataset below (or create a similar one with 100,000 rows using pd.concat) and:\nCheck memory usage before and after Convert city and department to category dtype Rewrite the salary filter (salary \u0026gt; 60000) using query() instead of boolean indexing import numpy as np n = 100000 df = pd.DataFrame({ \u0026#34;name\u0026#34;: [f\u0026#34;emp_{i}\u0026#34; for i in range(n)], \u0026#34;department\u0026#34;: np.random.choice([\u0026#34;Engineering\u0026#34;, \u0026#34;HR\u0026#34;, \u0026#34;Sales\u0026#34;, \u0026#34;Marketing\u0026#34;], n), \u0026#34;city\u0026#34;: np.random.choice([\u0026#34;Delhi\u0026#34;, \u0026#34;Mumbai\u0026#34;, \u0026#34;Bangalore\u0026#34;, \u0026#34;Hyderabad\u0026#34;, \u0026#34;Pune\u0026#34;], n), \u0026#34;salary\u0026#34;: np.random.randint(30000, 150000, n), \u0026#34;experience\u0026#34;: np.random.randint(0, 20, n) }) Q13 - Full Analysis Pipeline You are given sales data from a fictional Indian retail chain. Answer the following:\nWhich city has the highest average order value? What percentage of orders are \u0026ldquo;Delivered\u0026rdquo; vs \u0026ldquo;Cancelled\u0026rdquo;? Which product category generates the most revenue? Add a column revenue_tier: \u0026ldquo;High\u0026rdquo; (≥10000), \u0026ldquo;Medium\u0026rdquo; (3000–9999), \u0026ldquo;Low\u0026rdquo; (\u0026lt;3000) Find customers who have placed more than 2 orders Which month had peak revenue? np.random.seed(42) n = 200 sales = pd.DataFrame({ \u0026#34;order_id\u0026#34;: range(1001, 1001+n), \u0026#34;order_date\u0026#34;: pd.date_range(\u0026#34;2024-01-01\u0026#34;, periods=n, freq=\u0026#34;2D\u0026#34;), \u0026#34;customer\u0026#34;: np.random.choice([\u0026#34;Aarav\u0026#34;,\u0026#34;Priya\u0026#34;,\u0026#34;Rohan\u0026#34;,\u0026#34;Sneha\u0026#34;,\u0026#34;Karan\u0026#34;,\u0026#34;Meera\u0026#34;,\u0026#34;Arjun\u0026#34;,\u0026#34;Divya\u0026#34;], n), \u0026#34;city\u0026#34;: np.random.choice([\u0026#34;Delhi\u0026#34;,\u0026#34;Mumbai\u0026#34;,\u0026#34;Bangalore\u0026#34;,\u0026#34;Hyderabad\u0026#34;,\u0026#34;Pune\u0026#34;,\u0026#34;Chennai\u0026#34;], n), \u0026#34;category\u0026#34;: np.random.choice([\u0026#34;Electronics\u0026#34;,\u0026#34;Clothing\u0026#34;,\u0026#34;Books\u0026#34;,\u0026#34;Home\u0026#34;,\u0026#34;Beauty\u0026#34;], n), \u0026#34;amount\u0026#34;: np.random.randint(200, 50000, n), \u0026#34;status\u0026#34;: np.random.choice([\u0026#34;Delivered\u0026#34;,\u0026#34;Pending\u0026#34;,\u0026#34;Cancelled\u0026#34;], n, p=[0.75, 0.15, 0.10]) }) 22. What We Covered + What\u0026rsquo;s Next ✅ What This Masterclass Covered Section Topics Core Structures Series, DataFrame, index, dtypes Creating DataFrames From dict, list, NumPy, custom index Reading \u0026amp; Writing CSV, Excel, JSON, SQL Exploration head, info, describe, value_counts, isnull Selection Column access, loc, iloc Filtering Boolean masks, isin, between, query Column Operations Add, modify, rename, drop, reorder Sorting sort_values, sort_index, reset_index Missing Data isnull, dropna, fillna, interpolate Strings .str accessor - case, contains, split, replace Dates \u0026amp; Times to_datetime, dt accessors, resample apply / map map, apply (row + column), DataFrame.map GroupBy groupby, agg, transform, filter Merging merge (inner/left/right/outer), concat Pivoting pivot_table, crosstab Reshaping melt, pivot, stack, unstack MultiIndex Hierarchical indexing, xs Performance dtypes, category, vectorization, chunked reading 🚀 What to Explore Next 1. Matplotlib \u0026amp; Seaborn Visualize your Pandas DataFrames. Line charts, bar plots, histograms, heatmaps, pair plots. Data without visuals is half the story.\n2. Plotly / Plotly Express Interactive charts in the browser. Drag, zoom, hover - much better for dashboards and presentations.\n3. Real Datasets Go to Kaggle.com and pick a dataset you care about. IPL, Zomato Bangalore, Indian census, air quality - all available free. Real data will teach you more than any masterclass.\n4. Pandas + SQL pd.read_sql() and df.to_sql() let Pandas talk directly to databases. Combined with SQLAlchemy, you can build full data pipelines.\n5. Scikit-learn Once your data is clean and shaped with Pandas, Scikit-learn turns it into machine learning models. Feature engineering with Pandas + model training with Scikit-learn is a standard workflow.\n6. Polars A newer, faster alternative to Pandas written in Rust. If you ever need to process datasets with tens of millions of rows, Polars is worth learning - and the API is similar.\n7. dask Pandas for data that doesn\u0026rsquo;t fit in RAM. If you need to process a 50GB CSV, dask wraps Pandas in a parallel, lazy computation model.\nFinal thought: Pandas is where most data stories actually get told. The code patterns you\u0026rsquo;ve learned here - groupby, merge, pivot, filter - are the same ones used at every analytics team in the country.\nThe best next step is simple: open a real Indian dataset (IPL, Zomato, NIFTY 50, census) and start asking questions. The answers are in the data - Pandas is just the language to find them. 🇮🇳\n23. Practice Question Solutions Q1 \u0026ndash; DataFrame Creation and Exploration import pandas as pd df = pd.DataFrame({ \u0026#34;name\u0026#34;: [\u0026#34;Aarav\u0026#34;, \u0026#34;Priya\u0026#34;, \u0026#34;Rohan\u0026#34;, \u0026#34;Sneha\u0026#34;, \u0026#34;Karan\u0026#34;, \u0026#34;Meera\u0026#34;, \u0026#34;Arjun\u0026#34;, \u0026#34;Divya\u0026#34;], \u0026#34;city\u0026#34;: [\u0026#34;Delhi\u0026#34;, \u0026#34;Mumbai\u0026#34;, \u0026#34;Bangalore\u0026#34;, \u0026#34;Hyderabad\u0026#34;, \u0026#34;Pune\u0026#34;, \u0026#34;Chennai\u0026#34;, \u0026#34;Kolkata\u0026#34;, \u0026#34;Jaipur\u0026#34;], \u0026#34;marks_theory\u0026#34;: [78, 92, 65, 88, 55, 81, 70, 95], \u0026#34;marks_practical\u0026#34;: [82, 88, 70, 91, 60, 79, 74, 90], }) df[\u0026#34;total\u0026#34;] = df[\u0026#34;marks_theory\u0026#34;] + df[\u0026#34;marks_practical\u0026#34;] print(df) print(\u0026#34;\\n\u0026#34;) print(df.describe()) Q2 \u0026ndash; loc and iloc df = pd.DataFrame({ \u0026#34;emp_id\u0026#34;: [101, 102, 103, 104, 105], \u0026#34;name\u0026#34;: [\u0026#34;Vikram\u0026#34;, \u0026#34;Ananya\u0026#34;, \u0026#34;Suresh\u0026#34;, \u0026#34;Divya\u0026#34;, \u0026#34;Rahul\u0026#34;], \u0026#34;salary\u0026#34;: [75000, 55000, 48000, 82000, 60000], \u0026#34;city\u0026#34;: [\u0026#34;Bangalore\u0026#34;, \u0026#34;Mumbai\u0026#34;, \u0026#34;Delhi\u0026#34;, \u0026#34;Hyderabad\u0026#34;, \u0026#34;Chennai\u0026#34;] }).set_index(\u0026#34;emp_id\u0026#34;) # Rows where emp_id (index label) is between 102 and 104 -- loc is label inclusive print(df.loc[102:104]) # Last 3 rows by position print(df.iloc[-3:]) # Salary and city columns for the first 2 rows print(df.iloc[0:2][[\u0026#34;salary\u0026#34;, \u0026#34;city\u0026#34;]]) # or equivalently: print(df.loc[101:102, [\u0026#34;salary\u0026#34;, \u0026#34;city\u0026#34;]]) Q3 \u0026ndash; Filtering zomato = pd.DataFrame({ \u0026#34;name\u0026#34;: [\u0026#34;The Fatty Bao\u0026#34;, \u0026#34;Truffles\u0026#34;, \u0026#34;Meghana Foods\u0026#34;, \u0026#34;Smoke House Deli\u0026#34;, \u0026#34;Vidyarthi Bhavan\u0026#34;], \u0026#34;location\u0026#34;: [\u0026#34;Koramangala\u0026#34;, \u0026#34;Koramangala\u0026#34;, \u0026#34;Koramangala\u0026#34;, \u0026#34;Indiranagar\u0026#34;, \u0026#34;Gandhi Bazaar\u0026#34;], \u0026#34;cuisine\u0026#34;: [\u0026#34;Asian\u0026#34;, \u0026#34;American\u0026#34;, \u0026#34;North Indian\u0026#34;, \u0026#34;American\u0026#34;, \u0026#34;South Indian\u0026#34;], \u0026#34;rating\u0026#34;: [4.1, 4.4, 4.2, 4.0, 4.5], \u0026#34;votes\u0026#34;: [4200, 9800, 7600, 3200, 12000], \u0026#34;cost_for2\u0026#34;:[800, 600, 500, 1100, 150] }) # Koramangala restaurants with rating \u0026gt; 4.0 print(zomato[ (zomato[\u0026#34;location\u0026#34;] == \u0026#34;Koramangala\u0026#34;) \u0026amp; (zomato[\u0026#34;rating\u0026#34;] \u0026gt; 4.0) ]) # North Indian OR Chinese cuisine print(zomato[zomato[\u0026#34;cuisine\u0026#34;].isin([\u0026#34;North Indian\u0026#34;, \u0026#34;Chinese\u0026#34;])]) # Votes between 500 and 2000 # Note: none in this dataset fall in that range -- the result will be empty # The correct approach is: print(zomato[zomato[\u0026#34;votes\u0026#34;].between(500, 2000)]) Q4 \u0026ndash; Missing Data import numpy as np df = pd.DataFrame({ \u0026#34;name\u0026#34;: [\u0026#34;Arjun\u0026#34;, \u0026#34;Bhavna\u0026#34;, \u0026#34;Chirag\u0026#34;, \u0026#34;Divya\u0026#34;, \u0026#34;Esha\u0026#34;], \u0026#34;age\u0026#34;: [28, None, 32, None, 25], \u0026#34;city\u0026#34;: [\u0026#34;Pune\u0026#34;, None, \u0026#34;Delhi\u0026#34;, \u0026#34;Surat\u0026#34;, None], \u0026#34;salary\u0026#34;: [50000, 65000, None, None, 48000] }) # Count nulls print(df.isnull().sum()) # Fill age with median, city with \u0026#34;Unknown\u0026#34;, salary with mean df[\u0026#34;age\u0026#34;] = df[\u0026#34;age\u0026#34;].fillna(df[\u0026#34;age\u0026#34;].median()) df[\u0026#34;city\u0026#34;] = df[\u0026#34;city\u0026#34;].fillna(\u0026#34;Unknown\u0026#34;) df[\u0026#34;salary\u0026#34;] = df[\u0026#34;salary\u0026#34;].fillna(df[\u0026#34;salary\u0026#34;].mean()) print(df) # Drop rows where BOTH age and salary were originally null # (Divya has both null -- she would have been dropped before filling) df_original = pd.DataFrame({ \u0026#34;name\u0026#34;: [\u0026#34;Arjun\u0026#34;, \u0026#34;Bhavna\u0026#34;, \u0026#34;Chirag\u0026#34;, \u0026#34;Divya\u0026#34;, \u0026#34;Esha\u0026#34;], \u0026#34;age\u0026#34;: [28, None, 32, None, 25], \u0026#34;city\u0026#34;: [\u0026#34;Pune\u0026#34;, None, \u0026#34;Delhi\u0026#34;, \u0026#34;Surat\u0026#34;, None], \u0026#34;salary\u0026#34;: [50000, 65000, None, None, 48000] }) df_dropped = df_original.dropna(subset=[\u0026#34;age\u0026#34;, \u0026#34;salary\u0026#34;], how=\u0026#34;all\u0026#34;) print(df_dropped) Q5 \u0026ndash; String Operations df = pd.DataFrame({ \u0026#34;name\u0026#34;: [\u0026#34; arjun SHARMA\u0026#34;, \u0026#34;PRIYA patel \u0026#34;, \u0026#34;rohan VERMA\u0026#34;], \u0026#34;email\u0026#34;: [\u0026#34;arjun@gmail.com\u0026#34;, \u0026#34;priya@yahoo.in\u0026#34;, \u0026#34;rohan@gmail.com\u0026#34;] }) # Standardize name: strip whitespace then title case df[\u0026#34;name\u0026#34;] = df[\u0026#34;name\u0026#34;].str.strip().str.title() print(df[\u0026#34;name\u0026#34;]) # Arjun Sharma, Priya Patel, Rohan Verma # Extract domain (everything after @) df[\u0026#34;domain\u0026#34;] = df[\u0026#34;email\u0026#34;].str.split(\u0026#34;@\u0026#34;).str[1] print(df[\u0026#34;domain\u0026#34;]) # gmail.com, yahoo.in, gmail.com # Gmail customers gmail_customers = df[df[\u0026#34;email\u0026#34;].str.contains(\u0026#34;gmail\u0026#34;)] print(gmail_customers[[\u0026#34;name\u0026#34;, \u0026#34;email\u0026#34;]]) Q6 \u0026ndash; Date Operations df = pd.DataFrame({ \u0026#34;order_date\u0026#34;: [\u0026#34;2024-10-15\u0026#34;, \u0026#34;2024-10-22\u0026#34;, \u0026#34;2024-11-01\u0026#34;, \u0026#34;2024-10-28\u0026#34;, \u0026#34;2024-11-05\u0026#34;], \u0026#34;customer\u0026#34;: [\u0026#34;Ravi\u0026#34;, \u0026#34;Sita\u0026#34;, \u0026#34;Arjun\u0026#34;, \u0026#34;Deepa\u0026#34;, \u0026#34;Mohan\u0026#34;], \u0026#34;amount\u0026#34;: [5000, 3200, 8900, 4100, 2200] }) # Convert to datetime df[\u0026#34;order_date\u0026#34;] = pd.to_datetime(df[\u0026#34;order_date\u0026#34;]) # Extract month name and day of week df[\u0026#34;month_name\u0026#34;] = df[\u0026#34;order_date\u0026#34;].dt.month_name() df[\u0026#34;day_of_week\u0026#34;] = df[\u0026#34;order_date\u0026#34;].dt.day_name() print(df[[\u0026#34;order_date\u0026#34;, \u0026#34;month_name\u0026#34;, \u0026#34;day_of_week\u0026#34;]]) # Orders in October october_orders = df[df[\u0026#34;order_date\u0026#34;].dt.month == 10] print(october_orders) # Days ago (from today -- output will vary by when you run this) df[\u0026#34;days_ago\u0026#34;] = (pd.Timestamp.today() - df[\u0026#34;order_date\u0026#34;]).dt.days print(df[[\u0026#34;customer\u0026#34;, \u0026#34;order_date\u0026#34;, \u0026#34;days_ago\u0026#34;]]) Q7 \u0026ndash; apply() and map() employees = pd.DataFrame({ \u0026#34;name\u0026#34;: [\u0026#34;Aarav\u0026#34;, \u0026#34;Priya\u0026#34;, \u0026#34;Rohan\u0026#34;, \u0026#34;Sneha\u0026#34;, \u0026#34;Karan\u0026#34;], \u0026#34;city\u0026#34;: [\u0026#34;Delhi\u0026#34;, \u0026#34;Pune\u0026#34;, \u0026#34;Mumbai\u0026#34;, \u0026#34;Jaipur\u0026#34;, \u0026#34;Bangalore\u0026#34;], \u0026#34;experience\u0026#34;: [1, 3, 7, 4, 2] }) # experience_band using apply() def exp_band(yrs): if yrs \u0026lt; 2: return \u0026#34;Fresher\u0026#34; elif yrs \u0026lt;= 5: return \u0026#34;Mid-level\u0026#34; else: return \u0026#34;Senior\u0026#34; employees[\u0026#34;experience_band\u0026#34;] = employees[\u0026#34;experience\u0026#34;].apply(exp_band) # city_tier using map() with a dictionary tier_map = { \u0026#34;Delhi\u0026#34;: \u0026#34;Tier 1\u0026#34;, \u0026#34;Mumbai\u0026#34;: \u0026#34;Tier 1\u0026#34;, \u0026#34;Bangalore\u0026#34;: \u0026#34;Tier 1\u0026#34; } employees[\u0026#34;city_tier\u0026#34;] = employees[\u0026#34;city\u0026#34;].map(tier_map).fillna(\u0026#34;Tier 2\u0026#34;) print(employees) Q8 \u0026ndash; GroupBy ipl = pd.DataFrame({ \u0026#34;player\u0026#34;: [\u0026#34;Rohit\u0026#34;, \u0026#34;Kohli\u0026#34;, \u0026#34;Dhoni\u0026#34;, \u0026#34;Jadeja\u0026#34;, \u0026#34;Bumrah\u0026#34;, \u0026#34;Surya\u0026#34;, \u0026#34;Shubman\u0026#34;], \u0026#34;team\u0026#34;: [\u0026#34;MI\u0026#34;, \u0026#34;RCB\u0026#34;, \u0026#34;CSK\u0026#34;, \u0026#34;CSK\u0026#34;, \u0026#34;MI\u0026#34;, \u0026#34;MI\u0026#34;, \u0026#34;GT\u0026#34;], \u0026#34;runs\u0026#34;: [440, 639, 210, 180, 15, 380, 520], \u0026#34;fours\u0026#34;: [42, 55, 20, 18, 1, 35, 48], \u0026#34;sixes\u0026#34;: [20, 30, 10, 8, 0, 22, 25], \u0026#34;sr\u0026#34;: [148, 145, 125, 120, 100, 165, 140] }) # Total runs per team print(ipl.groupby(\u0026#34;team\u0026#34;)[\u0026#34;runs\u0026#34;].sum()) # Average strike rate per player (each player has one row, so this is just the sr column) print(ipl.groupby(\u0026#34;player\u0026#34;)[\u0026#34;sr\u0026#34;].mean()) # Team with most boundaries (fours + sixes) ipl[\u0026#34;boundaries\u0026#34;] = ipl[\u0026#34;fours\u0026#34;] + ipl[\u0026#34;sixes\u0026#34;] most_boundaries = ipl.groupby(\u0026#34;team\u0026#34;)[\u0026#34;boundaries\u0026#34;].sum() print(most_boundaries) print(\u0026#34;Most boundaries:\u0026#34;, most_boundaries.idxmax()) # transform: add team\u0026#39;s total runs to each player\u0026#39;s row ipl[\u0026#34;team_total_runs\u0026#34;] = ipl.groupby(\u0026#34;team\u0026#34;)[\u0026#34;runs\u0026#34;].transform(\u0026#34;sum\u0026#34;) print(ipl[[\u0026#34;player\u0026#34;, \u0026#34;team\u0026#34;, \u0026#34;runs\u0026#34;, \u0026#34;team_total_runs\u0026#34;]]) Q9 \u0026ndash; Merging students = pd.DataFrame({ \u0026#34;student_id\u0026#34;: [1, 2, 3, 4, 5], \u0026#34;name\u0026#34;: [\u0026#34;Aarav\u0026#34;, \u0026#34;Priya\u0026#34;, \u0026#34;Rohan\u0026#34;, \u0026#34;Sneha\u0026#34;, \u0026#34;Karan\u0026#34;], \u0026#34;class\u0026#34;: [\u0026#34;10A\u0026#34;, \u0026#34;10B\u0026#34;, \u0026#34;10A\u0026#34;, \u0026#34;10B\u0026#34;, \u0026#34;10A\u0026#34;] }) exam_results = pd.DataFrame({ \u0026#34;student_id\u0026#34;: [1, 2, 3, 5], \u0026#34;score\u0026#34;: [88, 91, 74, 65], \u0026#34;grade\u0026#34;: [\u0026#34;A\u0026#34;, \u0026#34;A+\u0026#34;, \u0026#34;B\u0026#34;, \u0026#34;C\u0026#34;] }) # All students even if they have no exam record (left join) all_students = pd.merge(students, exam_results, on=\u0026#34;student_id\u0026#34;, how=\u0026#34;left\u0026#34;) print(\u0026#34;Left join (all students):\u0026#34;) print(all_students) # Only students who appeared in the exam (inner join) appeared = pd.merge(students, exam_results, on=\u0026#34;student_id\u0026#34;, how=\u0026#34;inner\u0026#34;) print(\u0026#34;\\nInner join (appeared only):\u0026#34;) print(appeared) # Students with NO exam record # After left join, students with no match will have NaN in score no_record = all_students[all_students[\u0026#34;score\u0026#34;].isna()][[\u0026#34;student_id\u0026#34;, \u0026#34;name\u0026#34;, \u0026#34;class\u0026#34;]] print(\u0026#34;\\nNo exam record:\u0026#34;) print(no_record) # Sneha (student_id 4) has no exam record Q10 \u0026ndash; Pivot Table sales = pd.DataFrame({ \u0026#34;city\u0026#34;: [\u0026#34;Delhi\u0026#34;,\u0026#34;Delhi\u0026#34;,\u0026#34;Mumbai\u0026#34;,\u0026#34;Mumbai\u0026#34;,\u0026#34;Bangalore\u0026#34;,\u0026#34;Bangalore\u0026#34;,\u0026#34;Delhi\u0026#34;,\u0026#34;Mumbai\u0026#34;], \u0026#34;product\u0026#34;: [\u0026#34;Laptop\u0026#34;,\u0026#34;Phone\u0026#34;,\u0026#34;Laptop\u0026#34;,\u0026#34;Tablet\u0026#34;,\u0026#34;Phone\u0026#34;,\u0026#34;Laptop\u0026#34;,\u0026#34;Tablet\u0026#34;,\u0026#34;Phone\u0026#34;], \u0026#34;revenue\u0026#34;: [65000, 18000, 62000, 32000, 16000, 70000, 28000, 20000] }) pivot = sales.pivot_table( values=\u0026#34;revenue\u0026#34;, index=\u0026#34;product\u0026#34;, columns=\u0026#34;city\u0026#34;, aggfunc=\u0026#34;sum\u0026#34;, fill_value=0, margins=True, margins_name=\u0026#34;Total\u0026#34; ) print(pivot) Q11 \u0026ndash; melt() and Reshape wide = pd.DataFrame({ \u0026#34;student\u0026#34;: [\u0026#34;Aarav\u0026#34;, \u0026#34;Priya\u0026#34;, \u0026#34;Rohan\u0026#34;, \u0026#34;Sneha\u0026#34;], \u0026#34;Maths\u0026#34;: [78, 92, 65, 88], \u0026#34;Science\u0026#34;: [85, 76, 70, 91], \u0026#34;English\u0026#34;: [90, 88, 80, 85], \u0026#34;History\u0026#34;: [72, 80, 68, 79] }) # Melt to long format long = pd.melt( wide, id_vars=[\u0026#34;student\u0026#34;], value_vars=[\u0026#34;Maths\u0026#34;, \u0026#34;Science\u0026#34;, \u0026#34;English\u0026#34;, \u0026#34;History\u0026#34;], var_name=\u0026#34;subject\u0026#34;, value_name=\u0026#34;marks\u0026#34; ) print(long.head(8)) # Subject-wise average subject_avg = long.groupby(\u0026#34;subject\u0026#34;)[\u0026#34;marks\u0026#34;].mean().round(1) print(\u0026#34;\\nSubject-wise average:\u0026#34;) print(subject_avg) Q12 \u0026ndash; Performance Optimization import numpy as np n = 100000 df = pd.DataFrame({ \u0026#34;name\u0026#34;: [f\u0026#34;emp_{i}\u0026#34; for i in range(n)], \u0026#34;department\u0026#34;: np.random.choice([\u0026#34;Engineering\u0026#34;, \u0026#34;HR\u0026#34;, \u0026#34;Sales\u0026#34;, \u0026#34;Marketing\u0026#34;], n), \u0026#34;city\u0026#34;: np.random.choice([\u0026#34;Delhi\u0026#34;, \u0026#34;Mumbai\u0026#34;, \u0026#34;Bangalore\u0026#34;, \u0026#34;Hyderabad\u0026#34;, \u0026#34;Pune\u0026#34;], n), \u0026#34;salary\u0026#34;: np.random.randint(30000, 150000, n), \u0026#34;experience\u0026#34;: np.random.randint(0, 20, n) }) # Memory before print(\u0026#34;Before:\u0026#34;) print(df.memory_usage(deep=True).sum() / 1024**2, \u0026#34;MB\u0026#34;) # Convert string columns to category df[\u0026#34;department\u0026#34;] = df[\u0026#34;department\u0026#34;].astype(\u0026#34;category\u0026#34;) df[\u0026#34;city\u0026#34;] = df[\u0026#34;city\u0026#34;].astype(\u0026#34;category\u0026#34;) # Memory after print(\u0026#34;After category conversion:\u0026#34;) print(df.memory_usage(deep=True).sum() / 1024**2, \u0026#34;MB\u0026#34;) # query() instead of boolean indexing result_boolean = df[df[\u0026#34;salary\u0026#34;] \u0026gt; 60000] result_query = df.query(\u0026#34;salary \u0026gt; 60000\u0026#34;) print(f\u0026#34;\\nRows with salary \u0026gt; 60000: {len(result_query)}\u0026#34;) # Both produce the same result -- query() is more readable for complex filters Q13 \u0026ndash; Full Analysis Pipeline import numpy as np np.random.seed(42) n = 200 sales = pd.DataFrame({ \u0026#34;order_id\u0026#34;: range(1001, 1001+n), \u0026#34;order_date\u0026#34;: pd.date_range(\u0026#34;2024-01-01\u0026#34;, periods=n, freq=\u0026#34;2D\u0026#34;), \u0026#34;customer\u0026#34;: np.random.choice([\u0026#34;Aarav\u0026#34;,\u0026#34;Priya\u0026#34;,\u0026#34;Rohan\u0026#34;,\u0026#34;Sneha\u0026#34;,\u0026#34;Karan\u0026#34;,\u0026#34;Meera\u0026#34;,\u0026#34;Arjun\u0026#34;,\u0026#34;Divya\u0026#34;], n), \u0026#34;city\u0026#34;: np.random.choice([\u0026#34;Delhi\u0026#34;,\u0026#34;Mumbai\u0026#34;,\u0026#34;Bangalore\u0026#34;,\u0026#34;Hyderabad\u0026#34;,\u0026#34;Pune\u0026#34;,\u0026#34;Chennai\u0026#34;], n), \u0026#34;category\u0026#34;: np.random.choice([\u0026#34;Electronics\u0026#34;,\u0026#34;Clothing\u0026#34;,\u0026#34;Books\u0026#34;,\u0026#34;Home\u0026#34;,\u0026#34;Beauty\u0026#34;], n), \u0026#34;amount\u0026#34;: np.random.randint(200, 50000, n), \u0026#34;status\u0026#34;: np.random.choice([\u0026#34;Delivered\u0026#34;,\u0026#34;Pending\u0026#34;,\u0026#34;Cancelled\u0026#34;], n, p=[0.75, 0.15, 0.10]) }) # 1. City with highest average order value city_avg = sales.groupby(\u0026#34;city\u0026#34;)[\u0026#34;amount\u0026#34;].mean() print(\u0026#34;Highest avg order value city:\u0026#34;, city_avg.idxmax()) print(city_avg.sort_values(ascending=False)) # 2. Percentage of Delivered vs Cancelled status_pct = sales[\u0026#34;status\u0026#34;].value_counts(normalize=True) * 100 print(\u0026#34;\\nOrder status breakdown:\u0026#34;) print(status_pct.round(1)) # 3. Category with most revenue cat_revenue = sales.groupby(\u0026#34;category\u0026#34;)[\u0026#34;amount\u0026#34;].sum() print(\u0026#34;\\nTop revenue category:\u0026#34;, cat_revenue.idxmax()) print(cat_revenue.sort_values(ascending=False)) # 4. revenue_tier column sales[\u0026#34;revenue_tier\u0026#34;] = pd.cut( sales[\u0026#34;amount\u0026#34;], bins=[0, 2999, 9999, float(\u0026#34;inf\u0026#34;)], labels=[\u0026#34;Low\u0026#34;, \u0026#34;Medium\u0026#34;, \u0026#34;High\u0026#34;] ) # Alternative with np.where: # sales[\u0026#34;revenue_tier\u0026#34;] = np.where(sales[\u0026#34;amount\u0026#34;] \u0026gt;= 10000, \u0026#34;High\u0026#34;, # np.where(sales[\u0026#34;amount\u0026#34;] \u0026gt;= 3000, \u0026#34;Medium\u0026#34;, \u0026#34;Low\u0026#34;)) print(\u0026#34;\\nRevenue tier counts:\u0026#34;) print(sales[\u0026#34;revenue_tier\u0026#34;].value_counts()) # 5. Customers with more than 2 orders order_counts = sales.groupby(\u0026#34;customer\u0026#34;)[\u0026#34;order_id\u0026#34;].count() repeat_customers = order_counts[order_counts \u0026gt; 2] print(\u0026#34;\\nCustomers with more than 2 orders:\u0026#34;) print(repeat_customers) # 6. Month with peak revenue sales[\u0026#34;month\u0026#34;] = sales[\u0026#34;order_date\u0026#34;].dt.month_name() monthly_rev = sales.groupby(\u0026#34;month\u0026#34;)[\u0026#34;amount\u0026#34;].sum() print(\u0026#34;\\nPeak revenue month:\u0026#34;, monthly_rev.idxmax()) print(monthly_rev.sort_values(ascending=False).head(3)) Made with care for Codeverra learners | codeverra.com\n","permalink":"https://blogs.codeverra.com/blog/pandas-practice/","summary":"\u003ch1 id=\"-pandas-masterclass\"\u003e🐼 Pandas Masterclass\u003c/h1\u003e\n\u003ch3 id=\"from-raw-data-to-real-insights---a-beginner\"\u003e\u003cem\u003eFrom Raw Data to Real Insights - A Beginner\u0026rsquo;s Guide to Data Analysis in Python\u003c/em\u003e\u003c/h3\u003e\n\u003chr\u003e\n\u003cblockquote\u003e\n\u003cp\u003e\u003cstrong\u003eWho is this for?\u003c/strong\u003e\nYou\u0026rsquo;ve done the NumPy Masterclass (or know the basics of Python and arrays).\nNow you want to work with \u003cstrong\u003ereal tabular data\u003c/strong\u003e  -  rows, columns, missing values, merges, and group-level summaries.\nThat\u0026rsquo;s exactly what Pandas is for.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003chr\u003e\n\u003ch2 id=\"-table-of-contents\"\u003e📌 Table of Contents\u003c/h2\u003e\n\u003col\u003e\n\u003cli\u003e\u003ca href=\"#1-what-is-pandas-and-why-do-you-need-it\"\u003eWhat is Pandas and Why Do You Need It?\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#2-installing-and-importing-pandas\"\u003eInstalling and Importing Pandas\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#3-the-two-core-data-structures\"\u003eThe Two Core Data Structures\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#4-creating-dataframes\"\u003eCreating DataFrames\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#5-reading-and-writing-data\"\u003eReading and Writing Data\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#6-exploring-a-dataframe\"\u003eExploring a DataFrame\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#7-selecting-data--loc-iloc-and-column-access\"\u003eSelecting Data  -  loc, iloc, and Column Access\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#8-filtering-rows\"\u003eFiltering Rows\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#9-adding-modifying-and-dropping-columns\"\u003eAdding, Modifying, and Dropping Columns\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#10-sorting-data\"\u003eSorting Data\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#11-handling-missing-data\"\u003eHandling Missing Data\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#12-string-operations\"\u003eString Operations\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#13-working-with-dates-and-times\"\u003eWorking with Dates and Times\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#14-apply-map-and-applymap\"\u003eapply(), map(), and applymap()\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#15-groupby--aggregation-and-split-apply-combine\"\u003eGroupBy  -  Aggregation and Split-Apply-Combine\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#16-merging-and-joining-dataframes\"\u003eMerging and Joining DataFrames\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#17-pivot-tables-and-crosstabs\"\u003ePivot Tables and Crosstabs\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#18-reshaping--melt-and-stackunstack\"\u003eReshaping  -  melt() and stack()/unstack()\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#19-multiindex-dataframes\"\u003eMultiIndex DataFrames\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#20-performance-tips\"\u003ePerformance Tips\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#21-practice-questions\"\u003ePractice Questions\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#22-what-we-covered--whats-next\"\u003eWhat We Covered + What\u0026rsquo;s Next\u003c/a\u003e\u003c/li\u003e\n\u003c/ol\u003e\n\u003chr\u003e\n\u003ch2 id=\"1-what-is-pandas-and-why-do-you-need-it\"\u003e1. What is Pandas and Why Do You Need It?\u003c/h2\u003e\n\u003cp\u003eNumPy arrays are powerful, but they have one limitation: \u003cstrong\u003eevery column must be the same data type\u003c/strong\u003e, and there are no column names or row labels.\u003c/p\u003e","tags":["python","pandas","data-analysis"],"title":"Complete Guide to Pandas in Python"},{"content":"Python Collections - Practice Questions and Solutions List, Tuple, Dictionary, Set \u0026ndash; Complete Practice Set This practice set covers strings and all four major Python collections. Questions are grouped by collection type, then by difficulty within each section. The final section has mixed questions that combine multiple collections, loops, and conditionals. Try each question before reading the solution.\nTable of Contents Part 1 \u0026ndash; Lists Part 2 \u0026ndash; Strings Part 3 \u0026ndash; Tuples Part 4 \u0026ndash; Dictionaries Part 5 \u0026ndash; Sets Part 6 \u0026ndash; Mixed Questions Quick Reference Part 1 \u0026ndash; Lists A list is an ordered, mutable collection. It allows duplicate values and supports indexing, slicing, appending, and sorting.\nL1 \u0026ndash; Basic Indexing and Slicing Given the list below, write code to:\nPrint the first element Print the last element Print elements from index 2 to 5 (inclusive) Print every alternate element cities = [\u0026#34;Delhi\u0026#34;, \u0026#34;Mumbai\u0026#34;, \u0026#34;Bangalore\u0026#34;, \u0026#34;Chennai\u0026#34;, \u0026#34;Hyderabad\u0026#34;, \u0026#34;Pune\u0026#34;, \u0026#34;Kolkata\u0026#34;, \u0026#34;Ahmedabad\u0026#34;] Expected Output:\nFirst : Delhi Last : Ahmedabad Slice : [\u0026#39;Bangalore\u0026#39;, \u0026#39;Chennai\u0026#39;, \u0026#39;Hyderabad\u0026#39;, \u0026#39;Pune\u0026#39;] Every alternate : [\u0026#39;Delhi\u0026#39;, \u0026#39;Bangalore\u0026#39;, \u0026#39;Hyderabad\u0026#39;, \u0026#39;Kolkata\u0026#39;] Solution:\ncities = [\u0026#34;Delhi\u0026#34;, \u0026#34;Mumbai\u0026#34;, \u0026#34;Bangalore\u0026#34;, \u0026#34;Chennai\u0026#34;, \u0026#34;Hyderabad\u0026#34;, \u0026#34;Pune\u0026#34;, \u0026#34;Kolkata\u0026#34;, \u0026#34;Ahmedabad\u0026#34;] print(\u0026#34;First :\u0026#34;, cities[0]) print(\u0026#34;Last :\u0026#34;, cities[-1]) print(\u0026#34;Slice :\u0026#34;, cities[2:6]) print(\u0026#34;Every alternate :\u0026#34;, cities[::2]) L2 \u0026ndash; List Methods Start with the list below and apply the following operations in order. Print the list after each step.\nAppend \u0026ldquo;Jaipur\u0026rdquo; Insert \u0026ldquo;Surat\u0026rdquo; at index 2 Remove \u0026ldquo;Chennai\u0026rdquo; Sort the list alphabetically Reverse it cities = [\u0026#34;Delhi\u0026#34;, \u0026#34;Mumbai\u0026#34;, \u0026#34;Bangalore\u0026#34;, \u0026#34;Chennai\u0026#34;, \u0026#34;Hyderabad\u0026#34;] Solution:\ncities = [\u0026#34;Delhi\u0026#34;, \u0026#34;Mumbai\u0026#34;, \u0026#34;Bangalore\u0026#34;, \u0026#34;Chennai\u0026#34;, \u0026#34;Hyderabad\u0026#34;] cities.append(\u0026#34;Jaipur\u0026#34;) print(\u0026#34;After append :\u0026#34;, cities) cities.insert(2, \u0026#34;Surat\u0026#34;) print(\u0026#34;After insert :\u0026#34;, cities) cities.remove(\u0026#34;Chennai\u0026#34;) print(\u0026#34;After remove :\u0026#34;, cities) cities.sort() print(\u0026#34;After sort :\u0026#34;, cities) cities.reverse() print(\u0026#34;After reverse:\u0026#34;, cities) L3 \u0026ndash; List Comprehension with Transformation Given the list of product prices below, use a list comprehension to:\nCreate a new list with 18% GST added to each price Create another list with only prices above Rs.500 (after GST) prices = [120, 450, 800, 250, 1200, 60, 3500, 175] Solution:\nprices = [120, 450, 800, 250, 1200, 60, 3500, 175] with_gst = [round(p * 1.18, 2) for p in prices] print(\u0026#34;With GST:\u0026#34;, with_gst) expensive = [p for p in with_gst if p \u0026gt; 500] print(\u0026#34;Above 500:\u0026#34;, expensive) L4 \u0026ndash; Nested List (2D) You have a marks matrix for 4 students across 3 subjects. Write code to:\nPrint the marks of the second student (index 1) Print the third subject marks for all students Find the student with the highest total marks = [ [78, 85, 90], # Aarav [92, 76, 88], # Priya [65, 70, 80], # Rohan [88, 91, 95], # Sneha ] names = [\u0026#34;Aarav\u0026#34;, \u0026#34;Priya\u0026#34;, \u0026#34;Rohan\u0026#34;, \u0026#34;Sneha\u0026#34;] Solution:\nmarks = [ [78, 85, 90], [92, 76, 88], [65, 70, 80], [88, 91, 95], ] names = [\u0026#34;Aarav\u0026#34;, \u0026#34;Priya\u0026#34;, \u0026#34;Rohan\u0026#34;, \u0026#34;Sneha\u0026#34;] print(\u0026#34;Second student\u0026#39;s marks:\u0026#34;, marks[1]) print(\u0026#34;Third subject marks:\u0026#34;, [row[2] for row in marks]) totals = [sum(row) for row in marks] max_total = max(totals) topper = names[totals.index(max_total)] print(f\u0026#34;Topper: {topper} with {max_total} marks\u0026#34;) L5 \u0026ndash; List Operations Without Built-ins Write a function second_largest(lst) that returns the second largest unique number from a list. Do not use sorted() or max().\nnumbers = [45, 92, 78, 92, 55, 88, 78, 100, 63] Expected Output:\nSecond largest: 92 Solution:\ndef second_largest(lst): unique = [] for n in lst: if n not in unique: unique.append(n) # Find max manually first = unique[0] for n in unique: if n \u0026gt; first: first = n # Find second max (largest value that is not the max) second = None for n in unique: if n == first: continue if second is None or n \u0026gt; second: second = n return second numbers = [45, 92, 78, 92, 55, 88, 78, 100, 63] print(\u0026#34;Second largest:\u0026#34;, second_largest(numbers)) L6 \u0026ndash; Rotate a List Write a function rotate_left(lst, k) that rotates a list to the left by k positions without using any slicing tricks \u0026ndash; use a loop.\nitems = [10, 20, 30, 40, 50] rotate_left(items, 2) # Expected: [30, 40, 50, 10, 20] Solution:\ndef rotate_left(lst, k): k = k % len(lst) # handle k larger than list length result = [] for i in range(k, len(lst)): result.append(lst[i]) for i in range(k): result.append(lst[i]) return result items = [10, 20, 30, 40, 50] print(rotate_left(items, 2)) # [30, 40, 50, 10, 20] print(rotate_left(items, 7)) # same as rotating by 2 L7 \u0026ndash; Chunk a List Write a function chunk(lst, size) that splits a list into smaller lists of a given size. The last chunk may be smaller.\ndata = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10] chunk(data, 3) # Expected: [[1, 2, 3], [4, 5, 6], [7, 8, 9], [10]] Solution:\ndef chunk(lst, size): result = [] for i in range(0, len(lst), size): result.append(lst[i:i+size]) return result data = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10] print(chunk(data, 3)) print(chunk(data, 4)) Part 2 \u0026ndash; Strings A string is an ordered, immutable sequence of characters. Like a tuple, you can index and slice it, but you cannot change individual characters in place. Python has a rich set of built-in string methods for searching, formatting, and transforming text.\nST1 \u0026ndash; Basic Indexing and Slicing Given the string below, write code to:\nPrint the first character Print the last character Print characters from index 8 to 13 Print the string in reverse Print the length of the string message = \u0026#34;Welcome to Python\u0026#34; Expected Output:\nFirst : W Last : n Slice : to Py Reversed : nohtyP ot emocleW Length : 17 Solution:\nmessage = \u0026#34;Welcome to Python\u0026#34; print(\u0026#34;First :\u0026#34;, message[0]) print(\u0026#34;Last :\u0026#34;, message[-1]) print(\u0026#34;Slice :\u0026#34;, message[8:13]) print(\u0026#34;Reversed :\u0026#34;, message[::-1]) print(\u0026#34;Length :\u0026#34;, len(message)) ST2 \u0026ndash; String Methods Start with the string below and apply the following operations. Print the result after each step.\nConvert to uppercase Convert to lowercase Strip the extra whitespace from both ends Replace \u0026ldquo;Python\u0026rdquo; with \u0026ldquo;Coding\u0026rdquo; Split the sentence into a list of words text = \u0026#34; Learning Python is Fun \u0026#34; Solution:\ntext = \u0026#34; Learning Python is Fun \u0026#34; print(text.upper()) print(text.lower()) print(text.strip()) print(text.strip().replace(\u0026#34;Python\u0026#34;, \u0026#34;Coding\u0026#34;)) print(text.strip().split()) ST3 \u0026ndash; String Checking Methods Given the strings below, use appropriate string methods to answer each question.\nword1 = \u0026#34;Python3\u0026#34; word2 = \u0026#34;codeverra\u0026#34; word3 = \u0026#34; \u0026#34; sentence = \u0026#34;Learning Python is fun\u0026#34; Is word1 made of only alphabets? Is word1 alphanumeric? Does word2 start with \u0026quot;code\u0026quot;? Does sentence end with \u0026quot;fun\u0026quot;? Is \u0026quot;Python\u0026quot; present in sentence? Is word3 blank (only whitespace)? Solution:\nword1 = \u0026#34;Python3\u0026#34; word2 = \u0026#34;codeverra\u0026#34; word3 = \u0026#34; \u0026#34; sentence = \u0026#34;Learning Python is fun\u0026#34; print(word1.isalpha()) # False -- contains a digit print(word1.isalnum()) # True print(word2.startswith(\u0026#34;code\u0026#34;)) # True print(sentence.endswith(\u0026#34;fun\u0026#34;)) # True print(\u0026#34;Python\u0026#34; in sentence) # True print(word3.strip() == \u0026#34;\u0026#34;) # True ST4 \u0026ndash; Palindrome Checker Write a function is_palindrome(text) that returns True if the text reads the same forwards and backwards. Ignore spaces and case.\nis_palindrome(\u0026#34;racecar\u0026#34;) # True is_palindrome(\u0026#34;A man a plan a canal Panama\u0026#34;) # True is_palindrome(\u0026#34;hello\u0026#34;) # False is_palindrome(\u0026#34;Madam\u0026#34;) # True Solution:\ndef is_palindrome(text): cleaned = text.replace(\u0026#34; \u0026#34;, \u0026#34;\u0026#34;).lower() return cleaned == cleaned[::-1] print(is_palindrome(\u0026#34;racecar\u0026#34;)) # True print(is_palindrome(\u0026#34;A man a plan a canal Panama\u0026#34;)) # True print(is_palindrome(\u0026#34;hello\u0026#34;)) # False print(is_palindrome(\u0026#34;Madam\u0026#34;)) # True ST5 \u0026ndash; Count Vowels and Consonants Write a function count_letters(text) that counts vowels and consonants in a string. Ignore spaces, digits, and punctuation.\ntext = \u0026#34;Learning Python at Codeverra is Amazing\u0026#34; Expected Output:\nVowels : 13 Consonants : 21 Solution:\ndef count_letters(text): vowels = \u0026#34;aeiouAEIOU\u0026#34; v_count = 0 c_count = 0 for char in text: if char.isalpha(): if char in vowels: v_count += 1 else: c_count += 1 return v_count, c_count text = \u0026#34;Learning Python at Codeverra is Amazing\u0026#34; vowels, consonants = count_letters(text) print(f\u0026#34;Vowels : {vowels}\u0026#34;) print(f\u0026#34;Consonants : {consonants}\u0026#34;) Part 3 \u0026ndash; Tuples A tuple is an ordered, immutable collection. Once created, its values cannot be changed. Tuples are used for fixed data, function return values, and dictionary keys.\nT1 \u0026ndash; Tuple Basics and Unpacking employee = (\u0026#34;Vikram Nair\u0026#34;, \u0026#34;Engineering\u0026#34;, 85000, \u0026#34;Bangalore\u0026#34;) Unpack this tuple into four variables Print each variable on a separate line Try to change the salary (third element) \u0026ndash; explain what happens Solution:\nemployee = (\u0026#34;Vikram Nair\u0026#34;, \u0026#34;Engineering\u0026#34;, 85000, \u0026#34;Bangalore\u0026#34;) name, department, salary, city = employee print(\u0026#34;Name :\u0026#34;, name) print(\u0026#34;Department :\u0026#34;, department) print(\u0026#34;Salary :\u0026#34;, salary) print(\u0026#34;City :\u0026#34;, city) # Attempting to modify raises a TypeError # employee[2] = 90000 --\u0026gt; TypeError: \u0026#39;tuple\u0026#39; object does not support item assignment # The right way to \u0026#34;update\u0026#34; a tuple is to create a new one updated_employee = employee[:2] + (90000,) + employee[3:] print(\u0026#34;Updated:\u0026#34;, updated_employee) T2 \u0026ndash; Tuple of Tuples (Loop and Unpack) You have a list of tuples representing train journeys. Print a formatted summary for each journey.\njourneys = [ (\u0026#34;Rajdhani Express\u0026#34;, \u0026#34;Delhi\u0026#34;, \u0026#34;Mumbai\u0026#34;, 16), (\u0026#34;Shatabdi\u0026#34;, \u0026#34;Chennai\u0026#34;, \u0026#34;Bangalore\u0026#34;, 5), (\u0026#34;Duronto\u0026#34;, \u0026#34;Kolkata\u0026#34;, \u0026#34;Delhi\u0026#34;, 17), (\u0026#34;Vande Bharat\u0026#34;, \u0026#34;Hyderabad\u0026#34;, \u0026#34;Pune\u0026#34;, 8), ] Expected Output:\nRajdhani Express : Delhi -\u0026gt; Mumbai (16 hrs) Shatabdi : Chennai -\u0026gt; Bangalore (5 hrs) Duronto : Kolkata -\u0026gt; Delhi (17 hrs) Vande Bharat : Hyderabad -\u0026gt; Pune (8 hrs) Solution:\njourneys = [ (\u0026#34;Rajdhani Express\u0026#34;, \u0026#34;Delhi\u0026#34;, \u0026#34;Mumbai\u0026#34;, 16), (\u0026#34;Shatabdi\u0026#34;, \u0026#34;Chennai\u0026#34;, \u0026#34;Bangalore\u0026#34;, 5), (\u0026#34;Duronto\u0026#34;, \u0026#34;Kolkata\u0026#34;, \u0026#34;Delhi\u0026#34;, 17), (\u0026#34;Vande Bharat\u0026#34;, \u0026#34;Hyderabad\u0026#34;, \u0026#34;Pune\u0026#34;, 8), ] for train, origin, destination, hours in journeys: print(f\u0026#34;{train:\u0026lt;18}: {origin} -\u0026gt; {destination} ({hours} hrs)\u0026#34;) T3 \u0026ndash; Tuple as Dictionary Key Tuples can be dictionary keys because they are immutable. Create a dictionary that maps (city, year) tuples to population (in lakhs).\nThen write a loop to print cities whose population grew between 2020 and 2023.\npopulation = { (\u0026#34;Delhi\u0026#34;, 2020): 300, (\u0026#34;Delhi\u0026#34;, 2023): 320, (\u0026#34;Mumbai\u0026#34;, 2020): 205, (\u0026#34;Mumbai\u0026#34;, 2023): 210, (\u0026#34;Bangalore\u0026#34;, 2020): 120, (\u0026#34;Bangalore\u0026#34;, 2023): 140, (\u0026#34;Chennai\u0026#34;, 2020): 92, (\u0026#34;Chennai\u0026#34;, 2023): 90, } Solution:\npopulation = { (\u0026#34;Delhi\u0026#34;, 2020): 300, (\u0026#34;Delhi\u0026#34;, 2023): 320, (\u0026#34;Mumbai\u0026#34;, 2020): 205, (\u0026#34;Mumbai\u0026#34;, 2023): 210, (\u0026#34;Bangalore\u0026#34;, 2020): 120, (\u0026#34;Bangalore\u0026#34;, 2023): 140, (\u0026#34;Chennai\u0026#34;, 2020): 92, (\u0026#34;Chennai\u0026#34;, 2023): 90, } cities = {city for city, year in population.keys()} for city in sorted(cities): pop_2020 = population[(city, 2020)] pop_2023 = population[(city, 2023)] if pop_2023 \u0026gt; pop_2020: growth = pop_2023 - pop_2020 print(f\u0026#34;{city}: grew by {growth} lakhs\u0026#34;) T4 \u0026ndash; Named Tuple (collections.namedtuple) Use namedtuple to create a Student type. Create 3 student instances and print a report.\nSolution:\nfrom collections import namedtuple Student = namedtuple(\u0026#34;Student\u0026#34;, [\u0026#34;name\u0026#34;, \u0026#34;roll_no\u0026#34;, \u0026#34;marks\u0026#34;, \u0026#34;city\u0026#34;]) s1 = Student(\u0026#34;Aarav\u0026#34;, 101, 88, \u0026#34;Delhi\u0026#34;) s2 = Student(\u0026#34;Priya\u0026#34;, 102, 92, \u0026#34;Mumbai\u0026#34;) s3 = Student(\u0026#34;Rohan\u0026#34;, 103, 75, \u0026#34;Bangalore\u0026#34;) students = [s1, s2, s3] for s in students: grade = \u0026#34;A\u0026#34; if s.marks \u0026gt;= 85 else \u0026#34;B\u0026#34; if s.marks \u0026gt;= 70 else \u0026#34;C\u0026#34; print(f\u0026#34;Roll {s.roll_no} | {s.name:\u0026lt;8} | {s.marks} | {grade} | {s.city}\u0026#34;) Why namedtuple? You get tuple immutability but can access fields by name (s.marks) instead of index (s[2]). Much more readable.\nT5 \u0026ndash; Swap Without a Temp Variable Python allows elegant swapping using tuple unpacking. Demonstrate this and write a function that takes a list of tuples and swaps each pair.\npairs = [(1, 2), (10, 20), (\u0026#34;Delhi\u0026#34;, \u0026#34;Mumbai\u0026#34;), (True, False)] Expected Output:\n(2, 1) (20, 10) (\u0026#39;Mumbai\u0026#39;, \u0026#39;Delhi\u0026#39;) (False, True) Solution:\n# Classic swap using tuple unpacking a = 10 b = 20 a, b = b, a print(a, b) # 20 10 # Swap all pairs in a list pairs = [(1, 2), (10, 20), (\u0026#34;Delhi\u0026#34;, \u0026#34;Mumbai\u0026#34;), (True, False)] swapped = [(b, a) for a, b in pairs] print(swapped) Part 4 \u0026ndash; Dictionaries A dictionary is an ordered (Python 3.7+), mutable collection of key-value pairs. Keys must be unique and immutable. Dictionaries are optimised for fast lookup by key.\nD1 \u0026ndash; Dictionary Basics Given the dictionary below:\nAccess Kohli\u0026rsquo;s runs Add a new player \u0026ldquo;Hardik\u0026rdquo; with 320 runs Update Rohit\u0026rsquo;s runs to 580 Delete Dhoni\u0026rsquo;s entry Check if \u0026ldquo;Bumrah\u0026rdquo; is in the dictionary ipl_runs = { \u0026#34;Rohit\u0026#34;: 520, \u0026#34;Kohli\u0026#34;: 639, \u0026#34;Dhoni\u0026#34;: 210, \u0026#34;Jadeja\u0026#34;: 180, } Solution:\nipl_runs = { \u0026#34;Rohit\u0026#34;: 520, \u0026#34;Kohli\u0026#34;: 639, \u0026#34;Dhoni\u0026#34;: 210, \u0026#34;Jadeja\u0026#34;: 180, } print(ipl_runs[\u0026#34;Kohli\u0026#34;]) # 639 ipl_runs[\u0026#34;Hardik\u0026#34;] = 320 print(ipl_runs) ipl_runs[\u0026#34;Rohit\u0026#34;] = 580 print(ipl_runs[\u0026#34;Rohit\u0026#34;]) del ipl_runs[\u0026#34;Dhoni\u0026#34;] print(ipl_runs) print(\u0026#34;Bumrah\u0026#34; in ipl_runs) # False print(\u0026#34;Kohli\u0026#34; in ipl_runs) # True D2 \u0026ndash; Safe Access with get() and setdefault() The .get() method avoids KeyError when accessing a key that might not exist.\nGiven the inventory below, write code that:\nPrints the stock for \u0026ldquo;Laptop\u0026rdquo; and \u0026ldquo;Printer\u0026rdquo; using .get() with a default of 0 Uses setdefault() to add \u0026ldquo;Printer\u0026rdquo; with a stock of 5 if it does not exist inventory = { \u0026#34;Laptop\u0026#34;: 15, \u0026#34;Phone\u0026#34;: 42, \u0026#34;Tablet\u0026#34;: 8, \u0026#34;Monitor\u0026#34;: 20, } Solution:\ninventory = { \u0026#34;Laptop\u0026#34;: 15, \u0026#34;Phone\u0026#34;: 42, \u0026#34;Tablet\u0026#34;: 8, \u0026#34;Monitor\u0026#34;: 20, } print(inventory.get(\u0026#34;Laptop\u0026#34;, 0)) # 15 print(inventory.get(\u0026#34;Printer\u0026#34;, 0)) # 0 -- no error inventory.setdefault(\u0026#34;Printer\u0026#34;, 5) # adds only if key is missing print(inventory[\u0026#34;Printer\u0026#34;]) # 5 inventory.setdefault(\u0026#34;Laptop\u0026#34;, 100) # does NOT overwrite existing key print(inventory[\u0026#34;Laptop\u0026#34;]) # still 15 D3 \u0026ndash; Merging Dictionaries You have sales data from two regions. Merge them into one dictionary. For cities that appear in both, sum the values.\nnorth = {\u0026#34;Delhi\u0026#34;: 450, \u0026#34;Chandigarh\u0026#34;: 120, \u0026#34;Lucknow\u0026#34;: 200} south = {\u0026#34;Chennai\u0026#34;: 380, \u0026#34;Bangalore\u0026#34;: 520, \u0026#34;Delhi\u0026#34;: 150} Expected Output:\n{\u0026#39;Delhi\u0026#39;: 600, \u0026#39;Chandigarh\u0026#39;: 120, \u0026#39;Lucknow\u0026#39;: 200, \u0026#39;Chennai\u0026#39;: 380, \u0026#39;Bangalore\u0026#39;: 520} Solution:\nnorth = {\u0026#34;Delhi\u0026#34;: 450, \u0026#34;Chandigarh\u0026#34;: 120, \u0026#34;Lucknow\u0026#34;: 200} south = {\u0026#34;Chennai\u0026#34;: 380, \u0026#34;Bangalore\u0026#34;: 520, \u0026#34;Delhi\u0026#34;: 150} merged = dict(north) # start with a copy of north for city, sales in south.items(): if city in merged: merged[city] += sales # sum if city already exists else: merged[city] = sales # add new city print(merged) D4 \u0026ndash; Inverting a Dictionary Write a function that inverts a dictionary \u0026ndash; keys become values and values become keys. Handle cases where multiple keys share the same value by grouping them into a list.\ndepartments = { \u0026#34;Aarav\u0026#34;: \u0026#34;Engineering\u0026#34;, \u0026#34;Priya\u0026#34;: \u0026#34;HR\u0026#34;, \u0026#34;Rohan\u0026#34;: \u0026#34;Engineering\u0026#34;, \u0026#34;Sneha\u0026#34;: \u0026#34;Marketing\u0026#34;, \u0026#34;Karan\u0026#34;: \u0026#34;HR\u0026#34;, \u0026#34;Meera\u0026#34;: \u0026#34;Engineering\u0026#34;, } Expected Output:\n{\u0026#39;Engineering\u0026#39;: [\u0026#39;Aarav\u0026#39;, \u0026#39;Rohan\u0026#39;, \u0026#39;Meera\u0026#39;], \u0026#39;HR\u0026#39;: [\u0026#39;Priya\u0026#39;, \u0026#39;Karan\u0026#39;], \u0026#39;Marketing\u0026#39;: [\u0026#39;Sneha\u0026#39;]} Solution:\ndepartments = { \u0026#34;Aarav\u0026#34;: \u0026#34;Engineering\u0026#34;, \u0026#34;Priya\u0026#34;: \u0026#34;HR\u0026#34;, \u0026#34;Rohan\u0026#34;: \u0026#34;Engineering\u0026#34;, \u0026#34;Sneha\u0026#34;: \u0026#34;Marketing\u0026#34;, \u0026#34;Karan\u0026#34;: \u0026#34;HR\u0026#34;, \u0026#34;Meera\u0026#34;: \u0026#34;Engineering\u0026#34;, } inverted = {} for name, dept in departments.items(): if dept not in inverted: inverted[dept] = [] inverted[dept].append(name) print(inverted) D5 \u0026ndash; Nested Dictionary You have a nested dictionary representing student report cards. Write code to:\nPrint Priya\u0026rsquo;s science marks Find the overall class topper (highest average across all subjects) Count how many students scored above 80 in maths report_cards = { \u0026#34;Aarav\u0026#34;: {\u0026#34;maths\u0026#34;: 88, \u0026#34;science\u0026#34;: 92, \u0026#34;english\u0026#34;: 85}, \u0026#34;Priya\u0026#34;: {\u0026#34;maths\u0026#34;: 75, \u0026#34;science\u0026#34;: 68, \u0026#34;english\u0026#34;: 79}, \u0026#34;Rohan\u0026#34;: {\u0026#34;maths\u0026#34;: 55, \u0026#34;science\u0026#34;: 60, \u0026#34;english\u0026#34;: 72}, \u0026#34;Sneha\u0026#34;: {\u0026#34;maths\u0026#34;: 95, \u0026#34;science\u0026#34;: 98, \u0026#34;english\u0026#34;: 91}, } Solution:\nreport_cards = { \u0026#34;Aarav\u0026#34;: {\u0026#34;maths\u0026#34;: 88, \u0026#34;science\u0026#34;: 92, \u0026#34;english\u0026#34;: 85}, \u0026#34;Priya\u0026#34;: {\u0026#34;maths\u0026#34;: 75, \u0026#34;science\u0026#34;: 68, \u0026#34;english\u0026#34;: 79}, \u0026#34;Rohan\u0026#34;: {\u0026#34;maths\u0026#34;: 55, \u0026#34;science\u0026#34;: 60, \u0026#34;english\u0026#34;: 72}, \u0026#34;Sneha\u0026#34;: {\u0026#34;maths\u0026#34;: 95, \u0026#34;science\u0026#34;: 98, \u0026#34;english\u0026#34;: 91}, } # Priya\u0026#39;s science marks print(\u0026#34;Priya\u0026#39;s science:\u0026#34;, report_cards[\u0026#34;Priya\u0026#34;][\u0026#34;science\u0026#34;]) # Class topper by average topper = \u0026#34;\u0026#34; top_avg = 0 for name, subjects in report_cards.items(): avg = sum(subjects.values()) / len(subjects) if avg \u0026gt; top_avg: top_avg = avg topper = name print(f\u0026#34;Topper: {topper} with avg {top_avg:.1f}\u0026#34;) # Count students above 80 in maths above_80 = sum(1 for data in report_cards.values() if data[\u0026#34;maths\u0026#34;] \u0026gt; 80) print(f\u0026#34;Students above 80 in maths: {above_80}\u0026#34;) D6 \u0026ndash; Word Frequency with Dictionary Write a function that takes a sentence and returns a dictionary with each word and how many times it appears. Ignore case. Then print the top 3 most frequent words.\ntext = \u0026#34;to be or not to be that is the question to be is to live\u0026#34; Expected Output:\nTop 3 words: to : 4 be : 3 is : 2 Solution:\ndef word_frequency(text): freq = {} for word in text.lower().split(): freq[word] = freq.get(word, 0) + 1 return freq text = \u0026#34;to be or not to be that is the question to be is to live\u0026#34; freq = word_frequency(text) sorted_freq = sorted(freq.items(), key=lambda x: x[1], reverse=True) print(\u0026#34;Top 3 words:\u0026#34;) for word, count in sorted_freq[:3]: print(f\u0026#34;{word:\u0026lt;4}: {count}\u0026#34;) Part 5 \u0026ndash; Sets A set is an unordered, mutable collection of unique elements. It has no duplicates and no index. Sets shine when you need fast membership checks or mathematical set operations.\nS1 \u0026ndash; Set Basics and Deduplication Given a list with many duplicate values, use a set to find all unique values. Then convert back to a sorted list.\norders = [\u0026#34;Delhi\u0026#34;, \u0026#34;Mumbai\u0026#34;, \u0026#34;Delhi\u0026#34;, \u0026#34;Bangalore\u0026#34;, \u0026#34;Pune\u0026#34;, \u0026#34;Mumbai\u0026#34;, \u0026#34;Delhi\u0026#34;, \u0026#34;Chennai\u0026#34;, \u0026#34;Pune\u0026#34;, \u0026#34;Bangalore\u0026#34;] Expected Output:\nUnique cities (5): [\u0026#39;Bangalore\u0026#39;, \u0026#39;Chennai\u0026#39;, \u0026#39;Delhi\u0026#39;, \u0026#39;Mumbai\u0026#39;, \u0026#39;Pune\u0026#39;] Solution:\norders = [\u0026#34;Delhi\u0026#34;, \u0026#34;Mumbai\u0026#34;, \u0026#34;Delhi\u0026#34;, \u0026#34;Bangalore\u0026#34;, \u0026#34;Pune\u0026#34;, \u0026#34;Mumbai\u0026#34;, \u0026#34;Delhi\u0026#34;, \u0026#34;Chennai\u0026#34;, \u0026#34;Pune\u0026#34;, \u0026#34;Bangalore\u0026#34;] unique = sorted(set(orders)) print(f\u0026#34;Unique cities ({len(unique)}): {unique}\u0026#34;) S2 \u0026ndash; Set Operations (Union, Intersection, Difference) Two cricket teams have been announced. Use set operations to find:\nAll players across both teams (union) Players selected in both teams (intersection) Players in Team A but not Team B (difference) Players in exactly one team but not both (symmetric difference) team_a = {\u0026#34;Rohit\u0026#34;, \u0026#34;Virat\u0026#34;, \u0026#34;Shubman\u0026#34;, \u0026#34;Pant\u0026#34;, \u0026#34;Jadeja\u0026#34;, \u0026#34;Bumrah\u0026#34;, \u0026#34;Siraj\u0026#34;} team_b = {\u0026#34;Rohit\u0026#34;, \u0026#34;Shubman\u0026#34;, \u0026#34;Dhoni\u0026#34;, \u0026#34;Jadeja\u0026#34;, \u0026#34;Ashwin\u0026#34;, \u0026#34;Bumrah\u0026#34;, \u0026#34;Kuldeep\u0026#34;} Solution:\nteam_a = {\u0026#34;Rohit\u0026#34;, \u0026#34;Virat\u0026#34;, \u0026#34;Shubman\u0026#34;, \u0026#34;Pant\u0026#34;, \u0026#34;Jadeja\u0026#34;, \u0026#34;Bumrah\u0026#34;, \u0026#34;Siraj\u0026#34;} team_b = {\u0026#34;Rohit\u0026#34;, \u0026#34;Shubman\u0026#34;, \u0026#34;Dhoni\u0026#34;, \u0026#34;Jadeja\u0026#34;, \u0026#34;Ashwin\u0026#34;, \u0026#34;Bumrah\u0026#34;, \u0026#34;Kuldeep\u0026#34;} print(\u0026#34;All players (union) :\u0026#34;, sorted(team_a | team_b)) print(\u0026#34;In both teams (intersect) :\u0026#34;, sorted(team_a \u0026amp; team_b)) print(\u0026#34;Only in Team A (diff) :\u0026#34;, sorted(team_a - team_b)) print(\u0026#34;Only in Team B (diff) :\u0026#34;, sorted(team_b - team_a)) print(\u0026#34;Exactly one team (sym diff):\u0026#34;, sorted(team_a ^ team_b)) S3 \u0026ndash; Set Methods: add, remove, discard registered = {\u0026#34;Aarav\u0026#34;, \u0026#34;Priya\u0026#34;, \u0026#34;Rohan\u0026#34;, \u0026#34;Sneha\u0026#34;} Add \u0026ldquo;Karan\u0026rdquo; to the set Remove \u0026ldquo;Rohan\u0026rdquo; using remove() \u0026ndash; show what happens if the name is not there Remove \u0026ldquo;Meera\u0026rdquo; using discard() \u0026ndash; show that no error is raised Check if \u0026ldquo;Priya\u0026rdquo; is in the set Solution:\nregistered = {\u0026#34;Aarav\u0026#34;, \u0026#34;Priya\u0026#34;, \u0026#34;Rohan\u0026#34;, \u0026#34;Sneha\u0026#34;} registered.add(\u0026#34;Karan\u0026#34;) print(\u0026#34;After add:\u0026#34;, registered) registered.remove(\u0026#34;Rohan\u0026#34;) print(\u0026#34;After remove:\u0026#34;, registered) # registered.remove(\u0026#34;Meera\u0026#34;) # raises KeyError -- \u0026#34;Meera\u0026#34; is not in set registered.discard(\u0026#34;Meera\u0026#34;) # no error even if \u0026#34;Meera\u0026#34; does not exist print(\u0026#34;After discard:\u0026#34;, registered) print(\u0026#34;Priya in set:\u0026#34;, \u0026#34;Priya\u0026#34; in registered) S4 \u0026ndash; Subset and Superset You have a set of required skills for a job and a set of skills each candidate has. Check whether each candidate qualifies.\nrequired = {\u0026#34;Python\u0026#34;, \u0026#34;SQL\u0026#34;, \u0026#34;Pandas\u0026#34;} candidate_a = {\u0026#34;Python\u0026#34;, \u0026#34;SQL\u0026#34;, \u0026#34;Pandas\u0026#34;, \u0026#34;Excel\u0026#34;, \u0026#34;Tableau\u0026#34;} candidate_b = {\u0026#34;Python\u0026#34;, \u0026#34;SQL\u0026#34;} candidate_c = {\u0026#34;Python\u0026#34;, \u0026#34;SQL\u0026#34;, \u0026#34;Pandas\u0026#34;} candidate_d = {\u0026#34;Java\u0026#34;, \u0026#34;SQL\u0026#34;, \u0026#34;Spring\u0026#34;} Expected Output:\nCandidate A: Qualified (has all required skills + more) Candidate B: Not qualified (missing: {\u0026#39;Pandas\u0026#39;}) Candidate C: Qualified Candidate D: Not qualified (missing: {\u0026#39;Python\u0026#39;, \u0026#39;Pandas\u0026#39;}) Solution:\nrequired = {\u0026#34;Python\u0026#34;, \u0026#34;SQL\u0026#34;, \u0026#34;Pandas\u0026#34;} candidates = { \u0026#34;Candidate A\u0026#34;: {\u0026#34;Python\u0026#34;, \u0026#34;SQL\u0026#34;, \u0026#34;Pandas\u0026#34;, \u0026#34;Excel\u0026#34;, \u0026#34;Tableau\u0026#34;}, \u0026#34;Candidate B\u0026#34;: {\u0026#34;Python\u0026#34;, \u0026#34;SQL\u0026#34;}, \u0026#34;Candidate C\u0026#34;: {\u0026#34;Python\u0026#34;, \u0026#34;SQL\u0026#34;, \u0026#34;Pandas\u0026#34;}, \u0026#34;Candidate D\u0026#34;: {\u0026#34;Java\u0026#34;, \u0026#34;SQL\u0026#34;, \u0026#34;Spring\u0026#34;}, } for name, skills in candidates.items(): missing = required - skills if not missing: print(f\u0026#34;{name}: Qualified\u0026#34;) else: print(f\u0026#34;{name}: Not qualified (missing: {missing})\u0026#34;) S5 \u0026ndash; Frozen Set A frozenset is an immutable set \u0026ndash; useful as a dictionary key or when you want a set that cannot change.\nDemonstrate the difference between set and frozenset and use a frozenset as a dictionary key.\nSolution:\n# Regular set is mutable tags = set([\u0026#34;python\u0026#34;, \u0026#34;data\u0026#34;, \u0026#34;beginner\u0026#34;]) tags.add(\u0026#34;loops\u0026#34;) print(\u0026#34;Set:\u0026#34;, tags) # Frozenset is immutable frozen = frozenset([\u0026#34;python\u0026#34;, \u0026#34;data\u0026#34;, \u0026#34;beginner\u0026#34;]) # frozen.add(\u0026#34;loops\u0026#34;) --\u0026gt; AttributeError: \u0026#39;frozenset\u0026#39; object has no attribute \u0026#39;add\u0026#39; # frozenset can be used as a dictionary key (regular set cannot) course_levels = { frozenset([\u0026#34;python\u0026#34;, \u0026#34;beginner\u0026#34;]): \u0026#34;Python 101\u0026#34;, frozenset([\u0026#34;python\u0026#34;, \u0026#34;intermediate\u0026#34;]): \u0026#34;Python OOP\u0026#34;, frozenset([\u0026#34;sql\u0026#34;, \u0026#34;beginner\u0026#34;]): \u0026#34;SQL Basics\u0026#34;, } search = frozenset([\u0026#34;python\u0026#34;, \u0026#34;beginner\u0026#34;]) print(course_levels[search]) # Python 101 Part 6 \u0026ndash; Mixed Questions These questions combine multiple collections with loops, conditionals, and comprehensions.\nM1 \u0026ndash; Frequency Counter (List + Dictionary + Loop) Given a list of exam scores, write a program that:\nCounts how many students fall in each grade band Prints the result sorted from highest grade band to lowest scores = [88, 45, 72, 95, 60, 55, 83, 91, 38, 74, 66, 49, 87, 100, 52] Expected Output:\nDistinction (\u0026gt;=85) : 4 First Class (\u0026gt;=60) : 5 Pass (\u0026gt;=40) : 4 Fail (\u0026lt;40) : 1 Solution:\nscores = [88, 45, 72, 95, 60, 55, 83, 91, 38, 74, 66, 49, 87, 100, 52] bands = {\u0026#34;Distinction\u0026#34;: 0, \u0026#34;First Class\u0026#34;: 0, \u0026#34;Pass\u0026#34;: 0, \u0026#34;Fail\u0026#34;: 0} for s in scores: if s \u0026gt;= 85: bands[\u0026#34;Distinction\u0026#34;] += 1 elif s \u0026gt;= 60: bands[\u0026#34;First Class\u0026#34;] += 1 elif s \u0026gt;= 40: bands[\u0026#34;Pass\u0026#34;] += 1 else: bands[\u0026#34;Fail\u0026#34;] += 1 for band, count in bands.items(): print(f\u0026#34;{band:\u0026lt;12}: {count}\u0026#34;) M2 \u0026ndash; Anagram Checker (String + Set + Dictionary) Write a function are_anagrams(s1, s2) that returns True if two strings are anagrams of each other (same letters, different order). Ignore spaces and case.\nare_anagrams(\u0026#34;listen\u0026#34;, \u0026#34;silent\u0026#34;) # True are_anagrams(\u0026#34;Triangle\u0026#34;, \u0026#34;Integral\u0026#34;) # True are_anagrams(\u0026#34;hello\u0026#34;, \u0026#34;world\u0026#34;) # False are_anagrams(\u0026#34;Astronomer\u0026#34;, \u0026#34;Moon starer\u0026#34;) # True (ignore spaces) Solution:\ndef are_anagrams(s1, s2): # Remove spaces and convert to lowercase clean1 = s1.replace(\u0026#34; \u0026#34;, \u0026#34;\u0026#34;).lower() clean2 = s2.replace(\u0026#34; \u0026#34;, \u0026#34;\u0026#34;).lower() # Build frequency dictionaries freq1 = {} for char in clean1: freq1[char] = freq1.get(char, 0) + 1 freq2 = {} for char in clean2: freq2[char] = freq2.get(char, 0) + 1 return freq1 == freq2 print(are_anagrams(\u0026#34;listen\u0026#34;, \u0026#34;silent\u0026#34;)) # True print(are_anagrams(\u0026#34;Triangle\u0026#34;, \u0026#34;Integral\u0026#34;)) # True print(are_anagrams(\u0026#34;hello\u0026#34;, \u0026#34;world\u0026#34;)) # False print(are_anagrams(\u0026#34;Astronomer\u0026#34;, \u0026#34;Moon starer\u0026#34;)) # True M3 \u0026ndash; Group Transactions (List of Dicts + Nested Dict + Loop) Given a list of bank transactions, group them by category and compute:\nTotal spent per category Number of transactions per category Average transaction per category transactions = [ {\u0026#34;desc\u0026#34;: \u0026#34;Zomato order\u0026#34;, \u0026#34;amount\u0026#34;: 450, \u0026#34;category\u0026#34;: \u0026#34;Food\u0026#34;}, {\u0026#34;desc\u0026#34;: \u0026#34;Metro recharge\u0026#34;, \u0026#34;amount\u0026#34;: 200, \u0026#34;category\u0026#34;: \u0026#34;Travel\u0026#34;}, {\u0026#34;desc\u0026#34;: \u0026#34;Amazon purchase\u0026#34;, \u0026#34;amount\u0026#34;: 1800, \u0026#34;category\u0026#34;: \u0026#34;Shopping\u0026#34;}, {\u0026#34;desc\u0026#34;: \u0026#34;Swiggy order\u0026#34;, \u0026#34;amount\u0026#34;: 380, \u0026#34;category\u0026#34;: \u0026#34;Food\u0026#34;}, {\u0026#34;desc\u0026#34;: \u0026#34;Uber ride\u0026#34;, \u0026#34;amount\u0026#34;: 320, \u0026#34;category\u0026#34;: \u0026#34;Travel\u0026#34;}, {\u0026#34;desc\u0026#34;: \u0026#34;Grocery store\u0026#34;, \u0026#34;amount\u0026#34;: 950, \u0026#34;category\u0026#34;: \u0026#34;Food\u0026#34;}, {\u0026#34;desc\u0026#34;: \u0026#34;Flipkart order\u0026#34;, \u0026#34;amount\u0026#34;: 2200, \u0026#34;category\u0026#34;: \u0026#34;Shopping\u0026#34;}, {\u0026#34;desc\u0026#34;: \u0026#34;OLA ride\u0026#34;, \u0026#34;amount\u0026#34;: 180, \u0026#34;category\u0026#34;: \u0026#34;Travel\u0026#34;}, {\u0026#34;desc\u0026#34;: \u0026#34;Restaurant\u0026#34;, \u0026#34;amount\u0026#34;: 1100, \u0026#34;category\u0026#34;: \u0026#34;Food\u0026#34;}, ] Expected Output:\nCategory | Transactions | Total Spent | Avg per Transaction Food | 4 | Rs.2880 | Rs.720.0 Travel | 3 | Rs.700 | Rs.233.33 Shopping | 2 | Rs.4000 | Rs.2000.0 Solution:\ntransactions = [ {\u0026#34;desc\u0026#34;: \u0026#34;Zomato order\u0026#34;, \u0026#34;amount\u0026#34;: 450, \u0026#34;category\u0026#34;: \u0026#34;Food\u0026#34;}, {\u0026#34;desc\u0026#34;: \u0026#34;Metro recharge\u0026#34;, \u0026#34;amount\u0026#34;: 200, \u0026#34;category\u0026#34;: \u0026#34;Travel\u0026#34;}, {\u0026#34;desc\u0026#34;: \u0026#34;Amazon purchase\u0026#34;, \u0026#34;amount\u0026#34;: 1800, \u0026#34;category\u0026#34;: \u0026#34;Shopping\u0026#34;}, {\u0026#34;desc\u0026#34;: \u0026#34;Swiggy order\u0026#34;, \u0026#34;amount\u0026#34;: 380, \u0026#34;category\u0026#34;: \u0026#34;Food\u0026#34;}, {\u0026#34;desc\u0026#34;: \u0026#34;Uber ride\u0026#34;, \u0026#34;amount\u0026#34;: 320, \u0026#34;category\u0026#34;: \u0026#34;Travel\u0026#34;}, {\u0026#34;desc\u0026#34;: \u0026#34;Grocery store\u0026#34;, \u0026#34;amount\u0026#34;: 950, \u0026#34;category\u0026#34;: \u0026#34;Food\u0026#34;}, {\u0026#34;desc\u0026#34;: \u0026#34;Flipkart order\u0026#34;, \u0026#34;amount\u0026#34;: 2200, \u0026#34;category\u0026#34;: \u0026#34;Shopping\u0026#34;}, {\u0026#34;desc\u0026#34;: \u0026#34;OLA ride\u0026#34;, \u0026#34;amount\u0026#34;: 180, \u0026#34;category\u0026#34;: \u0026#34;Travel\u0026#34;}, {\u0026#34;desc\u0026#34;: \u0026#34;Restaurant\u0026#34;, \u0026#34;amount\u0026#34;: 1100, \u0026#34;category\u0026#34;: \u0026#34;Food\u0026#34;}, ] summary = {} for t in transactions: cat = t[\u0026#34;category\u0026#34;] if cat not in summary: summary[cat] = {\u0026#34;count\u0026#34;: 0, \u0026#34;total\u0026#34;: 0} summary[cat][\u0026#34;count\u0026#34;] += 1 summary[cat][\u0026#34;total\u0026#34;] += t[\u0026#34;amount\u0026#34;] print(f\u0026#34;{\u0026#39;Category\u0026#39;:\u0026lt;10} | {\u0026#39;Txns\u0026#39;:\u0026gt;5} | {\u0026#39;Total\u0026#39;:\u0026gt;12} | {\u0026#39;Avg\u0026#39;:\u0026gt;10}\u0026#34;) print(\u0026#34;-\u0026#34; * 50) for cat, data in summary.items(): avg = round(data[\u0026#34;total\u0026#34;] / data[\u0026#34;count\u0026#34;], 2) print(f\u0026#34;{cat:\u0026lt;10} | {data[\u0026#39;count\u0026#39;]:\u0026gt;5} | Rs.{data[\u0026#39;total\u0026#39;]:\u0026gt;9} | Rs.{avg:\u0026gt;9}\u0026#34;) M4 \u0026ndash; Find Common Elements Across Multiple Lists (Set + Loop) Given results from three different quiz rounds, find:\nStudents who passed all three rounds Students who passed exactly two rounds Students who passed only one round round1 = [\u0026#34;Aarav\u0026#34;, \u0026#34;Priya\u0026#34;, \u0026#34;Rohan\u0026#34;, \u0026#34;Sneha\u0026#34;, \u0026#34;Karan\u0026#34;, \u0026#34;Meera\u0026#34;] round2 = [\u0026#34;Priya\u0026#34;, \u0026#34;Sneha\u0026#34;, \u0026#34;Karan\u0026#34;, \u0026#34;Arjun\u0026#34;, \u0026#34;Divya\u0026#34;] round3 = [\u0026#34;Aarav\u0026#34;, \u0026#34;Priya\u0026#34;, \u0026#34;Sneha\u0026#34;, \u0026#34;Arjun\u0026#34;, \u0026#34;Vikram\u0026#34;] Solution:\nround1 = set([\u0026#34;Aarav\u0026#34;, \u0026#34;Priya\u0026#34;, \u0026#34;Rohan\u0026#34;, \u0026#34;Sneha\u0026#34;, \u0026#34;Karan\u0026#34;, \u0026#34;Meera\u0026#34;]) round2 = set([\u0026#34;Priya\u0026#34;, \u0026#34;Sneha\u0026#34;, \u0026#34;Karan\u0026#34;, \u0026#34;Arjun\u0026#34;, \u0026#34;Divya\u0026#34;]) round3 = set([\u0026#34;Aarav\u0026#34;, \u0026#34;Priya\u0026#34;, \u0026#34;Sneha\u0026#34;, \u0026#34;Arjun\u0026#34;, \u0026#34;Vikram\u0026#34;]) all_three = round1 \u0026amp; round2 \u0026amp; round3 exactly_two = ( (round1 \u0026amp; round2) | (round2 \u0026amp; round3) | (round1 \u0026amp; round3) ) - all_three all_students = round1 | round2 | round3 only_one = all_students - (round1 \u0026amp; round2) - (round2 \u0026amp; round3) - (round1 \u0026amp; round3) print(\u0026#34;Passed all three :\u0026#34;, sorted(all_three)) print(\u0026#34;Passed exactly two:\u0026#34;, sorted(exactly_two)) print(\u0026#34;Passed only one :\u0026#34;, sorted(only_one)) M5 \u0026ndash; Top N with Tuples and Sorting (List + Tuple + Sort) Given a list of product sales tuples, write a function top_n(sales, n) that returns the top n products by revenue. Each tuple is (product_name, units_sold, price_per_unit).\nsales = [ (\u0026#34;Laptop\u0026#34;, 120, 65000), (\u0026#34;Phone\u0026#34;, 450, 18000), (\u0026#34;Tablet\u0026#34;, 210, 32000), (\u0026#34;Headphones\u0026#34;,780, 2500), (\u0026#34;Monitor\u0026#34;, 95, 22000), (\u0026#34;Keyboard\u0026#34;, 640, 1200), (\u0026#34;Mouse\u0026#34;, 900, 800), ] Expected Output for top_n(sales, 3):\nRank 1: Phone -- Revenue: Rs.81,00,000 (450 units) Rank 2: Laptop -- Revenue: Rs.78,00,000 (120 units) Rank 3: Tablet -- Revenue: Rs.67,20,000 (210 units) Solution:\nsales = [ (\u0026#34;Laptop\u0026#34;, 120, 65000), (\u0026#34;Phone\u0026#34;, 450, 18000), (\u0026#34;Tablet\u0026#34;, 210, 32000), (\u0026#34;Headphones\u0026#34;,780, 2500), (\u0026#34;Monitor\u0026#34;, 95, 22000), (\u0026#34;Keyboard\u0026#34;, 640, 1200), (\u0026#34;Mouse\u0026#34;, 900, 800), ] def top_n(sales, n): # Create list of (name, revenue, units) tuples with_revenue = [] for name, units, price in sales: revenue = units * price with_revenue.append((name, revenue, units)) # Sort by revenue descending with_revenue.sort(key=lambda x: x[1], reverse=True) for rank, (name, revenue, units) in enumerate(with_revenue[:n], start=1): print(f\u0026#34;Rank {rank}: {name:\u0026lt;12} -- Revenue: Rs.{revenue:,} ({units} units)\u0026#34;) top_n(sales, 3) M6 \u0026ndash; Matrix Transpose Using Lists (Nested List + Comprehension) Write a function transpose(matrix) that returns the transpose of a 2D list (rows become columns and columns become rows). Do it in two ways: with a loop and with a list comprehension.\nmatrix = [ [1, 2, 3], [4, 5, 6], [7, 8, 9], ] # Expected transpose: # [[1, 4, 7], # [2, 5, 8], # [3, 6, 9]] Solution:\nmatrix = [ [1, 2, 3], [4, 5, 6], [7, 8, 9], ] # Method 1: loop def transpose_loop(matrix): rows = len(matrix) cols = len(matrix[0]) result = [[0] * rows for _ in range(cols)] for i in range(rows): for j in range(cols): result[j][i] = matrix[i][j] return result # Method 2: list comprehension def transpose_comprehension(matrix): return [[row[i] for row in matrix] for i in range(len(matrix[0]))] print(transpose_loop(matrix)) print(transpose_comprehension(matrix)) M7 \u0026ndash; Inventory Manager (Dict + List + Conditional + Loop) You manage a small shop. Write a program that:\nStarts with an inventory dictionary Processes a list of sales Reduces stock after each sale Prints a \u0026ldquo;LOW STOCK\u0026rdquo; warning if quantity falls below 5 Prints \u0026ldquo;OUT OF STOCK\u0026rdquo; and skips the sale if quantity is 0 inventory = { \u0026#34;Pen\u0026#34;: 50, \u0026#34;Notebook\u0026#34;: 20, \u0026#34;Eraser\u0026#34;: 8, \u0026#34;Ruler\u0026#34;: 3, \u0026#34;Compass\u0026#34;: 0, } sales = [ (\u0026#34;Pen\u0026#34;, 12), (\u0026#34;Notebook\u0026#34;, 5), (\u0026#34;Compass\u0026#34;, 1), (\u0026#34;Eraser\u0026#34;, 5), (\u0026#34;Ruler\u0026#34;, 2), (\u0026#34;Pen\u0026#34;, 40), (\u0026#34;Notebook\u0026#34;, 18), ] Solution:\ninventory = { \u0026#34;Pen\u0026#34;: 50, \u0026#34;Notebook\u0026#34;: 20, \u0026#34;Eraser\u0026#34;: 8, \u0026#34;Ruler\u0026#34;: 3, \u0026#34;Compass\u0026#34;: 0, } sales = [ (\u0026#34;Pen\u0026#34;, 12), (\u0026#34;Notebook\u0026#34;, 5), (\u0026#34;Compass\u0026#34;, 1), (\u0026#34;Eraser\u0026#34;, 5), (\u0026#34;Ruler\u0026#34;, 2), (\u0026#34;Pen\u0026#34;, 40), (\u0026#34;Notebook\u0026#34;, 18), ] for item, qty in sales: if inventory.get(item, 0) == 0: print(f\u0026#34;OUT OF STOCK : {item} -- sale of {qty} skipped\u0026#34;) continue if qty \u0026gt; inventory[item]: print(f\u0026#34;INSUFFICIENT : Only {inventory[item]} {item}(s) left, cannot sell {qty}\u0026#34;) continue inventory[item] -= qty remaining = inventory[item] if remaining \u0026lt; 5: print(f\u0026#34;Sold {qty} {item}(s). Remaining: {remaining} [LOW STOCK]\u0026#34;) else: print(f\u0026#34;Sold {qty} {item}(s). Remaining: {remaining}\u0026#34;) print(\u0026#34;\\nFinal Inventory:\u0026#34;) for item, qty in inventory.items(): print(f\u0026#34; {item:\u0026lt;10}: {qty}\u0026#34;) M8 \u0026ndash; Caesar Cipher (String + List + Dictionary + Loop) Write a function caesar_encrypt(text, shift) and caesar_decrypt(text, shift) that encrypts and decrypts text using the Caesar cipher. Only shift letters, leave spaces and punctuation unchanged.\nmessage = \u0026#34;Attack at Dawn\u0026#34; shift = 3 encrypted = caesar_encrypt(message, shift) decrypted = caesar_decrypt(encrypted, shift) # encrypted: \u0026#34;Dwwdfn dw Gdzq\u0026#34; # decrypted: \u0026#34;Attack at Dawn\u0026#34; Solution:\ndef caesar_encrypt(text, shift): result = [] for char in text: if char.isalpha(): base = ord(\u0026#34;A\u0026#34;) if char.isupper() else ord(\u0026#34;a\u0026#34;) shifted = chr((ord(char) - base + shift) % 26 + base) result.append(shifted) else: result.append(char) return \u0026#34;\u0026#34;.join(result) def caesar_decrypt(text, shift): return caesar_encrypt(text, -shift) message = \u0026#34;Attack at Dawn\u0026#34; shift = 3 encrypted = caesar_encrypt(message, shift) decrypted = caesar_decrypt(encrypted, shift) print(f\u0026#34;Original : {message}\u0026#34;) print(f\u0026#34;Encrypted: {encrypted}\u0026#34;) print(f\u0026#34;Decrypted: {decrypted}\u0026#34;) M9 \u0026ndash; Student Ranking System (Full Pipeline) Given a list of student records, write a complete program that:\nCalculates total and percentage for each student Assigns grade: \u0026ldquo;Distinction\u0026rdquo; (\u0026gt;=85%), \u0026ldquo;First Class\u0026rdquo; (\u0026gt;=60%), \u0026ldquo;Pass\u0026rdquo; (\u0026gt;=40%), \u0026ldquo;Fail\u0026rdquo; (\u0026lt;40%) Ranks students from topper to lowest Identifies students who improved from their previous percentage (given separately) Prints a complete ranked report students = [ {\u0026#34;name\u0026#34;: \u0026#34;Aarav\u0026#34;, \u0026#34;maths\u0026#34;: 88, \u0026#34;science\u0026#34;: 92, \u0026#34;english\u0026#34;: 85, \u0026#34;history\u0026#34;: 78, \u0026#34;cs\u0026#34;: 95}, {\u0026#34;name\u0026#34;: \u0026#34;Priya\u0026#34;, \u0026#34;maths\u0026#34;: 55, \u0026#34;science\u0026#34;: 48, \u0026#34;english\u0026#34;: 62, \u0026#34;history\u0026#34;: 50, \u0026#34;cs\u0026#34;: 58}, {\u0026#34;name\u0026#34;: \u0026#34;Rohan\u0026#34;, \u0026#34;maths\u0026#34;: 72, \u0026#34;science\u0026#34;: 68, \u0026#34;english\u0026#34;: 75, \u0026#34;history\u0026#34;: 70, \u0026#34;cs\u0026#34;: 65}, {\u0026#34;name\u0026#34;: \u0026#34;Sneha\u0026#34;, \u0026#34;maths\u0026#34;: 95, \u0026#34;science\u0026#34;: 98, \u0026#34;english\u0026#34;: 92, \u0026#34;history\u0026#34;: 88, \u0026#34;cs\u0026#34;: 97}, {\u0026#34;name\u0026#34;: \u0026#34;Karan\u0026#34;, \u0026#34;maths\u0026#34;: 35, \u0026#34;science\u0026#34;: 40, \u0026#34;english\u0026#34;: 38, \u0026#34;history\u0026#34;: 42, \u0026#34;cs\u0026#34;: 30}, {\u0026#34;name\u0026#34;: \u0026#34;Meera\u0026#34;, \u0026#34;maths\u0026#34;: 78, \u0026#34;science\u0026#34;: 82, \u0026#34;english\u0026#34;: 80, \u0026#34;history\u0026#34;: 75, \u0026#34;cs\u0026#34;: 84}, ] previous_pct = { \u0026#34;Aarav\u0026#34;: 80.0, \u0026#34;Priya\u0026#34;: 60.0, \u0026#34;Rohan\u0026#34;: 65.0, \u0026#34;Sneha\u0026#34;: 91.0, \u0026#34;Karan\u0026#34;: 42.0, \u0026#34;Meera\u0026#34;: 75.0, } Solution:\nstudents = [ {\u0026#34;name\u0026#34;: \u0026#34;Aarav\u0026#34;, \u0026#34;maths\u0026#34;: 88, \u0026#34;science\u0026#34;: 92, \u0026#34;english\u0026#34;: 85, \u0026#34;history\u0026#34;: 78, \u0026#34;cs\u0026#34;: 95}, {\u0026#34;name\u0026#34;: \u0026#34;Priya\u0026#34;, \u0026#34;maths\u0026#34;: 55, \u0026#34;science\u0026#34;: 48, \u0026#34;english\u0026#34;: 62, \u0026#34;history\u0026#34;: 50, \u0026#34;cs\u0026#34;: 58}, {\u0026#34;name\u0026#34;: \u0026#34;Rohan\u0026#34;, \u0026#34;maths\u0026#34;: 72, \u0026#34;science\u0026#34;: 68, \u0026#34;english\u0026#34;: 75, \u0026#34;history\u0026#34;: 70, \u0026#34;cs\u0026#34;: 65}, {\u0026#34;name\u0026#34;: \u0026#34;Sneha\u0026#34;, \u0026#34;maths\u0026#34;: 95, \u0026#34;science\u0026#34;: 98, \u0026#34;english\u0026#34;: 92, \u0026#34;history\u0026#34;: 88, \u0026#34;cs\u0026#34;: 97}, {\u0026#34;name\u0026#34;: \u0026#34;Karan\u0026#34;, \u0026#34;maths\u0026#34;: 35, \u0026#34;science\u0026#34;: 40, \u0026#34;english\u0026#34;: 38, \u0026#34;history\u0026#34;: 42, \u0026#34;cs\u0026#34;: 30}, {\u0026#34;name\u0026#34;: \u0026#34;Meera\u0026#34;, \u0026#34;maths\u0026#34;: 78, \u0026#34;science\u0026#34;: 82, \u0026#34;english\u0026#34;: 80, \u0026#34;history\u0026#34;: 75, \u0026#34;cs\u0026#34;: 84}, ] previous_pct = { \u0026#34;Aarav\u0026#34;: 80.0, \u0026#34;Priya\u0026#34;: 60.0, \u0026#34;Rohan\u0026#34;: 65.0, \u0026#34;Sneha\u0026#34;: 91.0, \u0026#34;Karan\u0026#34;: 42.0, \u0026#34;Meera\u0026#34;: 75.0, } subjects = [\u0026#34;maths\u0026#34;, \u0026#34;science\u0026#34;, \u0026#34;english\u0026#34;, \u0026#34;history\u0026#34;, \u0026#34;cs\u0026#34;] max_marks = 500 processed = [] for s in students: total = sum(s[sub] for sub in subjects) pct = round((total / max_marks) * 100, 1) if pct \u0026gt;= 85: grade = \u0026#34;Distinction\u0026#34; elif pct \u0026gt;= 60: grade = \u0026#34;First Class\u0026#34; elif pct \u0026gt;= 40: grade = \u0026#34;Pass\u0026#34; else: grade = \u0026#34;Fail\u0026#34; improved = pct \u0026gt; previous_pct.get(s[\u0026#34;name\u0026#34;], 0) processed.append((s[\u0026#34;name\u0026#34;], total, pct, grade, improved)) # Sort by percentage descending processed.sort(key=lambda x: x[2], reverse=True) print(\u0026#34;=\u0026#34; * 65) print(f\u0026#34;{\u0026#39;Rank\u0026#39;:\u0026lt;5} {\u0026#39;Name\u0026#39;:\u0026lt;8} {\u0026#39;Total\u0026#39;:\u0026gt;6} {\u0026#39;Pct\u0026#39;:\u0026gt;7} {\u0026#39;Grade\u0026#39;:\u0026lt;13} {\u0026#39;Trend\u0026#39;}\u0026#34;) print(\u0026#34;=\u0026#34; * 65) for rank, (name, total, pct, grade, improved) in enumerate(processed, start=1): trend = \u0026#34;Up\u0026#34; if improved else \u0026#34;Down\u0026#34; print(f\u0026#34;{rank:\u0026lt;5} {name:\u0026lt;8} {total:\u0026gt;6}/500 {pct:\u0026gt;6}% {grade:\u0026lt;13} {trend}\u0026#34;) print(\u0026#34;=\u0026#34; * 65) Quick Reference String s = \u0026#34;hello\u0026#34; s[0] # index s[1:4] # slice s[::-1] # reverse len(s) # length s.upper() # uppercase s.lower() # lowercase s.strip() # remove leading/trailing whitespace s.replace(\u0026#34;l\u0026#34;, \u0026#34;r\u0026#34;) # replace occurrences s.split(\u0026#34; \u0026#34;) # split into list \u0026#34; \u0026#34;.join([\u0026#34;a\u0026#34;, \u0026#34;b\u0026#34;]) # join list into string s.startswith(\u0026#34;he\u0026#34;) # check prefix s.endswith(\u0026#34;lo\u0026#34;) # check suffix s.count(\u0026#34;l\u0026#34;) # count occurrences \u0026#34;l\u0026#34; in s # membership check s.isalpha() # all letters? s.isdigit() # all digits? s.isalnum() # all letters or digits? List lst = [1, 2, 3] lst.append(4) # add to end lst.insert(1, 99) # insert at index lst.remove(2) # remove first occurrence lst.pop() # remove and return last lst.pop(0) # remove and return at index lst.sort() # sort in place lst.reverse() # reverse in place lst.index(99) # find index of value lst.count(3) # count occurrences lst.copy() # shallow copy lst[1:3] # slicing lst[::-1] # reversed copy [x*2 for x in lst] # list comprehension Tuple t = (1, 2, 3) t[0] # access (no modify) a, b, c = t # unpack t.count(1) # count occurrences t.index(2) # find index t1 + t2 # concatenate (new tuple) Dictionary d = {\u0026#34;a\u0026#34;: 1} d[\u0026#34;a\u0026#34;] # access d[\u0026#34;b\u0026#34;] = 2 # add / update del d[\u0026#34;a\u0026#34;] # delete d.get(\u0026#34;x\u0026#34;, 0) # safe access with default d.setdefault(\u0026#34;y\u0026#34;, 5) # add only if missing d.keys() # all keys d.values() # all values d.items() # all key-value pairs d.pop(\u0026#34;a\u0026#34;) # remove and return {k: v for k, v in d.items()} # dict comprehension Set s = {1, 2, 3} s.add(4) # add element s.remove(2) # remove (error if missing) s.discard(9) # remove (no error) s.pop() # remove arbitrary element s1 | s2 # union s1 \u0026amp; s2 # intersection s1 - s2 # difference s1 ^ s2 # symmetric difference s1.issubset(s2) # check subset s1.issuperset(s2) # check superset frozenset(s) # immutable version When to Use Which Collection Use when String Text data, characters, immutable sequences List Order matters, duplicates allowed, items change Tuple Fixed data, function return values, dict keys Dictionary Fast lookup by key, key-value relationships Set Uniqueness, fast membership check, set math Made with care for Codeverra learners | codeverra.com\n","permalink":"https://blogs.codeverra.com/blog/python-06-collections-practice/","summary":"\u003ch1 id=\"python-collections---practice-questions-and-solutions\"\u003ePython Collections - Practice Questions and Solutions\u003c/h1\u003e\n\u003ch3 id=\"list-tuple-dictionary-set--complete-practice-set\"\u003eList, Tuple, Dictionary, Set \u0026ndash; Complete Practice Set\u003c/h3\u003e\n\u003chr\u003e\n\u003cblockquote\u003e\n\u003cp\u003eThis practice set covers strings and all four major Python collections.\nQuestions are grouped by collection type, then by difficulty within each section.\nThe final section has mixed questions that combine multiple collections, loops, and conditionals.\nTry each question before reading the solution.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003chr\u003e\n\u003ch2 id=\"table-of-contents\"\u003eTable of Contents\u003c/h2\u003e\n\u003cul\u003e\n\u003cli\u003e\u003ca href=\"#part-1--lists\"\u003ePart 1 \u0026ndash; Lists\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#part-2--strings\"\u003ePart 2 \u0026ndash; Strings\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#part-3--tuples\"\u003ePart 3 \u0026ndash; Tuples\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#part-4--dictionaries\"\u003ePart 4 \u0026ndash; Dictionaries\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#part-5--sets\"\u003ePart 5 \u0026ndash; Sets\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#part-6--mixed-questions\"\u003ePart 6 \u0026ndash; Mixed Questions\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#quick-reference\"\u003eQuick Reference\u003c/a\u003e\u003c/li\u003e\n\u003c/ul\u003e\n\u003chr\u003e\n\u003ch2 id=\"part-1--lists\"\u003ePart 1 \u0026ndash; Lists\u003c/h2\u003e\n\u003cp\u003eA list is an \u003cstrong\u003eordered, mutable\u003c/strong\u003e collection. It allows duplicate values and supports indexing, slicing, appending, and sorting.\u003c/p\u003e","tags":["python"],"title":"Master Collections in Python"},{"content":"Python Loops - Practice Questions and Solutions 20 Questions from Easy to Hard All 20 questions are based on the Loops Masterclass. Try each question on your own before reading the solution. The questions are ordered by difficulty: Easy (1-7), Medium (8-14), Hard (15-20).\nTable of Contents Easy Questions (1-7) Medium Questions (8-14) Hard Questions (15-20) Easy Questions (1-7) Q1 \u0026ndash; Basic for Loop with range() Print all numbers from 1 to 30 that are divisible by both 3 and 5.\nExpected Output:\n15 30 Solution:\nfor i in range(1, 31): if i % 3 == 0 and i % 5 == 0: print(i) What this tests: range(), for loop, compound condition with and.\nQ2 \u0026ndash; String Loop Write a loop that counts the number of uppercase letters in the string below.\ntext = \u0026#34;Welcome To India, The Land Of DiversiTy\u0026#34; Expected Output:\nUppercase letters: 6 Solution:\ntext = \u0026#34;Welcome To India, The Land Of DiversiTy\u0026#34; count = 0 for char in text: if char.isupper(): count += 1 print(f\u0026#34;Uppercase letters: {count}\u0026#34;) What this tests: Iterating over a string character by character, .isupper() method, accumulator pattern.\nQ3 \u0026ndash; List Loop with Condition Print only the names from the list below that have more than 4 characters.\nnames = [\u0026#34;Ali\u0026#34;, \u0026#34;Priya\u0026#34;, \u0026#34;Raj\u0026#34;, \u0026#34;Sneha\u0026#34;, \u0026#34;Om\u0026#34;, \u0026#34;Vikram\u0026#34;, \u0026#34;Ria\u0026#34;] Expected Output:\nPriya Sneha Vikram Solution:\nnames = [\u0026#34;Ali\u0026#34;, \u0026#34;Priya\u0026#34;, \u0026#34;Raj\u0026#34;, \u0026#34;Sneha\u0026#34;, \u0026#34;Om\u0026#34;, \u0026#34;Vikram\u0026#34;, \u0026#34;Ria\u0026#34;] for name in names: if len(name) \u0026gt; 4: print(name) What this tests: Looping over a list, len() inside a condition.\nQ4 \u0026ndash; while Loop with Division Write a while loop that keeps dividing a number by 2 until it goes below 1. Print the value at each step.\nnumber = 64 Expected Output:\n64 32.0 16.0 8.0 4.0 2.0 1.0 Solution:\nnumber = 64 while number \u0026gt;= 1: print(number) number = number / 2 What this tests: while loop, updating the loop variable each iteration, float division.\nQ5 \u0026ndash; Dictionary Loop Print each item name and its price after a 15% discount. Round to 2 decimal places.\nmenu = { \u0026#34;Masala Chai\u0026#34;: 20, \u0026#34;Vada Pav\u0026#34;: 15, \u0026#34;Samosa\u0026#34;: 10, \u0026#34;Filter Coffee\u0026#34;: 25, \u0026#34;Idli\u0026#34;: 30, } Expected Output:\nMasala Chai : Rs.17.0 Vada Pav : Rs.12.75 Samosa : Rs.8.5 Filter Coffee : Rs.21.25 Idli : Rs.25.5 Solution:\nmenu = { \u0026#34;Masala Chai\u0026#34;: 20, \u0026#34;Vada Pav\u0026#34;: 15, \u0026#34;Samosa\u0026#34;: 10, \u0026#34;Filter Coffee\u0026#34;: 25, \u0026#34;Idli\u0026#34;: 30, } for item, price in menu.items(): discounted = round(price * 0.85, 2) print(f\u0026#34;{item} : Rs.{discounted}\u0026#34;) What this tests: .items() to loop over a dictionary, arithmetic inside a loop, f-strings.\nQ6 \u0026ndash; Countdown with while Write a countdown timer from 10 to 1, then print \u0026ldquo;Happy New Year!\u0026rdquo; at the end.\nExpected Output:\n10 9 8 7 6 5 4 3 2 1 Happy New Year! Solution:\ncount = 10 while count \u0026gt; 0: print(count) count -= 1 print(\u0026#34;Happy New Year!\u0026#34;) What this tests: while loop with decrement, code outside the loop runs once.\nQ7 \u0026ndash; Sum with for and range() Calculate the sum of all integers from 1 to 100 using a for loop. Do not use the sum() function.\nExpected Output:\nSum from 1 to 100: 5050 Solution:\ntotal = 0 for i in range(1, 101): total += i print(f\u0026#34;Sum from 1 to 100: {total}\u0026#34;) What this tests: Accumulator pattern, range() with both start and stop.\nMedium Questions (8-14) Q8 \u0026ndash; enumerate() with Logic Given a list of cricket scores for each ball, use enumerate() to:\nPrint each ball number (starting from 1) with its score Print the ball number on which the highest score was hit ball_scores = [1, 0, 4, 6, 1, 2, 6, 0, 4, 3] Expected Output:\nBall 1: 1 run Ball 2: 0 runs Ball 3: 4 runs Ball 4: 6 runs Ball 5: 1 run Ball 6: 2 runs Ball 7: 6 runs Ball 8: 0 runs Ball 9: 4 runs Ball 10: 3 runs Highest score of 6 was first hit on Ball 4 Solution:\nball_scores = [1, 0, 4, 6, 1, 2, 6, 0, 4, 3] for ball_num, score in enumerate(ball_scores, start=1): label = \u0026#34;run\u0026#34; if score == 1 else \u0026#34;runs\u0026#34; print(f\u0026#34;Ball {ball_num}: {score} {label}\u0026#34;) max_score = max(ball_scores) for ball_num, score in enumerate(ball_scores, start=1): if score == max_score: print(f\u0026#34;\\nHighest score of {max_score} was first hit on Ball {ball_num}\u0026#34;) break What this tests: enumerate() with custom start, finding max + its position, break to stop after first match.\nQ9 \u0026ndash; zip() and Comparison You have two lists: last year\u0026rsquo;s sales and this year\u0026rsquo;s sales for 6 cities. Use zip() to print, for each city, whether sales went up, went down, or stayed the same.\ncities = [\u0026#34;Delhi\u0026#34;, \u0026#34;Mumbai\u0026#34;, \u0026#34;Bangalore\u0026#34;, \u0026#34;Chennai\u0026#34;, \u0026#34;Hyderabad\u0026#34;, \u0026#34;Pune\u0026#34;] last_year = [120, 145, 98, 110, 85, 76] this_year = [135, 140, 115, 110, 92, 88] Expected Output:\nDelhi : Up (120 -\u0026gt; 135) Mumbai : Down (145 -\u0026gt; 140) Bangalore : Up (98 -\u0026gt; 115) Chennai : Same (110 -\u0026gt; 110) Hyderabad : Up (85 -\u0026gt; 92) Pune : Up (76 -\u0026gt; 88) Solution:\ncities = [\u0026#34;Delhi\u0026#34;, \u0026#34;Mumbai\u0026#34;, \u0026#34;Bangalore\u0026#34;, \u0026#34;Chennai\u0026#34;, \u0026#34;Hyderabad\u0026#34;, \u0026#34;Pune\u0026#34;] last_year = [120, 145, 98, 110, 85, 76] this_year = [135, 140, 115, 110, 92, 88] for city, prev, curr in zip(cities, last_year, this_year): if curr \u0026gt; prev: trend = \u0026#34;Up\u0026#34; elif curr \u0026lt; prev: trend = \u0026#34;Down\u0026#34; else: trend = \u0026#34;Same\u0026#34; print(f\u0026#34;{city:\u0026lt;10}: {trend} ({prev} -\u0026gt; {curr})\u0026#34;) What this tests: zip() over three lists at once, comparison logic, string formatting with alignment.\nQ10 \u0026ndash; break and continue Together Given a list of transactions, skip any with status = \u0026quot;pending\u0026quot;, and stop all processing the moment a status = \u0026quot;fraud\u0026quot; is encountered. Print all transactions that were actually processed.\ntransactions = [ {\u0026#34;id\u0026#34;: 1, \u0026#34;amount\u0026#34;: 5000, \u0026#34;status\u0026#34;: \u0026#34;success\u0026#34;}, {\u0026#34;id\u0026#34;: 2, \u0026#34;amount\u0026#34;: 1200, \u0026#34;status\u0026#34;: \u0026#34;pending\u0026#34;}, {\u0026#34;id\u0026#34;: 3, \u0026#34;amount\u0026#34;: 8800, \u0026#34;status\u0026#34;: \u0026#34;success\u0026#34;}, {\u0026#34;id\u0026#34;: 4, \u0026#34;amount\u0026#34;: 3200, \u0026#34;status\u0026#34;: \u0026#34;fraud\u0026#34;}, {\u0026#34;id\u0026#34;: 5, \u0026#34;amount\u0026#34;: 4400, \u0026#34;status\u0026#34;: \u0026#34;success\u0026#34;}, ] Expected Output:\nProcessed transaction #1 : Rs.5000 Skipping pending transaction #2 Processed transaction #3 : Rs.8800 Fraud detected on transaction #4. Halting all processing. Solution:\ntransactions = [ {\u0026#34;id\u0026#34;: 1, \u0026#34;amount\u0026#34;: 5000, \u0026#34;status\u0026#34;: \u0026#34;success\u0026#34;}, {\u0026#34;id\u0026#34;: 2, \u0026#34;amount\u0026#34;: 1200, \u0026#34;status\u0026#34;: \u0026#34;pending\u0026#34;}, {\u0026#34;id\u0026#34;: 3, \u0026#34;amount\u0026#34;: 8800, \u0026#34;status\u0026#34;: \u0026#34;success\u0026#34;}, {\u0026#34;id\u0026#34;: 4, \u0026#34;amount\u0026#34;: 3200, \u0026#34;status\u0026#34;: \u0026#34;fraud\u0026#34;}, {\u0026#34;id\u0026#34;: 5, \u0026#34;amount\u0026#34;: 4400, \u0026#34;status\u0026#34;: \u0026#34;success\u0026#34;}, ] for t in transactions: if t[\u0026#34;status\u0026#34;] == \u0026#34;pending\u0026#34;: print(f\u0026#34;Skipping pending transaction #{t[\u0026#39;id\u0026#39;]}\u0026#34;) continue if t[\u0026#34;status\u0026#34;] == \u0026#34;fraud\u0026#34;: print(f\u0026#34;Fraud detected on transaction #{t[\u0026#39;id\u0026#39;]}. Halting all processing.\u0026#34;) break print(f\u0026#34;Processed transaction #{t[\u0026#39;id\u0026#39;]} : Rs.{t[\u0026#39;amount\u0026#39;]}\u0026#34;) What this tests: break and continue in the same loop, accessing dict keys inside a loop, order of conditions matters.\nQ11 \u0026ndash; Nested Loop Pattern Print the following number pattern using nested loops:\n1 2 2 3 3 3 4 4 4 4 5 5 5 5 5 Solution:\nfor i in range(1, 6): for j in range(i): print(i, end=\u0026#34; \u0026#34;) print() What this tests: Nested loops, end=\u0026quot; \u0026quot; to print on same line, outer loop controls rows, inner loop controls columns.\nQ12 \u0026ndash; List Comprehension with Multiple Conditions Using a single list comprehension, extract all words from the sentence below that:\nAre longer than 3 characters Start with a vowel sentence = \u0026#34;An elephant ate orange apples in an orchard outside the city\u0026#34; Expected Output:\n[\u0026#39;elephant\u0026#39;, \u0026#39;ate\u0026#39;, \u0026#39;orange\u0026#39;, \u0026#39;apples\u0026#39;, \u0026#39;orchard\u0026#39;, \u0026#39;outside\u0026#39;] Solution:\nsentence = \u0026#34;An elephant ate orange apples in an orchard outside the city\u0026#34; result = [ word for word in sentence.split() if len(word) \u0026gt; 3 and word[0].lower() in \u0026#34;aeiou\u0026#34; ] print(result) What this tests: List comprehension with two conditions using and, .split() to tokenize, checking first character.\nQ13 \u0026ndash; Dictionary Comprehension with Grade Logic Given the two lists below, create a dictionary where each student\u0026rsquo;s name maps to their grade using the rules: \u0026ldquo;A\u0026rdquo; if marks \u0026gt;= 80, \u0026ldquo;B\u0026rdquo; if 60-79, \u0026ldquo;C\u0026rdquo; otherwise.\nstudents = [\u0026#34;Aarav\u0026#34;, \u0026#34;Priya\u0026#34;, \u0026#34;Rohan\u0026#34;, \u0026#34;Sneha\u0026#34;, \u0026#34;Karan\u0026#34;, \u0026#34;Meera\u0026#34;] marks = [88, 55, 73, 91, 62, 47] Expected Output:\n{\u0026#39;Aarav\u0026#39;: \u0026#39;A\u0026#39;, \u0026#39;Priya\u0026#39;: \u0026#39;C\u0026#39;, \u0026#39;Rohan\u0026#39;: \u0026#39;B\u0026#39;, \u0026#39;Sneha\u0026#39;: \u0026#39;A\u0026#39;, \u0026#39;Karan\u0026#39;: \u0026#39;B\u0026#39;, \u0026#39;Meera\u0026#39;: \u0026#39;C\u0026#39;} Solution:\nstudents = [\u0026#34;Aarav\u0026#34;, \u0026#34;Priya\u0026#34;, \u0026#34;Rohan\u0026#34;, \u0026#34;Sneha\u0026#34;, \u0026#34;Karan\u0026#34;, \u0026#34;Meera\u0026#34;] marks = [88, 55, 73, 91, 62, 47] grades = { name: (\u0026#34;A\u0026#34; if m \u0026gt;= 80 else \u0026#34;B\u0026#34; if m \u0026gt;= 60 else \u0026#34;C\u0026#34;) for name, m in zip(students, marks) } print(grades) What this tests: Dictionary comprehension, nested ternary in a comprehension, zip() to pair two lists.\nQ14 \u0026ndash; else with a Loop (Prime Checker) Write a function is_prime(n) that uses a for loop with an else block to check whether a number is prime. Test it on a list of numbers.\nnumbers = [2, 7, 10, 13, 25, 29, 36, 41] Expected Output:\n2 is prime 7 is prime 10 is not prime 13 is prime 25 is not prime 29 is prime 36 is not prime 41 is prime Solution:\ndef is_prime(n): if n \u0026lt; 2: return False for i in range(2, n): if n % i == 0: return False return True numbers = [2, 7, 10, 13, 25, 29, 36, 41] for num in numbers: status = \u0026#34;is prime\u0026#34; if is_prime(num) else \u0026#34;is not prime\u0026#34; print(f\u0026#34;{num:\u0026lt;2} {status}\u0026#34;) Note: The else block on a for loop is an alternative way to write this:\ndef is_prime_with_else(n): if n \u0026lt; 2: return False for i in range(2, n): if n % i == 0: print(f\u0026#34;{n} is not prime (divisible by {i})\u0026#34;) break else: # This else runs only if the loop finished without hitting break print(f\u0026#34;{n} is prime\u0026#34;) is_prime_with_else(13) is_prime_with_else(15) What this tests: for-else, returning from a function inside a loop, looping over a list and calling a function.\nHard Questions (15-20) Q15 \u0026ndash; Loop with Data Grouping Given the list of orders below, write a loop to build a summary dictionary that tracks total revenue and number of orders per city. Then print each city\u0026rsquo;s summary.\norders = [ {\u0026#34;city\u0026#34;: \u0026#34;Mumbai\u0026#34;, \u0026#34;amount\u0026#34;: 1500}, {\u0026#34;city\u0026#34;: \u0026#34;Delhi\u0026#34;, \u0026#34;amount\u0026#34;: 3200}, {\u0026#34;city\u0026#34;: \u0026#34;Mumbai\u0026#34;, \u0026#34;amount\u0026#34;: 800}, {\u0026#34;city\u0026#34;: \u0026#34;Bangalore\u0026#34;, \u0026#34;amount\u0026#34;: 4500}, {\u0026#34;city\u0026#34;: \u0026#34;Delhi\u0026#34;, \u0026#34;amount\u0026#34;: 1100}, {\u0026#34;city\u0026#34;: \u0026#34;Mumbai\u0026#34;, \u0026#34;amount\u0026#34;: 2200}, {\u0026#34;city\u0026#34;: \u0026#34;Bangalore\u0026#34;, \u0026#34;amount\u0026#34;: 3800}, {\u0026#34;city\u0026#34;: \u0026#34;Delhi\u0026#34;, \u0026#34;amount\u0026#34;: 900}, ] Expected Output:\nMumbai | Orders: 3 | Revenue: Rs.4500 Delhi | Orders: 3 | Revenue: Rs.5200 Bangalore | Orders: 2 | Revenue: Rs.8300 Solution:\norders = [ {\u0026#34;city\u0026#34;: \u0026#34;Mumbai\u0026#34;, \u0026#34;amount\u0026#34;: 1500}, {\u0026#34;city\u0026#34;: \u0026#34;Delhi\u0026#34;, \u0026#34;amount\u0026#34;: 3200}, {\u0026#34;city\u0026#34;: \u0026#34;Mumbai\u0026#34;, \u0026#34;amount\u0026#34;: 800}, {\u0026#34;city\u0026#34;: \u0026#34;Bangalore\u0026#34;, \u0026#34;amount\u0026#34;: 4500}, {\u0026#34;city\u0026#34;: \u0026#34;Delhi\u0026#34;, \u0026#34;amount\u0026#34;: 1100}, {\u0026#34;city\u0026#34;: \u0026#34;Mumbai\u0026#34;, \u0026#34;amount\u0026#34;: 2200}, {\u0026#34;city\u0026#34;: \u0026#34;Bangalore\u0026#34;, \u0026#34;amount\u0026#34;: 3800}, {\u0026#34;city\u0026#34;: \u0026#34;Delhi\u0026#34;, \u0026#34;amount\u0026#34;: 900}, ] summary = {} for order in orders: city = order[\u0026#34;city\u0026#34;] if city not in summary: summary[city] = {\u0026#34;orders\u0026#34;: 0, \u0026#34;revenue\u0026#34;: 0} summary[city][\u0026#34;orders\u0026#34;] += 1 summary[city][\u0026#34;revenue\u0026#34;] += order[\u0026#34;amount\u0026#34;] for city, data in summary.items(): print(f\u0026#34;{city:\u0026lt;10}| Orders: {data[\u0026#39;orders\u0026#39;]} | Revenue: Rs.{data[\u0026#39;revenue\u0026#39;]}\u0026#34;) What this tests: Building a nested dictionary inside a loop, the \u0026ldquo;key not in dict\u0026rdquo; initialization pattern, formatting output from a nested dict.\nQ16 \u0026ndash; while Loop with Validation Logic Write a number guessing game. The secret number is 33. The player gets 6 attempts. After each wrong guess, tell the player if the answer is higher or lower. If all attempts are exhausted without a correct guess, reveal the number.\nExpected interaction:\nGuess a number between 1 and 50: 25 Too low! 5 attempts left. Guess a number between 1 and 50: 40 Too high! 4 attempts left. Guess a number between 1 and 50: 33 Correct! You got it in 3 attempt(s). Solution:\nsecret = 33 attempts_allowed = 6 attempts_used = 0 while attempts_used \u0026lt; attempts_allowed: guess = int(input(\u0026#34;Guess a number between 1 and 50: \u0026#34;)) attempts_used += 1 remaining = attempts_allowed - attempts_used if guess == secret: print(f\u0026#34;Correct! You got it in {attempts_used} attempt(s).\u0026#34;) break elif guess \u0026lt; secret: print(f\u0026#34;Too low! {remaining} attempt(s) left.\u0026#34;) else: print(f\u0026#34;Too high! {remaining} attempt(s) left.\u0026#34;) else: # while-else: runs only if the while condition became False (never broke out) print(f\u0026#34;Out of attempts! The number was {secret}.\u0026#34;) What this tests: while-else, tracking attempts with a counter, break inside while, multi-branch elif.\nQ17 \u0026ndash; Nested Loop with Diamond Pattern Print a diamond shape of stars for a given n. For n = 4, the output should be:\n* * * * * * * * * * * * * * * * * * * * Solution:\nn = 4 # Top half (rows 1 to n) for i in range(1, n + 1): spaces = n - i print(\u0026#34; \u0026#34; * spaces + \u0026#34;* \u0026#34; * i) # Bottom half (rows n-1 down to 1) for i in range(n - 1, 0, -1): spaces = n - i print(\u0026#34; \u0026#34; * spaces + \u0026#34;* \u0026#34; * i) What this tests: Two separate loops for top and bottom halves, calculating spaces as a function of row number, range() going downward, string multiplication.\nQ18 \u0026ndash; Flatten and Deduplicate Given the nested list of city lists from different survey sources below, write a loop to:\nFlatten it into a single list Remove duplicates while preserving the first-seen order Print the final clean list survey_data = [ [\u0026#34;Delhi\u0026#34;, \u0026#34;Mumbai\u0026#34;, \u0026#34;Pune\u0026#34;], [\u0026#34;Bangalore\u0026#34;, \u0026#34;Mumbai\u0026#34;, \u0026#34;Hyderabad\u0026#34;], [\u0026#34;Chennai\u0026#34;, \u0026#34;Delhi\u0026#34;, \u0026#34;Kochi\u0026#34;], [\u0026#34;Pune\u0026#34;, \u0026#34;Kochi\u0026#34;, \u0026#34;Surat\u0026#34;], ] Expected Output:\n[\u0026#39;Delhi\u0026#39;, \u0026#39;Mumbai\u0026#39;, \u0026#39;Pune\u0026#39;, \u0026#39;Bangalore\u0026#39;, \u0026#39;Hyderabad\u0026#39;, \u0026#39;Chennai\u0026#39;, \u0026#39;Kochi\u0026#39;, \u0026#39;Surat\u0026#39;] Solution:\nsurvey_data = [ [\u0026#34;Delhi\u0026#34;, \u0026#34;Mumbai\u0026#34;, \u0026#34;Pune\u0026#34;], [\u0026#34;Bangalore\u0026#34;, \u0026#34;Mumbai\u0026#34;, \u0026#34;Hyderabad\u0026#34;], [\u0026#34;Chennai\u0026#34;, \u0026#34;Delhi\u0026#34;, \u0026#34;Kochi\u0026#34;], [\u0026#34;Pune\u0026#34;, \u0026#34;Kochi\u0026#34;, \u0026#34;Surat\u0026#34;], ] seen = [] for sublist in survey_data: for city in sublist: if city not in seen: seen.append(city) print(seen) Alternative using a set to track seen items (faster for large data):\nseen_set = set() result = [] for sublist in survey_data: for city in sublist: if city not in seen_set: result.append(city) seen_set.add(city) print(result) What this tests: Nested loops over a list of lists, deduplication while preserving order, two approaches (list vs set for membership check).\nQ19 \u0026ndash; Word Frequency Counter Write a program that counts how many times each word appears in the paragraph below. Ignore case and punctuation. Print the results sorted from most frequent to least frequent. Show only words that appear more than once.\nparagraph = \u0026#34;\u0026#34;\u0026#34; Cricket is the most popular sport in India. India has produced many great cricket players. The cricket World Cup is watched by millions in India and around the world. Players from India have won the World Cup twice for India. \u0026#34;\u0026#34;\u0026#34; Expected Output (order may vary for equal counts):\nindia : 6 cricket : 4 the : 4 world : 3 cup : 2 players : 2 has : 1 \u0026lt;- only shown if you choose to include count \u0026gt;= 1 (Show only words with count \u0026gt; 1 in the final output.)\nSolution:\nparagraph = \u0026#34;\u0026#34;\u0026#34; Cricket is the most popular sport in India. India has produced many great cricket players. The cricket World Cup is watched by millions in India and around the world. Players from India have won the World Cup twice for India. \u0026#34;\u0026#34;\u0026#34; # Clean and split cleaned = \u0026#34;\u0026#34; for char in paragraph.lower(): if char.isalpha() or char == \u0026#34; \u0026#34;: cleaned += char words = cleaned.split() # Count frequencies freq = {} for word in words: if word not in freq: freq[word] = 0 freq[word] += 1 # Sort by count descending sorted_words = sorted(freq.items(), key=lambda x: x[1], reverse=True) # Print words that appear more than once for word, count in sorted_words: if count \u0026gt; 1: print(f\u0026#34;{word:\u0026lt;10}: {count}\u0026#34;) What this tests: Nested loops, building a frequency dictionary, sorting a list of tuples by value, string cleaning with a loop, lambda as a sort key.\nQ20 \u0026ndash; Full Student Report Pipeline You are given student records. Write a complete program that:\nCalculates each student\u0026rsquo;s total marks and percentage (out of 500) Assigns a grade: \u0026ldquo;Distinction\u0026rdquo; (\u0026gt;= 85%), \u0026ldquo;First Class\u0026rdquo; (\u0026gt;= 60%), \u0026ldquo;Pass\u0026rdquo; (\u0026gt;= 40%), \u0026ldquo;Fail\u0026rdquo; (\u0026lt; 40%) Counts how many students got each grade Finds the topper (highest percentage) Prints the full formatted report students = [ {\u0026#34;name\u0026#34;: \u0026#34;Aarav\u0026#34;, \u0026#34;maths\u0026#34;: 88, \u0026#34;science\u0026#34;: 92, \u0026#34;english\u0026#34;: 85, \u0026#34;history\u0026#34;: 78, \u0026#34;cs\u0026#34;: 95}, {\u0026#34;name\u0026#34;: \u0026#34;Priya\u0026#34;, \u0026#34;maths\u0026#34;: 55, \u0026#34;science\u0026#34;: 48, \u0026#34;english\u0026#34;: 62, \u0026#34;history\u0026#34;: 50, \u0026#34;cs\u0026#34;: 58}, {\u0026#34;name\u0026#34;: \u0026#34;Rohan\u0026#34;, \u0026#34;maths\u0026#34;: 72, \u0026#34;science\u0026#34;: 68, \u0026#34;english\u0026#34;: 75, \u0026#34;history\u0026#34;: 70, \u0026#34;cs\u0026#34;: 65}, {\u0026#34;name\u0026#34;: \u0026#34;Sneha\u0026#34;, \u0026#34;maths\u0026#34;: 95, \u0026#34;science\u0026#34;: 98, \u0026#34;english\u0026#34;: 92, \u0026#34;history\u0026#34;: 88, \u0026#34;cs\u0026#34;: 97}, {\u0026#34;name\u0026#34;: \u0026#34;Karan\u0026#34;, \u0026#34;maths\u0026#34;: 35, \u0026#34;science\u0026#34;: 40, \u0026#34;english\u0026#34;: 38, \u0026#34;history\u0026#34;: 42, \u0026#34;cs\u0026#34;: 30}, {\u0026#34;name\u0026#34;: \u0026#34;Meera\u0026#34;, \u0026#34;maths\u0026#34;: 78, \u0026#34;science\u0026#34;: 82, \u0026#34;english\u0026#34;: 80, \u0026#34;history\u0026#34;: 75, \u0026#34;cs\u0026#34;: 84}, {\u0026#34;name\u0026#34;: \u0026#34;Arjun\u0026#34;, \u0026#34;maths\u0026#34;: 45, \u0026#34;science\u0026#34;: 55, \u0026#34;english\u0026#34;: 50, \u0026#34;history\u0026#34;: 48, \u0026#34;cs\u0026#34;: 52}, ] Expected Output:\n===== STUDENT REPORT ===== Aarav | Total: 438/500 | 87.6% | Distinction Priya | Total: 273/500 | 54.6% | Pass Rohan | Total: 350/500 | 70.0% | First Class Sneha | Total: 470/500 | 94.0% | Distinction Karan | Total: 185/500 | 37.0% | Fail Meera | Total: 399/500 | 79.8% | First Class Arjun | Total: 250/500 | 50.0% | Pass ===== GRADE SUMMARY ===== Distinction : 2 First Class : 2 Pass : 2 Fail : 1 ===== TOPPER ===== Sneha with 94.0% Solution:\nstudents = [ {\u0026#34;name\u0026#34;: \u0026#34;Aarav\u0026#34;, \u0026#34;maths\u0026#34;: 88, \u0026#34;science\u0026#34;: 92, \u0026#34;english\u0026#34;: 85, \u0026#34;history\u0026#34;: 78, \u0026#34;cs\u0026#34;: 95}, {\u0026#34;name\u0026#34;: \u0026#34;Priya\u0026#34;, \u0026#34;maths\u0026#34;: 55, \u0026#34;science\u0026#34;: 48, \u0026#34;english\u0026#34;: 62, \u0026#34;history\u0026#34;: 50, \u0026#34;cs\u0026#34;: 58}, {\u0026#34;name\u0026#34;: \u0026#34;Rohan\u0026#34;, \u0026#34;maths\u0026#34;: 72, \u0026#34;science\u0026#34;: 68, \u0026#34;english\u0026#34;: 75, \u0026#34;history\u0026#34;: 70, \u0026#34;cs\u0026#34;: 65}, {\u0026#34;name\u0026#34;: \u0026#34;Sneha\u0026#34;, \u0026#34;maths\u0026#34;: 95, \u0026#34;science\u0026#34;: 98, \u0026#34;english\u0026#34;: 92, \u0026#34;history\u0026#34;: 88, \u0026#34;cs\u0026#34;: 97}, {\u0026#34;name\u0026#34;: \u0026#34;Karan\u0026#34;, \u0026#34;maths\u0026#34;: 35, \u0026#34;science\u0026#34;: 40, \u0026#34;english\u0026#34;: 38, \u0026#34;history\u0026#34;: 42, \u0026#34;cs\u0026#34;: 30}, {\u0026#34;name\u0026#34;: \u0026#34;Meera\u0026#34;, \u0026#34;maths\u0026#34;: 78, \u0026#34;science\u0026#34;: 82, \u0026#34;english\u0026#34;: 80, \u0026#34;history\u0026#34;: 75, \u0026#34;cs\u0026#34;: 84}, {\u0026#34;name\u0026#34;: \u0026#34;Arjun\u0026#34;, \u0026#34;maths\u0026#34;: 45, \u0026#34;science\u0026#34;: 55, \u0026#34;english\u0026#34;: 50, \u0026#34;history\u0026#34;: 48, \u0026#34;cs\u0026#34;: 52}, ] subjects = [\u0026#34;maths\u0026#34;, \u0026#34;science\u0026#34;, \u0026#34;english\u0026#34;, \u0026#34;history\u0026#34;, \u0026#34;cs\u0026#34;] max_marks = 500 grade_count = {\u0026#34;Distinction\u0026#34;: 0, \u0026#34;First Class\u0026#34;: 0, \u0026#34;Pass\u0026#34;: 0, \u0026#34;Fail\u0026#34;: 0} topper_name = \u0026#34;\u0026#34; topper_pct = 0 print(\u0026#34;===== STUDENT REPORT =====\u0026#34;) for student in students: total = sum(student[sub] for sub in subjects) pct = round((total / max_marks) * 100, 1) if pct \u0026gt;= 85: grade = \u0026#34;Distinction\u0026#34; elif pct \u0026gt;= 60: grade = \u0026#34;First Class\u0026#34; elif pct \u0026gt;= 40: grade = \u0026#34;Pass\u0026#34; else: grade = \u0026#34;Fail\u0026#34; grade_count[grade] += 1 if pct \u0026gt; topper_pct: topper_pct = pct topper_name = student[\u0026#34;name\u0026#34;] print(f\u0026#34;{student[\u0026#39;name\u0026#39;]:\u0026lt;6} | Total: {total}/{max_marks} | {pct}% | {grade}\u0026#34;) print(\u0026#34;\\n===== GRADE SUMMARY =====\u0026#34;) for grade, count in grade_count.items(): print(f\u0026#34;{grade:\u0026lt;12}: {count}\u0026#34;) print(\u0026#34;\\n===== TOPPER =====\u0026#34;) print(f\u0026#34;{topper_name} with {topper_pct}%\u0026#34;) What this tests: Everything. Loop over list of dicts, inner loop to sum subject scores, grade assignment with if-elif-else, accumulator for grade counts, tracking max value and its label across iterations, formatted multi-section output.\nQuick Reference Concept One-liner reminder for loop for item in collection: while loop while condition: range range(start, stop, step) break Exit the loop immediately continue Skip this iteration, keep going pass Do nothing, placeholder else on loop Runs only if break was never hit enumerate for i, val in enumerate(lst, start=1): zip for a, b in zip(list1, list2): list comprehension [expr for x in lst if condition] dict comprehension {k: v for k, v in zip(keys, vals)} dict loop for key, val in d.items(): nested loop Inner loop runs fully for each outer iteration Made with care for Codeverra learners | codeverra.com\n","permalink":"https://blogs.codeverra.com/blog/python-04-loops-practice/","summary":"\u003ch1 id=\"python-loops---practice-questions-and-solutions\"\u003ePython Loops - Practice Questions and Solutions\u003c/h1\u003e\n\u003ch3 id=\"20-questions-from-easy-to-hard\"\u003e20 Questions from Easy to Hard\u003c/h3\u003e\n\u003chr\u003e\n\u003cblockquote\u003e\n\u003cp\u003eAll 20 questions are based on the Loops Masterclass.\nTry each question on your own before reading the solution.\nThe questions are ordered by difficulty: Easy (1-7), Medium (8-14), Hard (15-20).\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003chr\u003e\n\u003ch2 id=\"table-of-contents\"\u003eTable of Contents\u003c/h2\u003e\n\u003cul\u003e\n\u003cli\u003e\u003ca href=\"#easy-questions-1-7\"\u003eEasy Questions (1-7)\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#medium-questions-8-14\"\u003eMedium Questions (8-14)\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#hard-questions-15-20\"\u003eHard Questions (15-20)\u003c/a\u003e\u003c/li\u003e\n\u003c/ul\u003e\n\u003chr\u003e\n\u003ch2 id=\"easy-questions-1-7\"\u003eEasy Questions (1-7)\u003c/h2\u003e\n\u003chr\u003e\n\u003ch3 id=\"q1--basic-for-loop-with-range\"\u003eQ1 \u0026ndash; Basic for Loop with range()\u003c/h3\u003e\n\u003cp\u003ePrint all numbers from 1 to 30 that are divisible by both 3 and 5.\u003c/p\u003e","tags":["python"],"title":"Master Loops in Python"},{"content":"NumPy Masterclass - From Basics to Advanced Prerequisites: Python basics, lists, loops, functions\nInstall: pip install numpy\nImport convention: import numpy as np\nTable of Contents What is NumPy and Why Use It? Creating Arrays Array Properties Indexing and Slicing Modifying Arrays Reshaping and Stacking Math Operations and Broadcasting Universal Functions (ufuncs) Aggregations and Statistics Sorting and Searching Boolean Operations and Masking Random Number Generation Linear Algebra Working with Files Performance Tips and Best Practices 1. What is NumPy and Why Use It? NumPy (Numerical Python) is the foundation of the entire Python data science stack. Pandas, Matplotlib, Scikit-learn, and TensorFlow all build on top of it.\nWhy not just use Python lists? import numpy as np import time # Python list - slow loop-based python_list = list(range(1_000_000)) start = time.time() result = [x * 2 for x in python_list] print(f\u0026#34;List: {time.time() - start:.4f}s\u0026#34;) # NumPy array - vectorized C operations np_array = np.arange(1_000_000) start = time.time() result = np_array * 2 print(f\u0026#34;NumPy: {time.time() - start:.4f}s\u0026#34;) # NumPy is typically 10–100x faster for numerical operations Key differences from Python lists Feature Python List NumPy Array Data types Mixed (int, str, etc.) All same type Speed Slow (Python loop) Fast (C under the hood) Memory More overhead Compact Operations Need explicit loops Vectorized (no loops) Dimensions Only 1D natively Any number of dimensions 2. Creating Arrays From Python sequences import numpy as np # From a list - 1D array a = np.array([1, 2, 3, 4, 5]) print(a) # [1 2 3 4 5] print(type(a)) # \u0026lt;class \u0026#39;numpy.ndarray\u0026#39;\u0026gt; # From a list of lists - 2D array (matrix) m = np.array([[1, 2, 3], [4, 5, 6]]) print(m) # [[1 2 3] # [4 5 6]] # From a tuple b = np.array((10, 20, 30)) print(b) # [10 20 30] # Specify data type explicitly c = np.array([1, 2, 3], dtype=float) print(c) # [1. 2. 3.] d = np.array([1.7, 2.9, 3.1], dtype=int) print(d) # [1 2 3] (truncated, not rounded!) Built-in array creation functions # Zeros - all elements are 0.0 np.zeros(5) # [0. 0. 0. 0. 0.] np.zeros((3, 4)) # 3 rows, 4 cols of zeros # Ones - all elements are 1.0 np.ones(4) # [1. 1. 1. 1.] np.ones((2, 3)) # 2×3 matrix of ones # Full - fill with a specific value np.full(5, 7) # [7 7 7 7 7] np.full((2, 2), 3.14) # [[3.14 3.14] # [3.14 3.14]] # Identity matrix (n×n, 1s on diagonal) np.eye(3) # [[1. 0. 0.] # [0. 1. 0.] # [0. 0. 1.]] # Uninitialized - fast but contains garbage values (use carefully) np.empty((2, 3)) # Like another array (same shape) a = np.array([[1, 2], [3, 4]]) np.zeros_like(a) # [[0 0] [0 0]] np.ones_like(a) # [[1 1] [1 1]] np.full_like(a, 9) # [[9 9] [9 9]] Range-based arrays # np.arange(start, stop, step) - like Python range() np.arange(5) # [0 1 2 3 4] np.arange(1, 10) # [1 2 3 4 5 6 7 8 9] np.arange(0, 1, 0.2) # [0. 0.2 0.4 0.6 0.8] np.arange(10, 0, -2) # [10 8 6 4 2] # np.linspace(start, stop, num) - exactly num evenly spaced points # NOTE: stop IS included (unlike arange) np.linspace(0, 1, 5) # [0. 0.25 0.5 0.75 1. ] np.linspace(0, 10, 11) # [0. 1. 2. 3. 4. 5. 6. 7. 8. 9. 10.] # Logarithmically spaced np.logspace(0, 3, 4) # [ 1. 10. 100. 1000.] (10^0 to 10^3) When to use arange vs linspace:\nUse arange when you need a specific step size.\nUse linspace when you need a specific number of points (great for plotting x-axes).\n3. Array Properties a = np.array([[1, 2, 3, 4], [5, 6, 7, 8], [9,10,11,12]]) # Shape - (rows, cols) for 2D print(a.shape) # (3, 4) # Number of dimensions print(a.ndim) # 2 # Total number of elements print(a.size) # 12 # Data type of elements print(a.dtype) # int64 # Memory size of each element in bytes print(a.itemsize) # 8 (int64 = 8 bytes) # Total memory in bytes print(a.nbytes) # 96 (12 elements × 8 bytes) Common dtypes np.array([1, 2, 3], dtype=np.int8) # -128 to 127 np.array([1, 2, 3], dtype=np.int32) # ±2 billion np.array([1, 2, 3], dtype=np.int64) # default int on most systems np.array([1.0], dtype=np.float32) # single precision np.array([1.0], dtype=np.float64) # double precision (default) np.array([True, False], dtype=np.bool_) # boolean np.array([\u0026#39;a\u0026#39;, \u0026#39;b\u0026#39;], dtype=np.str_) # unicode string # Convert dtype a = np.array([1, 2, 3]) a.astype(float) # [1. 2. 3.] a.astype(np.float32) 4. Indexing and Slicing 1D indexing a = np.array([10, 20, 30, 40, 50]) # Single element (0-based) a[0] # 10 a[4] # 50 a[-1] # 50 (last element) a[-2] # 40 (second to last) # Slicing [start:stop:step] - stop is EXCLUSIVE a[1:4] # [20 30 40] a[:3] # [10 20 30] (from beginning) a[2:] # [30 40 50] (to end) a[:] # [10 20 30 40 50] (all) a[::2] # [10 30 50] (every 2nd) a[::-1] # [50 40 30 20 10] (reversed!) a[1::2] # [20 40] (start at 1, every 2nd) 2D indexing m = np.array([[1, 2, 3, 4], [5, 6, 7, 8], [9, 10, 11, 12]]) # Single element [row, col] m[0, 0] # 1 (top-left) m[2, 3] # 12 (bottom-right) m[1, 2] # 7 m[-1, -1] # 12 # Entire row m[0] # [1 2 3 4] m[1, :] # [5 6 7 8] m[-1, :] # [9 10 11 12] # Entire column m[:, 0] # [1 5 9] m[:, 2] # [3 7 11] # Submatrix [row_slice, col_slice] m[0:2, 1:3] # [[2 3] [6 7]] m[1:, 2:] # [[7 8] [11 12]] m[:2, :2] # [[1 2] [5 6]] Fancy indexing (index with an array) a = np.array([10, 20, 30, 40, 50]) # Index with a list of positions - returns a COPY (not a view) idx = [0, 2, 4] a[idx] # [10 30 50] a[[1, 3]] # [20 40] # 2D fancy indexing m = np.arange(12).reshape(3, 4) rows = [0, 2] cols = [1, 3] m[rows, cols] # [m[0,1], m[2,3]] = [1, 11] Views vs Copies - important! a = np.array([1, 2, 3, 4, 5]) # Slices return VIEWS - modifying changes the original! b = a[1:4] b[0] = 99 print(a) # [1 99 3 4 5] ← original changed! # Use .copy() to avoid this c = a[1:4].copy() c[0] = 0 print(a) # unchanged # Check if an array is a view b.base is a # True (b is a view of a) c.base is a # False (c is independent) 5. Modifying Arrays Setting values a = np.array([1, 2, 3, 4, 5]) # Set single element a[0] = 99 # Set a slice a[1:3] = [20, 30] # Set multiple positions with fancy indexing a[[0, 4]] = 0 # 2D assignment m = np.zeros((3, 3)) m[0, :] = [1, 2, 3] # set first row m[:, 1] = 9 # set second column to all 9s m[1:3, 1:3] = [[5, 6], [7, 8]] # set submatrix Adding and removing elements a = np.array([1, 2, 3]) # Append - returns a NEW array (doesn\u0026#39;t modify in place) np.append(a, 4) # [1 2 3 4] np.append(a, [4, 5, 6]) # [1 2 3 4 5 6] # Insert at position np.insert(a, 1, 99) # [1 99 2 3] (insert 99 at index 1) # Delete by index np.delete(a, 0) # [2 3] np.delete(a, [0, 2]) # [2] # Note: all of these return NEW arrays 6. Reshaping and Stacking Reshaping a = np.arange(12) # [0 1 2 3 4 5 6 7 8 9 10 11] # reshape(rows, cols) - total elements must match m = a.reshape(3, 4) # [[ 0 1 2 3] # [ 4 5 6 7] # [ 8 9 10 11]] a.reshape(2, 6) # 2 rows × 6 cols a.reshape(4, 3) # 4 rows × 3 cols a.reshape(2, 2, 3) # 3D: 2 blocks × 2 rows × 3 cols # Use -1 to let NumPy infer one dimension a.reshape(4, -1) # 4 rows, NumPy figures out 3 cols a.reshape(-1, 3) # NumPy figures out 4 rows, 3 cols a.reshape(-1) # flatten to 1D # np.reshape is the same np.reshape(a, (3, 4)) Flattening m = np.array([[1, 2, 3], [4, 5, 6]]) m.flatten() # [1 2 3 4 5 6] - returns a COPY m.ravel() # [1 2 3 4 5 6] - returns a VIEW (faster) m.reshape(-1) # same as ravel Adding dimensions a = np.array([1, 2, 3]) # shape (3,) # Add dimension to make it a row vector a[np.newaxis, :] # shape (1, 3) a.reshape(1, -1) # same # Add dimension to make it a column vector a[:, np.newaxis] # shape (3, 1) a.reshape(-1, 1) # same # np.expand_dims np.expand_dims(a, axis=0) # (1, 3) np.expand_dims(a, axis=1) # (3, 1) Transpose m = np.array([[1, 2, 3], [4, 5, 6]]) # shape (2, 3) m.T # shape (3, 2) - rows become cols np.transpose(m) # same # For higher dimensions - specify axis order a = np.arange(24).reshape(2, 3, 4) a.T.shape # (4, 3, 2) np.transpose(a, axes=(0, 2, 1)) # custom axis order Stacking and splitting a = np.array([1, 2, 3]) b = np.array([4, 5, 6]) # Stack vertically (add rows) np.vstack([a, b]) # [[1 2 3] [4 5 6]] shape (2, 3) # Stack horizontally (extend columns / join end-to-end for 1D) np.hstack([a, b]) # [1 2 3 4 5 6] shape (6,) # Stack along new axis np.stack([a, b], axis=0) # [[1 2 3] [4 5 6]] axis=0 → new first axis np.stack([a, b], axis=1) # [[1 4] [2 5] [3 6]] # column_stack - makes each 1D array a column np.column_stack([a, b]) # [[1 4] [2 5] [3 6]] # General concatenate np.concatenate([a, b]) # 1D: [1 2 3 4 5 6] np.concatenate([m1, m2], axis=0) # stack rows np.concatenate([m1, m2], axis=1) # stack cols # Splitting a = np.arange(12) np.split(a, 3) # 3 equal arrays of 4 np.split(a, [3, 7]) # split at positions 3 and 7 m = np.arange(12).reshape(3, 4) np.vsplit(m, 3) # split into 3 row-blocks np.hsplit(m, 2) # split into 2 col-blocks 7. Math Operations and Broadcasting Element-wise operations a = np.array([1, 2, 3, 4]) b = np.array([10, 20, 30, 40]) # Arithmetic (all element-wise) a + b # [11 22 33 44] a - b # [-9 -18 -27 -36] a * b # [10 40 90 160] a / b # [0.1 0.1 0.1 0.1] a // b # [0 0 0 0] integer division a % b # [1 2 3 4] remainder a ** 2 # [1 4 9 16] exponentiation # Comparison (returns boolean array) a \u0026gt; 2 # [False False True True] a == 2 # [False True False False] a != 3 # [True True False True] a \u0026gt;= 3 # [False False True True] Scalar broadcasting a = np.array([1, 2, 3, 4]) # Scalar stretches to match the array a + 10 # [11 12 13 14] a * 2 # [2 4 6 8] a / 2.0 # [0.5 1.0 1.5 2.0] a ** 2 # [1 4 9 16] 10 - a # [9 8 7 6] Broadcasting with arrays of different shapes # Rule: dimensions are compared trailing → leading # Sizes must match OR one of them must be 1 m = np.array([[1, 2, 3], [4, 5, 6]]) # shape (2, 3) row = np.array([10, 20, 30]) # shape (3,) # row broadcasts to (2, 3) - added to every row m + row # [[11 22 33] # [14 25 36]] col = np.array([[100], [200]]) # shape (2, 1) # col broadcasts to (2, 3) - added to every column m + col # [[101 102 103] # [204 205 206]] # Both broadcast simultaneously row + col # [[110 120 130] # [210 220 230]] 8. Universal Functions (ufuncs) Universal functions operate element-wise and are very fast (C-implemented).\na = np.array([1, 4, 9, 16, 25]) # Math np.sqrt(a) # [1. 2. 3. 4. 5.] np.square(a) # [1 16 81 256 625] np.abs([-1, -2, 3]) # [1 2 3] np.log(a) # natural log np.log2(a) # base-2 log np.log10(a) # base-10 log np.exp(a) # e^x np.exp2(a) # 2^x # Trig (angles in RADIANS) x = np.array([0, np.pi/6, np.pi/4, np.pi/2, np.pi]) np.sin(x) # [0. 0.5 0.707 1. 0.] np.cos(x) np.tan(x) np.arcsin([0, 1]) # inverse sin → [0. 1.5707...] np.degrees(x) # convert radians to degrees np.radians([0, 90, 180]) # degrees to radians # Rounding a = np.array([1.2, 2.5, 3.7, -1.5]) np.round(a) # [1. 2. 4. -2.] (banker\u0026#39;s rounding) np.floor(a) # [1. 2. 3. -2.] (round down) np.ceil(a) # [2. 3. 4. -1.] (round up) np.trunc(a) # [1. 2. 3. -1.] (truncate toward zero) np.clip(a, 0, 3) # [1.2 2.5 3.0 0.0] (clamp to range) # Min / max between two arrays a = np.array([1, 5, 3]) b = np.array([4, 2, 6]) np.maximum(a, b) # [4 5 6] (element-wise max) np.minimum(a, b) # [1 2 3] (element-wise min) # Sign and modulo np.sign([-3, 0, 5]) # [-1 0 1] np.mod(a, 2) # remainder after dividing by 2 np.divmod(a, 2) # quotient and remainder together 9. Aggregations and Statistics Basic aggregations a = np.array([[1, 2, 3], [4, 5, 6]]) # Aggregate entire array a.sum() # 21 a.min() # 1 a.max() # 6 a.mean() # 3.5 a.std() # standard deviation ≈ 1.708 a.var() # variance = std² a.prod() # product of all elements = 720 The axis parameter - critical concept a = np.array([[1, 2, 3], [4, 5, 6]]) # axis=0 → collapse ROWS → result is per COLUMN (↓ vertical) a.sum(axis=0) # [5 7 9] (1+4, 2+5, 3+6) a.min(axis=0) # [1 2 3] a.max(axis=0) # [4 5 6] a.mean(axis=0) # [2.5 3.5 4.5] # axis=1 → collapse COLS → result is per ROW (→ horizontal) a.sum(axis=1) # [6 15] (1+2+3, 4+5+6) a.min(axis=1) # [1 4] a.max(axis=1) # [3 6] a.mean(axis=1) # [2. 5.] # keepdims - preserve dimensions for broadcasting a.sum(axis=1, keepdims=True) # [[6], [15]] shape (2,1) Memory trick: axis=0 collapses rows (goes down ↓), axis=1 collapses cols (goes across →).\nIndex of min/max a = np.array([3, 1, 4, 1, 5, 9, 2, 6]) np.argmin(a) # 1 (index of minimum value 1) np.argmax(a) # 5 (index of maximum value 9) # For 2D - flattened index by default m = np.array([[3, 1], [4, 1]]) np.argmax(m) # 2 (flat index) np.argmax(m, axis=0) # [1 0] (index per column) np.argmax(m, axis=1) # [0 0] (index per row) Cumulative operations a = np.array([1, 2, 3, 4, 5]) np.cumsum(a) # [1 3 6 10 15] running total np.cumprod(a) # [1 2 6 24 120] running product # 2D cumulative m = np.array([[1, 2], [3, 4]]) np.cumsum(m, axis=0) # cumulative down each column np.cumsum(m, axis=1) # cumulative across each row Statistical functions a = np.array([2, 4, 4, 4, 5, 5, 7, 9]) np.mean(a) # 5.0 np.median(a) # 4.5 np.std(a) # 2.0 (population std) np.var(a) # 4.0 # Percentile / quantile np.percentile(a, 25) # Q1 = 4.0 np.percentile(a, 50) # median = 4.5 np.percentile(a, 75) # Q3 = 5.5 np.percentile(a, [25, 50, 75]) # all at once np.quantile(a, 0.25) # same as percentile 25 # Correlation and covariance x = np.array([1, 2, 3, 4, 5]) y = np.array([2, 4, 5, 4, 5]) np.corrcoef(x, y) # correlation matrix np.cov(x, y) # covariance matrix # Histogram counts, bin_edges = np.histogram(a, bins=5) 10. Sorting and Searching a = np.array([3, 1, 4, 1, 5, 9, 2, 6]) # Sort - returns a NEW sorted array np.sort(a) # [1 1 2 3 4 5 6 9] np.sort(a)[::-1] # descending: [9 6 5 4 3 2 1 1] # Sort IN PLACE (modifies original) a.sort() # argsort - returns indices that would sort the array np.argsort(a) # [1 3 6 0 2 4 7 5] a[np.argsort(a)] # same as np.sort(a) # 2D sorting m = np.array([[3, 1, 2], [6, 4, 5]]) np.sort(m, axis=0) # sort each column np.sort(m, axis=1) # sort each row # Partial sort - find the k smallest np.partition(a, 3) # first 3 positions are the 3 smallest (unordered) np.argpartition(a, 3) # indices version # Searching np.searchsorted([1, 3, 5, 7], 4) # 2 (insert 4 before index 2) np.where(a \u0026gt; 3) # indices where condition is True np.nonzero(a) # indices of non-zero elements # Count occurrences np.count_nonzero(a \u0026gt; 3) # how many elements are \u0026gt; 3 11. Boolean Operations and Masking a = np.array([1, 5, 3, 8, 2, 9, 4, 7]) # Conditions produce boolean arrays mask = a \u0026gt; 4 # [False True False True False True False True] print(mask) # Use mask to select elements a[mask] # [5 8 9 7] a[a \u0026gt; 4] # same shorthand # Modify elements based on condition a[a \u0026gt; 4] = 0 # set all \u0026gt;4 to zero a[a \u0026lt; 3] = -1 # set all \u0026lt;3 to -1 # np.where(condition, if_true, if_false) - vectorized if/else np.where(a \u0026gt; 4, a, 0) # keep value if \u0026gt;4, else 0 np.where(a \u0026gt; 0, a, -a) # absolute value! np.where(a % 2 == 0, \u0026#39;even\u0026#39;, \u0026#39;odd\u0026#39;) # Combining conditions (use \u0026amp;, |, ~ NOT and/or/not) a[(a \u0026gt; 2) \u0026amp; (a \u0026lt; 8)] # between 2 and 8 a[(a \u0026lt; 2) | (a \u0026gt; 7)] # less than 2 or greater than 7 a[~(a == 5)] # everything EXCEPT 5 # Logical functions np.logical_and(a \u0026gt; 2, a \u0026lt; 8) np.logical_or(a \u0026lt; 2, a \u0026gt; 7) np.logical_not(a \u0026gt; 5) # Any / all np.any(a \u0026gt; 5) # True if at least one element satisfies np.all(a \u0026gt; 0) # True if ALL elements satisfy np.any(a \u0026gt; 5, axis=0) # per column for 2D # Finding unique values np.unique(a) # sorted unique values vals, counts = np.unique(a, return_counts=True) # with counts np.isin(a, [2, 4, 6]) # True where element is in list 12. Random Number Generation # Modern API - always use default_rng with a seed for reproducibility rng = np.random.default_rng(seed=42) # Uniform floats [0, 1) rng.random() # single float rng.random(5) # 1D array of 5 floats rng.random((3, 4)) # 3×4 matrix # Uniform floats in a range [low, high) rng.uniform(0, 10, 100) # Random integers [low, high) rng.integers(1, 7, 10) # 10 dice rolls (1-6) rng.integers(0, 100, (3, 3)) # 3×3 matrix of ints # Normal (Gaussian) distribution rng.normal(loc=0, scale=1, size=1000) # μ=0, σ=1 rng.normal(loc=100, scale=15, size=500) # IQ-like scores # Other distributions rng.binomial(n=10, p=0.5, size=100) # binomial rng.poisson(lam=3, size=100) # Poisson rng.exponential(scale=1.0, size=100) # exponential rng.choice([1, 2, 3, 4, 5], size=10) # sample from array rng.choice([1, 2, 3], size=5, replace=False) # without replacement # Shuffle a = np.arange(10) rng.shuffle(a) # in-place shuffle rng.permutation(10) # returns new shuffled array # Seed for reproducibility - same seed = same numbers rng1 = np.random.default_rng(42) rng2 = np.random.default_rng(42) print(rng1.random(3) == rng2.random(3)) # [True True True] Always set a seed when writing code others will run, or when you need reproducible results for testing.\n13. Linear Algebra A = np.array([[1, 2], [3, 4]]) B = np.array([[5, 6], [7, 8]]) # Matrix multiplication - use @ operator or np.dot A @ B # [[19 22] # [43 50]] np.dot(A, B) # same result # WARNING: * is element-wise, NOT matrix multiply! A * B # [[5 12] [21 32]] ← wrong for matrix math # Matrix-vector multiplication v = np.array([1, 2]) A @ v # [5 11] # Determinant np.linalg.det(A) # -2.0 # Matrix inverse (A must be square and non-singular) np.linalg.inv(A) # [[-2. 1. ] # [ 1.5 -0.5]] # Verify: A @ inv(A) should give identity A @ np.linalg.inv(A) # [[1. 0.] [0. 1.]] # Solving linear systems: Ax = b b = np.array([5, 11]) x = np.linalg.solve(A, b) # [1. 2.] print(A @ x) # [5. 11.] ← verify # Eigenvalues and eigenvectors eigenvalues, eigenvectors = np.linalg.eig(A) print(eigenvalues) # [-0.372 5.372] # Norm (magnitude) v = np.array([3, 4]) np.linalg.norm(v) # 5.0 (Euclidean = sqrt(3²+4²)) np.linalg.norm(A) # Frobenius norm (default for matrices) np.linalg.norm(v, ord=1) # L1 norm = 7.0 np.linalg.norm(v, ord=np.inf) # max norm = 4.0 # Rank np.linalg.matrix_rank(A) # 2 # Trace (sum of diagonal) np.trace(A) # 5 (1 + 4) # Singular Value Decomposition U, s, Vt = np.linalg.svd(A) # Cross product and dot product a = np.array([1, 0, 0]) b = np.array([0, 1, 0]) np.cross(a, b) # [0 0 1] (z-axis) np.dot(a, b) # 0 (orthogonal vectors) 14. Working with Files # Save a single array (binary .npy format) a = np.array([1, 2, 3, 4, 5]) np.save(\u0026#39;my_array.npy\u0026#39;, a) loaded = np.load(\u0026#39;my_array.npy\u0026#39;) # Save multiple arrays (binary .npz format) x = np.arange(10) y = np.linspace(0, 1, 10) np.savez(\u0026#39;my_data.npz\u0026#39;, x=x, y=y) data = np.load(\u0026#39;my_data.npz\u0026#39;) data[\u0026#39;x\u0026#39;] # retrieve x data[\u0026#39;y\u0026#39;] # retrieve y # Compressed version np.savez_compressed(\u0026#39;my_data.npz\u0026#39;, x=x, y=y) # Save to text / CSV m = np.array([[1, 2, 3], [4, 5, 6]]) np.savetxt(\u0026#39;matrix.csv\u0026#39;, m, delimiter=\u0026#39;,\u0026#39;, fmt=\u0026#39;%d\u0026#39;) np.savetxt(\u0026#39;matrix.txt\u0026#39;, m, header=\u0026#39;col1 col2 col3\u0026#39;) # Load from text / CSV loaded = np.loadtxt(\u0026#39;matrix.csv\u0026#39;, delimiter=\u0026#39;,\u0026#39;) loaded = np.genfromtxt(\u0026#39;data.csv\u0026#39;, delimiter=\u0026#39;,\u0026#39;, skip_header=1, # skip header row filling_values=0) # fill NaNs with 0 15. Performance Tips and Best Practices Vectorize - avoid Python loops # SLOW - Python loop a = np.arange(1_000_000) result = [] for x in a: result.append(x * 2 + 1) # FAST - vectorized result = a * 2 + 1 Use views over copies when possible a = np.arange(1_000_000) # Returns a VIEW (fast, no memory copy) b = a[::2] b = a.ravel() # Returns a COPY (slower, allocates memory) c = a[::2].copy() c = a.flatten() Pre-allocate arrays # SLOW - growing array in a loop result = np.array([]) for i in range(1000): result = np.append(result, i**2) # new array every iteration! # FAST - pre-allocate result = np.empty(1000) for i in range(1000): result[i] = i**2 # BEST - vectorize entirely result = np.arange(1000) ** 2 Choose the right dtype # Use smaller dtypes when you don\u0026#39;t need full precision a = np.arange(100, dtype=np.int8) # 100 bytes b = np.arange(100, dtype=np.int64) # 800 bytes - 8x larger! c = np.zeros((1000, 1000), dtype=np.float32) # 4 MB d = np.zeros((1000, 1000), dtype=np.float64) # 8 MB Useful utility functions # Check if two arrays are equal (including floats) np.array_equal(a, b) # exact equality np.allclose(a, b) # approximate equality (for floats) np.allclose(a, b, atol=1e-6, rtol=1e-5) # Copy a.copy() # explicit copy np.copy(a) # same # Check for NaN and inf np.isnan(a) # boolean mask where NaN np.isinf(a) # boolean mask where ±inf np.isfinite(a) # boolean mask where finite np.nan_to_num(a) # replace NaN with 0, inf with large number # Type checking np.isscalar(5) # True np.isreal(a) # True for all-real arrays Quick Reference Card # ── CREATION ───────────────────────────────────────── np.array([1,2,3]) # from list np.zeros((r,c)) # zeros np.ones((r,c)) # ones np.eye(n) # identity np.arange(start,stop,step) # range np.linspace(start,stop,n) # n evenly spaced # ── PROPERTIES ─────────────────────────────────────── a.shape a.ndim a.size a.dtype a.itemsize # ── INDEXING ───────────────────────────────────────── a[i] a[i:j] a[::step] a[-1] m[r,c] m[r,:] m[:,c] m[r1:r2, c1:c2] # ── RESHAPING ──────────────────────────────────────── a.reshape(r,c) a.T a.flatten() a.ravel() np.vstack([a,b]) np.hstack([a,b]) np.concatenate([a,b]) # ── MATH ───────────────────────────────────────────── a + b a * b a ** 2 a / b # element-wise A @ B np.dot(A,B) # matrix multiply np.sqrt(a) np.abs(a) np.exp(a) # ufuncs # ── AGGREGATIONS ───────────────────────────────────── a.sum(axis=0) a.mean() a.std() a.min() a.max() a.argmin() a.argmax() np.cumsum(a) np.percentile(a, 75) # ── BOOLEAN / MASKING ──────────────────────────────── a[a \u0026gt; 5] # filter np.where(a \u0026gt; 5, a, 0) # conditional np.any(a \u0026gt; 5) np.all(a \u0026gt; 0) # ── RANDOM ─────────────────────────────────────────── rng = np.random.default_rng(42) rng.random(n) rng.integers(lo,hi,n) rng.normal(μ,σ,n) # ── LINEAR ALGEBRA ─────────────────────────────────── np.linalg.inv(A) np.linalg.det(A) np.linalg.solve(A,b) np.linalg.eig(A) End of NumPy Masterclass\n","permalink":"https://blogs.codeverra.com/blog/numpy-masterclass/","summary":"\u003ch1 id=\"numpy-masterclass---from-basics-to-advanced\"\u003eNumPy Masterclass - From Basics to Advanced\u003c/h1\u003e\n\u003cblockquote\u003e\n\u003cp\u003e\u003cstrong\u003ePrerequisites:\u003c/strong\u003e Python basics, lists, loops, functions\u003cbr\u003e\n\u003cstrong\u003eInstall:\u003c/strong\u003e \u003ccode\u003epip install numpy\u003c/code\u003e\u003cbr\u003e\n\u003cstrong\u003eImport convention:\u003c/strong\u003e \u003ccode\u003eimport numpy as np\u003c/code\u003e\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003chr\u003e\n\u003ch2 id=\"table-of-contents\"\u003eTable of Contents\u003c/h2\u003e\n\u003col\u003e\n\u003cli\u003e\u003ca href=\"#1-what-is-numpy-and-why-use-it\"\u003eWhat is NumPy and Why Use It?\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#2-creating-arrays\"\u003eCreating Arrays\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#3-array-properties\"\u003eArray Properties\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#4-indexing-and-slicing\"\u003eIndexing and Slicing\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#5-modifying-arrays\"\u003eModifying Arrays\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#6-reshaping-and-stacking\"\u003eReshaping and Stacking\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#7-math-operations-and-broadcasting\"\u003eMath Operations and Broadcasting\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#8-universal-functions-ufuncs\"\u003eUniversal Functions (ufuncs)\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#9-aggregations-and-statistics\"\u003eAggregations and Statistics\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#10-sorting-and-searching\"\u003eSorting and Searching\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#11-boolean-operations-and-masking\"\u003eBoolean Operations and Masking\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#12-random-number-generation\"\u003eRandom Number Generation\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#13-linear-algebra\"\u003eLinear Algebra\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#14-working-with-files\"\u003eWorking with Files\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#15-performance-tips-and-best-practices\"\u003ePerformance Tips and Best Practices\u003c/a\u003e\u003c/li\u003e\n\u003c/ol\u003e\n\u003chr\u003e\n\u003ch2 id=\"1-what-is-numpy-and-why-use-it\"\u003e1. What is NumPy and Why Use It?\u003c/h2\u003e\n\u003cp\u003eNumPy (Numerical Python) is the foundation of the entire Python data science stack. Pandas, Matplotlib, Scikit-learn, and TensorFlow all build on top of it.\u003c/p\u003e","tags":["python","numpy","data-analysis"],"title":"Numpy Masterclass - Beginner to Advanced"},{"content":"Pandas Masterclass - From Basics to Advanced Prerequisites: Python basics, NumPy fundamentals\nInstall: pip install pandas openpyxl xlrd\nImport convention: import pandas as pd\nTable of Contents What is Pandas and Why Use It? Series - The 1D Structure DataFrame - The 2D Structure Loading and Saving Data Exploring Your Data Selecting and Filtering Adding and Modifying Columns Sorting Cleaning Data - Handling Missing Values Cleaning Data - Duplicates and Types String Operations Datetime Operations Groupby and Aggregation Merging, Joining and Concatenating Pivot Tables and Reshaping Apply, Map and Lambda Functions Plotting with Pandas Performance Tips 1. What is Pandas and Why Use It? Pandas provides two core data structures - Series (1D) and DataFrame (2D) - that make working with labeled, tabular data as natural as working with a spreadsheet, but with the full power of Python.\nWhat Pandas gives you Read data from CSV, Excel, JSON, SQL, HTML, and more Clean and transform messy real-world data Filter, group, aggregate, join datasets Time series analysis Integrates seamlessly with NumPy, Matplotlib, and Scikit-learn Setup import pandas as pd import numpy as np # Check version print(pd.__version__) 2. Series - The 1D Structure A Series is a one-dimensional labeled array. Think of it as a column with an index.\nCreating a Series # From a list - index defaults to 0, 1, 2, ... s = pd.Series([10, 20, 30, 40, 50]) print(s) # 0 10 # 1 20 # 2 30 # 3 40 # 4 50 # dtype: int64 # With a custom index s = pd.Series([10, 20, 30], index=[\u0026#39;a\u0026#39;, \u0026#39;b\u0026#39;, \u0026#39;c\u0026#39;]) print(s) # a 10 # b 20 # c 30 # From a dictionary (keys become index) s = pd.Series({\u0026#39;python\u0026#39;: 90, \u0026#39;java\u0026#39;: 75, \u0026#39;c++\u0026#39;: 60}) print(s) # python 90 # java 75 # c++ 60 # From a scalar (fills all positions) s = pd.Series(42, index=[\u0026#39;x\u0026#39;, \u0026#39;y\u0026#39;, \u0026#39;z\u0026#39;]) # x 42 # y 42 # z 42 Accessing Series values s = pd.Series([10, 20, 30, 40], index=[\u0026#39;a\u0026#39;, \u0026#39;b\u0026#39;, \u0026#39;c\u0026#39;, \u0026#39;d\u0026#39;]) # By label (index name) s[\u0026#39;a\u0026#39;] # 10 s[[\u0026#39;a\u0026#39;, \u0026#39;c\u0026#39;]] # returns Series with \u0026#39;a\u0026#39; and \u0026#39;c\u0026#39; # By position s.iloc[0] # 10 s.iloc[-1] # 40 s.iloc[1:3] # \u0026#39;b\u0026#39; and \u0026#39;c\u0026#39; # Attribute access (if index is a valid Python identifier) s.a # 10 Series operations s = pd.Series([1, 2, 3, 4, 5]) # All NumPy-style operations work s * 2 # 2 4 6 8 10 s + 10 # 11 12 13 14 15 s ** 2 # 1 4 9 16 25 s[s \u0026gt; 3] # 4, 5 # Alignment on index - missing = NaN s1 = pd.Series([1, 2, 3], index=[\u0026#39;a\u0026#39;, \u0026#39;b\u0026#39;, \u0026#39;c\u0026#39;]) s2 = pd.Series([10, 20, 30], index=[\u0026#39;b\u0026#39;, \u0026#39;c\u0026#39;, \u0026#39;d\u0026#39;]) s1 + s2 # a=NaN, b=22, c=53, d=NaN # Statistics s.sum() # 15 s.mean() # 3.0 s.std() # 1.58 s.min() # 1 s.max() # 5 s.describe() # summary stats # Value operations s.unique() # array of unique values s.nunique() # count of unique values s.value_counts() # frequency of each value s.sort_values() # sort by value s.sort_index() # sort by index 3. DataFrame - The 2D Structure A DataFrame is a 2D table - rows and columns, each column being a Series sharing the same index.\nCreating a DataFrame # From a dictionary (most common) df = pd.DataFrame({ \u0026#39;name\u0026#39;: [\u0026#39;Ramesh\u0026#39;, \u0026#39;Mahesh\u0026#39;, \u0026#39;Suresh\u0026#39;, \u0026#39;Devesh\u0026#39;, \u0026#39;Rajesh\u0026#39;], \u0026#39;age\u0026#39;: [25, 30, 35, 28, 22], \u0026#39;salary\u0026#39;: [55000, 72000, 85000, 61000, 48000], \u0026#39;dept\u0026#39;: [\u0026#39;Eng\u0026#39;, \u0026#39;Mkt\u0026#39;, \u0026#39;Eng\u0026#39;, \u0026#39;Mkt\u0026#39;, \u0026#39;HR\u0026#39;], \u0026#39;active\u0026#39;: [True, True, True, False, True] }) # From a list of dicts (each dict = one row) df = pd.DataFrame([ {\u0026#39;name\u0026#39;: \u0026#39;Ramesh\u0026#39;, \u0026#39;age\u0026#39;: 25, \u0026#39;city\u0026#39;: \u0026#39;NYC\u0026#39;}, {\u0026#39;name\u0026#39;: \u0026#39;Mahesh\u0026#39;, \u0026#39;age\u0026#39;: 30, \u0026#39;city\u0026#39;: \u0026#39;LA\u0026#39;}, {\u0026#39;name\u0026#39;: \u0026#39;Suresh\u0026#39;, \u0026#39;age\u0026#39;: 35}, # missing \u0026#39;city\u0026#39; → NaN ]) # From a NumPy array arr = np.random.randint(0, 100, (5, 3)) df = pd.DataFrame(arr, columns=[\u0026#39;A\u0026#39;, \u0026#39;B\u0026#39;, \u0026#39;C\u0026#39;]) # From a CSV (covered in section 4) df = pd.read_csv(\u0026#39;file.csv\u0026#39;) # Empty DataFrame with defined columns df = pd.DataFrame(columns=[\u0026#39;name\u0026#39;, \u0026#39;age\u0026#39;, \u0026#39;salary\u0026#39;]) DataFrame structure df = pd.DataFrame({ \u0026#39;name\u0026#39;: [\u0026#39;Ramesh\u0026#39;, \u0026#39;Mahesh\u0026#39;, \u0026#39;Suresh\u0026#39;], \u0026#39;age\u0026#39;: [25, 30, 35], \u0026#39;salary\u0026#39;: [55000, 72000, 85000] }) df.columns # Index([\u0026#39;name\u0026#39;, \u0026#39;age\u0026#39;, \u0026#39;salary\u0026#39;]) df.index # RangeIndex(start=0, stop=3, step=1) df.dtypes # name: object, age: int64, salary: int64 df.values # NumPy array of all values df.shape # (3, 3) df.size # 9 (total elements) df.ndim # 2 4. Loading and Saving Data Reading files # CSV - most common df = pd.read_csv(\u0026#39;data.csv\u0026#39;) # With options df = pd.read_csv(\u0026#39;data.csv\u0026#39;, sep=\u0026#39;;\u0026#39;, # different separator (e.g., semicolon) header=0, # row number to use as column names index_col=\u0026#39;id\u0026#39;, # use this column as the index usecols=[\u0026#39;name\u0026#39;, \u0026#39;age\u0026#39;], # only load specific columns nrows=100, # only load first 100 rows skiprows=[1, 2], # skip specific rows dtype={\u0026#39;age\u0026#39;: int, \u0026#39;salary\u0026#39;: float}, # force column types parse_dates=[\u0026#39;date\u0026#39;, \u0026#39;created_at\u0026#39;], # auto-parse as datetime na_values=[\u0026#39;?\u0026#39;, \u0026#39;N/A\u0026#39;, \u0026#39;none\u0026#39;], # treat as NaN encoding=\u0026#39;utf-8\u0026#39;, # file encoding thousands=\u0026#39;,\u0026#39;, # \u0026#39;1,000\u0026#39; → 1000 decimal=\u0026#39;.\u0026#39;, # decimal separator ) # Excel df = pd.read_excel(\u0026#39;data.xlsx\u0026#39;) df = pd.read_excel(\u0026#39;data.xlsx\u0026#39;, sheet_name=\u0026#39;Sheet2\u0026#39;) df = pd.read_excel(\u0026#39;data.xlsx\u0026#39;, sheet_name=0) # first sheet # JSON df = pd.read_json(\u0026#39;data.json\u0026#39;) # From clipboard (great for pasting from spreadsheets!) df = pd.read_clipboard() # SQL import sqlite3 conn = sqlite3.connect(\u0026#39;database.db\u0026#39;) df = pd.read_sql(\u0026#39;SELECT * FROM employees\u0026#39;, conn) # HTML table on a webpage tables = pd.read_html(\u0026#39;https://example.com/table-page\u0026#39;) df = tables[0] # first table found # From a URL directly df = pd.read_csv(\u0026#39;https://raw.githubusercontent.com/..../data.csv\u0026#39;) # Parquet (fast columnar format) df = pd.read_parquet(\u0026#39;data.parquet\u0026#39;) Saving files # CSV df.to_csv(\u0026#39;output.csv\u0026#39;, index=False) # index=False avoids saving the row numbers df.to_csv(\u0026#39;output.csv\u0026#39;, sep=\u0026#39;\\t\u0026#39;) # tab-separated df.to_csv(\u0026#39;output.csv\u0026#39;, columns=[\u0026#39;name\u0026#39;, \u0026#39;age\u0026#39;]) # only these columns # Excel df.to_excel(\u0026#39;output.xlsx\u0026#39;, index=False) df.to_excel(\u0026#39;output.xlsx\u0026#39;, sheet_name=\u0026#39;Results\u0026#39;, index=False) # Multiple sheets with pd.ExcelWriter(\u0026#39;output.xlsx\u0026#39;) as writer: df1.to_excel(writer, sheet_name=\u0026#39;Sheet1\u0026#39;, index=False) df2.to_excel(writer, sheet_name=\u0026#39;Sheet2\u0026#39;, index=False) # JSON df.to_json(\u0026#39;output.json\u0026#39;, orient=\u0026#39;records\u0026#39;) # list of records # Parquet df.to_parquet(\u0026#39;output.parquet\u0026#39;, index=False) # SQL df.to_sql(\u0026#39;table_name\u0026#39;, conn, if_exists=\u0026#39;replace\u0026#39;, index=False) 5. Exploring Your Data Always do these steps after loading new data.\ndf = pd.read_csv(\u0026#39;employees.csv\u0026#39;) # --- Shape and size --- df.shape # (rows, cols) e.g. (1000, 8) len(df) # number of rows df.columns.tolist() # list of column names # --- First/last rows --- df.head() # first 5 rows (default) df.head(10) # first 10 rows df.tail(3) # last 3 rows df.sample(5) # 5 random rows df.sample(frac=0.1) # 10% random sample # --- Data types and memory --- df.info() # \u0026lt;class \u0026#39;pandas.core.frame.DataFrame\u0026#39;\u0026gt; # RangeIndex: 1000 entries, 0 to 999 # Data columns (total 4 columns): # # Column Non-Null Count Dtype # --- ------ -------------- ----- # 0 name 1000 non-null object # 1 age 980 non-null float64 ← 20 missing values! # 2 salary 1000 non-null int64 # 3 dept 1000 non-null object df.dtypes # column data types df.memory_usage(deep=True) # memory per column # --- Statistics --- df.describe() # count, mean, std, min, 25%, 50%, 75%, max for numeric cols df.describe(include=\u0026#39;all\u0026#39;) # include non-numeric df.describe(include=[\u0026#39;object\u0026#39;]) # only string columns df[\u0026#39;salary\u0026#39;].describe() # for one column # --- Missing data --- df.isnull().sum() # count NaN per column df.isnull().mean() * 100 # % missing per column df.isnull().any(axis=1) # True for rows with any NaN df.notnull().all(axis=1) # True for complete rows # --- Unique values --- df[\u0026#39;dept\u0026#39;].unique() # array of unique values df[\u0026#39;dept\u0026#39;].nunique() # count of unique values df[\u0026#39;dept\u0026#39;].value_counts() # frequency table (sorted) df[\u0026#39;dept\u0026#39;].value_counts(normalize=True) # as proportions df[\u0026#39;dept\u0026#39;].value_counts(dropna=False) # include NaN count # --- Correlations --- df.corr() # correlation matrix (numeric cols) df[\u0026#39;salary\u0026#39;].corr(df[\u0026#39;age\u0026#39;]) # correlation between two cols 6. Selecting and Filtering Column selection # Single column → returns Series df[\u0026#39;name\u0026#39;] df.name # dot notation (only works if name is a valid Python identifier) # Multiple columns → returns DataFrame df[[\u0026#39;name\u0026#39;, \u0026#39;salary\u0026#39;]] df[[\u0026#39;name\u0026#39;, \u0026#39;age\u0026#39;, \u0026#39;dept\u0026#39;]] Row selection with .loc (label-based) # .loc[row_selector, col_selector] # row_selector: index label, list, slice, or boolean array # col_selector: column name, list, or slice df.loc[0] # row with index label 0 df.loc[2] # row with index label 2 df.loc[[0, 2, 4]] # rows with labels 0, 2, 4 df.loc[0:3] # rows 0 THROUGH 3 (INCLUSIVE, unlike Python!) df.loc[0, \u0026#39;name\u0026#39;] # single value df.loc[0, [\u0026#39;name\u0026#39;, \u0026#39;age\u0026#39;]] # row 0, specific columns df.loc[:, \u0026#39;age\u0026#39;] # all rows, \u0026#39;age\u0026#39; column df.loc[:, \u0026#39;age\u0026#39;:\u0026#39;salary\u0026#39;] # all rows, columns age through salary df.loc[0:2, \u0026#39;name\u0026#39;:\u0026#39;dept\u0026#39;] # rows 0-2, columns name through dept Row selection with .iloc (position-based) # .iloc[row_pos, col_pos] - always uses integers, end is EXCLUSIVE df.iloc[0] # first row df.iloc[-1] # last row df.iloc[[0, 2, 4]] # rows at positions 0, 2, 4 df.iloc[0:3] # rows 0, 1, 2 (3 is EXCLUDED) df.iloc[0, 1] # row 0, col 1 df.iloc[:, 0] # all rows, first column df.iloc[0:3, 1:4] # rows 0-2, cols 1-3 df.iloc[-5:] # last 5 rows loc vs iloc summary:\n.loc uses labels (index values, column names) - stop is inclusive\n.iloc uses integers (positions 0, 1, 2\u0026hellip;) - stop is exclusive\nBoolean filtering # Condition creates a boolean Series df[df[\u0026#39;salary\u0026#39;] \u0026gt; 60000] df[df[\u0026#39;dept\u0026#39;] == \u0026#39;Eng\u0026#39;] df[df[\u0026#39;active\u0026#39;] == True] df[df[\u0026#39;name\u0026#39;] != \u0026#39;Bob\u0026#39;] # Multiple conditions - use \u0026amp; (and), | (or), ~ (not) # MUST use parentheses around each condition! df[(df[\u0026#39;salary\u0026#39;] \u0026gt; 60000) \u0026amp; (df[\u0026#39;dept\u0026#39;] == \u0026#39;Eng\u0026#39;)] df[(df[\u0026#39;age\u0026#39;] \u0026lt; 30) | (df[\u0026#39;salary\u0026#39;] \u0026gt; 80000)] df[~(df[\u0026#39;dept\u0026#39;] == \u0026#39;HR\u0026#39;)] # all except HR # .query() - cleaner syntax for complex filters df.query(\u0026#39;salary \u0026gt; 60000\u0026#39;) df.query(\u0026#39;dept == \u0026#34;Eng\u0026#34;\u0026#39;) df.query(\u0026#39;age \u0026gt; 25 and salary \u0026gt; 60000\u0026#39;) df.query(\u0026#39;dept in [\u0026#34;Eng\u0026#34;, \u0026#34;HR\u0026#34;]\u0026#39;) df.query(\u0026#39;salary \u0026gt; @threshold\u0026#39;) # reference a Python variable with @ # .isin() - check membership df[df[\u0026#39;dept\u0026#39;].isin([\u0026#39;Eng\u0026#39;, \u0026#39;Mkt\u0026#39;])] df[~df[\u0026#39;dept\u0026#39;].isin([\u0026#39;HR\u0026#39;])] # exclude HR # .between() - range filter (inclusive) df[df[\u0026#39;age\u0026#39;].between(25, 35)] df[df[\u0026#39;salary\u0026#39;].between(50000, 80000)] # String methods in filters df[df[\u0026#39;name\u0026#39;].str.startswith(\u0026#39;A\u0026#39;)] df[df[\u0026#39;name\u0026#39;].str.contains(\u0026#39;ali\u0026#39;, case=False)] 7. Adding and Modifying Columns df = pd.DataFrame({ \u0026#39;name\u0026#39;: [\u0026#39;Alice\u0026#39;, \u0026#39;Bob\u0026#39;, \u0026#39;Carol\u0026#39;], \u0026#39;salary\u0026#39;: [55000, 72000, 85000], \u0026#39;age\u0026#39;: [25, 30, 35] }) # Add a new column df[\u0026#39;bonus\u0026#39;] = df[\u0026#39;salary\u0026#39;] * 0.10 df[\u0026#39;total\u0026#39;] = df[\u0026#39;salary\u0026#39;] + df[\u0026#39;bonus\u0026#39;] df[\u0026#39;senior\u0026#39;] = df[\u0026#39;age\u0026#39;] \u0026gt;= 30 # boolean column df[\u0026#39;constant\u0026#39;] = 100 # scalar broadcasts # Modify existing column df[\u0026#39;salary\u0026#39;] = df[\u0026#39;salary\u0026#39;] * 1.05 # 5% raise df[\u0026#39;name\u0026#39;] = df[\u0026#39;name\u0026#39;].str.upper() # Conditional column with np.where df[\u0026#39;level\u0026#39;] = np.where(df[\u0026#39;salary\u0026#39;] \u0026gt; 70000, \u0026#39;Senior\u0026#39;, \u0026#39;Junior\u0026#39;) # Multiple conditions with np.select conditions = [ df[\u0026#39;salary\u0026#39;] \u0026lt; 60000, df[\u0026#39;salary\u0026#39;].between(60000, 80000), df[\u0026#39;salary\u0026#39;] \u0026gt; 80000 ] choices = [\u0026#39;Junior\u0026#39;, \u0026#39;Mid\u0026#39;, \u0026#39;Senior\u0026#39;] df[\u0026#39;level\u0026#39;] = np.select(conditions, choices, default=\u0026#39;Unknown\u0026#39;) # rename columns df.rename(columns={\u0026#39;name\u0026#39;: \u0026#39;full_name\u0026#39;, \u0026#39;salary\u0026#39;: \u0026#39;annual_salary\u0026#39;}, inplace=True) # rename all columns at once df.columns = [\u0026#39;full_name\u0026#39;, \u0026#39;annual_salary\u0026#39;, \u0026#39;age\u0026#39;, \u0026#39;bonus\u0026#39;, \u0026#39;total\u0026#39;, \u0026#39;senior\u0026#39;, \u0026#39;constant\u0026#39;, \u0026#39;level\u0026#39;] # drop columns df.drop(columns=[\u0026#39;constant\u0026#39;, \u0026#39;bonus\u0026#39;], inplace=True) # or equivalently: df = df.drop(columns=[\u0026#39;constant\u0026#39;]) # reorder columns df = df[[\u0026#39;name\u0026#39;, \u0026#39;dept\u0026#39;, \u0026#39;salary\u0026#39;, \u0026#39;age\u0026#39;]] # select only these, in this order # insert column at specific position df.insert(2, \u0026#39;city\u0026#39;, [\u0026#39;NYC\u0026#39;, \u0026#39;LA\u0026#39;, \u0026#39;Chicago\u0026#39;]) 8. Sorting df = pd.DataFrame({ \u0026#39;name\u0026#39;: [\u0026#39;Ramesh\u0026#39;, \u0026#39;Mahesh\u0026#39;, \u0026#39;Suresh\u0026#39;, \u0026#39;Devesh\u0026#39;], \u0026#39;age\u0026#39;: [25, 30, 25, 28], \u0026#39;salary\u0026#39;: [55000, 72000, 85000, 61000], \u0026#39;dept\u0026#39;: [\u0026#39;Eng\u0026#39;, \u0026#39;Mkt\u0026#39;, \u0026#39;Eng\u0026#39;, \u0026#39;Mkt\u0026#39;] }) # Sort by a single column df.sort_values(\u0026#39;salary\u0026#39;) # ascending (default) df.sort_values(\u0026#39;salary\u0026#39;, ascending=False) # descending # Sort by multiple columns df.sort_values([\u0026#39;dept\u0026#39;, \u0026#39;salary\u0026#39;]) # by dept, then salary within dept df.sort_values([\u0026#39;dept\u0026#39;, \u0026#39;salary\u0026#39;], ascending=[True, False]) # dept asc, salary desc # Sort by index df.sort_index() df.sort_index(ascending=False) # inplace vs returning new df.sort_values(\u0026#39;salary\u0026#39;, inplace=True) # modifies df new_df = df.sort_values(\u0026#39;salary\u0026#39;) # returns new sorted df # nlargest / nsmallest - get top/bottom N rows df.nlargest(3, \u0026#39;salary\u0026#39;) # top 3 by salary df.nsmallest(2, \u0026#39;age\u0026#39;) # bottom 2 by age df.nlargest(3, [\u0026#39;salary\u0026#39;, \u0026#39;age\u0026#39;]) # sort by multiple cols # reset index after sorting df.sort_values(\u0026#39;salary\u0026#39;).reset_index(drop=True) 9. Cleaning Data - Handling Missing Values Missing data in Pandas is represented as NaN (Not a Number) for numeric columns, or None/NaN for objects.\nDetecting missing values df.isnull() # boolean DataFrame, True where NaN df.notnull() # opposite df.isnull().sum() # count NaN per column df.isnull().sum(axis=1) # count NaN per row df.isnull().any() # True if column has ANY NaN df.isnull().all() # True if column is ALL NaN # Find rows with any missing data df[df.isnull().any(axis=1)] # Percentage missing (df.isnull().sum() / len(df)) * 100 Dropping missing values # Drop rows with ANY NaN df.dropna() # Drop rows where a SPECIFIC column is NaN df.dropna(subset=[\u0026#39;age\u0026#39;]) df.dropna(subset=[\u0026#39;age\u0026#39;, \u0026#39;salary\u0026#39;]) # drop if EITHER is NaN # Drop rows only if ALL values are NaN df.dropna(how=\u0026#39;all\u0026#39;) # Keep rows with at least N non-NaN values df.dropna(thresh=3) # keep row only if \u0026gt;= 3 non-null values # Drop COLUMNS with any NaN df.dropna(axis=1) # inplace df.dropna(inplace=True) Filling missing values # Fill all NaN with a single value df.fillna(0) df.fillna(\u0026#39;unknown\u0026#39;) df.fillna(False) # Fill specific columns differently df.fillna({\u0026#39;age\u0026#39;: 0, \u0026#39;salary\u0026#39;: df[\u0026#39;salary\u0026#39;].mean(), \u0026#39;name\u0026#39;: \u0026#39;Unknown\u0026#39;}) # Fill with column statistics df[\u0026#39;age\u0026#39;].fillna(df[\u0026#39;age\u0026#39;].mean()) df[\u0026#39;age\u0026#39;].fillna(df[\u0026#39;age\u0026#39;].median()) df[\u0026#39;dept\u0026#39;].fillna(df[\u0026#39;dept\u0026#39;].mode()[0]) # most common value # Forward fill - propagate last valid value forward df.fillna(method=\u0026#39;ffill\u0026#39;) # fill NaN with the value above it df[\u0026#39;salary\u0026#39;].ffill() # forward fill a column # Backward fill - propagate next valid value backward df.fillna(method=\u0026#39;bfill\u0026#39;) # Interpolate - for time series / sequential data df[\u0026#39;temperature\u0026#39;].interpolate() # linear interpolation df[\u0026#39;temperature\u0026#39;].interpolate(method=\u0026#39;polynomial\u0026#39;, order=2) # Limit - only fill a certain number of consecutive NaNs df.fillna(method=\u0026#39;ffill\u0026#39;, limit=2) 10. Cleaning Data - Duplicates and Types Handling duplicates # Find duplicate rows df.duplicated() # True for duplicate rows df.duplicated(subset=[\u0026#39;name\u0026#39;]) # duplicated based on \u0026#39;name\u0026#39; only df.duplicated(keep=\u0026#39;first\u0026#39;) # keep=first: marks all but first as dup df.duplicated(keep=\u0026#39;last\u0026#39;) # keep=last: marks all but last df.duplicated(keep=False) # marks ALL copies as duplicates # Count duplicates df.duplicated().sum() # View duplicated rows df[df.duplicated()] # Drop duplicates df.drop_duplicates() # drop all duplicate rows df.drop_duplicates(subset=[\u0026#39;name\u0026#39;]) # drop based on \u0026#39;name\u0026#39; column df.drop_duplicates(subset=[\u0026#39;name\u0026#39;, \u0026#39;dept\u0026#39;]) df.drop_duplicates(keep=\u0026#39;last\u0026#39;) # keep the last occurrence df.drop_duplicates(inplace=True) Type conversion df.dtypes # inspect current types # Convert to numeric df[\u0026#39;salary\u0026#39;] = df[\u0026#39;salary\u0026#39;].astype(int) df[\u0026#39;salary\u0026#39;] = df[\u0026#39;salary\u0026#39;].astype(float) df[\u0026#39;price\u0026#39;] = pd.to_numeric(df[\u0026#39;price\u0026#39;]) df[\u0026#39;price\u0026#39;] = pd.to_numeric(df[\u0026#39;price\u0026#39;], errors=\u0026#39;coerce\u0026#39;) # NaN on failure # Convert to string df[\u0026#39;code\u0026#39;] = df[\u0026#39;code\u0026#39;].astype(str) # Convert to boolean df[\u0026#39;active\u0026#39;] = df[\u0026#39;active\u0026#39;].astype(bool) # Convert to datetime df[\u0026#39;date\u0026#39;] = pd.to_datetime(df[\u0026#39;date\u0026#39;]) df[\u0026#39;date\u0026#39;] = pd.to_datetime(df[\u0026#39;date\u0026#39;], format=\u0026#39;%Y-%m-%d\u0026#39;) df[\u0026#39;date\u0026#39;] = pd.to_datetime(df[\u0026#39;date\u0026#39;], errors=\u0026#39;coerce\u0026#39;) # NaN on failure # Convert to category (saves memory for low-cardinality string cols) df[\u0026#39;dept\u0026#39;] = df[\u0026#39;dept\u0026#39;].astype(\u0026#39;category\u0026#39;) df[\u0026#39;dept\u0026#39;].cat.categories # view categories df[\u0026#39;dept\u0026#39;].cat.codes # numeric code for each category # Replace values df.replace(\u0026#39;N/A\u0026#39;, np.nan) df.replace({\u0026#39;dept\u0026#39;: {\u0026#39;Eng\u0026#39;: \u0026#39;Engineering\u0026#39;, \u0026#39;Mkt\u0026#39;: \u0026#39;Marketing\u0026#39;}}) df[\u0026#39;salary\u0026#39;].replace(0, np.nan) 11. String Operations Access string methods via the .str accessor on a Series.\ns = pd.Series([\u0026#39;Alice Smith\u0026#39;, \u0026#39; Bob Jones \u0026#39;, \u0026#39;carol Brown\u0026#39;, \u0026#39;DAVE WILSON\u0026#39;]) # Case s.str.upper() # \u0026#39;ALICE SMITH\u0026#39;, ... s.str.lower() # \u0026#39;alice smith\u0026#39;, ... s.str.title() # \u0026#39;Alice Smith\u0026#39;, ... # Strip whitespace s.str.strip() # remove leading/trailing spaces s.str.lstrip() # remove leading only s.str.rstrip() # remove trailing only # Check contents (returns boolean Series) s.str.contains(\u0026#39;Smith\u0026#39;) # case-sensitive s.str.contains(\u0026#39;smith\u0026#39;, case=False) # case-insensitive s.str.contains(r\u0026#39;\\d+\u0026#39;, regex=True) # regex: contains digits s.str.startswith(\u0026#39;A\u0026#39;) s.str.endswith(\u0026#39;n\u0026#39;) s.str.isdigit() s.str.isalpha() s.str.isnumeric() # Extract and modify s.str.replace(\u0026#39;Smith\u0026#39;, \u0026#39;S.\u0026#39;) s.str.replace(r\u0026#39;\\s+\u0026#39;, \u0026#39; \u0026#39;, regex=True) # collapse multiple spaces s.str.split(\u0026#39; \u0026#39;) # list of words s.str.split(\u0026#39; \u0026#39;, expand=True) # split into separate columns s.str.split(\u0026#39; \u0026#39;).str[0] # first word only s.str.get(0) # first character # Length and padding s.str.len() s.str.pad(width=20, side=\u0026#39;right\u0026#39;, fillchar=\u0026#39;-\u0026#39;) s.str.zfill(5) # zero-pad (for numbers as strings) # Slicing s.str[0:5] # first 5 characters s.str[-3:] # last 3 characters # Extract with regex emails = pd.Series([\u0026#39;alice@example.com\u0026#39;, \u0026#39;bob@gmail.com\u0026#39;]) emails.str.extract(r\u0026#39;(\\w+)@(\\w+\\.\\w+)\u0026#39;) # extract groups emails.str.extractall(r\u0026#39;(\\w+)\u0026#39;) # extract all matches emails.str.findall(r\u0026#39;\\w+\u0026#39;) # find all matches # Count occurrences s.str.count(\u0026#39;l\u0026#39;) # count of \u0026#39;l\u0026#39; in each string # Join parts = pd.Series([[\u0026#39;hello\u0026#39;, \u0026#39;world\u0026#39;], [\u0026#39;foo\u0026#39;, \u0026#39;bar\u0026#39;]]) parts.str.join(\u0026#39;-\u0026#39;) # \u0026#39;hello-world\u0026#39;, \u0026#39;foo-bar\u0026#39; 12. Datetime Operations # Parse dates df[\u0026#39;date\u0026#39;] = pd.to_datetime(df[\u0026#39;date\u0026#39;]) # Create date range pd.date_range(start=\u0026#39;2024-01-01\u0026#39;, end=\u0026#39;2024-12-31\u0026#39;, freq=\u0026#39;D\u0026#39;) # daily pd.date_range(start=\u0026#39;2024-01-01\u0026#39;, periods=12, freq=\u0026#39;ME\u0026#39;) # monthly end pd.date_range(start=\u0026#39;2024-01-01\u0026#39;, periods=52, freq=\u0026#39;W\u0026#39;) # weekly # Access components via .dt accessor df[\u0026#39;date\u0026#39;].dt.year df[\u0026#39;date\u0026#39;].dt.month df[\u0026#39;date\u0026#39;].dt.day df[\u0026#39;date\u0026#39;].dt.hour df[\u0026#39;date\u0026#39;].dt.minute df[\u0026#39;date\u0026#39;].dt.second df[\u0026#39;date\u0026#39;].dt.day_of_week # 0=Monday, 6=Sunday df[\u0026#39;date\u0026#39;].dt.day_name() # \u0026#39;Monday\u0026#39;, \u0026#39;Tuesday\u0026#39;, ... df[\u0026#39;date\u0026#39;].dt.month_name() # \u0026#39;January\u0026#39;, ... df[\u0026#39;date\u0026#39;].dt.quarter # 1, 2, 3, or 4 df[\u0026#39;date\u0026#39;].dt.is_month_end # boolean df[\u0026#39;date\u0026#39;].dt.is_month_start # Date arithmetic df[\u0026#39;date\u0026#39;] + pd.Timedelta(days=7) # add 7 days df[\u0026#39;date\u0026#39;] - pd.Timedelta(weeks=1) # subtract 1 week df[\u0026#39;end_date\u0026#39;] - df[\u0026#39;start_date\u0026#39;] # timedelta between dates # Time offsets df[\u0026#39;date\u0026#39;] + pd.DateOffset(months=1) # add 1 month df[\u0026#39;date\u0026#39;] + pd.DateOffset(years=1) # add 1 year # Filtering by date df[df[\u0026#39;date\u0026#39;] \u0026gt; \u0026#39;2024-06-01\u0026#39;] df[df[\u0026#39;date\u0026#39;].between(\u0026#39;2024-01-01\u0026#39;, \u0026#39;2024-06-30\u0026#39;)] df[df[\u0026#39;date\u0026#39;].dt.year == 2024] df[df[\u0026#39;date\u0026#39;].dt.month == 3] # all March rows # Resample - aggregate by time period (requires datetime index) df = df.set_index(\u0026#39;date\u0026#39;) df.resample(\u0026#39;ME\u0026#39;).sum() # monthly totals df.resample(\u0026#39;W\u0026#39;).mean() # weekly averages df.resample(\u0026#39;Q\u0026#39;).max() # quarterly max df.resample(\u0026#39;h\u0026#39;).count() # hourly counts # Rolling window df[\u0026#39;sales\u0026#39;].rolling(7).mean() # 7-day moving average df[\u0026#39;sales\u0026#39;].rolling(30).sum() # 30-day rolling total df[\u0026#39;sales\u0026#39;].expanding().mean() # cumulative mean 13. Groupby and Aggregation groupby is one of Pandas\u0026rsquo; most powerful features. The pattern is: split → apply → combine.\nBasic groupby df = pd.DataFrame({ \u0026#39;name\u0026#39;: [\u0026#39;Alice\u0026#39;, \u0026#39;Bob\u0026#39;, \u0026#39;Carol\u0026#39;, \u0026#39;Dave\u0026#39;, \u0026#39;Eve\u0026#39;, \u0026#39;Frank\u0026#39;], \u0026#39;dept\u0026#39;: [\u0026#39;Eng\u0026#39;, \u0026#39;Mkt\u0026#39;, \u0026#39;Eng\u0026#39;, \u0026#39;Mkt\u0026#39;, \u0026#39;HR\u0026#39;, \u0026#39;Eng\u0026#39;], \u0026#39;city\u0026#39;: [\u0026#39;NYC\u0026#39;, \u0026#39;NYC\u0026#39;, \u0026#39;LA\u0026#39;, \u0026#39;LA\u0026#39;, \u0026#39;NYC\u0026#39;, \u0026#39;LA\u0026#39;], \u0026#39;salary\u0026#39;: [80000, 72000, 95000, 61000, 48000, 88000], \u0026#39;age\u0026#39;: [25, 30, 35, 28, 22, 32] }) # Group by one column, apply one aggregation df.groupby(\u0026#39;dept\u0026#39;)[\u0026#39;salary\u0026#39;].mean() # dept # Eng 87666.67 # HR 48000.00 # Mkt 66500.00 df.groupby(\u0026#39;dept\u0026#39;)[\u0026#39;salary\u0026#39;].sum() df.groupby(\u0026#39;dept\u0026#39;)[\u0026#39;salary\u0026#39;].max() df.groupby(\u0026#39;dept\u0026#39;)[\u0026#39;salary\u0026#39;].count() df.groupby(\u0026#39;dept\u0026#39;)[\u0026#39;salary\u0026#39;].std() # Multiple columns df.groupby(\u0026#39;dept\u0026#39;)[[\u0026#39;salary\u0026#39;, \u0026#39;age\u0026#39;]].mean() df.groupby(\u0026#39;dept\u0026#39;).size() # number of rows per group Multiple aggregations with .agg() # Multiple aggregations on one column df.groupby(\u0026#39;dept\u0026#39;)[\u0026#39;salary\u0026#39;].agg([\u0026#39;mean\u0026#39;, \u0026#39;min\u0026#39;, \u0026#39;max\u0026#39;, \u0026#39;count\u0026#39;]) # Different aggregations per column df.groupby(\u0026#39;dept\u0026#39;).agg({ \u0026#39;salary\u0026#39;: [\u0026#39;mean\u0026#39;, \u0026#39;max\u0026#39;, \u0026#39;std\u0026#39;], \u0026#39;age\u0026#39;: [\u0026#39;mean\u0026#39;, \u0026#39;min\u0026#39;, \u0026#39;max\u0026#39;], \u0026#39;name\u0026#39;: \u0026#39;count\u0026#39; }) # Custom aggregations with lambda df.groupby(\u0026#39;dept\u0026#39;)[\u0026#39;salary\u0026#39;].agg( avg_salary=\u0026#39;mean\u0026#39;, highest=\u0026#39;max\u0026#39;, range=lambda x: x.max() - x.min() ) # Named aggregations (cleaner column names) df.groupby(\u0026#39;dept\u0026#39;).agg( avg_salary=(\u0026#39;salary\u0026#39;, \u0026#39;mean\u0026#39;), max_age=(\u0026#39;age\u0026#39;, \u0026#39;max\u0026#39;), headcount=(\u0026#39;name\u0026#39;, \u0026#39;count\u0026#39;) ) Group by multiple columns df.groupby([\u0026#39;dept\u0026#39;, \u0026#39;city\u0026#39;])[\u0026#39;salary\u0026#39;].mean() df.groupby([\u0026#39;dept\u0026#39;, \u0026#39;city\u0026#39;]).agg({\u0026#39;salary\u0026#39;: \u0026#39;mean\u0026#39;, \u0026#39;age\u0026#39;: \u0026#39;count\u0026#39;}) # Access a specific group grouped = df.groupby(\u0026#39;dept\u0026#39;) grouped.get_group(\u0026#39;Eng\u0026#39;) # all Eng rows # Iterate over groups for dept_name, group_df in df.groupby(\u0026#39;dept\u0026#39;): print(f\u0026#34;\\n{dept_name}:\u0026#34;) print(group_df) transform - keep original shape # transform returns a Series with same length as the original df # Perfect for adding computed columns back df[\u0026#39;dept_avg_salary\u0026#39;] = df.groupby(\u0026#39;dept\u0026#39;)[\u0026#39;salary\u0026#39;].transform(\u0026#39;mean\u0026#39;) df[\u0026#39;dept_total\u0026#39;] = df.groupby(\u0026#39;dept\u0026#39;)[\u0026#39;salary\u0026#39;].transform(\u0026#39;sum\u0026#39;) df[\u0026#39;salary_rank\u0026#39;] = df.groupby(\u0026#39;dept\u0026#39;)[\u0026#39;salary\u0026#39;].transform(\u0026#39;rank\u0026#39;) # Salary above or below dept average? df[\u0026#39;above_avg\u0026#39;] = df[\u0026#39;salary\u0026#39;] \u0026gt; df.groupby(\u0026#39;dept\u0026#39;)[\u0026#39;salary\u0026#39;].transform(\u0026#39;mean\u0026#39;) # Normalize salary within department df[\u0026#39;salary_norm\u0026#39;] = df.groupby(\u0026#39;dept\u0026#39;)[\u0026#39;salary\u0026#39;].transform( lambda x: (x - x.mean()) / x.std() ) filter - include or exclude entire groups # Keep only departments with more than 2 employees df.groupby(\u0026#39;dept\u0026#39;).filter(lambda x: len(x) \u0026gt; 2) # Keep only groups where max salary \u0026gt; 80000 df.groupby(\u0026#39;dept\u0026#39;).filter(lambda x: x[\u0026#39;salary\u0026#39;].max() \u0026gt; 80000) 14. Merging, Joining and Concatenating pd.merge - SQL-style joins employees = pd.DataFrame({ \u0026#39;emp_id\u0026#39;: [1, 2, 3, 4], \u0026#39;name\u0026#39;: [\u0026#39;Alice\u0026#39;, \u0026#39;Bob\u0026#39;, \u0026#39;Carol\u0026#39;, \u0026#39;Dave\u0026#39;], \u0026#39;dept_id\u0026#39;: [10, 20, 10, 30] }) departments = pd.DataFrame({ \u0026#39;dept_id\u0026#39;: [10, 20, 40], \u0026#39;dept_name\u0026#39;: [\u0026#39;Engineering\u0026#39;, \u0026#39;Marketing\u0026#39;, \u0026#39;Finance\u0026#39;] }) # INNER join - only matching rows in both (default) pd.merge(employees, departments, on=\u0026#39;dept_id\u0026#39;) # Alice, Bob, Carol - Dave is excluded (dept_id 30 not in departments) # LEFT join - all rows from left, NaN where no match on right pd.merge(employees, departments, on=\u0026#39;dept_id\u0026#39;, how=\u0026#39;left\u0026#39;) # All 4 employees, Dave gets NaN for dept_name # RIGHT join - all rows from right, NaN where no match on left pd.merge(employees, departments, on=\u0026#39;dept_id\u0026#39;, how=\u0026#39;right\u0026#39;) # Engineering, Marketing, Finance - Finance has no employees (NaN) # OUTER join - all rows from both pd.merge(employees, departments, on=\u0026#39;dept_id\u0026#39;, how=\u0026#39;outer\u0026#39;) # Join on columns with different names pd.merge(employees, departments, left_on=\u0026#39;dept_id\u0026#39;, right_on=\u0026#39;id\u0026#39;) # Join on multiple columns pd.merge(df1, df2, on=[\u0026#39;name\u0026#39;, \u0026#39;date\u0026#39;]) # Suffix when column names clash pd.merge(df1, df2, on=\u0026#39;id\u0026#39;, suffixes=(\u0026#39;_left\u0026#39;, \u0026#39;_right\u0026#39;)) pd.concat - stacking DataFrames # Stack vertically (add rows) - same columns df_all = pd.concat([df_jan, df_feb, df_mar]) # Reset index (avoid duplicate index values) df_all = pd.concat([df_jan, df_feb], ignore_index=True) # Add a key to identify each source df_all = pd.concat([df1, df2], keys=[\u0026#39;file1\u0026#39;, \u0026#39;file2\u0026#39;]) # Stack horizontally (add columns) - same rows df_wide = pd.concat([df1, df2], axis=1) # Handling mismatched columns df_all = pd.concat([df1, df2], join=\u0026#39;outer\u0026#39;) # NaN for missing (default) df_all = pd.concat([df1, df2], join=\u0026#39;inner\u0026#39;) # only common columns .join - index-based merge df1 = pd.DataFrame({\u0026#39;salary\u0026#39;: [80000, 72000]}, index=[\u0026#39;Alice\u0026#39;, \u0026#39;Bob\u0026#39;]) df2 = pd.DataFrame({\u0026#39;age\u0026#39;: [25, 30]}, index=[\u0026#39;Alice\u0026#39;, \u0026#39;Bob\u0026#39;]) df1.join(df2) # join on index df1.join(df2, how=\u0026#39;left\u0026#39;) 15. Pivot Tables and Reshaping Pivot table df = pd.DataFrame({ \u0026#39;dept\u0026#39;: [\u0026#39;Eng\u0026#39;, \u0026#39;Eng\u0026#39;, \u0026#39;Mkt\u0026#39;, \u0026#39;Mkt\u0026#39;, \u0026#39;HR\u0026#39;], \u0026#39;city\u0026#39;: [\u0026#39;NYC\u0026#39;, \u0026#39;LA\u0026#39;, \u0026#39;NYC\u0026#39;, \u0026#39;LA\u0026#39;, \u0026#39;NYC\u0026#39;], \u0026#39;salary\u0026#39;: [80000, 95000, 72000, 61000, 48000], \u0026#39;headcount\u0026#39;: [5, 3, 4, 2, 3] }) # Like Excel pivot tables pd.pivot_table(df, values=\u0026#39;salary\u0026#39;, # what to aggregate index=\u0026#39;dept\u0026#39;, # row grouping columns=\u0026#39;city\u0026#39;, # column grouping aggfunc=\u0026#39;mean\u0026#39;, # aggregation function fill_value=0 # replace NaN with 0 ) # city LA NYC # dept # Eng 95000.0 80000.0 # HR 0.0 48000.0 # Mkt 61000.0 72000.0 # Multiple values pd.pivot_table(df, values=[\u0026#39;salary\u0026#39;, \u0026#39;headcount\u0026#39;], index=\u0026#39;dept\u0026#39;, columns=\u0026#39;city\u0026#39;, aggfunc={\u0026#39;salary\u0026#39;: \u0026#39;mean\u0026#39;, \u0026#39;headcount\u0026#39;: \u0026#39;sum\u0026#39;} ) melt - wide to long format # Wide format (one row per person, multiple time columns) wide = pd.DataFrame({ \u0026#39;name\u0026#39;: [\u0026#39;Alice\u0026#39;, \u0026#39;Bob\u0026#39;], \u0026#39;Jan\u0026#39;: [100, 200], \u0026#39;Feb\u0026#39;: [150, 250], \u0026#39;Mar\u0026#39;: [120, 230] }) # Convert to long (tidy) format long = pd.melt(wide, id_vars=[\u0026#39;name\u0026#39;], # columns to keep value_vars=[\u0026#39;Jan\u0026#39;, \u0026#39;Feb\u0026#39;, \u0026#39;Mar\u0026#39;], # columns to melt var_name=\u0026#39;month\u0026#39;, # name for the new \u0026#34;variable\u0026#34; column value_name=\u0026#39;sales\u0026#39; # name for the new \u0026#34;value\u0026#34; column ) # name month sales # 0 Alice Jan 100 # 1 Bob Jan 200 # 2 Alice Feb 150 # ... stack and unstack # unstack - move row index level to columns df.set_index([\u0026#39;dept\u0026#39;, \u0026#39;city\u0026#39;])[\u0026#39;salary\u0026#39;].unstack() # stack - move column level to row index df.stack() # columns become row labels crosstab - frequency table pd.crosstab(df[\u0026#39;dept\u0026#39;], df[\u0026#39;city\u0026#39;]) # counts pd.crosstab(df[\u0026#39;dept\u0026#39;], df[\u0026#39;city\u0026#39;], values=df[\u0026#39;salary\u0026#39;], aggfunc=\u0026#39;mean\u0026#39;) # with values pd.crosstab(df[\u0026#39;dept\u0026#39;], df[\u0026#39;city\u0026#39;], normalize=\u0026#39;index\u0026#39;) # row percentages 16. Apply, Map and Lambda Functions .apply() - apply a function to rows or columns df = pd.DataFrame({ \u0026#39;salary\u0026#39;: [55000, 72000, 85000], \u0026#39;tax_rate\u0026#39;: [0.2, 0.25, 0.3] }) # Apply to a column (element-wise for Series) df[\u0026#39;salary\u0026#39;].apply(lambda x: x * 1.1) # 10% raise df[\u0026#39;salary\u0026#39;].apply(lambda x: f\u0026#39;${x:,}\u0026#39;) # format as string # Apply a named function def categorize(salary): if salary \u0026lt; 60000: return \u0026#39;Junior\u0026#39; elif salary \u0026lt; 80000: return \u0026#39;Mid\u0026#39; else: return \u0026#39;Senior\u0026#39; df[\u0026#39;level\u0026#39;] = df[\u0026#39;salary\u0026#39;].apply(categorize) # Apply across rows (axis=1) - each row is passed as a Series df[\u0026#39;take_home\u0026#39;] = df.apply( lambda row: row[\u0026#39;salary\u0026#39;] * (1 - row[\u0026#39;tax_rate\u0026#39;]), axis=1 ) # Apply across columns (axis=0 default) - returns aggregate per column df.apply(\u0026#39;mean\u0026#39;) # mean of each column df.apply(lambda x: x.max() - x.min()) # range of each column .map() - element-wise mapping on Series s = pd.Series([\u0026#39;Eng\u0026#39;, \u0026#39;Mkt\u0026#39;, \u0026#39;HR\u0026#39;, \u0026#39;Eng\u0026#39;]) # Map with a dictionary (like a lookup/replace) dept_map = {\u0026#39;Eng\u0026#39;: \u0026#39;Engineering\u0026#39;, \u0026#39;Mkt\u0026#39;: \u0026#39;Marketing\u0026#39;, \u0026#39;HR\u0026#39;: \u0026#39;Human Resources\u0026#39;} s.map(dept_map) # Map with a function s.map(lambda x: x.lower()) s.map(len) # length of each string # Missing values in dict → NaN s.map({\u0026#39;Eng\u0026#39;: \u0026#39;Engineering\u0026#39;}) # Mkt and HR → NaN .applymap() / .map() on DataFrame - element-wise # Apply a function to every single element in a DataFrame df.map(lambda x: round(x, 2) if isinstance(x, float) else x) df.map(str) # convert everything to string Vectorized operations vs apply - performance # SLOW - use apply only when necessary df[\u0026#39;salary\u0026#39;].apply(lambda x: x * 1.1) # FAST - use vectorized operations instead df[\u0026#39;salary\u0026#39;] * 1.1 # apply is needed when: # 1. Your logic is too complex for vectorized operations # 2. You need to use multiple columns per row (axis=1) # 3. You\u0026#39;re calling an external function 17. Plotting with Pandas Pandas has built-in plotting that wraps Matplotlib. Always import Matplotlib too.\nimport pandas as pd import matplotlib.pyplot as plt df = pd.DataFrame({ \u0026#39;month\u0026#39;: [\u0026#39;Jan\u0026#39;, \u0026#39;Feb\u0026#39;, \u0026#39;Mar\u0026#39;, \u0026#39;Apr\u0026#39;, \u0026#39;May\u0026#39;, \u0026#39;Jun\u0026#39;], \u0026#39;revenue\u0026#39;: [12000, 15000, 11000, 18000, 21000, 17000], \u0026#39;expenses\u0026#39;: [8000, 9000, 10000, 11000, 13000, 12000] }) # Line plot df.plot(x=\u0026#39;month\u0026#39;, y=\u0026#39;revenue\u0026#39;, figsize=(10, 5)) df.plot(x=\u0026#39;month\u0026#39;, y=[\u0026#39;revenue\u0026#39;, \u0026#39;expenses\u0026#39;]) # multiple lines plt.title(\u0026#39;Monthly Revenue\u0026#39;) plt.show() # Bar chart df.plot.bar(x=\u0026#39;month\u0026#39;, y=\u0026#39;revenue\u0026#39;) df.plot.barh(x=\u0026#39;month\u0026#39;, y=\u0026#39;revenue\u0026#39;) # horizontal df.plot.bar(x=\u0026#39;month\u0026#39;, stacked=True) # stacked bars # Scatter df.plot.scatter(x=\u0026#39;expenses\u0026#39;, y=\u0026#39;revenue\u0026#39;) # Histogram df[\u0026#39;revenue\u0026#39;].plot.hist(bins=20) # Box plot df[[\u0026#39;revenue\u0026#39;, \u0026#39;expenses\u0026#39;]].plot.box() # Pie chart df[\u0026#39;revenue\u0026#39;].plot.pie() # Area chart df.plot.area(x=\u0026#39;month\u0026#39;, y=[\u0026#39;revenue\u0026#39;, \u0026#39;expenses\u0026#39;]) # Plot directly on a Series s = pd.Series([1, 4, 9, 16, 25]) s.plot() s.plot.bar() # Correlation heatmap (with seaborn) import seaborn as sns numeric_df = df[[\u0026#39;revenue\u0026#39;, \u0026#39;expenses\u0026#39;]] sns.heatmap(numeric_df.corr(), annot=True, cmap=\u0026#39;coolwarm\u0026#39;) plt.show() 18. Performance Tips Use vectorized operations # SLOW for i, row in df.iterrows(): df.at[i, \u0026#39;bonus\u0026#39;] = row[\u0026#39;salary\u0026#39;] * 0.1 # FAST df[\u0026#39;bonus\u0026#39;] = df[\u0026#39;salary\u0026#39;] * 0.1 Use efficient types # Check memory usage df.memory_usage(deep=True) # Convert string columns with few unique values to category df[\u0026#39;dept\u0026#39;] = df[\u0026#39;dept\u0026#39;].astype(\u0026#39;category\u0026#39;) # can save up to 90% memory # Use smaller integer/float types when possible df[\u0026#39;age\u0026#39;] = df[\u0026#39;age\u0026#39;].astype(\u0026#39;int8\u0026#39;) # age fits in int8 (-128 to 127) df[\u0026#39;salary\u0026#39;] = df[\u0026#39;salary\u0026#39;].astype(\u0026#39;float32\u0026#39;) # less precision, less memory Prefer query/loc over chained indexing # SLOW and may cause SettingWithCopyWarning df[df[\u0026#39;dept\u0026#39;] == \u0026#39;Eng\u0026#39;][\u0026#39;salary\u0026#39;] = 90000 # BAD # CORRECT df.loc[df[\u0026#39;dept\u0026#39;] == \u0026#39;Eng\u0026#39;, \u0026#39;salary\u0026#39;] = 90000 # GOOD Avoid iterrows - use vectorized or apply # SLOW (Python-speed loop) for i, row in df.iterrows(): ... # MUCH FASTER df.apply(func, axis=1) # still Python, but optimized df[\u0026#39;col\u0026#39;] * 2 # fully vectorized (fastest) Read large files efficiently # Read in chunks for huge files chunk_size = 100_000 chunks = [] for chunk in pd.read_csv(\u0026#39;huge_file.csv\u0026#39;, chunksize=chunk_size): # process each chunk chunk = chunk[chunk[\u0026#39;salary\u0026#39;] \u0026gt; 50000] chunks.append(chunk) df = pd.concat(chunks, ignore_index=True) # Use parquet for fast I/O on large datasets df.to_parquet(\u0026#39;data.parquet\u0026#39;) # save df = pd.read_parquet(\u0026#39;data.parquet\u0026#39;) # load (10–100x faster than CSV) Quick Reference Card # ── CREATING ───────────────────────────────────────── pd.Series([1,2,3], index=[\u0026#39;a\u0026#39;,\u0026#39;b\u0026#39;,\u0026#39;c\u0026#39;]) pd.DataFrame({\u0026#39;col1\u0026#39;: [...], \u0026#39;col2\u0026#39;: [...]}) pd.read_csv(\u0026#39;f.csv\u0026#39;) pd.read_excel(\u0026#39;f.xlsx\u0026#39;) # ── EXPLORING ──────────────────────────────────────── df.head() df.tail() df.sample(n) df.shape df.info() df.describe() df.dtypes df.isnull().sum() df[\u0026#39;col\u0026#39;].value_counts() # ── SELECTING ──────────────────────────────────────── df[\u0026#39;col\u0026#39;] df[[\u0026#39;a\u0026#39;,\u0026#39;b\u0026#39;]] df.loc[0, \u0026#39;col\u0026#39;] df.loc[0:3, \u0026#39;a\u0026#39;:\u0026#39;c\u0026#39;] df.iloc[0, 1] df.iloc[0:3, 1:4] df[df[\u0026#39;col\u0026#39;] \u0026gt; 5] df.query(\u0026#39;col \u0026gt; 5\u0026#39;) df[df[\u0026#39;col\u0026#39;].isin([\u0026#39;x\u0026#39;,\u0026#39;y\u0026#39;])] # ── ADDING/MODIFYING ───────────────────────────────── df[\u0026#39;new\u0026#39;] = df[\u0026#39;a\u0026#39;] * 2 df[\u0026#39;level\u0026#39;] = np.where(df[\u0026#39;sal\u0026#39;]\u0026gt;60000, \u0026#39;Senior\u0026#39;, \u0026#39;Junior\u0026#39;) df.rename(columns={\u0026#39;old\u0026#39;: \u0026#39;new\u0026#39;}) df.drop(columns=[\u0026#39;col\u0026#39;]) # ── CLEANING ───────────────────────────────────────── df.isnull().sum() df.dropna() df.fillna(0) df.fillna(df[\u0026#39;col\u0026#39;].mean()) df.drop_duplicates() df[\u0026#39;col\u0026#39;].astype(int) df.replace(\u0026#39;?\u0026#39;, np.nan) # ── STRINGS ────────────────────────────────────────── df[\u0026#39;col\u0026#39;].str.lower() .str.strip() .str.contains(\u0026#39;x\u0026#39;) df[\u0026#39;col\u0026#39;].str.split(\u0026#39; \u0026#39;, expand=True) .str.extract(r\u0026#39;(\\w+)\u0026#39;) # ── GROUPBY ────────────────────────────────────────── df.groupby(\u0026#39;col\u0026#39;)[\u0026#39;val\u0026#39;].mean() df.groupby(\u0026#39;col\u0026#39;).agg({\u0026#39;val\u0026#39;: [\u0026#39;mean\u0026#39;,\u0026#39;max\u0026#39;]}) df.groupby(\u0026#39;col\u0026#39;)[\u0026#39;val\u0026#39;].transform(\u0026#39;mean\u0026#39;) # ── MERGING ────────────────────────────────────────── pd.merge(df1, df2, on=\u0026#39;key\u0026#39;, how=\u0026#39;left\u0026#39;) pd.concat([df1, df2], ignore_index=True) # ── RESHAPING ──────────────────────────────────────── pd.pivot_table(df, values=\u0026#39;v\u0026#39;, index=\u0026#39;r\u0026#39;, columns=\u0026#39;c\u0026#39;) pd.melt(df, id_vars=[\u0026#39;name\u0026#39;], value_vars=[\u0026#39;Jan\u0026#39;,\u0026#39;Feb\u0026#39;]) # ── SAVING ─────────────────────────────────────────── df.to_csv(\u0026#39;f.csv\u0026#39;, index=False) df.to_excel(\u0026#39;f.xlsx\u0026#39;, index=False) End of Pandas Masterclass\n","permalink":"https://blogs.codeverra.com/blog/pandas-masterclass/","summary":"\u003ch1 id=\"pandas-masterclass---from-basics-to-advanced\"\u003ePandas Masterclass - From Basics to Advanced\u003c/h1\u003e\n\u003cblockquote\u003e\n\u003cp\u003e\u003cstrong\u003ePrerequisites:\u003c/strong\u003e Python basics, NumPy fundamentals\u003cbr\u003e\n\u003cstrong\u003eInstall:\u003c/strong\u003e \u003ccode\u003epip install pandas openpyxl xlrd\u003c/code\u003e\u003cbr\u003e\n\u003cstrong\u003eImport convention:\u003c/strong\u003e \u003ccode\u003eimport pandas as pd\u003c/code\u003e\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003chr\u003e\n\u003ch2 id=\"table-of-contents\"\u003eTable of Contents\u003c/h2\u003e\n\u003col\u003e\n\u003cli\u003e\u003ca href=\"#1-what-is-pandas-and-why-use-it\"\u003eWhat is Pandas and Why Use It?\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#2-series--the-1d-structure\"\u003eSeries  -  The 1D Structure\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#3-dataframe--the-2d-structure\"\u003eDataFrame  -  The 2D Structure\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#4-loading-and-saving-data\"\u003eLoading and Saving Data\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#5-exploring-your-data\"\u003eExploring Your Data\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#6-selecting-and-filtering\"\u003eSelecting and Filtering\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#7-adding-and-modifying-columns\"\u003eAdding and Modifying Columns\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#8-sorting\"\u003eSorting\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#9-cleaning-data--handling-missing-values\"\u003eCleaning Data  -  Handling Missing Values\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#10-cleaning-data--duplicates-and-types\"\u003eCleaning Data  -  Duplicates and Types\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#11-string-operations\"\u003eString Operations\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#12-datetime-operations\"\u003eDatetime Operations\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#13-groupby-and-aggregation\"\u003eGroupby and Aggregation\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#14-merging-joining-and-concatenating\"\u003eMerging, Joining and Concatenating\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#15-pivot-tables-and-reshaping\"\u003ePivot Tables and Reshaping\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#16-apply-map-and-lambda-functions\"\u003eApply, Map and Lambda Functions\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#17-plotting-with-pandas\"\u003ePlotting with Pandas\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#18-performance-tips\"\u003ePerformance Tips\u003c/a\u003e\u003c/li\u003e\n\u003c/ol\u003e\n\u003chr\u003e\n\u003ch2 id=\"1-what-is-pandas-and-why-use-it\"\u003e1. What is Pandas and Why Use It?\u003c/h2\u003e\n\u003cp\u003ePandas provides two core data structures  -  \u003ccode\u003eSeries\u003c/code\u003e (1D) and \u003ccode\u003eDataFrame\u003c/code\u003e (2D)  -  that make working with labeled, tabular data as natural as working with a spreadsheet, but with the full power of Python.\u003c/p\u003e","tags":["python","pandas","data-analysis"],"title":"Pandas Masterclass - Beginner to Advanced"},{"content":"Python Exception Handling A Complete Reference Guide Table of Contents The Big Picture What Are Exceptions? The Exception Hierarchy try / except - The Foundation else and finally Clauses Raising Exceptions Custom Exceptions Exception Chaining Context Managers and Exceptions Logging Exceptions Common Mistakes to Avoid Real-Life Patterns Practice Questions \u0026amp; Answers Quick Reference Cheat Sheet The Big Picture ┌─────────────────────────────────────────────────────────────────┐ │ EXCEPTION HANDLING FLOW │ │ │ │ try: │ │ ── risky code runs here ──────────────────────────────┐ │ │ │ │ │ except SomeError: ◄── exception caught here │ │ │ ── handle it ── │ │ │ ▼ │ │ except (TypeError, ValueError): ◄── multiple types │ │ ── handle them ── │ │ │ │ except Exception as e: ◄── catch with reference │ │ ── inspect e ── │ │ │ │ else: ◄── runs ONLY if NO exception │ │ ── success path ── │ │ │ │ finally: ◄── ALWAYS runs, no matter what │ │ ── cleanup ── │ └─────────────────────────────────────────────────────────────────┘ Core Philosophy: Exceptions are not bugs - they are Python\u0026rsquo;s way of saying \u0026ldquo;something unexpected happened here, and I\u0026rsquo;m passing control to whoever knows how to handle it.\u0026rdquo;\nWhat Are Exceptions? An exception is a signal that something went wrong during program execution. When Python encounters an error it can\u0026rsquo;t resolve, it raises (throws) an exception object and starts looking for a handler.\n# Without exception handling - program crashes result = 10 / 0 print(\u0026#34;This never prints\u0026#34;) # ZeroDivisionError: division by zero # With exception handling - program recovers try: result = 10 / 0 except ZeroDivisionError: result = float(\u0026#39;inf\u0026#39;) print(\u0026#34;Caught division by zero, using infinity instead\u0026#34;) print(f\u0026#34;Result: {result}\u0026#34;) # Result: inf Errors vs Exceptions In Python, everything is an exception - even errors. There\u0026rsquo;s one key distinction:\nType Description Example SyntaxError Caught at parse time, before code runs if x == Exception Raised at runtime, can be caught 1 / 0, int(\u0026quot;abc\u0026quot;) # SyntaxError - Python won\u0026#39;t even run the file # if x ==: ← this fails before execution, cannot be caught by try/except # RuntimeError - happens during execution, CAN be caught try: x = int(\u0026#34;not a number\u0026#34;) except ValueError as e: print(f\u0026#34;Caught: {e}\u0026#34;) # Caught: invalid literal for int() with base 10: \u0026#39;not a number\u0026#39; The Exception Hierarchy Python\u0026rsquo;s built-in exceptions form a tree. Understanding this hierarchy is critical because catching a parent catches all its children.\nBaseException ├── SystemExit ← raised by sys.exit() ├── KeyboardInterrupt ← raised by Ctrl+C ├── GeneratorExit ← raised when generator is closed └── Exception ← base for all \u0026#34;normal\u0026#34; exceptions ├── ArithmeticError │ ├── ZeroDivisionError │ ├── OverflowError │ └── FloatingPointError ├── AttributeError ├── ImportError │ └── ModuleNotFoundError ├── LookupError │ ├── IndexError │ └── KeyError ├── NameError │ └── UnboundLocalError ├── OSError (IOError) │ ├── FileNotFoundError │ ├── PermissionError │ └── TimeoutError ├── TypeError ├── ValueError │ └── UnicodeError ├── RuntimeError │ └── RecursionError └── StopIteration # Catching a parent catches all children try: my_list = [1, 2, 3] print(my_list[99]) except LookupError as e: print(f\u0026#34;LookupError caught: {type(e).__name__}: {e}\u0026#34;) # LookupError caught: IndexError: list index out of range # The same handler also catches KeyError (another LookupError child) try: d = {\u0026#34;a\u0026#34;: 1} print(d[\u0026#34;z\u0026#34;]) except LookupError as e: print(f\u0026#34;LookupError caught: {type(e).__name__}: {e}\u0026#34;) # LookupError caught: KeyError: \u0026#39;z\u0026#39; try / except - The Foundation Basic Syntax try: # Code that might raise an exception risky_operation() except ExceptionType: # Code that runs if ExceptionType is raised handle_the_problem() Catching Specific Exceptions Always prefer catching specific exceptions over broad ones.\ndef divide(a, b): try: result = a / b return result except ZeroDivisionError: print(\u0026#34;Cannot divide by zero!\u0026#34;) return None print(divide(10, 2)) # 5.0 print(divide(10, 0)) # Cannot divide by zero! → None Catching Multiple Exception Types def parse_and_index(data, index): try: number = int(data[index]) return number * 2 except IndexError: print(f\u0026#34;Index {index} doesn\u0026#39;t exist in the list.\u0026#34;) except ValueError: print(f\u0026#34;\u0026#39;{data[index]}\u0026#39; cannot be converted to int.\u0026#34;) except TypeError: print(\u0026#34;Data must be a sequence.\u0026#34;) parse_and_index([\u0026#34;1\u0026#34;, \u0026#34;two\u0026#34;, \u0026#34;3\u0026#34;], 1) # ValueError parse_and_index([\u0026#34;1\u0026#34;, \u0026#34;2\u0026#34;, \u0026#34;3\u0026#34;], 5) # IndexError parse_and_index(None, 0) # TypeError Catching Multiple Exceptions in One Handler def safe_convert(value): try: return int(value) except (ValueError, TypeError) as e: print(f\u0026#34;Conversion failed: {e}\u0026#34;) return 0 print(safe_convert(\u0026#34;42\u0026#34;)) # 42 print(safe_convert(\u0026#34;abc\u0026#34;)) # Conversion failed: ... → 0 print(safe_convert(None)) # Conversion failed: ... → 0 Using as e - Inspecting the Exception # Every exception object has useful attributes try: int(\u0026#34;bad\u0026#34;) except ValueError as e: print(e.args) # (\u0026#34;invalid literal for int() with base 10: \u0026#39;bad\u0026#39;\u0026#34;,) print(str(e)) # invalid literal for int() with base 10: \u0026#39;bad\u0026#39; print(repr(e)) # ValueError(\u0026#34;invalid literal for int() with base 10: \u0026#39;bad\u0026#39;\u0026#34;) print(type(e).__name__) # ValueError The except Exception Catch-All (Use Carefully) # Acceptable when you log the error and re-raise or handle gracefully def run_job(job_func): try: job_func() except Exception as e: print(f\u0026#34;Job failed: {type(e).__name__}: {e}\u0026#34;) raise # re-raise so the caller knows something went wrong else and finally Clauses The else Clause - Runs Only on Success def read_config(filepath): try: f = open(filepath, \u0026#39;r\u0026#39;) except FileNotFoundError: print(f\u0026#34;Config file not found: {filepath}\u0026#34;) return {} else: # Only runs if open() succeeded - no exception was raised config = f.read() f.close() print(\u0026#34;Config loaded successfully.\u0026#34;) return config # Why use else instead of putting code in try? # Code in else is NOT protected by the except clause. # If parsing raises an error, it won\u0026#39;t be silently swallowed by the except above. The finally Clause - Always Runs def connect_to_db(host): connection = None try: connection = create_connection(host) # might fail result = connection.query(\u0026#34;SELECT 1\u0026#34;) # might fail return result except ConnectionError as e: print(f\u0026#34;Connection failed: {e}\u0026#34;) return None finally: # Runs whether connection succeeded, failed, or even if we `return` if connection: connection.close() print(\u0026#34;Connection closed.\u0026#34;) All Four Clauses Together def process_file(filepath): file_handle = None try: file_handle = open(filepath, \u0026#39;r\u0026#39;) content = file_handle.read() data = parse(content) # might raise ValueError except FileNotFoundError: print(\u0026#34;File not found.\u0026#34;) return None except ValueError as e: print(f\u0026#34;Parse error: {e}\u0026#34;) return None else: print(\u0026#34;File processed successfully.\u0026#34;) return data finally: if file_handle: file_handle.close() # ALWAYS close the file # Execution flow: # ✓ No exception → try → else → finally # ✗ FileNotFound → try (partial) → except FileNotFoundError → finally # ✗ ValueError → try (partial) → except ValueError → finally finally Even Beats return def tricky(): try: return \u0026#34;from try\u0026#34; finally: return \u0026#34;from finally\u0026#34; # This OVERRIDES the try\u0026#39;s return! print(tricky()) # \u0026#34;from finally\u0026#34; # Warning: avoid putting return in finally - it suppresses exceptions too Raising Exceptions raise - Raising a New Exception def set_age(age): if not isinstance(age, int): raise TypeError(f\u0026#34;Age must be an int, got {type(age).__name__}\u0026#34;) if age \u0026lt; 0 or age \u0026gt; 150: raise ValueError(f\u0026#34;Age must be between 0 and 150, got {age}\u0026#34;) return age set_age(25) # fine set_age(-5) # ValueError: Age must be between 0 and 150, got -5 set_age(\u0026#34;old\u0026#34;) # TypeError: Age must be an int, got str raise Without Arguments - Re-raising Inside an except block, bare raise re-raises the current exception unchanged, preserving the original traceback.\ndef step_three(): raise RuntimeError(\u0026#34;Something broke in step 3\u0026#34;) def step_two(): try: step_three() except RuntimeError as e: print(f\u0026#34;Logging error: {e}\u0026#34;) raise # re-raise the SAME exception with SAME traceback def step_one(): try: step_two() except RuntimeError as e: print(f\u0026#34;Step one caught: {e}\u0026#34;) step_one() # Logging error: Something broke in step 3 # Step one caught: Something broke in step 3 raise ... from ... - Exception Chaining def load_config(path): try: with open(path) as f: import json return json.load(f) except FileNotFoundError as e: raise RuntimeError(f\u0026#34;Cannot start app: config file missing\u0026#34;) from e try: load_config(\u0026#34;missing.json\u0026#34;) except RuntimeError as e: print(e) # Cannot start app: config file missing print(e.__cause__) # [Errno 2] No such file or directory: \u0026#39;missing.json\u0026#39; Custom Exceptions Why Create Custom Exceptions? Give callers precise control over what they catch Carry domain-specific context Document your error taxonomy explicitly Basic Custom Exception class InsufficientFundsError(Exception): \u0026#34;\u0026#34;\u0026#34;Raised when a bank account has insufficient balance.\u0026#34;\u0026#34;\u0026#34; pass class BankAccount: def __init__(self, owner, balance=0): self.owner = owner self.balance = balance def withdraw(self, amount): if amount \u0026gt; self.balance: raise InsufficientFundsError( f\u0026#34;{self.owner} only has Rs.{self.balance}, tried to withdraw Rs.{amount}\u0026#34; ) self.balance -= amount return amount account = BankAccount(\u0026#34;Ravi\u0026#34;, balance=500) try: account.withdraw(1000) except InsufficientFundsError as e: print(f\u0026#34;Transaction failed: {e}\u0026#34;) Custom Exception with Extra Attributes class APIError(Exception): \u0026#34;\u0026#34;\u0026#34;Raised when an external API returns an unexpected response.\u0026#34;\u0026#34;\u0026#34; def __init__(self, message, status_code=None, endpoint=None): super().__init__(message) self.status_code = status_code self.endpoint = endpoint def __str__(self): return ( f\u0026#34;APIError [{self.status_code}] on {self.endpoint}: \u0026#34; f\u0026#34;{super().__str__()}\u0026#34; ) try: raise APIError(\u0026#34;Rate limit exceeded\u0026#34;, status_code=429, endpoint=\u0026#34;/api/data\u0026#34;) except APIError as e: print(e) # APIError [429] on /api/data: Rate limit exceeded print(e.status_code) # 429 print(e.endpoint) # /api/data if e.status_code == 429: print(\u0026#34;Back off and retry later.\u0026#34;) Exception Hierarchy for a Domain Design a mini exception hierarchy for your application:\n# Base exception for the entire application class AppError(Exception): \u0026#34;\u0026#34;\u0026#34;Base class for all application errors.\u0026#34;\u0026#34;\u0026#34; pass # Category-level exceptions class DatabaseError(AppError): \u0026#34;\u0026#34;\u0026#34;Any error related to database operations.\u0026#34;\u0026#34;\u0026#34; pass class ValidationError(AppError): \u0026#34;\u0026#34;\u0026#34;Raised when user-provided data fails validation.\u0026#34;\u0026#34;\u0026#34; def __init__(self, field, message): self.field = field super().__init__(f\u0026#34;Validation failed on \u0026#39;{field}\u0026#39;: {message}\u0026#34;) class AuthenticationError(AppError): \u0026#34;\u0026#34;\u0026#34;Raised when login or token verification fails.\u0026#34;\u0026#34;\u0026#34; pass # Specific exceptions class RecordNotFoundError(DatabaseError): def __init__(self, model, pk): super().__init__(f\u0026#34;{model} with id={pk} not found\u0026#34;) class DuplicateEntryError(DatabaseError): pass class WeakPasswordError(ValidationError): def __init__(self): super().__init__(\u0026#34;password\u0026#34;, \u0026#34;Must be at least 8 characters with a digit\u0026#34;) # Usage def get_user(user_id): if user_id != 1: raise RecordNotFoundError(\u0026#34;User\u0026#34;, user_id) return {\u0026#34;id\u0026#34;: 1, \u0026#34;name\u0026#34;: \u0026#34;Priya\u0026#34;} try: user = get_user(99) except RecordNotFoundError as e: print(e) # User with id=99 not found except DatabaseError as e: print(f\u0026#34;DB issue: {e}\u0026#34;) # catches any DatabaseError, including RecordNotFoundError except AppError as e: print(f\u0026#34;App issue: {e}\u0026#34;) # catches everything in the app Exception Chaining Python tracks the chain of exceptions so debugging is easier.\nImplicit Chaining (__context__) When an exception is raised inside an except block, Python automatically links them:\ntry: int(\u0026#34;bad\u0026#34;) except ValueError: raise RuntimeError(\u0026#34;processing failed\u0026#34;) # During handling of ValueError, RuntimeError was raised. # The ValueError is stored as RuntimeError.__context__ Explicit Chaining (__cause__) - raise ... from class ConfigError(Exception): pass def load_settings(path): try: with open(path) as f: import json return json.load(f) except json.JSONDecodeError as e: raise ConfigError(f\u0026#34;Invalid JSON in {path}\u0026#34;) from e except FileNotFoundError as e: raise ConfigError(f\u0026#34;Settings file not found: {path}\u0026#34;) from e try: load_settings(\u0026#34;settings.json\u0026#34;) except ConfigError as e: print(f\u0026#34;Config Error: {e}\u0026#34;) if e.__cause__: print(f\u0026#34;Caused by: {type(e.__cause__).__name__}: {e.__cause__}\u0026#34;) Suppressing the Chain - raise ... from None def get_value(d, key): try: return d[key] except KeyError: raise ValueError(f\u0026#34;Key \u0026#39;{key}\u0026#39; is not valid\u0026#34;) from None # Hides the KeyError from the traceback - cleaner user-facing error try: get_value({\u0026#34;a\u0026#34;: 1}, \u0026#34;z\u0026#34;) except ValueError as e: print(e) # Key \u0026#39;z\u0026#39; is not valid (no KeyError traceback shown) Context Managers and Exceptions with Statement - The Right Way to Handle Resources # Without context manager f = None try: f = open(\u0026#34;data.txt\u0026#34;, \u0026#34;r\u0026#34;) content = f.read() finally: if f: f.close() # With context manager - identical behavior, much cleaner with open(\u0026#34;data.txt\u0026#34;, \u0026#34;r\u0026#34;) as f: content = f.read() # f.close() is called automatically, even on exception Writing a Custom Context Manager with __enter__ / __exit__ class ManagedTransaction: \u0026#34;\u0026#34;\u0026#34;A database transaction that commits on success and rolls back on error.\u0026#34;\u0026#34;\u0026#34; def __init__(self, connection): self.connection = connection def __enter__(self): print(\u0026#34;BEGIN TRANSACTION\u0026#34;) return self # value bound to the `as` variable def __exit__(self, exc_type, exc_val, exc_tb): if exc_type is None: print(\u0026#34;COMMIT\u0026#34;) else: print(f\u0026#34;ROLLBACK - caused by {exc_type.__name__}: {exc_val}\u0026#34;) return False # False = do NOT suppress the exception # Return True to suppress it silently class FakeConnection: pass conn = FakeConnection() # Successful transaction with ManagedTransaction(conn) as txn: print(\u0026#34;Doing work...\u0026#34;) # Output: BEGIN TRANSACTION → Doing work... → COMMIT # Failed transaction try: with ManagedTransaction(conn) as txn: print(\u0026#34;Doing work...\u0026#34;) raise ValueError(\u0026#34;Something went wrong!\u0026#34;) except ValueError: pass # Output: BEGIN TRANSACTION → Doing work... → ROLLBACK - caused by ValueError Context Manager with contextlib from contextlib import contextmanager, suppress # Generator-based context manager @contextmanager def timer(label): import time start = time.perf_counter() try: yield # code inside `with` block runs here finally: elapsed = time.perf_counter() - start print(f\u0026#34;{label}: {elapsed:.4f}s\u0026#34;) with timer(\u0026#34;Data processing\u0026#34;): total = sum(range(10_000_000)) # contextlib.suppress - silently ignore specific exceptions with suppress(FileNotFoundError): import os os.remove(\u0026#34;maybe_exists.txt\u0026#34;) # No error if file doesn\u0026#39;t exist # Equivalent to: # try: # os.remove(...) # except FileNotFoundError: # pass Logging Exceptions In production, you should log exceptions, not just print them.\nimport logging import traceback logging.basicConfig( level=logging.DEBUG, format=\u0026#39;%(asctime)s [%(levelname)s] %(name)s: %(message)s\u0026#39; ) logger = logging.getLogger(__name__) def process_order(order_id): try: # ... order logic ... pass except ValueError as e: logger.warning(f\u0026#34;Invalid order {order_id}: {e}\u0026#34;) return {\u0026#34;status\u0026#34;: \u0026#34;rejected\u0026#34;, \u0026#34;reason\u0026#34;: str(e)} except Exception as e: # exc_info=True attaches the full traceback to the log record logger.error(f\u0026#34;Unexpected error processing order {order_id}\u0026#34;, exc_info=True) raise # re-raise so the caller knows something went wrong # Capturing full traceback as a string try: int(\u0026#34;bad\u0026#34;) except ValueError: tb = traceback.format_exc() print(tb) # Full formatted traceback as a string # Useful for sending to a monitoring system like Sentry Common Mistakes to Avoid Mistake 1 - Bare except (Catches Everything Including KeyboardInterrupt) # BAD - catches Ctrl+C, SystemExit, and all runtime errors silently try: do_something() except: pass # BUG: you have silenced KeyboardInterrupt and SystemExit! # GOOD - catch only what you expect try: do_something() except ValueError: pass Mistake 2 - Swallowing Exceptions Silently # BAD - errors disappear, debugging becomes a nightmare try: result = complex_calculation() except Exception: pass # What went wrong? Nobody will ever know. # GOOD - at minimum, log it try: result = complex_calculation() except Exception as e: logger.error(f\u0026#34;Calculation failed: {e}\u0026#34;, exc_info=True) result = default_value Mistake 3 - Using Exceptions for Normal Flow Control # UNNECESSARY - using exceptions as a substitute for if/else def get_discount(user): try: return user[\u0026#34;discount\u0026#34;] except KeyError: return 0 # BETTER - use dict.get() when the key might legitimately be absent def get_discount(user): return user.get(\u0026#34;discount\u0026#34;, 0) # NOTE: Python EAFP style (try/except) IS preferred for cases where # you expect the key to usually exist. Use judgment based on frequency. Mistake 4 - Catching a Broad Exception Instead of the Specific One # BAD - catching Exception when you only expect ValueError def parse_age(text): try: return int(text) except Exception: # hides MemoryError, AttributeError, etc. return -1 # GOOD def parse_age(text): try: return int(text) except ValueError: return -1 Mistake 5 - Too Much Code in a Single try Block # BAD - which line raised the ValueError? try: age = int(data[\u0026#34;age\u0026#34;]) score = float(data[\u0026#34;score\u0026#34;]) result = compute(age, score) save_to_db(result) except ValueError: print(\u0026#34;Something failed...\u0026#34;) # impossible to tell where # GOOD - narrow try blocks, specific messages def safe_parse(data): try: age = int(data[\u0026#34;age\u0026#34;]) except ValueError: raise ValueError(f\u0026#34;\u0026#39;age\u0026#39; must be a number, got: {data[\u0026#39;age\u0026#39;]!r}\u0026#34;) try: score = float(data[\u0026#34;score\u0026#34;]) except ValueError: raise ValueError(f\u0026#34;\u0026#39;score\u0026#39; must be a decimal, got: {data[\u0026#39;score\u0026#39;]!r}\u0026#34;) return age, score Mistake 6 - Wrong Exception Order (General Before Specific) # BAD - ValueError is never reached try: int(\u0026#34;abc\u0026#34;) except Exception as e: print(\u0026#34;General handler\u0026#34;) # always runs except ValueError as e: # DEAD CODE - never reached print(\u0026#34;ValueError handler\u0026#34;) # GOOD - most specific first, most general last try: int(\u0026#34;abc\u0026#34;) except ValueError as e: print(f\u0026#34;ValueError: {e}\u0026#34;) except TypeError as e: print(f\u0026#34;TypeError: {e}\u0026#34;) except Exception as e: print(f\u0026#34;Unexpected: {e}\u0026#34;) Mistake 7 - Raising an Exception Inside finally # TRICKY: An exception in finally silently replaces the original exception def buggy(): try: raise ValueError(\u0026#34;original error\u0026#34;) finally: raise RuntimeError(\u0026#34;cleanup error\u0026#34;) # Replaces the ValueError! try: buggy() except RuntimeError as e: print(e) # \u0026#34;cleanup error\u0026#34; - the ValueError is completely lost! # GOOD - keep finally clean; wrap risky cleanup in its own try def safe(): try: raise ValueError(\u0026#34;original error\u0026#34;) finally: try: risky_cleanup() except Exception as cleanup_err: logger.warning(f\u0026#34;Cleanup failed: {cleanup_err}\u0026#34;) # Original ValueError still propagates normally Mistake 8 - Raising the Base Exception Class # BAD - generic, unhelpful, hard to catch specifically def set_username(name): if not name: raise Exception(\u0026#34;Bad input\u0026#34;) # How should callers catch this? # GOOD - use the most specific built-in or a custom exception def set_username(name): if not isinstance(name, str): raise TypeError(f\u0026#34;Username must be str, not {type(name).__name__}\u0026#34;) if not name.strip(): raise ValueError(\u0026#34;Username cannot be empty or whitespace\u0026#34;) Real-Life Patterns Pattern 1 - Retry Logic with Exponential Backoff import time import random def with_retry(func, max_attempts=3, base_delay=1.0, exceptions=(Exception,)): \u0026#34;\u0026#34;\u0026#34; Calls `func` up to `max_attempts` times. Waits exponentially longer between retries. \u0026#34;\u0026#34;\u0026#34; for attempt in range(1, max_attempts + 1): try: return func() except exceptions as e: if attempt == max_attempts: raise # give up, re-raise the last exception delay = base_delay * (2 ** (attempt - 1)) + random.uniform(0, 0.5) print(f\u0026#34;Attempt {attempt} failed: {e}. Retrying in {delay:.2f}s...\u0026#34;) time.sleep(delay) # Usage call_count = 0 def flaky_api_call(): global call_count call_count += 1 if call_count \u0026lt; 3: raise ConnectionError(\u0026#34;Server not responding\u0026#34;) return {\u0026#34;data\u0026#34;: \u0026#34;success\u0026#34;} result = with_retry(flaky_api_call, max_attempts=5, exceptions=(ConnectionError,)) print(result) # {\u0026#39;data\u0026#39;: \u0026#39;success\u0026#39;} on the 3rd attempt Pattern 2 - Result Object (No Exception Propagation Across Layers) from dataclasses import dataclass from typing import Any, Optional @dataclass class Result: \u0026#34;\u0026#34;\u0026#34;Encapsulates success/failure without raising exceptions across layers.\u0026#34;\u0026#34;\u0026#34; success: bool value: Any = None error: Optional[str] = None @classmethod def ok(cls, value): return cls(success=True, value=value) @classmethod def fail(cls, error): return cls(success=False, error=error) def divide(a, b) -\u0026gt; Result: try: return Result.ok(a / b) except ZeroDivisionError as e: return Result.fail(str(e)) r = divide(10, 2) if r.success: print(f\u0026#34;Result: {r.value}\u0026#34;) # Result: 5.0 r = divide(10, 0) if not r.success: print(f\u0026#34;Failed: {r.error}\u0026#34;) # Failed: division by zero Pattern 3 - Validation with Accumulated Errors class MultiValidationError(Exception): def __init__(self, errors): self.errors = errors super().__init__(f\u0026#34;{len(errors)} validation error(s): {\u0026#39;; \u0026#39;.join(errors)}\u0026#34;) def validate_user(data): errors = [] if not data.get(\u0026#34;name\u0026#34;): errors.append(\u0026#34;Name is required\u0026#34;) elif len(data[\u0026#34;name\u0026#34;]) \u0026lt; 2: errors.append(\u0026#34;Name must be at least 2 characters\u0026#34;) if not data.get(\u0026#34;email\u0026#34;) or \u0026#34;@\u0026#34; not in data.get(\u0026#34;email\u0026#34;, \u0026#34;\u0026#34;): errors.append(\u0026#34;A valid email is required\u0026#34;) age = data.get(\u0026#34;age\u0026#34;) if age is None: errors.append(\u0026#34;Age is required\u0026#34;) elif not isinstance(age, int) or age \u0026lt; 0 or age \u0026gt; 120: errors.append(\u0026#34;Age must be an integer between 0 and 120\u0026#34;) if errors: raise MultiValidationError(errors) return True try: validate_user({\u0026#34;name\u0026#34;: \u0026#34;A\u0026#34;, \u0026#34;email\u0026#34;: \u0026#34;notanemail\u0026#34;, \u0026#34;age\u0026#34;: -5}) except MultiValidationError as e: for err in e.errors: print(f\u0026#34; - {err}\u0026#34;) # - Name must be at least 2 characters # - A valid email is required # - Age must be an integer between 0 and 120 Pattern 4 - Safe Resource Cleanup with contextlib from contextlib import contextmanager @contextmanager def managed_resource(name): \u0026#34;\u0026#34;\u0026#34;Generic resource manager with exception-safe cleanup.\u0026#34;\u0026#34;\u0026#34; resource = None try: print(f\u0026#34;Acquiring {name}...\u0026#34;) resource = acquire(name) # your resource acquisition logic yield resource except Exception as e: print(f\u0026#34;Error while using {name}: {e}\u0026#34;) raise finally: if resource: print(f\u0026#34;Releasing {name}...\u0026#34;) release(resource) with managed_resource(\u0026#34;database_connection\u0026#34;) as db: db.query(\u0026#34;SELECT * FROM users\u0026#34;) Practice Questions \u0026amp; Answers Question 1 - Predict the Output Q: What is the output of the following code?\ntry: print(\u0026#34;A\u0026#34;) x = 1 / 0 print(\u0026#34;B\u0026#34;) except ZeroDivisionError: print(\u0026#34;C\u0026#34;) else: print(\u0026#34;D\u0026#34;) finally: print(\u0026#34;E\u0026#34;) A:\nA C E \u0026quot;A\u0026quot; prints, then ZeroDivisionError is raised so \u0026quot;B\u0026quot; is skipped. The except block prints \u0026quot;C\u0026quot;. The else block is skipped because an exception occurred. The finally block always runs, printing \u0026quot;E\u0026quot;.\nQuestion 2 - Fix the Bug Q: What is wrong with this code? Fix it.\ntry: result = int(input(\u0026#34;Enter a number: \u0026#34;)) except Exception: print(\u0026#34;Error!\u0026#34;) except ValueError: print(\u0026#34;Not a number!\u0026#34;) A:\nThe except Exception appears before except ValueError. Since ValueError is a subclass of Exception, it will always be caught by the first handler - the ValueError branch is dead code and will never execute.\n# Fixed: most specific first try: result = int(input(\u0026#34;Enter a number: \u0026#34;)) except ValueError: print(\u0026#34;Not a number!\u0026#34;) except Exception as e: print(f\u0026#34;Unexpected error: {e}\u0026#34;) Question 3 - Custom Exception Q: Create a TemperatureError exception. Write set_temperature(temp) that raises it if temp is below -273.15 (absolute zero). The exception should include the invalid value in its message.\nA:\nclass TemperatureError(ValueError): \u0026#34;\u0026#34;\u0026#34;Raised when a physically impossible temperature is provided.\u0026#34;\u0026#34;\u0026#34; ABSOLUTE_ZERO = -273.15 def __init__(self, temp): self.temp = temp super().__init__( f\u0026#34;{temp}C is below absolute zero ({self.ABSOLUTE_ZERO}C)\u0026#34; ) def set_temperature(temp): if temp \u0026lt; TemperatureError.ABSOLUTE_ZERO: raise TemperatureError(temp) return temp try: set_temperature(-300) except TemperatureError as e: print(e) # -300C is below absolute zero (-273.15C) print(e.temp) # -300 Question 4 - finally Guarantee Q: Write safe_open(filepath) that opens a file, reads its content, and always closes the handle regardless of what goes wrong.\nA:\ndef safe_open(filepath): f = None try: f = open(filepath, \u0026#39;r\u0026#39;, encoding=\u0026#39;utf-8\u0026#39;) return f.read() except FileNotFoundError: print(f\u0026#34;File not found: {filepath}\u0026#34;) return None except PermissionError: print(f\u0026#34;No permission to read: {filepath}\u0026#34;) return None finally: if f is not None: f.close() print(f\u0026#34;File \u0026#39;{filepath}\u0026#39; closed.\u0026#34;) # Even better: use a context manager which handles this automatically def safe_open_v2(filepath): try: with open(filepath, \u0026#39;r\u0026#39;, encoding=\u0026#39;utf-8\u0026#39;) as f: return f.read() except FileNotFoundError: print(f\u0026#34;File not found: {filepath}\u0026#34;) return None Question 5 - Exception Chaining Q: Write load_user_age(data, key) that looks up a key in a dict and converts it to int. Raise a clear ValueError if it fails, and chain the original exception.\nA:\ndef load_user_age(data, key): try: raw = data[key] except KeyError as e: raise ValueError(f\u0026#34;Missing required field: \u0026#39;{key}\u0026#39;\u0026#34;) from e try: return int(raw) except (ValueError, TypeError) as e: raise ValueError( f\u0026#34;Field \u0026#39;{key}\u0026#39; must be a valid integer, got: {raw!r}\u0026#34; ) from e try: load_user_age({\u0026#34;age\u0026#34;: \u0026#34;not_a_number\u0026#34;}, \u0026#34;age\u0026#34;) except ValueError as e: print(e) # Field \u0026#39;age\u0026#39; must be a valid integer, got: \u0026#39;not_a_number\u0026#39; print(e.__cause__) # invalid literal for int() with base 10: \u0026#39;not_a_number\u0026#39; Question 6 - Context Manager Q: Write a context manager class SuppressAndLog that suppresses given exception types and prints a log line instead of crashing.\nA:\nclass SuppressAndLog: def __init__(self, *exception_types, message=\u0026#34;Exception suppressed\u0026#34;): self.exception_types = exception_types self.message = message def __enter__(self): return self def __exit__(self, exc_type, exc_val, exc_tb): if exc_type and issubclass(exc_type, self.exception_types): print(f\u0026#34;[LOG] {self.message}: {exc_type.__name__}: {exc_val}\u0026#34;) return True # suppress the exception return False # re-raise anything else with SuppressAndLog(ZeroDivisionError, message=\u0026#34;Math error caught\u0026#34;): result = 1 / 0 # [LOG] Math error caught: ZeroDivisionError: division by zero print(\u0026#34;Program continues normally.\u0026#34;) Question 7 - Accumulate Multiple Errors Q: Write extract_fields(record, *fields) that extracts multiple keys from a dict. Collect all missing fields into a single KeyError instead of failing on the first one.\nA:\ndef extract_fields(record, *fields): result = {} missing = [] for field in fields: if field in record: result[field] = record[field] else: missing.append(field) if missing: raise KeyError(f\u0026#34;Missing required fields: {missing}\u0026#34;) return result data = {\u0026#34;name\u0026#34;: \u0026#34;Arjun\u0026#34;, \u0026#34;email\u0026#34;: \u0026#34;arjun@example.com\u0026#34;} try: info = extract_fields(data, \u0026#34;name\u0026#34;, \u0026#34;email\u0026#34;, \u0026#34;age\u0026#34;, \u0026#34;phone\u0026#34;) except KeyError as e: print(e) # \u0026#34;Missing required fields: [\u0026#39;age\u0026#39;, \u0026#39;phone\u0026#39;]\u0026#34; Question 8 - Tricky: Predict the Output Q: What does this print?\ndef foo(): try: return 1 finally: return 2 def bar(): try: raise ValueError(\u0026#34;oops\u0026#34;) except ValueError: return 3 finally: print(\u0026#34;finally in bar\u0026#34;) print(foo()) print(bar()) A:\n2 finally in bar 3 In foo(): the try block queues return value 1, but finally runs before delivery and its return 2 overrides it. foo() returns 2.\nIn bar(): ValueError is caught, return 3 is queued. finally runs first and prints \u0026quot;finally in bar\u0026quot;. Then bar() returns 3.\nQuestion 9 - Retry Decorator Q: Write a @retry(times=3) decorator that retries a function up to times attempts before letting the exception propagate.\nA:\nimport functools def retry(times=3): def decorator(func): @functools.wraps(func) def wrapper(*args, **kwargs): last_exception = None for attempt in range(1, times + 1): try: return func(*args, **kwargs) except Exception as e: last_exception = e print(f\u0026#34;Attempt {attempt}/{times} failed: {e}\u0026#34;) raise last_exception # all attempts exhausted return wrapper return decorator attempt_count = 0 @retry(times=3) def unstable_function(): global attempt_count attempt_count += 1 if attempt_count \u0026lt; 3: raise ConnectionError(\u0026#34;Not ready yet\u0026#34;) return \u0026#34;Success!\u0026#34; print(unstable_function()) # Attempt 1/3 failed: Not ready yet # Attempt 2/3 failed: Not ready yet # Success! Quick Reference Cheat Sheet ┌──────────────────────────────────────────────────────────────────────┐ │ EXCEPTION HANDLING QUICK REFERENCE │ ├───────────────────────┬──────────────────────────────────────────────┤ │ CLAUSE │ WHEN IT RUNS │ ├───────────────────────┼──────────────────────────────────────────────┤ │ try │ Always (starts here) │ │ except ExcType │ Only if ExcType (or subclass) was raised │ │ except (A, B) as e │ If A or B was raised; e = exception object │ │ else │ Only if NO exception was raised in try │ │ finally │ ALWAYS - even after return or exception │ ├───────────────────────┼──────────────────────────────────────────────┤ │ KEYWORD │ PURPOSE │ ├───────────────────────┼──────────────────────────────────────────────┤ │ raise ExcType(msg) │ Raise a new exception │ │ raise │ Re-raise current exception (same traceback) │ │ raise B from A │ Chain: B caused by A (A stored as __cause__) │ │ raise B from None │ Suppress original exception from traceback │ ├───────────────────────┼──────────────────────────────────────────────┤ │ BEST PRACTICES │ ├───────────────────────┬──────────────────────────────────────────────┤ │ DO │ Catch specific exceptions │ │ DO │ Keep try blocks narrow │ │ DO │ Use finally for cleanup │ │ DO │ Log before swallowing │ │ DO │ Create custom exceptions for your domain │ │ DO │ Use raise...from to chain exceptions │ │ AVOID │ Bare except: │ │ AVOID │ Silently passing on Exception │ │ AVOID │ return inside finally │ │ AVOID │ Ordering general handlers before specific │ └───────────────────────┴──────────────────────────────────────────────┘ EXCEPTION OBJECT ATTRIBUTES e.args → tuple of arguments passed to the exception e.__cause__ → explicitly chained exception (raise B from A) e.__context__ → implicitly chained exception e.__traceback__ → traceback object USEFUL STDLIB TOOLS traceback.format_exc() → current traceback as string traceback.print_exc() → print current traceback to stderr contextlib.suppress(ExcType) → silently ignore an exception logging.exception(msg) → log message + full traceback End of Lesson - Exception Handling in Python Codeverra - codeverra.com\n","permalink":"https://blogs.codeverra.com/blog/python-08-exception-handling/","summary":"\u003ch1 id=\"python-exception-handling\"\u003ePython Exception Handling\u003c/h1\u003e\n\u003ch3 id=\"a-complete-reference-guide\"\u003eA Complete Reference Guide\u003c/h3\u003e\n\u003chr\u003e\n\u003ch2 id=\"table-of-contents\"\u003eTable of Contents\u003c/h2\u003e\n\u003col\u003e\n\u003cli\u003e\u003ca href=\"#the-big-picture\"\u003eThe Big Picture\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#what-are-exceptions\"\u003eWhat Are Exceptions?\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#the-exception-hierarchy\"\u003eThe Exception Hierarchy\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#try--except--the-foundation\"\u003etry / except  -  The Foundation\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#else-and-finally-clauses\"\u003eelse and finally Clauses\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#raising-exceptions\"\u003eRaising Exceptions\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#custom-exceptions\"\u003eCustom Exceptions\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#exception-chaining\"\u003eException Chaining\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#context-managers-and-exceptions\"\u003eContext Managers and Exceptions\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#logging-exceptions\"\u003eLogging Exceptions\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#common-mistakes-to-avoid\"\u003eCommon Mistakes to Avoid\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#real-life-patterns\"\u003eReal-Life Patterns\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#practice-questions--answers\"\u003ePractice Questions \u0026amp; Answers\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#quick-reference-cheat-sheet\"\u003eQuick Reference Cheat Sheet\u003c/a\u003e\u003c/li\u003e\n\u003c/ol\u003e\n\u003chr\u003e\n\u003ch2 id=\"the-big-picture\"\u003eThe Big Picture\u003c/h2\u003e\n\u003cpre tabindex=\"0\"\u003e\u003ccode\u003e┌─────────────────────────────────────────────────────────────────┐\n│                    EXCEPTION HANDLING FLOW                      │\n│                                                                 │\n│   try:                                                          │\n│     ── risky code runs here ──────────────────────────────┐    │\n│                                                           │    │\n│   except SomeError:          ◄── exception caught here   │    │\n│     ── handle it ──                                       │    │\n│                                                           ▼    │\n│   except (TypeError, ValueError):  ◄── multiple types        │\n│     ── handle them ──                                          │\n│                                                                 │\n│   except Exception as e:    ◄── catch with reference          │\n│     ── inspect e ──                                            │\n│                                                                 │\n│   else:                     ◄── runs ONLY if NO exception      │\n│     ── success path ──                                         │\n│                                                                 │\n│   finally:                  ◄── ALWAYS runs, no matter what   │\n│     ── cleanup ──                                              │\n└─────────────────────────────────────────────────────────────────┘\n\u003c/code\u003e\u003c/pre\u003e\u003cblockquote\u003e\n\u003cp\u003e\u003cstrong\u003eCore Philosophy:\u003c/strong\u003e Exceptions are not bugs  -  they are Python\u0026rsquo;s way of saying \u0026ldquo;something unexpected happened here, and I\u0026rsquo;m passing control to whoever knows how to handle it.\u0026rdquo;\u003c/p\u003e","tags":["python"],"title":"Exception Handling in Python - Complete Guide"},{"content":"Working with Files in Python - The Complete Guide Everything you need to read, write, parse, and manage files like a professional Why Files Matter More Than You Think Every meaningful program eventually needs to talk to the outside world. Not through a web API or a database - just a plain file sitting on disk. Your data science pipeline reads a CSV. Your web scraper writes results to JSON. Your deployment script reads a config file. Your ETL job processes gigabytes of logs.\nBeginners often learn open(), write a few lines, close the file, and move on. But there\u0026rsquo;s a lot more to it - and the gaps in knowledge show up at the worst times. The file that never gets closed because an exception was raised. The CSV that silently corrupts because of encoding. The config file that works on your Mac and breaks on the Linux server. The 10 GB log file that gets loaded into a list and crashes the machine.\nThis guide covers all of it - from the fundamentals of how Python opens files, through every format you\u0026rsquo;ll encounter in real work, to patterns for handling large data, and finally the pathlib module that should replace most of your string-based path manipulation.\nBy the end, you\u0026rsquo;ll know not just how to work with files, but why Python\u0026rsquo;s file model works the way it does.\nThe Mental Model: What Happens When You Open a File? Before touching any code, understand what\u0026rsquo;s actually happening when you call open().\nYour Python Script │ │ open(\u0026#34;data.txt\u0026#34;, \u0026#34;r\u0026#34;) ▼ ┌─────────────────────┐ │ Operating System │ ← Python delegates to the OS │ File System API │ └─────────────────────┘ │ │ Returns a file descriptor (an integer handle) ▼ ┌─────────────────────┐ │ File Object │ ← Python wraps it in a file object │ (file handle) │ with methods: read(), write(), etc. │ │ │ - position pointer │ ← tracks where you are in the file │ - read/write buffer│ ← data cached in memory │ - encoding info │ ← how bytes map to characters └─────────────────────┘ │ │ When you close() (or `with` exits) ▼ Buffer flushed to disk File descriptor released back to OS The key things to take away from this:\nA file descriptor is a limited OS resource. Systems have a maximum number of open files. If you forget to close files, you will eventually hit Too many open files errors. The position pointer moves as you read. If you read the whole file and try to read again, you get an empty string - the pointer is at the end. You need to seek(0) to go back to the start. The buffer means writes may not immediately reach disk. Calling close() or flush() forces the buffer to disk. Part 1: The Fundamentals open() - The Entry Point # Full signature open(file, mode=\u0026#39;r\u0026#39;, buffering=-1, encoding=None, errors=None, newline=None) The two arguments you\u0026rsquo;ll use on every call are file (the path) and mode.\nFile Modes - The Complete Picture ┌──────────┬──────────────────────────────────────────────┬──────────┬─────────┐ │ Mode │ Meaning │ Read? │ Write? │ ├──────────┼──────────────────────────────────────────────┼──────────┼─────────┤ │ \u0026#39;r\u0026#39; │ Read (default). File must exist. │ Yes │ No │ │ \u0026#39;w\u0026#39; │ Write. Creates file or TRUNCATES existing. │ No │ Yes │ │ \u0026#39;a\u0026#39; │ Append. Creates file or appends to end. │ No │ Yes │ │ \u0026#39;x\u0026#39; │ Exclusive create. Fails if file exists. │ No │ Yes │ │ \u0026#39;r+\u0026#39; │ Read and write. File must exist. │ Yes │ Yes │ │ \u0026#39;w+\u0026#39; │ Read and write. Creates or TRUNCATES. │ Yes │ Yes │ │ \u0026#39;a+\u0026#39; │ Read and append. Creates if needed. │ Yes │ Yes │ ├──────────┼──────────────────────────────────────────────┼──────────┼─────────┤ │ + \u0026#39;b\u0026#39; │ Binary mode. No encoding/decoding. │ │ │ │ │ Add to any mode: \u0026#39;rb\u0026#39;, \u0026#39;wb\u0026#39;, \u0026#39;ab\u0026#39;, etc. │ │ │ ├──────────┼──────────────────────────────────────────────┼──────────┼─────────┤ │ + \u0026#39;t\u0026#39; │ Text mode (default). Handles encoding. │ │ │ │ │ \u0026#39;rt\u0026#39; == \u0026#39;r\u0026#39;, \u0026#39;wt\u0026#39; == \u0026#39;w\u0026#39;, etc. │ │ │ └──────────┴──────────────────────────────────────────────┴──────────┴─────────┘ # Mode examples f = open(\u0026#34;data.txt\u0026#34;, \u0026#34;r\u0026#34;) # read text (default) f = open(\u0026#34;data.txt\u0026#34;, \u0026#34;w\u0026#34;) # write text - WARNING: destroys existing content! f = open(\u0026#34;data.txt\u0026#34;, \u0026#34;a\u0026#34;) # append text f = open(\u0026#34;data.txt\u0026#34;, \u0026#34;x\u0026#34;) # create new, fail if exists - safe write f = open(\u0026#34;image.png\u0026#34;, \u0026#34;rb\u0026#34;) # read binary f = open(\u0026#34;data.bin\u0026#34;, \u0026#34;wb\u0026#34;) # write binary Always Use with - The Context Manager Pattern The single most important rule in file handling is this: always open files with a with statement.\n# NEVER do this in real code f = open(\u0026#34;data.txt\u0026#34;, \u0026#34;r\u0026#34;) content = f.read() f.close() # What if read() raised an exception? close() is never called. # ALWAYS do this with open(\u0026#34;data.txt\u0026#34;, \u0026#34;r\u0026#34;) as f: content = f.read() # f.close() is called automatically, even if an exception occurred inside the block The with statement calls __exit__ on the file object when the block ends - whether normally or via exception. The file is always closed.\n# Multiple files at once - single with statement with open(\u0026#34;input.txt\u0026#34;, \u0026#34;r\u0026#34;) as src, open(\u0026#34;output.txt\u0026#34;, \u0026#34;w\u0026#34;) as dst: for line in src: dst.write(line.upper()) # Both files are closed when the block exits Reading Files - All the Ways # 1. read() - entire file as one string with open(\u0026#34;story.txt\u0026#34;, \u0026#34;r\u0026#34;, encoding=\u0026#34;utf-8\u0026#34;) as f: content = f.read() print(type(content)) # \u0026lt;class \u0026#39;str\u0026#39;\u0026gt; print(len(content)) # total character count # 2. read(n) - read exactly n characters with open(\u0026#34;story.txt\u0026#34;, \u0026#34;r\u0026#34;, encoding=\u0026#34;utf-8\u0026#34;) as f: first_100 = f.read(100) # first 100 characters next_100 = f.read(100) # next 100 characters # 3. readline() - one line at a time with open(\u0026#34;data.txt\u0026#34;, \u0026#34;r\u0026#34;, encoding=\u0026#34;utf-8\u0026#34;) as f: first_line = f.readline() # includes the \u0026#39;\\n\u0026#39; second_line = f.readline() print(repr(first_line)) # \u0026#39;Hello, World!\\n\u0026#39; # 4. readlines() - all lines as a list with open(\u0026#34;data.txt\u0026#34;, \u0026#34;r\u0026#34;, encoding=\u0026#34;utf-8\u0026#34;) as f: lines = f.readlines() print(lines) # [\u0026#39;line 1\\n\u0026#39;, \u0026#39;line 2\\n\u0026#39;, \u0026#39;line 3\\n\u0026#39;] # 5. Iterating directly - most Pythonic and memory-efficient with open(\u0026#34;data.txt\u0026#34;, \u0026#34;r\u0026#34;, encoding=\u0026#34;utf-8\u0026#34;) as f: for line in f: # file object is its own iterator print(line.rstrip(\u0026#39;\\n\u0026#39;)) # strip trailing newline # 6. List comprehension over file with open(\u0026#34;data.txt\u0026#34;, \u0026#34;r\u0026#34;, encoding=\u0026#34;utf-8\u0026#34;) as f: lines = [line.strip() for line in f if line.strip()] # non-empty lines only Writing Files - All the Ways # 1. write(string) - write a string, returns number of characters written with open(\u0026#34;output.txt\u0026#34;, \u0026#34;w\u0026#34;, encoding=\u0026#34;utf-8\u0026#34;) as f: chars_written = f.write(\u0026#34;Hello, Hyderabad!\\n\u0026#34;) print(chars_written) # 18 # 2. writelines(iterable) - write each item; no newlines added automatically lines = [\u0026#34;line 1\\n\u0026#34;, \u0026#34;line 2\\n\u0026#34;, \u0026#34;line 3\\n\u0026#34;] with open(\u0026#34;output.txt\u0026#34;, \u0026#34;w\u0026#34;, encoding=\u0026#34;utf-8\u0026#34;) as f: f.writelines(lines) # does NOT add \u0026#39;\\n\u0026#39; between items - include it yourself # 3. print() to a file - convenient for formatted output with open(\u0026#34;report.txt\u0026#34;, \u0026#34;w\u0026#34;, encoding=\u0026#34;utf-8\u0026#34;) as f: print(\u0026#34;Sales Report\u0026#34;, file=f) print(\u0026#34;=\u0026#34; * 40, file=f) print(f\u0026#34;Total: Rs.{15000:,}\u0026#34;, file=f) # 4. Appending - never overwrites, always adds at end with open(\u0026#34;log.txt\u0026#34;, \u0026#34;a\u0026#34;, encoding=\u0026#34;utf-8\u0026#34;) as f: f.write(\u0026#34;2024-01-15 10:32:00 - User logged in\\n\u0026#34;) The Position Pointer and seek() / tell() with open(\u0026#34;data.txt\u0026#34;, \u0026#34;r\u0026#34;, encoding=\u0026#34;utf-8\u0026#34;) as f: # tell() returns the current byte position print(f.tell()) # 0 - at the start first = f.read(10) # read 10 characters print(f.tell()) # 10 (for ASCII; could be more for multi-byte chars) # seek(offset, whence=0) # whence=0: from start of file (default) # whence=1: from current position # whence=2: from end of file f.seek(0) # go back to the beginning everything = f.read() # now reads the full file again f.seek(0, 2) # seek to end of file size = f.tell() # file size in bytes print(f\u0026#34;File size: {size} bytes\u0026#34;) f.seek(-10, 2) # 10 bytes before the end last_10 = f.read() # last 10 bytes Part 2: Encoding - The Source of Silent Bugs Encoding is the most common source of file-related bugs, especially when files move between systems or contain non-ASCII characters. Understanding it prevents hours of debugging.\nWhat Is Encoding? A file on disk is just bytes - a sequence of integers from 0 to 255. Encoding is the mapping that converts those bytes into characters (and back). When you open a text file, Python uses an encoding to decode bytes into a Python string. When you write, it encodes strings back into bytes.\nFile on disk: E0 A4 A8 E0 A4 AE E0 A4 B8 (bytes) Encoding: UTF-8 ←───────────────────────────────── Python string: \u0026#39;नमस्\u0026#39; (characters) Always Specify encoding=\u0026quot;utf-8\u0026quot; # RISKY - uses the system\u0026#39;s default encoding (varies by OS and locale) with open(\u0026#34;data.txt\u0026#34;, \u0026#34;r\u0026#34;) as f: content = f.read() # Works on your Linux machine (UTF-8 default), fails on Windows (cp1252 default) # SAFE - explicit is always better with open(\u0026#34;data.txt\u0026#34;, \u0026#34;r\u0026#34;, encoding=\u0026#34;utf-8\u0026#34;) as f: content = f.read() # Writing with explicit encoding with open(\u0026#34;output.txt\u0026#34;, \u0026#34;w\u0026#34;, encoding=\u0026#34;utf-8\u0026#34;) as f: f.write(\u0026#34;नमस्कार दुनिया\\n\u0026#34;) # Hindi text - requires UTF-8 Common Encodings ┌──────────────────┬────────────────────────────────────────────────────┐ │ Encoding │ When you encounter it │ ├──────────────────┼────────────────────────────────────────────────────┤ │ utf-8 │ Default for Linux/Mac, modern web, Python source │ │ │ Supports every character in existence │ │ utf-8-sig │ UTF-8 with a BOM (Byte Order Mark) at the start │ │ │ Common in Excel-exported CSVs from Windows │ │ cp1252 / latin-1 │ Windows legacy encoding, common in older CSVs │ │ │ Extended ASCII, covers Western European chars │ │ ascii │7-bit only (0-127); fails on any accented char │ │ utf-16 │ Used in some Windows APIs and older files │ └──────────────────┴────────────────────────────────────────────────────┘ Handling Encoding Errors # errors parameter controls what happens on a bad byte # \u0026#39;strict\u0026#39; (default) - raises UnicodeDecodeError on bad byte with open(\u0026#34;file.txt\u0026#34;, \u0026#34;r\u0026#34;, encoding=\u0026#34;utf-8\u0026#34;, errors=\u0026#34;strict\u0026#34;) as f: content = f.read() # \u0026#39;ignore\u0026#39; - silently skips bad bytes (data loss, but no crash) with open(\u0026#34;file.txt\u0026#34;, \u0026#34;r\u0026#34;, encoding=\u0026#34;utf-8\u0026#34;, errors=\u0026#34;ignore\u0026#34;) as f: content = f.read() # \u0026#39;replace\u0026#39; - replaces bad bytes with the replacement char (U+FFFD: \u0026#39;?\u0026#39;) with open(\u0026#34;file.txt\u0026#34;, \u0026#34;r\u0026#34;, encoding=\u0026#34;utf-8\u0026#34;, errors=\u0026#34;replace\u0026#34;) as f: content = f.read() # \u0026#39;backslashreplace\u0026#39; - replaces bad bytes with \\xNN escape sequences with open(\u0026#34;file.txt\u0026#34;, \u0026#34;r\u0026#34;, encoding=\u0026#34;utf-8\u0026#34;, errors=\u0026#34;backslashreplace\u0026#34;) as f: content = f.read() # Detecting encoding when you don\u0026#39;t know it # pip install chardet import chardet with open(\u0026#34;mystery_file.txt\u0026#34;, \u0026#34;rb\u0026#34;) as f: raw = f.read(10000) # sample first 10 KB result = chardet.detect(raw) detected = result[\u0026#34;encoding\u0026#34;] confidence = result[\u0026#34;confidence\u0026#34;] print(f\u0026#34;Detected: {detected} ({confidence:.0%} confidence)\u0026#34;) with open(\u0026#34;mystery_file.txt\u0026#34;, \u0026#34;r\u0026#34;, encoding=detected) as f: content = f.read() Part 3: Working with Text Files Reading Line by Line - The Right Way for Large Files # MEMORY-EFFICIENT: iterate over the file object directly # Only one line is in memory at a time - works for files of any size def count_lines(filepath): count = 0 with open(filepath, \u0026#34;r\u0026#34;, encoding=\u0026#34;utf-8\u0026#34;) as f: for _ in f: count += 1 return count def grep(filepath, search_term): \u0026#34;\u0026#34;\u0026#34;Yield lines containing search_term.\u0026#34;\u0026#34;\u0026#34; with open(filepath, \u0026#34;r\u0026#34;, encoding=\u0026#34;utf-8\u0026#34;) as f: for line_num, line in enumerate(f, start=1): if search_term in line: yield line_num, line.rstrip(\u0026#34;\\n\u0026#34;) # Usage for num, line in grep(\u0026#34;server.log\u0026#34;, \u0026#34;ERROR\u0026#34;): print(f\u0026#34;Line {num}: {line}\u0026#34;) # MEMORY-EXPENSIVE: only use when you need all lines at once with open(\u0026#34;small_file.txt\u0026#34;, \u0026#34;r\u0026#34;, encoding=\u0026#34;utf-8\u0026#34;) as f: lines = f.readlines() # loads entire file into a list # BETTER alternative to readlines() - list comprehension strips newlines too with open(\u0026#34;small_file.txt\u0026#34;, \u0026#34;r\u0026#34;, encoding=\u0026#34;utf-8\u0026#34;) as f: lines = [line.rstrip(\u0026#34;\\n\u0026#34;) for line in f] Writing Logs and Reports import datetime def write_report(filepath, data): \u0026#34;\u0026#34;\u0026#34;Write a formatted report to a text file.\u0026#34;\u0026#34;\u0026#34; timestamp = datetime.datetime.now().strftime(\u0026#34;%Y-%m-%d %H:%M:%S\u0026#34;) with open(filepath, \u0026#34;w\u0026#34;, encoding=\u0026#34;utf-8\u0026#34;) as f: f.write(f\u0026#34;Report generated: {timestamp}\\n\u0026#34;) f.write(\u0026#34;=\u0026#34; * 60 + \u0026#34;\\n\\n\u0026#34;) for section_title, rows in data.items(): f.write(f\u0026#34;{section_title}\\n\u0026#34;) f.write(\u0026#34;-\u0026#34; * len(section_title) + \u0026#34;\\n\u0026#34;) for row in rows: f.write(f\u0026#34; {row}\\n\u0026#34;) f.write(\u0026#34;\\n\u0026#34;) data = { \u0026#34;Top Scorers\u0026#34;: [\u0026#34;Virat Kohli: 520 runs\u0026#34;, \u0026#34;Rohit Sharma: 480 runs\u0026#34;], \u0026#34;Top Wicket Takers\u0026#34;: [\u0026#34;Jasprit Bumrah: 22 wickets\u0026#34;, \u0026#34;R Ashwin: 18 wickets\u0026#34;], } write_report(\u0026#34;ipl_report.txt\u0026#34;, data) # Append-only logging def log_event(logfile, level, message): \u0026#34;\u0026#34;\u0026#34;Append a timestamped log entry.\u0026#34;\u0026#34;\u0026#34; timestamp = datetime.datetime.now().isoformat() with open(logfile, \u0026#34;a\u0026#34;, encoding=\u0026#34;utf-8\u0026#34;) as f: f.write(f\u0026#34;{timestamp} [{level.upper()}] {message}\\n\u0026#34;) log_event(\u0026#34;app.log\u0026#34;, \u0026#34;INFO\u0026#34;, \u0026#34;Server started on port 8000\u0026#34;) log_event(\u0026#34;app.log\u0026#34;, \u0026#34;WARNING\u0026#34;, \u0026#34;Memory usage above 80%\u0026#34;) log_event(\u0026#34;app.log\u0026#34;, \u0026#34;ERROR\u0026#34;, \u0026#34;Database connection refused\u0026#34;) Safe Write Pattern - Write-Then-Rename (Atomic Writes) A common problem: you write to a file, but the process is killed halfway through. Now the file is corrupt. The solution is to write to a temp file, then atomically rename it.\nimport os import tempfile def safe_write(filepath, content): \u0026#34;\u0026#34;\u0026#34; Write content to filepath safely: 1. Write to a temp file in the same directory 2. Rename temp → target (atomic on most systems) This ensures the target file is never partially written. \u0026#34;\u0026#34;\u0026#34; directory = os.path.dirname(os.path.abspath(filepath)) with tempfile.NamedTemporaryFile( mode=\u0026#34;w\u0026#34;, encoding=\u0026#34;utf-8\u0026#34;, dir=directory, delete=False, suffix=\u0026#34;.tmp\u0026#34; ) as tmp: tmp.write(content) tmp_path = tmp.name os.replace(tmp_path, filepath) # atomic on POSIX; near-atomic on Windows print(f\u0026#34;Written safely to {filepath}\u0026#34;) safe_write(\u0026#34;config.txt\u0026#34;, \u0026#34;host=localhost\\nport=5432\\n\u0026#34;) Part 4: CSV Files CSV (Comma-Separated Values) is the most common format for tabular data. Python\u0026rsquo;s csv module handles quoting, special characters, and delimiters correctly - never split CSV lines with .split(\u0026quot;,\u0026quot;).\nReading CSV Files import csv # Basic reading with open(\u0026#34;players.csv\u0026#34;, \u0026#34;r\u0026#34;, encoding=\u0026#34;utf-8\u0026#34;, newline=\u0026#34;\u0026#34;) as f: reader = csv.reader(f) header = next(reader) # skip the header row print(\u0026#34;Columns:\u0026#34;, header) for row in reader: print(row) # each row is a list of strings # DictReader - rows as dicts (column names as keys) with open(\u0026#34;players.csv\u0026#34;, \u0026#34;r\u0026#34;, encoding=\u0026#34;utf-8\u0026#34;, newline=\u0026#34;\u0026#34;) as f: reader = csv.DictReader(f) for row in reader: print(dict(row)) # {\u0026#39;name\u0026#39;: \u0026#39;Rohit Sharma\u0026#39;, \u0026#39;team\u0026#39;: \u0026#39;MI\u0026#39;, \u0026#39;runs\u0026#39;: \u0026#39;1200\u0026#39;, \u0026#39;avg\u0026#39;: \u0026#39;48.5\u0026#39;} Why newline=\u0026quot;\u0026quot;? The csv module handles its own line endings. If you let Python\u0026rsquo;s text mode translate \\r\\n to \\n first, the csv module can get confused on some Windows-written files. Always use newline=\u0026quot;\u0026quot; when opening files for csv.reader or csv.writer.\nWriting CSV Files import csv players = [ [\u0026#34;name\u0026#34;, \u0026#34;team\u0026#34;, \u0026#34;runs\u0026#34;, \u0026#34;wickets\u0026#34;], [\u0026#34;Rohit Sharma\u0026#34;, \u0026#34;MI\u0026#34;, 1200, 0], [\u0026#34;Virat Kohli\u0026#34;, \u0026#34;RCB\u0026#34;, 980, 0], [\u0026#34;Jasprit Bumrah\u0026#34;,\u0026#34;MI\u0026#34;, 45, 28], ] with open(\u0026#34;output.csv\u0026#34;, \u0026#34;w\u0026#34;, encoding=\u0026#34;utf-8\u0026#34;, newline=\u0026#34;\u0026#34;) as f: writer = csv.writer(f) writer.writerows(players) # writerow() for one row, writerows() for many # DictWriter - write from dicts fieldnames = [\u0026#34;name\u0026#34;, \u0026#34;team\u0026#34;, \u0026#34;runs\u0026#34;, \u0026#34;wickets\u0026#34;] records = [ {\u0026#34;name\u0026#34;: \u0026#34;Rohit Sharma\u0026#34;, \u0026#34;team\u0026#34;: \u0026#34;MI\u0026#34;, \u0026#34;runs\u0026#34;: 1200, \u0026#34;wickets\u0026#34;: 0}, {\u0026#34;name\u0026#34;: \u0026#34;Virat Kohli\u0026#34;, \u0026#34;team\u0026#34;: \u0026#34;RCB\u0026#34;, \u0026#34;runs\u0026#34;: 980, \u0026#34;wickets\u0026#34;: 0}, {\u0026#34;name\u0026#34;: \u0026#34;Jasprit Bumrah\u0026#34;,\u0026#34;team\u0026#34;: \u0026#34;MI\u0026#34;, \u0026#34;runs\u0026#34;: 45, \u0026#34;wickets\u0026#34;: 28}, ] with open(\u0026#34;players_dict.csv\u0026#34;, \u0026#34;w\u0026#34;, encoding=\u0026#34;utf-8\u0026#34;, newline=\u0026#34;\u0026#34;) as f: writer = csv.DictWriter(f, fieldnames=fieldnames) writer.writeheader() # writes the column header row writer.writerows(records) Handling Real-World CSV Complications import csv # Custom delimiter (TSV, pipe-separated, etc.) with open(\u0026#34;data.tsv\u0026#34;, \u0026#34;r\u0026#34;, encoding=\u0026#34;utf-8\u0026#34;, newline=\u0026#34;\u0026#34;) as f: reader = csv.reader(f, delimiter=\u0026#34;\\t\u0026#34;) # tab-separated for row in reader: print(row) # Quoting - fields with commas or newlines are automatically quoted with open(\u0026#34;tricky.csv\u0026#34;, \u0026#34;w\u0026#34;, encoding=\u0026#34;utf-8\u0026#34;, newline=\u0026#34;\u0026#34;) as f: writer = csv.writer(f, quoting=csv.QUOTE_MINIMAL) # default writer.writerow([\u0026#34;name\u0026#34;, \u0026#34;address\u0026#34;, \u0026#34;notes\u0026#34;]) writer.writerow([\u0026#34;Ravi\u0026#34;, \u0026#34;123, MG Road, Bangalore\u0026#34;, \u0026#34;Has a comma in address\u0026#34;]) # Output: Ravi,\u0026#34;123, MG Road, Bangalore\u0026#34;,Has a comma in address # Skipping blank lines and comments with open(\u0026#34;dirty.csv\u0026#34;, \u0026#34;r\u0026#34;, encoding=\u0026#34;utf-8\u0026#34;, newline=\u0026#34;\u0026#34;) as f: reader = csv.reader(f) cleaned = [ row for row in reader if row and not row[0].startswith(\u0026#34;#\u0026#34;) # skip empty and comment rows ] # Excel-exported CSVs often have UTF-8-BOM encoding with open(\u0026#34;excel_export.csv\u0026#34;, \u0026#34;r\u0026#34;, encoding=\u0026#34;utf-8-sig\u0026#34;, newline=\u0026#34;\u0026#34;) as f: reader = csv.DictReader(f) for row in reader: print(row) # column names won\u0026#39;t have a BOM prefix # Type conversion - CSV values are always strings; convert explicitly with open(\u0026#34;stats.csv\u0026#34;, \u0026#34;r\u0026#34;, encoding=\u0026#34;utf-8\u0026#34;, newline=\u0026#34;\u0026#34;) as f: reader = csv.DictReader(f) for row in reader: name = row[\u0026#34;name\u0026#34;] runs = int(row[\u0026#34;runs\u0026#34;]) # explicit conversion avg = float(row[\u0026#34;average\u0026#34;]) active = row[\u0026#34;active\u0026#34;].lower() == \u0026#34;true\u0026#34; Processing Large CSV Files Without pandas import csv from collections import defaultdict def summarize_csv(filepath): \u0026#34;\u0026#34;\u0026#34; Read a large CSV and compute per-team totals without loading the whole file into memory. \u0026#34;\u0026#34;\u0026#34; team_runs = defaultdict(int) with open(filepath, \u0026#34;r\u0026#34;, encoding=\u0026#34;utf-8\u0026#34;, newline=\u0026#34;\u0026#34;) as f: reader = csv.DictReader(f) for row in reader: team = row[\u0026#34;team\u0026#34;] runs = int(row[\u0026#34;runs\u0026#34;]) team_runs[team] += runs return dict(sorted(team_runs.items(), key=lambda x: -x[1])) # Each row is processed and discarded - constant memory usage regardless of file size Part 5: JSON Files JSON is the default format for configuration files, API responses, and data exchange between services.\nReading and Writing JSON import json # Write a Python object to JSON data = { \u0026#34;tournament\u0026#34;: \u0026#34;IPL 2024\u0026#34;, \u0026#34;teams\u0026#34;: [\u0026#34;MI\u0026#34;, \u0026#34;CSK\u0026#34;, \u0026#34;RCB\u0026#34;, \u0026#34;KKR\u0026#34;, \u0026#34;SRH\u0026#34;], \u0026#34;matches\u0026#34;: 74, \u0026#34;champion\u0026#34;: \u0026#34;KKR\u0026#34;, \u0026#34;stats\u0026#34;: { \u0026#34;highest_score\u0026#34;: 287, \u0026#34;total_sixes\u0026#34;: 1234, }, } with open(\u0026#34;ipl2024.json\u0026#34;, \u0026#34;w\u0026#34;, encoding=\u0026#34;utf-8\u0026#34;) as f: json.dump(data, f, indent=4, ensure_ascii=False) # indent=4 → pretty-printed with 4-space indentation # ensure_ascii=False → allows non-ASCII characters (Hindi, etc.) as-is # Read a JSON file back with open(\u0026#34;ipl2024.json\u0026#34;, \u0026#34;r\u0026#34;, encoding=\u0026#34;utf-8\u0026#34;) as f: loaded = json.load(f) print(loaded[\u0026#34;champion\u0026#34;]) # KKR print(loaded[\u0026#34;stats\u0026#34;][\u0026#34;total_sixes\u0026#34;]) # 1234 # JSON serialization (to string) vs deserialization (from string) json_string = json.dumps(data, indent=2) # dumps → to string print(json_string[:80]) back_to_dict = json.loads(json_string) # loads → from string JSON Type Mapping Python JSON ───────────────────────────────── dict → object {} list, tuple → array [] str → string \u0026#34;\u0026#34; int, float → number True / False → true / false None → null Handling Non-Serializable Types import json import datetime from decimal import Decimal from pathlib import Path # By default, datetime, Decimal, and Path objects are not JSON-serializable data = { \u0026#34;name\u0026#34;: \u0026#34;Rohit Sharma\u0026#34;, \u0026#34;dob\u0026#34;: datetime.date(1987, 4, 30), # ← not serializable \u0026#34;salary\u0026#34;: Decimal(\u0026#34;15000000.50\u0026#34;), # ← not serializable } # json.dumps(data) → TypeError! # Solution 1: Custom default function def json_serializer(obj): if isinstance(obj, (datetime.date, datetime.datetime)): return obj.isoformat() if isinstance(obj, Decimal): return float(obj) if isinstance(obj, Path): return str(obj) raise TypeError(f\u0026#34;Object of type {type(obj).__name__} is not JSON serializable\u0026#34;) json_string = json.dumps(data, default=json_serializer, indent=2) print(json_string) # Solution 2: Custom JSONEncoder subclass (better for repeated use) class AppJSONEncoder(json.JSONEncoder): def default(self, obj): if isinstance(obj, (datetime.date, datetime.datetime)): return obj.isoformat() if isinstance(obj, Decimal): return str(obj) # str preserves precision; float may lose it if isinstance(obj, set): return sorted(list(obj)) return super().default(obj) with open(\u0026#34;data.json\u0026#34;, \u0026#34;w\u0026#34;, encoding=\u0026#34;utf-8\u0026#34;) as f: json.dump(data, f, cls=AppJSONEncoder, indent=2) JSON Lines Format (JSONL) - One JSON Object Per Line JSONL is ideal for streaming or appending records because each line is independent.\nimport json # Writing JSONL - append records one by one records = [ {\u0026#34;id\u0026#34;: 1, \u0026#34;player\u0026#34;: \u0026#34;Rohit Sharma\u0026#34;, \u0026#34;score\u0026#34;: 89}, {\u0026#34;id\u0026#34;: 2, \u0026#34;player\u0026#34;: \u0026#34;Virat Kohli\u0026#34;, \u0026#34;score\u0026#34;: 112}, {\u0026#34;id\u0026#34;: 3, \u0026#34;player\u0026#34;: \u0026#34;MS Dhoni\u0026#34;, \u0026#34;score\u0026#34;: 67}, ] with open(\u0026#34;scores.jsonl\u0026#34;, \u0026#34;w\u0026#34;, encoding=\u0026#34;utf-8\u0026#34;) as f: for record in records: f.write(json.dumps(record) + \u0026#34;\\n\u0026#34;) # one JSON object per line # Reading JSONL - lazy, memory-efficient def read_jsonl(filepath): with open(filepath, \u0026#34;r\u0026#34;, encoding=\u0026#34;utf-8\u0026#34;) as f: for line in f: line = line.strip() if line: # skip blank lines yield json.loads(line) for record in read_jsonl(\u0026#34;scores.jsonl\u0026#34;): print(record) # {\u0026#39;id\u0026#39;: 1, \u0026#39;player\u0026#39;: \u0026#39;Rohit Sharma\u0026#39;, \u0026#39;score\u0026#39;: 89} # ... Part 6: Binary Files Some files aren\u0026rsquo;t text - images, PDFs, executables, compressed archives. You handle these with binary mode.\nReading and Writing Binary Files # Copy a binary file def copy_file(src_path, dst_path, chunk_size=65536): \u0026#34;\u0026#34;\u0026#34;Copy a file in binary chunks - works for files of any size.\u0026#34;\u0026#34;\u0026#34; with open(src_path, \u0026#34;rb\u0026#34;) as src, open(dst_path, \u0026#34;wb\u0026#34;) as dst: while True: chunk = src.read(chunk_size) if not chunk: # empty bytes = end of file break dst.write(chunk) copy_file(\u0026#34;photo.jpg\u0026#34;, \u0026#34;photo_backup.jpg\u0026#34;) # Get file size without loading content import os size_bytes = os.path.getsize(\u0026#34;photo.jpg\u0026#34;) print(f\u0026#34;File size: {size_bytes:,} bytes ({size_bytes / 1024 / 1024:.2f} MB)\u0026#34;) # Read the first few bytes - magic bytes for file type detection def detect_file_type(filepath): \u0026#34;\u0026#34;\u0026#34;Detect file type by reading magic bytes.\u0026#34;\u0026#34;\u0026#34; signatures = { b\u0026#39;\\xff\\xd8\\xff\u0026#39;: \u0026#39;JPEG image\u0026#39;, b\u0026#39;\\x89PNG\\r\\n\\x1a\\n\u0026#39;: \u0026#39;PNG image\u0026#39;, b\u0026#39;%PDF\u0026#39;: \u0026#39;PDF document\u0026#39;, b\u0026#39;PK\\x03\\x04\u0026#39;: \u0026#39;ZIP archive\u0026#39;, b\u0026#39;\\x1f\\x8b\u0026#39;: \u0026#39;GZIP compressed\u0026#39;, b\u0026#39;RIFF\u0026#39;: \u0026#39;RIFF file (WAV/AVI)\u0026#39;, } with open(filepath, \u0026#34;rb\u0026#34;) as f: header = f.read(8) for signature, file_type in signatures.items(): if header.startswith(signature): return file_type return \u0026#34;Unknown file type\u0026#34; print(detect_file_type(\u0026#34;report.pdf\u0026#34;)) # PDF document print(detect_file_type(\u0026#34;photo.jpg\u0026#34;)) # JPEG image Working with struct - Reading Structured Binary Data import struct # Write structured binary data (a simple binary format) # Format: name (20 chars), score (int), average (float) players = [ (\u0026#34;Rohit Sharma\u0026#34;, 1200, 48.50), (\u0026#34;Virat Kohli\u0026#34;, 980, 52.10), (\u0026#34;Jasprit Bumrah\u0026#34;, 45, 9.00), ] record_format = \u0026#34;20s i f\u0026#34; # 20-char string, int (4 bytes), float (4 bytes) record_size = struct.calcsize(record_format) # = 28 bytes with open(\u0026#34;players.bin\u0026#34;, \u0026#34;wb\u0026#34;) as f: for name, runs, avg in players: packed = struct.pack(record_format, name.encode(\u0026#34;utf-8\u0026#34;).ljust(20), runs, avg) f.write(packed) # Read it back with open(\u0026#34;players.bin\u0026#34;, \u0026#34;rb\u0026#34;) as f: while True: data = f.read(record_size) if not data: break name_bytes, runs, avg = struct.unpack(record_format, data) name = name_bytes.decode(\u0026#34;utf-8\u0026#34;).rstrip(\u0026#34;\\x00\u0026#34;) print(f\u0026#34;{name}: {runs} runs, avg {avg:.2f}\u0026#34;) Part 7: Configuration Files with configparser INI-style config files are common for application settings. Python\u0026rsquo;s configparser module handles them natively.\nReading and Writing Config Files # config.ini # [database] # host = localhost # port = 5432 # name = myapp_db # pool_size = 10 # # [server] # host = 0.0.0.0 # port = 8000 # debug = true # allowed_origins = http://localhost:3000,https://myapp.com import configparser config = configparser.ConfigParser() config.read(\u0026#34;config.ini\u0026#34;, encoding=\u0026#34;utf-8\u0026#34;) # Access values - always returns strings db_host = config[\u0026#34;database\u0026#34;][\u0026#34;host\u0026#34;] db_port = config.getint(\u0026#34;database\u0026#34;, \u0026#34;port\u0026#34;) # parse as int pool = config.getint(\u0026#34;database\u0026#34;, \u0026#34;pool_size\u0026#34;) debug = config.getboolean(\u0026#34;server\u0026#34;, \u0026#34;debug\u0026#34;) # \u0026#39;true\u0026#39;/\u0026#39;false\u0026#39; → bool origins = config[\u0026#34;server\u0026#34;][\u0026#34;allowed_origins\u0026#34;].split(\u0026#34;,\u0026#34;) print(f\u0026#34;Database: {db_host}:{db_port}/{config[\u0026#39;database\u0026#39;][\u0026#39;name\u0026#39;]}\u0026#34;) print(f\u0026#34;Debug mode: {debug}\u0026#34;) # Safe access with fallback log_level = config.get(\u0026#34;logging\u0026#34;, \u0026#34;level\u0026#34;, fallback=\u0026#34;INFO\u0026#34;) # Writing a config file config2 = configparser.ConfigParser() config2[\u0026#34;database\u0026#34;] = { \u0026#34;host\u0026#34;: \u0026#34;prod-db.example.com\u0026#34;, \u0026#34;port\u0026#34;: \u0026#34;5432\u0026#34;, \u0026#34;name\u0026#34;: \u0026#34;production\u0026#34;, \u0026#34;pool_size\u0026#34;: \u0026#34;20\u0026#34;, } config2[\u0026#34;server\u0026#34;] = { \u0026#34;host\u0026#34;: \u0026#34;0.0.0.0\u0026#34;, \u0026#34;port\u0026#34;: \u0026#34;8000\u0026#34;, \u0026#34;debug\u0026#34;: \u0026#34;false\u0026#34;, } with open(\u0026#34;prod_config.ini\u0026#34;, \u0026#34;w\u0026#34;, encoding=\u0026#34;utf-8\u0026#34;) as f: config2.write(f) Part 8: pathlib - The Modern Way to Handle Paths Python 3.4 introduced pathlib, which replaces string-based path operations with an object-oriented API. It\u0026rsquo;s cleaner, more readable, and cross-platform by default. Prefer pathlib over os.path in all new code.\nPath Objects - The Basics from pathlib import Path # Create a Path object - works on Windows, Mac, and Linux p = Path(\u0026#34;data/reports/ipl2024.csv\u0026#34;) # Path components print(p.name) # \u0026#39;ipl2024.csv\u0026#39; ← filename with extension print(p.stem) # \u0026#39;ipl2024\u0026#39; ← filename without extension print(p.suffix) # \u0026#39;.csv\u0026#39; ← extension print(p.suffixes) # [\u0026#39;.csv\u0026#39;] ← all extensions print(p.parent) # data/reports ← parent directory print(p.parents[0]) # data/reports print(p.parents[1]) # data print(p.parts) # (\u0026#39;data\u0026#39;, \u0026#39;reports\u0026#39;, \u0026#39;ipl2024.csv\u0026#39;) # Build paths with / operator - works cross-platform base = Path(\u0026#34;data\u0026#34;) reports = base / \u0026#34;reports\u0026#34; file = reports / \u0026#34;ipl2024.csv\u0026#34; print(file) # data/reports/ipl2024.csv # Absolute path print(Path.cwd()) # current working directory print(file.resolve())# fully resolved absolute path Checking, Creating, and Deleting from pathlib import Path p = Path(\u0026#34;data/reports/ipl2024.csv\u0026#34;) # Checking print(p.exists()) # True/False - does it exist? print(p.is_file()) # True if it\u0026#39;s a regular file print(p.is_dir()) # True if it\u0026#39;s a directory print(p.is_absolute()) # True if path is absolute # Create directories Path(\u0026#34;data/reports\u0026#34;).mkdir(parents=True, exist_ok=True) # parents=True → creates intermediate directories (like mkdir -p) # exist_ok=True → no error if directory already exists # Create a file Path(\u0026#34;data/notes.txt\u0026#34;).touch() # create empty file, or update mtime if exists # Delete Path(\u0026#34;data/temp.txt\u0026#34;).unlink() # delete a file Path(\u0026#34;data/temp.txt\u0026#34;).unlink(missing_ok=True) # no error if doesn\u0026#39;t exist (3.8+) Path(\u0026#34;data/empty_dir\u0026#34;).rmdir() # delete empty directory # Rename / move p.rename(\u0026#34;data/reports/ipl2024_backup.csv\u0026#34;) p.replace(\u0026#34;data/archive/ipl2024.csv\u0026#34;) # replace() works even if destination exists Listing and Searching Files from pathlib import Path project = Path(\u0026#34;my_project\u0026#34;) # List immediate children for item in project.iterdir(): kind = \u0026#34;DIR \u0026#34; if item.is_dir() else \u0026#34;FILE\u0026#34; print(f\u0026#34;{kind} {item.name}\u0026#34;) # Find files matching a pattern - one level deep for csv_file in project.glob(\u0026#34;*.csv\u0026#34;): print(csv_file) # Recursive search - all levels for py_file in project.rglob(\u0026#34;*.py\u0026#34;): print(py_file) # Find all Python files, skip hidden directories and __pycache__ py_files = [ f for f in project.rglob(\u0026#34;*.py\u0026#34;) if \u0026#34;__pycache__\u0026#34; not in f.parts and not any(part.startswith(\u0026#34;.\u0026#34;) for part in f.parts) ] # Get file metadata p = Path(\u0026#34;data.csv\u0026#34;) stat = p.stat() print(f\u0026#34;Size: {stat.st_size:,} bytes\u0026#34;) print(f\u0026#34;Modified: {datetime.datetime.fromtimestamp(stat.st_mtime)}\u0026#34;) print(f\u0026#34;Created: {datetime.datetime.fromtimestamp(stat.st_ctime)}\u0026#34;) Reading and Writing with pathlib Path objects have their own read/write methods for convenience:\nfrom pathlib import Path p = Path(\u0026#34;notes.txt\u0026#34;) # Write and read text p.write_text(\u0026#34;Hello, Python!\\nSecond line.\\n\u0026#34;, encoding=\u0026#34;utf-8\u0026#34;) content = p.read_text(encoding=\u0026#34;utf-8\u0026#34;) print(content) # Write and read bytes p_bin = Path(\u0026#34;data.bin\u0026#34;) p_bin.write_bytes(b\u0026#34;\\x00\\x01\\x02\\x03\u0026#34;) data = p_bin.read_bytes() print(data) # b\u0026#39;\\x00\\x01\\x02\\x03\u0026#39; # For more control (large files, line-by-line), use open() as before with p.open(\u0026#34;r\u0026#34;, encoding=\u0026#34;utf-8\u0026#34;) as f: for line in f: print(line.rstrip()) os.path vs pathlib Side-by-Side import os from pathlib import Path filepath = \u0026#34;data/reports/ipl2024.csv\u0026#34; # --- String-based (old way) --- dirname = os.path.dirname(filepath) # \u0026#39;data/reports\u0026#39; basename = os.path.basename(filepath) # \u0026#39;ipl2024.csv\u0026#39; stem, ext = os.path.splitext(basename) # (\u0026#39;ipl2024\u0026#39;, \u0026#39;.csv\u0026#39;) joined = os.path.join(\u0026#34;data\u0026#34;, \u0026#34;reports\u0026#34;, \u0026#34;f\u0026#34;) # \u0026#39;data/reports/f\u0026#39; exists = os.path.exists(filepath) abspath = os.path.abspath(filepath) # --- pathlib (new way) --- p = Path(filepath) dirname = p.parent # Path(\u0026#39;data/reports\u0026#39;) basename = p.name # \u0026#39;ipl2024.csv\u0026#39; stem = p.stem # \u0026#39;ipl2024\u0026#39; ext = p.suffix # \u0026#39;.csv\u0026#39; joined = Path(\u0026#34;data\u0026#34;) / \u0026#34;reports\u0026#34; / \u0026#34;f\u0026#34; # Path(\u0026#39;data/reports/f\u0026#39;) exists = p.exists() abspath = p.resolve() Part 9: Working with os and shutil - File System Operations For operations beyond reading and writing - copying, moving, deleting, directory management - use os and shutil.\nimport os import shutil from pathlib import Path # --- File operations --- os.rename(\u0026#34;old.txt\u0026#34;, \u0026#34;new.txt\u0026#34;) # rename (same filesystem) os.replace(\u0026#34;src.txt\u0026#34;, \u0026#34;dst.txt\u0026#34;) # rename, overwriting dst if it exists os.remove(\u0026#34;unwanted.txt\u0026#34;) # delete a file # shutil for copying shutil.copy(\u0026#34;src.txt\u0026#34;, \u0026#34;dst.txt\u0026#34;) # copy file (no metadata) shutil.copy2(\u0026#34;src.txt\u0026#34;, \u0026#34;dst.txt\u0026#34;) # copy file + metadata (timestamps) shutil.copyfileobj(src_file, dst_file) # copy between open file objects # --- Directory operations --- os.mkdir(\u0026#34;new_dir\u0026#34;) # create one directory os.makedirs(\u0026#34;a/b/c\u0026#34;, exist_ok=True) # create full path os.rmdir(\u0026#34;empty_dir\u0026#34;) # delete empty directory shutil.rmtree(\u0026#34;full_dir\u0026#34;) # delete directory and all contents (careful!) # --- Move --- shutil.move(\u0026#34;src_path\u0026#34;, \u0026#34;dst_path\u0026#34;) # move file or directory # --- Disk usage --- total, used, free = shutil.disk_usage(\u0026#34;/\u0026#34;) print(f\u0026#34;Total: {total // (2**30):.1f} GB\u0026#34;) print(f\u0026#34;Used: {used // (2**30):.1f} GB\u0026#34;) print(f\u0026#34;Free: {free // (2**30):.1f} GB\u0026#34;) # --- Environment and current directory --- print(os.getcwd()) # current working directory os.chdir(\u0026#34;/tmp\u0026#34;) # change directory print(os.environ.get(\u0026#34;HOME\u0026#34;, \u0026#34;unknown\u0026#34;)) # environment variables Temporary Files and Directories import tempfile import os # Temporary file - deleted when closed with tempfile.NamedTemporaryFile(mode=\u0026#34;w\u0026#34;, suffix=\u0026#34;.csv\u0026#34;, encoding=\u0026#34;utf-8\u0026#34;, delete=True) as tmp: tmp.write(\u0026#34;name,score\\nRavi,95\\n\u0026#34;) print(f\u0026#34;Temp file: {tmp.name}\u0026#34;) # File is accessible here by name # Temporary directory - deleted when the context exits with tempfile.TemporaryDirectory() as tmpdir: tmpdir_path = Path(tmpdir) output_file = tmpdir_path / \u0026#34;results.json\u0026#34; output_file.write_text(\u0026#39;{\u0026#34;status\u0026#34;: \u0026#34;ok\u0026#34;}\u0026#39;, encoding=\u0026#34;utf-8\u0026#34;) # Process files... # tmpdir and all its contents are deleted here # Persistent temp file (you manage deletion) fd, path = tempfile.mkstemp(suffix=\u0026#34;.txt\u0026#34;) os.close(fd) # close the low-level file descriptor with open(path, \u0026#34;w\u0026#34;, encoding=\u0026#34;utf-8\u0026#34;) as f: f.write(\u0026#34;temporary content\u0026#34;) # ... use the file ... os.unlink(path) # you must delete it manually Part 10: Common Mistakes to Avoid Mistake 1 - Forgetting to Close Files (Not Using with) # BAD - if an exception occurs, the file is never closed f = open(\u0026#34;data.txt\u0026#34;, \u0026#34;r\u0026#34;) content = f.read() process(content) # raises an exception! f.close() # never reached → file descriptor leaked # GOOD - with guarantees closure with open(\u0026#34;data.txt\u0026#34;, \u0026#34;r\u0026#34;, encoding=\u0026#34;utf-8\u0026#34;) as f: content = f.read() process(content) Mistake 2 - Opening in Write Mode Accidentally Destroys Data # CATASTROPHIC - \u0026#39;w\u0026#39; truncates the file immediately on open with open(\u0026#34;important_data.csv\u0026#34;, \u0026#34;w\u0026#34;) as f: # The file is NOW EMPTY, even before you write anything! f.write(new_data) # SAFE - if the file might exist, use \u0026#39;a\u0026#39; or \u0026#39;x\u0026#39; depending on intent with open(\u0026#34;log.csv\u0026#34;, \u0026#34;a\u0026#34;) as f: # append, never destroys f.write(new_row) with open(\u0026#34;new_file.csv\u0026#34;, \u0026#34;x\u0026#34;) as f: # fails if file already exists f.write(data) Mistake 3 - Splitting CSV Lines Manually # BAD - breaks on values containing commas line = \u0026#39;Ravi,\u0026#34;123, MG Road, Bangalore\u0026#34;,Hyderabad\u0026#39; parts = line.split(\u0026#34;,\u0026#34;) # WRONG: gives 4 parts instead of 3 # GOOD - use the csv module which handles quoting correctly import csv import io reader = csv.reader(io.StringIO(line)) parts = next(reader) # [\u0026#39;Ravi\u0026#39;, \u0026#39;123, MG Road, Bangalore\u0026#39;, \u0026#39;Hyderabad\u0026#39;] Mistake 4 - Not Specifying Encoding # BAD - works on your machine, breaks elsewhere with open(\u0026#34;hindi_content.txt\u0026#34;, \u0026#34;r\u0026#34;) as f: content = f.read() # might raise UnicodeDecodeError on different systems # GOOD - always explicit with open(\u0026#34;hindi_content.txt\u0026#34;, \u0026#34;r\u0026#34;, encoding=\u0026#34;utf-8\u0026#34;) as f: content = f.read() Mistake 5 - Loading Huge Files Entirely Into Memory # BAD - loads a 5 GB log file into a single list with open(\u0026#34;huge.log\u0026#34;, \u0026#34;r\u0026#34;, encoding=\u0026#34;utf-8\u0026#34;) as f: lines = f.readlines() # 5 GB in RAM! # GOOD - process one line at a time with open(\u0026#34;huge.log\u0026#34;, \u0026#34;r\u0026#34;, encoding=\u0026#34;utf-8\u0026#34;) as f: for line in f: process(line) # constant memory, any file size Mistake 6 - Reading a File Object Twice Without Seeking # BUG - second read returns empty string with open(\u0026#34;data.txt\u0026#34;, \u0026#34;r\u0026#34;, encoding=\u0026#34;utf-8\u0026#34;) as f: first_read = f.read() # reads all content, pointer at end second_read = f.read() # returns \u0026#39;\u0026#39; - pointer is at end! print(bool(second_read)) # False # FIX - seek back to the start with open(\u0026#34;data.txt\u0026#34;, \u0026#34;r\u0026#34;, encoding=\u0026#34;utf-8\u0026#34;) as f: first_read = f.read() f.seek(0) # reset pointer to beginning second_read = f.read() # reads full content again print(first_read == second_read) # True Mistake 7 - String Concatenation for Path Building import os from pathlib import Path # BAD - breaks on Windows (\u0026#39;\\\u0026#39; vs \u0026#39;/\u0026#39;) path = base_dir + \u0026#34;/\u0026#34; + subdir + \u0026#34;/\u0026#34; + filename # ALSO BAD - verbose and error-prone path = os.path.join(base_dir, subdir, filename) # GOOD - use pathlib path = Path(base_dir) / subdir / filename Mistake 8 - Forgetting newline=\u0026quot;\u0026quot; for CSV Files import csv # BAD - on Windows, Python\u0026#39;s universal newline translation # + csv\u0026#39;s own newline handling can create blank rows in output with open(\u0026#34;data.csv\u0026#34;, \u0026#34;w\u0026#34;) as f: writer = csv.writer(f) writer.writerow([\u0026#34;a\u0026#34;, \u0026#34;b\u0026#34;]) # GOOD - always use newline=\u0026#34;\u0026#34; with csv module with open(\u0026#34;data.csv\u0026#34;, \u0026#34;w\u0026#34;, encoding=\u0026#34;utf-8\u0026#34;, newline=\u0026#34;\u0026#34;) as f: writer = csv.writer(f) writer.writerow([\u0026#34;a\u0026#34;, \u0026#34;b\u0026#34;]) Part 11: Real-World Patterns Pattern 1 - Config Loader with Multiple Formats import json import configparser from pathlib import Path def load_config(config_path): \u0026#34;\u0026#34;\u0026#34; Load configuration from JSON or INI file. Returns a plain dict in either case. \u0026#34;\u0026#34;\u0026#34; p = Path(config_path) if not p.exists(): raise FileNotFoundError(f\u0026#34;Config file not found: {p}\u0026#34;) if p.suffix == \u0026#34;.json\u0026#34;: with p.open(\u0026#34;r\u0026#34;, encoding=\u0026#34;utf-8\u0026#34;) as f: return json.load(f) elif p.suffix in (\u0026#34;.ini\u0026#34;, \u0026#34;.cfg\u0026#34;): config = configparser.ConfigParser() config.read(p, encoding=\u0026#34;utf-8\u0026#34;) return { section: dict(config[section]) for section in config.sections() } else: raise ValueError(f\u0026#34;Unsupported config format: {p.suffix}\u0026#34;) # Usage config = load_config(\u0026#34;app.json\u0026#34;) db_host = config[\u0026#34;database\u0026#34;][\u0026#34;host\u0026#34;] Pattern 2 - Streaming File Processor Process large files efficiently by reading in chunks, never loading the whole file.\nfrom pathlib import Path from itertools import islice def read_in_batches(filepath, batch_size=1000): \u0026#34;\u0026#34;\u0026#34;Yield batches of `batch_size` lines from a file.\u0026#34;\u0026#34;\u0026#34; with open(filepath, \u0026#34;r\u0026#34;, encoding=\u0026#34;utf-8\u0026#34;) as f: while True: batch = list(islice(f, batch_size)) if not batch: break yield [line.rstrip(\u0026#34;\\n\u0026#34;) for line in batch] def process_large_log(filepath): \u0026#34;\u0026#34;\u0026#34;Count error occurrences by error code in a large log file.\u0026#34;\u0026#34;\u0026#34; from collections import Counter error_counts = Counter() for batch in read_in_batches(filepath, batch_size=500): for line in batch: if \u0026#34;ERROR\u0026#34; in line: # Parse error code from line like: \u0026#34;2024-01-15 [ERROR] E1234 message\u0026#34; parts = line.split() if len(parts) \u0026gt;= 3: error_counts[parts[2]] += 1 return error_counts.most_common(10) Pattern 3 - Directory Organizer import shutil from pathlib import Path def organize_downloads(source_dir, target_dir): \u0026#34;\u0026#34;\u0026#34; Move files from source_dir to categorized subdirectories in target_dir based on file extension. \u0026#34;\u0026#34;\u0026#34; CATEGORIES = { \u0026#34;Images\u0026#34;: {\u0026#34;.jpg\u0026#34;, \u0026#34;.jpeg\u0026#34;, \u0026#34;.png\u0026#34;, \u0026#34;.gif\u0026#34;, \u0026#34;.webp\u0026#34;, \u0026#34;.svg\u0026#34;}, \u0026#34;Documents\u0026#34;: {\u0026#34;.pdf\u0026#34;, \u0026#34;.docx\u0026#34;, \u0026#34;.doc\u0026#34;, \u0026#34;.txt\u0026#34;, \u0026#34;.md\u0026#34;, \u0026#34;.xlsx\u0026#34;}, \u0026#34;Data\u0026#34;: {\u0026#34;.csv\u0026#34;, \u0026#34;.json\u0026#34;, \u0026#34;.xml\u0026#34;, \u0026#34;.parquet\u0026#34;, \u0026#34;.sql\u0026#34;}, \u0026#34;Code\u0026#34;: {\u0026#34;.py\u0026#34;, \u0026#34;.js\u0026#34;, \u0026#34;.ts\u0026#34;, \u0026#34;.html\u0026#34;, \u0026#34;.css\u0026#34;, \u0026#34;.sh\u0026#34;}, \u0026#34;Archives\u0026#34;: {\u0026#34;.zip\u0026#34;, \u0026#34;.tar\u0026#34;, \u0026#34;.gz\u0026#34;, \u0026#34;.rar\u0026#34;, \u0026#34;.7z\u0026#34;}, } source = Path(source_dir) target = Path(target_dir) moved = 0 for file in source.iterdir(): if not file.is_file(): continue category = \u0026#34;Other\u0026#34; for cat_name, extensions in CATEGORIES.items(): if file.suffix.lower() in extensions: category = cat_name break dest_dir = target / category dest_dir.mkdir(parents=True, exist_ok=True) shutil.move(str(file), dest_dir / file.name) moved += 1 print(f\u0026#34; {file.name} → {category}/\u0026#34;) print(f\u0026#34;\\nOrganized {moved} files.\u0026#34;) organize_downloads(\u0026#34;~/Downloads\u0026#34;, \u0026#34;~/Downloads/Organized\u0026#34;) Pattern 4 - File Watcher import time import os from pathlib import Path def watch_file(filepath, poll_interval=1.0): \u0026#34;\u0026#34;\u0026#34; Generator that yields the file content whenever it changes. Useful for watching log files or config reloads. \u0026#34;\u0026#34;\u0026#34; p = Path(filepath) last_mtime = None while True: try: current_mtime = p.stat().st_mtime if current_mtime != last_mtime: last_mtime = current_mtime yield p.read_text(encoding=\u0026#34;utf-8\u0026#34;) except FileNotFoundError: pass # file may not exist yet time.sleep(poll_interval) # Usage (in a separate thread or async context) # for content in watch_file(\u0026#34;app.log\u0026#34;): # process_new_content(content) Practice Problems Problem 1 - Word Frequency Counter Task: Read a text file, count the frequency of each word (case-insensitive, ignoring punctuation), and write the top 10 most common words to a new file, one word per line with its count.\nSolution:\nimport re from collections import Counter from pathlib import Path def word_frequency(input_path, output_path, top_n=10): # Read and clean text = Path(input_path).read_text(encoding=\u0026#34;utf-8\u0026#34;).lower() words = re.findall(r\u0026#34;\\b[a-z]+\\b\u0026#34;, text) # only alphabetic words counts = Counter(words) # Write top N with open(output_path, \u0026#34;w\u0026#34;, encoding=\u0026#34;utf-8\u0026#34;) as f: f.write(f\u0026#34;Top {top_n} words\\n\u0026#34;) f.write(\u0026#34;-\u0026#34; * 25 + \u0026#34;\\n\u0026#34;) for word, count in counts.most_common(top_n): f.write(f\u0026#34;{word:\u0026lt;20} {count:\u0026gt;5}\\n\u0026#34;) word_frequency(\u0026#34;novel.txt\u0026#34;, \u0026#34;word_freq.txt\u0026#34;, top_n=10) Problem 2 - CSV Filter and Transform Task: Read a CSV of cricket players. Keep only players with average \u0026gt; 40. Add a new column grade (A for avg \u0026gt; 50, B for avg \u0026gt; 40). Write the result to a new CSV, sorted by average descending.\nSolution:\nimport csv def filter_and_grade(input_csv, output_csv, min_avg=40.0): results = [] with open(input_csv, \u0026#34;r\u0026#34;, encoding=\u0026#34;utf-8\u0026#34;, newline=\u0026#34;\u0026#34;) as f: reader = csv.DictReader(f) for row in reader: avg = float(row[\u0026#34;average\u0026#34;]) if avg \u0026gt; min_avg: row[\u0026#34;grade\u0026#34;] = \u0026#34;A\u0026#34; if avg \u0026gt; 50 else \u0026#34;B\u0026#34; results.append(row) results.sort(key=lambda r: float(r[\u0026#34;average\u0026#34;]), reverse=True) if not results: print(\u0026#34;No qualifying players.\u0026#34;) return fieldnames = list(results[0].keys()) with open(output_csv, \u0026#34;w\u0026#34;, encoding=\u0026#34;utf-8\u0026#34;, newline=\u0026#34;\u0026#34;) as f: writer = csv.DictWriter(f, fieldnames=fieldnames) writer.writeheader() writer.writerows(results) print(f\u0026#34;Wrote {len(results)} players to {output_csv}\u0026#34;) filter_and_grade(\u0026#34;players.csv\u0026#34;, \u0026#34;top_players.csv\u0026#34;) Problem 3 - JSON Config Merge Task: Given a defaults.json and an overrides.json, merge them so that overrides take precedence. Write the merged config to final.json. Handle missing files gracefully.\nSolution:\nimport json from pathlib import Path def merge_configs(defaults_path, overrides_path, output_path): def load_json_safe(path): p = Path(path) if not p.exists(): print(f\u0026#34;File not found: {path}, using empty dict\u0026#34;) return {} with p.open(\u0026#34;r\u0026#34;, encoding=\u0026#34;utf-8\u0026#34;) as f: return json.load(f) defaults = load_json_safe(defaults_path) overrides = load_json_safe(overrides_path) # Deep merge: overrides take precedence, nested dicts are merged recursively def deep_merge(base, override): result = base.copy() for key, value in override.items(): if key in result and isinstance(result[key], dict) and isinstance(value, dict): result[key] = deep_merge(result[key], value) else: result[key] = value return result merged = deep_merge(defaults, overrides) with open(output_path, \u0026#34;w\u0026#34;, encoding=\u0026#34;utf-8\u0026#34;) as f: json.dump(merged, f, indent=2, ensure_ascii=False) print(f\u0026#34;Merged config written to {output_path}\u0026#34;) merge_configs(\u0026#34;defaults.json\u0026#34;, \u0026#34;overrides.json\u0026#34;, \u0026#34;final.json\u0026#34;) Problem 4 - Log File Analyzer Task: Read a log file where each line has the format YYYY-MM-DD HH:MM:SS [LEVEL] message. Count entries per level (INFO, WARNING, ERROR, etc.). Print a summary and write all ERROR lines to a separate file.\nSolution:\nimport re from collections import Counter from pathlib import Path LOG_PATTERN = re.compile( r\u0026#34;(\\d{4}-\\d{2}-\\d{2} \\d{2}:\\d{2}:\\d{2}) \\[(\\w+)\\] (.+)\u0026#34; ) def analyze_logs(log_path, error_output_path): level_counts = Counter() error_lines = [] with open(log_path, \u0026#34;r\u0026#34;, encoding=\u0026#34;utf-8\u0026#34;) as f: for line_num, line in enumerate(f, start=1): match = LOG_PATTERN.match(line.strip()) if match: timestamp, level, message = match.groups() level_counts[level] += 1 if level == \u0026#34;ERROR\u0026#34;: error_lines.append(line.strip()) else: print(f\u0026#34; Unparseable line {line_num}: {line[:60]!r}\u0026#34;) # Summary print(\u0026#34;\\nLog Summary\u0026#34;) print(\u0026#34;=\u0026#34; * 30) total = sum(level_counts.values()) for level, count in sorted(level_counts.items()): pct = count / total * 100 print(f\u0026#34; {level:\u0026lt;10} {count:\u0026gt;5} ({pct:5.1f}%)\u0026#34;) print(f\u0026#34; {\u0026#39;TOTAL\u0026#39;:\u0026lt;10} {total:\u0026gt;5}\u0026#34;) # Write errors if error_lines: with open(error_output_path, \u0026#34;w\u0026#34;, encoding=\u0026#34;utf-8\u0026#34;) as f: f.write(f\u0026#34;# {len(error_lines)} ERROR entries\\n\\n\u0026#34;) f.write(\u0026#34;\\n\u0026#34;.join(error_lines) + \u0026#34;\\n\u0026#34;) print(f\u0026#34;\\nWrote {len(error_lines)} errors to {error_output_path}\u0026#34;) analyze_logs(\u0026#34;app.log\u0026#34;, \u0026#34;errors_only.log\u0026#34;) Problem 5 - Directory Diff Task: Given two directories, find files that exist in the first but not the second (missing), files that exist in both (common), and files that exist only in the second (new). Use pathlib.\nSolution:\nfrom pathlib import Path def directory_diff(dir_a, dir_b): \u0026#34;\u0026#34;\u0026#34; Compare two directories by filename. Returns (only_in_a, common, only_in_b) as sets of filenames. \u0026#34;\u0026#34;\u0026#34; a = Path(dir_a) b = Path(dir_b) files_a = {f.name for f in a.iterdir() if f.is_file()} files_b = {f.name for f in b.iterdir() if f.is_file()} only_in_a = files_a - files_b common = files_a \u0026amp; files_b only_in_b = files_b - files_a print(f\u0026#34;Only in {a.name}/ ({len(only_in_a)}): {sorted(only_in_a)}\u0026#34;) print(f\u0026#34;Common ({len(common)}): {sorted(common)}\u0026#34;) print(f\u0026#34;Only in {b.name}/ ({len(only_in_b)}): {sorted(only_in_b)}\u0026#34;) return only_in_a, common, only_in_b directory_diff(\u0026#34;project_v1\u0026#34;, \u0026#34;project_v2\u0026#34;) Problem 6 - Bulk File Renamer Task: Write a function that renames all .txt files in a directory by adding a timestamp prefix to their names. Use pathlib. Dry-run mode should print what would happen without actually renaming.\nSolution:\nimport datetime from pathlib import Path def bulk_rename(directory, pattern=\u0026#34;*.txt\u0026#34;, dry_run=True): \u0026#34;\u0026#34;\u0026#34; Rename all files matching `pattern` in `directory`, prepending today\u0026#39;s date as YYYYMMDD_. \u0026#34;\u0026#34;\u0026#34; base = Path(directory) today = datetime.date.today().strftime(\u0026#34;%Y%m%d\u0026#34;) renamed = 0 for filepath in sorted(base.glob(pattern)): new_name = f\u0026#34;{today}_{filepath.name}\u0026#34; new_path = filepath.parent / new_name if new_path.exists(): print(f\u0026#34; SKIP (target exists): {filepath.name} → {new_name}\u0026#34;) continue if dry_run: print(f\u0026#34; DRY RUN: {filepath.name} → {new_name}\u0026#34;) else: filepath.rename(new_path) print(f\u0026#34; RENAMED: {filepath.name} → {new_name}\u0026#34;) renamed += 1 if dry_run: print(\u0026#34;\\nDry run complete. Pass dry_run=False to apply changes.\u0026#34;) else: print(f\u0026#34;\\nRenamed {renamed} file(s).\u0026#34;) # Preview first bulk_rename(\u0026#34;my_notes\u0026#34;, pattern=\u0026#34;*.txt\u0026#34;, dry_run=True) # Then apply # bulk_rename(\u0026#34;my_notes\u0026#34;, pattern=\u0026#34;*.txt\u0026#34;, dry_run=False) Quick Reference Cheat Sheet ┌──────────────────────────────────────────────────────────────────────────┐ │ FILE HANDLING QUICK REFERENCE │ ├──────────────────────────┬───────────────────────────────────────────────┤ │ OPEN MODES │ │ ├──────────────────────────┼───────────────────────────────────────────────┤ │ \u0026#39;r\u0026#39; │ Read text. File must exist. │ │ \u0026#39;w\u0026#39; │ Write text. Creates or TRUNCATES. │ │ \u0026#39;a\u0026#39; │ Append text. Creates or adds to end. │ │ \u0026#39;x\u0026#39; │ Create. Fails if already exists. │ │ \u0026#39;rb\u0026#39; / \u0026#39;wb\u0026#39; │ Binary read / write. │ ├──────────────────────────┼───────────────────────────────────────────────┤ │ READING METHODS │ │ ├──────────────────────────┼───────────────────────────────────────────────┤ │ f.read() │ Entire file as one string. │ │ f.read(n) │ Next n characters. │ │ f.readline() │ One line (with \\n). │ │ f.readlines() │ All lines as a list. │ │ for line in f: │ One line at a time. BEST for large files. │ ├──────────────────────────┼───────────────────────────────────────────────┤ │ WRITING METHODS │ │ ├──────────────────────────┼───────────────────────────────────────────────┤ │ f.write(s) │ Write string s. Returns char count. │ │ f.writelines(lst) │ Write each item. No newlines added. │ │ print(..., file=f) │ Formatted write to file. │ ├──────────────────────────┼───────────────────────────────────────────────┤ │ POSITION │ │ ├──────────────────────────┼───────────────────────────────────────────────┤ │ f.tell() │ Current byte position. │ │ f.seek(0) │ Go to start of file. │ │ f.seek(0, 2) │ Go to end of file. │ ├──────────────────────────┼───────────────────────────────────────────────┤ │ CSV MODULE │ │ ├──────────────────────────┼───────────────────────────────────────────────┤ │ csv.reader(f) │ Rows as lists. │ │ csv.DictReader(f) │ Rows as dicts. │ │ csv.writer(f) │ Write rows as lists. │ │ csv.DictWriter(f, ...) │ Write rows as dicts. │ │ Always newline=\u0026#34;\u0026#34; │ When opening files for csv module. │ │ Always encoding utf-8-sig│ For Excel-exported CSVs. │ ├──────────────────────────┼───────────────────────────────────────────────┤ │ JSON MODULE │ │ ├──────────────────────────┼───────────────────────────────────────────────┤ │ json.load(f) │ File → Python object. │ │ json.dump(obj, f) │ Python object → file. │ │ json.loads(s) │ String → Python object. │ │ json.dumps(obj) │ Python object → string. │ ├──────────────────────────┼───────────────────────────────────────────────┤ │ PATHLIB │ │ ├──────────────────────────┼───────────────────────────────────────────────┤ │ Path(str) │ Create a path object. │ │ p / \u0026#34;subdir\u0026#34; / \u0026#34;file\u0026#34; │ Join paths with /. │ │ p.exists() │ Check if path exists. │ │ p.is_file() / is_dir() │ Check type. │ │ p.mkdir(parents=True, │ Create directory tree. │ │ exist_ok=True) │ │ │ p.glob(\u0026#34;*.csv\u0026#34;) │ Match files in one directory. │ │ p.rglob(\u0026#34;*.csv\u0026#34;) │ Match files recursively. │ │ p.read_text(encoding=..) │ Read entire file to string. │ │ p.write_text(s, enc=..) │ Write string to file. │ │ p.rename(new) │ Rename/move path. │ │ p.unlink() │ Delete file. │ │ p.stat().st_size │ File size in bytes. │ ├──────────────────────────┼───────────────────────────────────────────────┤ │ GOLDEN RULES │ │ ├──────────────────────────┼───────────────────────────────────────────────┤ │ ALWAYS │ Use `with` statement to open files. │ │ ALWAYS │ Specify encoding=\u0026#34;utf-8\u0026#34; explicitly. │ │ ALWAYS │ Use newline=\u0026#34;\u0026#34; with csv module. │ │ ALWAYS │ Iterate over file object for large files. │ │ NEVER │ Use \u0026#39;w\u0026#39; without confirming overwrite is ok. │ │ NEVER │ Split CSV lines with .split(\u0026#39;,\u0026#39;). │ │ PREFER │ pathlib.Path over os.path string methods. │ └──────────────────────────┴───────────────────────────────────────────────┘ End of Guide - Working with Files in Python Codeverra - codeverra.com\n","permalink":"https://blogs.codeverra.com/blog/python-09-file-handling/","summary":"\u003ch1 id=\"working-with-files-in-python-----the-complete-guide\"\u003eWorking with Files in Python  -  The Complete Guide\u003c/h1\u003e\n\u003ch3 id=\"everything-you-need-to-read-write-parse-and-manage-files-like-a-professional\"\u003eEverything you need to read, write, parse, and manage files like a professional\u003c/h3\u003e\n\u003chr\u003e\n\u003ch2 id=\"why-files-matter-more-than-you-think\"\u003eWhy Files Matter More Than You Think\u003c/h2\u003e\n\u003cp\u003eEvery meaningful program eventually needs to talk to the outside world. Not through a web API or a database  -  just a plain file sitting on disk. Your data science pipeline reads a CSV. Your web scraper writes results to JSON. Your deployment script reads a config file. Your ETL job processes gigabytes of logs.\u003c/p\u003e","tags":["python"],"title":"File Handling in Python - Complete Guide"},{"content":"Lesson 2 - Filtering \u0026amp; Querying Data Theory + Practice | ShopDB - Indian E-commerce Dataset Before You Start Run setup.sql in pgAdmin to create and populate ShopDB. Verify you see 5 tables with the correct row counts. Then work through this file top to bottom. What you\u0026rsquo;ll learn in this lesson:\nConcept What it does DISTINCT Remove duplicate values from results IN / NOT IN Match against a list of values BETWEEN Filter within a range LIKE / ILIKE Pattern matching on text IS NULL / IS NOT NULL Handle missing values CASE WHEN Conditional logic inside a query Date functions Filter and extract parts of dates COALESCE Replace NULL with a fallback value All queries in this lesson run on ShopDB. The dataset has:\n20 customers from 12+ Indian cities 6 product categories and 25 products 15 orders with 30 order line items Part 1 - Quick Refresh: The ShopDB Schema customers orders ┌────┬───────┬───────┬───────┐ ┌────┬─────────────┬────────────┬──────────┐ │ id │ name │ city │ state │ │ id │ customer_id │ order_date │ status │ └────┴───────┴───────┴───────┘ └────┴─────────────┴────────────┴──────────┘ │ │ │ └───────────── FK ──────────────────┘ │ │ order_items products │ categories ┌────┬──────────┬────────────┬──────┐ ┌────┬──┴────┬───────────┐ ┌────┬──────────┐ │ id │ order_id │ product_id │ qty │ │ id │ name │ price_inr │ │ id │ name │ └────┴──────────┴────────────┴──────┘ └────┴───────┴───────────┘ └────┴──────────┘ │ │ │ FK └── FK ─────────┘ └──────────────────────────────┘ The flow of a customer purchase: customer places an order → order contains order_items → each item references a product → product belongs to a category\nPart 2 - DISTINCT: Remove Duplicates DISTINCT removes duplicate rows from results.\n-- Which states do our customers come from? -- Without DISTINCT, a state like Maharashtra appears once per customer SELECT state FROM customers ORDER BY state; -- With DISTINCT - each state appears only once SELECT DISTINCT state FROM customers ORDER BY state; -- Which payment methods have been used? (across all orders) SELECT DISTINCT payment_method FROM orders ORDER BY payment_method; -- How many distinct states are our customers from? -- Combining COUNT with DISTINCT SELECT COUNT(DISTINCT state) AS unique_states FROM customers; DISTINCT applies to the entire row when multiple columns are listed. SELECT DISTINCT city, state returns distinct combinations of city + state.\n-- Distinct city-state combinations SELECT DISTINCT city, state FROM customers ORDER BY state, city; Part 3 - IN and NOT IN: Match a List IN is a cleaner way to write multiple OR conditions.\n-- Customers from Maharashtra OR Gujarat OR Delhi -- The long way: SELECT name, city, state FROM customers WHERE state = \u0026#39;Maharashtra\u0026#39; OR state = \u0026#39;Gujarat\u0026#39; OR state = \u0026#39;Delhi\u0026#39;; -- The clean way: SELECT name, city, state FROM customers WHERE state IN (\u0026#39;Maharashtra\u0026#39;, \u0026#39;Gujarat\u0026#39;, \u0026#39;Delhi\u0026#39;) ORDER BY state, name; -- Orders that are NOT delivered and NOT cancelled SELECT id, customer_id, order_date, status FROM orders WHERE status NOT IN (\u0026#39;Delivered\u0026#39;, \u0026#39;Cancelled\u0026#39;) ORDER BY order_date; -- Products in the Electronics or Sports category -- (category_id 1 = Electronics, 5 = Sports) SELECT name, price_inr, stock_quantity FROM products WHERE category_id IN (1, 5) ORDER BY category_id, price_inr DESC; ⚠️ NULL trap with NOT IN: If the list contains even one NULL, NOT IN returns no rows. Always make sure the values you compare against cannot be NULL. WHERE column NOT IN (1, 2, NULL) → always empty!\nPart 4 - BETWEEN: Filter a Range BETWEEN low AND high is inclusive - both endpoints are included.\n-- Products priced between ₹500 and ₹3000 SELECT name, price_inr FROM products WHERE price_inr BETWEEN 500 AND 3000 ORDER BY price_inr; -- Orders placed in the first quarter of 2024 (Jan–Mar) SELECT id, customer_id, order_date, status FROM orders WHERE order_date BETWEEN \u0026#39;2024-01-01\u0026#39; AND \u0026#39;2024-03-31\u0026#39; ORDER BY order_date; -- Products NOT in the ₹500–₹3000 range (budget or premium) SELECT name, price_inr FROM products WHERE price_inr NOT BETWEEN 500 AND 3000 ORDER BY price_inr; BETWEEN with dates includes the full day for both endpoints. BETWEEN '2024-01-01' AND '2024-03-31' includes all of March 31st.\n✏️ Practice Set 1 - DISTINCT, IN, BETWEEN Q1. List all distinct roles (job roles) in the players table from CricketDB. (Switch to cricketdb for this one: \\c cricketdb or reconnect in pgAdmin)\nActually - stay in ShopDB for all questions in this lesson.\nQ1. List all distinct cities our customers come from, alphabetically.\nQ2. Show all orders placed by customers with id IN (1, 3, 5, 7). Show order_id, customer_id, order_date, and status.\nQ3. Find all products priced between ₹1000 and ₹10000. Show product name and price, ordered by price ascending.\nQ4. Which customers joined the platform between 1st April 2023 and 30th September 2023? Show name, city, and joined_on date.\nQ5. Find all orders with status NOT IN (\u0026lsquo;Delivered\u0026rsquo;, \u0026lsquo;Cancelled\u0026rsquo;). How many are there? (Use COUNT)\nPart 5 - LIKE and ILIKE: Pattern Matching LIKE matches text patterns using wildcards:\n% matches any sequence of characters (including none) _ matches exactly one character -- Products whose name starts with \u0026#34;boAt\u0026#34; SELECT name, price_inr FROM products WHERE name LIKE \u0026#39;boAt%\u0026#39;; -- Products with \u0026#34;Cricket\u0026#34; anywhere in the name SELECT name, price_inr FROM products WHERE name LIKE \u0026#39;%Cricket%\u0026#39;; -- Customers whose name ends in \u0026#34;Sharma\u0026#34; SELECT name, email, city FROM customers WHERE name LIKE \u0026#39;%Sharma\u0026#39;; -- Customers whose email is from gmail.com SELECT name, email FROM customers WHERE email LIKE \u0026#39;%@gmail.com\u0026#39;; -- Products with exactly 5 characters before a space (pattern: _____ %) -- e.g. \u0026#34;Nivia Football\u0026#34; - 5 chars then space SELECT name FROM products WHERE name LIKE \u0026#39;_____ %\u0026#39;; ILIKE - Case-Insensitive LIKE (PostgreSQL only) -- Find products with \u0026#34;samsung\u0026#34; in the name - regardless of capitalisation SELECT name, price_inr FROM products WHERE name ILIKE \u0026#39;%samsung%\u0026#39;; LIKE is case-sensitive: 'Samsung' ≠ 'samsung' ILIKE is case-insensitive (PostgreSQL-specific) Standard SQL alternative: LOWER(name) LIKE '%samsung%'\nPart 6 - IS NULL and IS NOT NULL Some columns allow NULL - batting_style in CricketDB, or optional fields. In ShopDB, categories.description can be NULL.\n-- Categories that have no description filled in SELECT name FROM categories WHERE description IS NULL; -- Categories that DO have a description SELECT name, description FROM categories WHERE description IS NOT NULL; -- COALESCE: show a fallback when a value is NULL -- Returns first non-NULL argument SELECT name, COALESCE(description, \u0026#39;No description provided\u0026#39;) AS description FROM categories; Rule: Never write WHERE column = NULL - it always returns 0 rows. SQL uses three-value logic: TRUE, FALSE, and NULL (unknown). NULL = NULL evaluates to NULL, not TRUE.\nPart 7 - CASE WHEN: Conditional Logic CASE WHEN is SQL\u0026rsquo;s version of if/else. It creates a new column based on conditions evaluated row by row.\n-- Label products as budget / mid-range / premium based on price SELECT name, price_inr, CASE WHEN price_inr \u0026lt; 1000 THEN \u0026#39;Budget\u0026#39; WHEN price_inr BETWEEN 1000 AND 9999 THEN \u0026#39;Mid-range\u0026#39; ELSE \u0026#39;Premium\u0026#39; END AS price_segment FROM products ORDER BY price_inr; -- Show order status with an emoji label SELECT id, order_date, CASE status WHEN \u0026#39;Delivered\u0026#39; THEN \u0026#39;✅ Delivered\u0026#39; WHEN \u0026#39;Shipped\u0026#39; THEN \u0026#39;🚚 On the Way\u0026#39; WHEN \u0026#39;Pending\u0026#39; THEN \u0026#39;⏳ Pending\u0026#39; WHEN \u0026#39;Cancelled\u0026#39; THEN \u0026#39;❌ Cancelled\u0026#39; END AS status_label FROM orders ORDER BY order_date; -- Count orders in each status using CASE + SUM (pivot pattern) SELECT COUNT(*) AS total_orders, SUM(CASE WHEN status = \u0026#39;Delivered\u0026#39; THEN 1 ELSE 0 END) AS delivered, SUM(CASE WHEN status = \u0026#39;Shipped\u0026#39; THEN 1 ELSE 0 END) AS shipped, SUM(CASE WHEN status = \u0026#39;Pending\u0026#39; THEN 1 ELSE 0 END) AS pending, SUM(CASE WHEN status = \u0026#39;Cancelled\u0026#39; THEN 1 ELSE 0 END) AS cancelled FROM orders; This last pattern is called a pivot - turning row values into columns. Very useful for dashboards and summary reports.\n-- Flag whether each product is available or out of stock SELECT name, price_inr, stock_quantity, CASE WHEN is_available = FALSE THEN \u0026#39;Out of Stock\u0026#39; WHEN stock_quantity = 0 THEN \u0026#39;Out of Stock\u0026#39; WHEN stock_quantity \u0026lt; 20 THEN \u0026#39;Low Stock\u0026#39; ELSE \u0026#39;In Stock\u0026#39; END AS stock_status FROM products ORDER BY stock_quantity; ✏️ Practice Set 2 - LIKE, IS NULL, CASE WHEN Q6. Find all products whose name contains the word \u0026ldquo;Maths\u0026rdquo; or \u0026ldquo;Cricket\u0026rdquo;. Hint: use two LIKE conditions with OR.\nQ7. How many customers have email addresses ending in \u0026lsquo;@gmail.com\u0026rsquo;? Use LIKE and COUNT.\nQ8. Write a query that shows every product\u0026rsquo;s name, price_inr, and a new column called affordability with these labels:\n'Under ₹500' - price below 500 '₹500–₹5000' - price between 500 and 5000 'Above ₹5000' - price above 5000 Q9. List categories that have a description (IS NOT NULL). Show name and description.\nQ10. Write a query that shows the total number of orders for each payment method using GROUP BY. Then rewrite it using the CASE + SUM pivot pattern to show all four payment methods in a single row.\nPart 8 - Date Functions PostgreSQL has rich built-in date functions. Our orders and customers tables both have date columns.\nExtracting Parts of a Date -- Extract the month and year from order_date SELECT id, order_date, EXTRACT(YEAR FROM order_date) AS order_year, EXTRACT(MONTH FROM order_date) AS order_month, EXTRACT(DAY FROM order_date) AS order_day FROM orders ORDER BY order_date; -- How many orders were placed each month? SELECT EXTRACT(MONTH FROM order_date) AS month, COUNT(*) AS order_count FROM orders GROUP BY month ORDER BY month; -- Format the date for display SELECT id, TO_CHAR(order_date, \u0026#39;DD Mon YYYY\u0026#39;) AS formatted_date, TO_CHAR(order_date, \u0026#39;Month YYYY\u0026#39;) AS month_year FROM orders; Comparing and Calculating Dates -- Orders placed in the last 6 months from today -- CURRENT_DATE = today\u0026#39;s date in PostgreSQL SELECT id, customer_id, order_date, status FROM orders WHERE order_date \u0026gt;= CURRENT_DATE - INTERVAL \u0026#39;6 months\u0026#39; ORDER BY order_date DESC; -- How many days ago was each order placed? SELECT id, order_date, CURRENT_DATE - order_date AS days_ago FROM orders ORDER BY days_ago; -- Customers who joined in 2023 Q4 (October–December) SELECT name, city, joined_on FROM customers WHERE joined_on BETWEEN \u0026#39;2023-10-01\u0026#39; AND \u0026#39;2023-12-31\u0026#39; ORDER BY joined_on; -- The most recent and oldest orders SELECT MIN(order_date) AS first_order, MAX(order_date) AS latest_order FROM orders; Part 9 - Putting It All Together: Multi-Condition Queries Real queries combine multiple clauses. Here are some realistic business queries on ShopDB.\n-- All active, in-stock products under ₹5000 in the Sports or Books category SELECT p.name, c.name AS category, p.price_inr, p.stock_quantity FROM products AS p JOIN categories AS c ON p.category_id = c.id WHERE p.is_available = TRUE AND p.stock_quantity \u0026gt; 0 AND p.price_inr \u0026lt; 5000 AND c.name IN (\u0026#39;Sports\u0026#39;, \u0026#39;Books\u0026#39;) ORDER BY p.price_inr; -- Customers from Maharashtra or Kerala who joined after June 2023 SELECT name, city, state, joined_on FROM customers WHERE state IN (\u0026#39;Maharashtra\u0026#39;, \u0026#39;Kerala\u0026#39;) AND joined_on \u0026gt; \u0026#39;2023-06-30\u0026#39; ORDER BY state, joined_on; -- Orders placed in 2024 that are still not delivered -- (Pending or Shipped) SELECT o.id AS order_id, c.name AS customer, c.city, o.order_date, o.status FROM orders AS o JOIN customers AS c ON o.customer_id = c.id WHERE EXTRACT(YEAR FROM o.order_date) = 2024 AND o.status NOT IN (\u0026#39;Delivered\u0026#39;, \u0026#39;Cancelled\u0026#39;) ORDER BY o.order_date; -- Products containing \u0026#34;Maths\u0026#34; or \u0026#34;Cricket\u0026#34; or priced above ₹50000 SELECT name, price_inr, stock_quantity FROM products WHERE name ILIKE \u0026#39;%maths%\u0026#39; OR name ILIKE \u0026#39;%cricket%\u0026#39; OR price_inr \u0026gt; 50000 ORDER BY price_inr DESC; ✏️ Practice Set 3 - Date Functions \u0026amp; Combined Queries Q11. How many orders were placed in each month of 2024? Show month number and order_count, sorted by month.\nQ12. Write a query to find all customers who joined before 1st July 2023. Show name, city, and joined_on date.\nQ13. Find all Delivered orders that were placed using UPI. Show order_id, customer_id, order_date.\nQ14. Write a query that labels each order with a quarter:\n\u0026lsquo;Q1\u0026rsquo; - January to March \u0026lsquo;Q2\u0026rsquo; - April to June \u0026lsquo;Q3\u0026rsquo; - July to September \u0026lsquo;Q4\u0026rsquo; - October to December Show order_id, order_date, and the quarter label. Q15. Find products where the name starts with \u0026lsquo;B\u0026rsquo; AND the price is under ₹1000. Show name and price_inr.\nPart 10 - Practice Set Answers Answers: Practice Set 1 Q1. Distinct cities:\nSELECT DISTINCT city FROM customers ORDER BY city; Q2. Orders by specific customers:\nSELECT id AS order_id, customer_id, order_date, status FROM orders WHERE customer_id IN (1, 3, 5, 7) ORDER BY order_date; Q3. Products between ₹1000 and ₹10000:\nSELECT name, price_inr FROM products WHERE price_inr BETWEEN 1000 AND 10000 ORDER BY price_inr; Q4. Customers who joined Apr–Sep 2023:\nSELECT name, city, joined_on FROM customers WHERE joined_on BETWEEN \u0026#39;2023-04-01\u0026#39; AND \u0026#39;2023-09-30\u0026#39; ORDER BY joined_on; Q5. Non-delivered, non-cancelled orders:\nSELECT COUNT(*) AS count FROM orders WHERE status NOT IN (\u0026#39;Delivered\u0026#39;, \u0026#39;Cancelled\u0026#39;); Answers: Practice Set 2 Q6. Products with \u0026ldquo;Maths\u0026rdquo; or \u0026ldquo;Cricket\u0026rdquo; in the name:\nSELECT name, price_inr FROM products WHERE name LIKE \u0026#39;%Maths%\u0026#39; OR name LIKE \u0026#39;%Cricket%\u0026#39;; Q7. Gmail customers:\nSELECT COUNT(*) AS gmail_customers FROM customers WHERE email LIKE \u0026#39;%@gmail.com\u0026#39;; Q8. Affordability label:\nSELECT name, price_inr, CASE WHEN price_inr \u0026lt; 500 THEN \u0026#39;Under ₹500\u0026#39; WHEN price_inr BETWEEN 500 AND 5000 THEN \u0026#39;₹500–₹5000\u0026#39; ELSE \u0026#39;Above ₹5000\u0026#39; END AS affordability FROM products ORDER BY price_inr; Q9. Categories with a description:\nSELECT name, description FROM categories WHERE description IS NOT NULL; Q10a. Payment method breakdown (GROUP BY):\nSELECT payment_method, COUNT(*) AS order_count FROM orders GROUP BY payment_method ORDER BY order_count DESC; Q10b. Same result as a pivot (single row):\nSELECT SUM(CASE WHEN payment_method = \u0026#39;UPI\u0026#39; THEN 1 ELSE 0 END) AS upi, SUM(CASE WHEN payment_method = \u0026#39;Card\u0026#39; THEN 1 ELSE 0 END) AS card, SUM(CASE WHEN payment_method = \u0026#39;COD\u0026#39; THEN 1 ELSE 0 END) AS cod, SUM(CASE WHEN payment_method = \u0026#39;NetBanking\u0026#39; THEN 1 ELSE 0 END) AS netbanking FROM orders; Answers: Practice Set 3 Q11. Orders per month:\nSELECT EXTRACT(MONTH FROM order_date) AS month, COUNT(*) AS order_count FROM orders WHERE EXTRACT(YEAR FROM order_date) = 2024 GROUP BY month ORDER BY month; Q12. Customers who joined before July 2023:\nSELECT name, city, joined_on FROM customers WHERE joined_on \u0026lt; \u0026#39;2023-07-01\u0026#39; ORDER BY joined_on; Q13. Delivered + UPI orders:\nSELECT id AS order_id, customer_id, order_date FROM orders WHERE status = \u0026#39;Delivered\u0026#39; AND payment_method = \u0026#39;UPI\u0026#39; ORDER BY order_date; Q14. Quarter labels using CASE WHEN:\nSELECT id AS order_id, order_date, CASE WHEN EXTRACT(MONTH FROM order_date) BETWEEN 1 AND 3 THEN \u0026#39;Q1\u0026#39; WHEN EXTRACT(MONTH FROM order_date) BETWEEN 4 AND 6 THEN \u0026#39;Q2\u0026#39; WHEN EXTRACT(MONTH FROM order_date) BETWEEN 7 AND 9 THEN \u0026#39;Q3\u0026#39; ELSE \u0026#39;Q4\u0026#39; END AS quarter FROM orders ORDER BY order_date; Q15. Products starting with \u0026lsquo;B\u0026rsquo; under ₹1000:\nSELECT name, price_inr FROM products WHERE name LIKE \u0026#39;B%\u0026#39; AND price_inr \u0026lt; 1000 ORDER BY price_inr; What\u0026rsquo;s Next You have covered:\n✅ DISTINCT - remove duplicate rows ✅ IN / NOT IN - match against a list of values ✅ BETWEEN - filter by range (inclusive) ✅ LIKE / ILIKE - pattern matching with % and _ ✅ IS NULL / IS NOT NULL - handle missing values ✅ COALESCE - fallback value for NULLs ✅ CASE WHEN - conditional logic and pivot pattern ✅ Date functions - EXTRACT, TO_CHAR, CURRENT_DATE, INTERVAL ✅ Multi-condition queries combining all of the above In Lesson 2.3 we go deep on Aggregations \u0026amp; Grouping:\nMulti-column GROUP BY HAVING with complex conditions ROLLUP for subtotals FILTER clause with aggregates A first look at window functions (ROW_NUMBER, RANK, SUM OVER) All on CricketDB + ShopDB together ","permalink":"https://blogs.codeverra.com/blog/sql-part2/","summary":"\u003ch1 id=\"lesson-2---filtering--querying-data\"\u003eLesson 2 - Filtering \u0026amp; Querying Data\u003c/h1\u003e\n\u003ch3 id=\"theory--practice--shopdb-----indian-e-commerce-dataset\"\u003eTheory + Practice | ShopDB  -  Indian E-commerce Dataset\u003c/h3\u003e\n\u003chr\u003e\n\u003ch2 id=\"before-you-start\"\u003eBefore You Start\u003c/h2\u003e\n\u003col\u003e\n\u003cli\u003eRun \u003ccode\u003esetup.sql\u003c/code\u003e in pgAdmin to create and populate ShopDB.\u003c/li\u003e\n\u003cli\u003eVerify you see 5 tables with the correct row counts.\u003c/li\u003e\n\u003cli\u003eThen work through this file top to bottom.\u003c/li\u003e\n\u003c/ol\u003e\n\u003cp\u003e\u003cstrong\u003eWhat you\u0026rsquo;ll learn in this lesson:\u003c/strong\u003e\u003c/p\u003e\n\u003ctable\u003e\n  \u003cthead\u003e\n      \u003ctr\u003e\n          \u003cth\u003eConcept\u003c/th\u003e\n          \u003cth\u003eWhat it does\u003c/th\u003e\n      \u003c/tr\u003e\n  \u003c/thead\u003e\n  \u003ctbody\u003e\n      \u003ctr\u003e\n          \u003ctd\u003e\u003ccode\u003eDISTINCT\u003c/code\u003e\u003c/td\u003e\n          \u003ctd\u003eRemove duplicate values from results\u003c/td\u003e\n      \u003c/tr\u003e\n      \u003ctr\u003e\n          \u003ctd\u003e\u003ccode\u003eIN\u003c/code\u003e / \u003ccode\u003eNOT IN\u003c/code\u003e\u003c/td\u003e\n          \u003ctd\u003eMatch against a list of values\u003c/td\u003e\n      \u003c/tr\u003e\n      \u003ctr\u003e\n          \u003ctd\u003e\u003ccode\u003eBETWEEN\u003c/code\u003e\u003c/td\u003e\n          \u003ctd\u003eFilter within a range\u003c/td\u003e\n      \u003c/tr\u003e\n      \u003ctr\u003e\n          \u003ctd\u003e\u003ccode\u003eLIKE\u003c/code\u003e / \u003ccode\u003eILIKE\u003c/code\u003e\u003c/td\u003e\n          \u003ctd\u003ePattern matching on text\u003c/td\u003e\n      \u003c/tr\u003e\n      \u003ctr\u003e\n          \u003ctd\u003e\u003ccode\u003eIS NULL\u003c/code\u003e / \u003ccode\u003eIS NOT NULL\u003c/code\u003e\u003c/td\u003e\n          \u003ctd\u003eHandle missing values\u003c/td\u003e\n      \u003c/tr\u003e\n      \u003ctr\u003e\n          \u003ctd\u003e\u003ccode\u003eCASE WHEN\u003c/code\u003e\u003c/td\u003e\n          \u003ctd\u003eConditional logic inside a query\u003c/td\u003e\n      \u003c/tr\u003e\n      \u003ctr\u003e\n          \u003ctd\u003eDate functions\u003c/td\u003e\n          \u003ctd\u003eFilter and extract parts of dates\u003c/td\u003e\n      \u003c/tr\u003e\n      \u003ctr\u003e\n          \u003ctd\u003e\u003ccode\u003eCOALESCE\u003c/code\u003e\u003c/td\u003e\n          \u003ctd\u003eReplace NULL with a fallback value\u003c/td\u003e\n      \u003c/tr\u003e\n  \u003c/tbody\u003e\n\u003c/table\u003e\n\u003cp\u003eAll queries in this lesson run on ShopDB. The dataset has:\u003c/p\u003e","tags":["sql","data-analysis"],"title":"Filtering and Querying data"},{"content":"Python Functions Masterclass A Complete Guide from Basics to Advanced Concepts Who is this for? You know Python basics and have some experience with loops and collections. This guide covers every important concept around functions in Python \u0026ndash; from defining your first function all the way through decorators and closures. Every concept is explained with a simple example first.\nTable of Contents What is a Function and Why Use One? Defining and Calling Functions Parameters and Arguments Default Parameters *args \u0026ndash; Variable Positional Arguments **kwargs \u0026ndash; Variable Keyword Arguments Combining All Argument Types Return Values Scope and the LEGB Rule global and nonlocal Keywords Docstrings and Function Annotations Lambda Functions Higher Order Functions map(), filter(), and reduce() Closures Decorators Decorators with Arguments Chaining Multiple Decorators Recursion Generator Functions and yield Practice Questions Solutions What We Covered and What is Next 1. What is a Function and Why Use One? Imagine you need to calculate GST on product prices in three different parts of your program. Without a function, you write the same calculation three times. If the GST rate changes, you update it in three places \u0026ndash; and likely miss one.\nA function lets you write that logic once, name it, and call it wherever you need it.\n# Without a function -- repeated logic price1_with_gst = 299 * 1.18 price2_with_gst = 499 * 1.18 price3_with_gst = 999 * 1.18 # With a function -- write once, use anywhere def add_gst(price): return round(price * 1.18, 2) price1_with_gst = add_gst(299) price2_with_gst = add_gst(499) price3_with_gst = add_gst(999) Functions give you:\nReusability \u0026ndash; write once, call many times Readability \u0026ndash; give a name to a block of logic Testability \u0026ndash; test one piece of behavior in isolation Maintainability \u0026ndash; change the logic in one place only 2. Defining and Calling Functions Basic Syntax def function_name(parameters): \u0026#34;\u0026#34;\u0026#34;Optional docstring describing what the function does.\u0026#34;\u0026#34;\u0026#34; # function body return value # optional The def keyword starts the definition. The body is indented. The function does nothing until you call it.\ndef greet(name): message = f\u0026#34;Namaste, {name}!\u0026#34; print(message) # Calling the function greet(\u0026#34;Aarav\u0026#34;) # Namaste, Aarav! greet(\u0026#34;Priya\u0026#34;) # Namaste, Priya! greet(\u0026#34;Rohan\u0026#34;) # Namaste, Rohan! Functions are Objects In Python, functions are first-class objects. You can assign them to variables, store them in lists, and pass them to other functions.\ndef say_hello(): print(\u0026#34;Hello!\u0026#34;) # Assign to a variable -- no parentheses, we are not calling it greeting = say_hello greeting() # Hello! -- now calling it through the variable 3. Parameters and Arguments A parameter is the variable name in the function definition. An argument is the actual value you pass when calling the function.\n# parameter def square(number): return number ** 2 # argument result = square(7) # 49 Positional Arguments By default, arguments are matched to parameters by their position.\ndef describe_player(name, team, runs): print(f\u0026#34;{name} plays for {team} and has scored {runs} runs.\u0026#34;) describe_player(\u0026#34;Kohli\u0026#34;, \u0026#34;RCB\u0026#34;, 639) # Kohli plays for RCB and has scored 639 runs. # Order matters -- swap and you get wrong results describe_player(\u0026#34;RCB\u0026#34;, 639, \u0026#34;Kohli\u0026#34;) # RCB plays for 639 and has scored Kohli runs. \u0026lt;-- wrong Keyword Arguments You can name arguments explicitly when calling \u0026ndash; order no longer matters.\ndescribe_player(runs=639, name=\u0026#34;Kohli\u0026#34;, team=\u0026#34;RCB\u0026#34;) # Kohli plays for RCB and has scored 639 runs. \u0026lt;-- correct You can mix positional and keyword arguments, but positional arguments must come first.\ndescribe_player(\u0026#34;Kohli\u0026#34;, runs=639, team=\u0026#34;RCB\u0026#34;) # valid describe_player(name=\u0026#34;Kohli\u0026#34;, \u0026#34;RCB\u0026#34;, 639) # SyntaxError -- keyword before positional 4. Default Parameters A default parameter has a pre-set value that is used when the caller does not provide that argument.\ndef create_profile(name, city=\u0026#34;Bangalore\u0026#34;, role=\u0026#34;Student\u0026#34;): print(f\u0026#34;{name} | {city} | {role}\u0026#34;) create_profile(\u0026#34;Aarav\u0026#34;) # Aarav | Bangalore | Student create_profile(\u0026#34;Priya\u0026#34;, \u0026#34;Mumbai\u0026#34;) # Priya | Mumbai | Student create_profile(\u0026#34;Rohan\u0026#34;, \u0026#34;Delhi\u0026#34;, \u0026#34;Engineer\u0026#34;) # Rohan | Delhi | Engineer create_profile(\u0026#34;Sneha\u0026#34;, role=\u0026#34;Manager\u0026#34;) # Sneha | Bangalore | Manager The Mutable Default Argument Trap Never use a mutable object (list, dict) as a default argument. It is created once when the function is defined, not each time the function is called.\n# WRONG -- the list is shared across all calls def add_score(player, scores=[]): scores.append(player) return scores print(add_score(\u0026#34;Rohit\u0026#34;)) # [\u0026#39;Rohit\u0026#39;] print(add_score(\u0026#34;Kohli\u0026#34;)) # [\u0026#39;Rohit\u0026#39;, \u0026#39;Kohli\u0026#39;] \u0026lt;-- unexpected! print(add_score(\u0026#34;Dhoni\u0026#34;)) # [\u0026#39;Rohit\u0026#39;, \u0026#39;Kohli\u0026#39;, \u0026#39;Dhoni\u0026#39;] \u0026lt;-- still growing! # CORRECT -- use None and create a fresh list inside def add_score(player, scores=None): if scores is None: scores = [] scores.append(player) return scores print(add_score(\u0026#34;Rohit\u0026#34;)) # [\u0026#39;Rohit\u0026#39;] print(add_score(\u0026#34;Kohli\u0026#34;)) # [\u0026#39;Kohli\u0026#39;] \u0026lt;-- fresh list each time 5. *args \u0026ndash; Variable Positional Arguments *args lets a function accept any number of positional arguments. Inside the function, args is a tuple.\ndef total_runs(*scores): print(type(scores)) # \u0026lt;class \u0026#39;tuple\u0026#39;\u0026gt; return sum(scores) print(total_runs(45, 82, 67)) # 194 print(total_runs(10, 20, 30, 40, 50)) # 150 print(total_runs(100)) # 100 # Mix of fixed and variable arguments def match_summary(team, *scores): total = sum(scores) print(f\u0026#34;{team} scored: {scores}\u0026#34;) print(f\u0026#34;Total: {total}\u0026#34;) match_summary(\u0026#34;India\u0026#34;, 45, 0, 6, 82, 37, 54) # India scored: (45, 0, 6, 82, 37, 54) # Total: 224 The name args is a convention. The star * is what matters. You could write *numbers or *values \u0026ndash; it works the same way.\nUnpacking with * when calling The * operator can also unpack a list/tuple into positional arguments.\ndef add(a, b, c): return a + b + c numbers = [10, 20, 30] print(add(*numbers)) # 60 -- unpacks list into three arguments 6. **kwargs \u0026ndash; Variable Keyword Arguments **kwargs lets a function accept any number of keyword arguments. Inside the function, kwargs is a dictionary.\ndef print_profile(**details): print(type(details)) # \u0026lt;class \u0026#39;dict\u0026#39;\u0026gt; for key, value in details.items(): print(f\u0026#34; {key}: {value}\u0026#34;) print_profile(name=\u0026#34;Aarav\u0026#34;, city=\u0026#34;Delhi\u0026#34;, score=88) # name: Aarav # city: Delhi # score: 88 print_profile(name=\u0026#34;Priya\u0026#34;, team=\u0026#34;RCB\u0026#34;, matches=16, runs=639) # name: Priya # team: RCB # matches: 16 # runs: 639 # Practical use -- build a configuration dict def create_order(product, **options): order = {\u0026#34;product\u0026#34;: product} order.update(options) return order order = create_order(\u0026#34;Laptop\u0026#34;, color=\u0026#34;Silver\u0026#34;, warranty=\u0026#34;2yr\u0026#34;, delivery=\u0026#34;express\u0026#34;) print(order) # {\u0026#39;product\u0026#39;: \u0026#39;Laptop\u0026#39;, \u0026#39;color\u0026#39;: \u0026#39;Silver\u0026#39;, \u0026#39;warranty\u0026#39;: \u0026#39;2yr\u0026#39;, \u0026#39;delivery\u0026#39;: \u0026#39;express\u0026#39;} Unpacking with ** when calling def greet(name, city, role): print(f\u0026#34;{name} from {city} is a {role}\u0026#34;) user = {\u0026#34;name\u0026#34;: \u0026#34;Rohan\u0026#34;, \u0026#34;city\u0026#34;: \u0026#34;Mumbai\u0026#34;, \u0026#34;role\u0026#34;: \u0026#34;Engineer\u0026#34;} greet(**user) # Rohan from Mumbai is a Engineer 7. Combining All Argument Types Python allows all four argument types in one function. The order must be:\ndef func(positional, default, *args, keyword_only, **kwargs) def register(name, city=\u0026#34;Unknown\u0026#34;, *scores, role, **extras): print(f\u0026#34;Name : {name}\u0026#34;) print(f\u0026#34;City : {city}\u0026#34;) print(f\u0026#34;Scores : {scores}\u0026#34;) print(f\u0026#34;Role : {role}\u0026#34;) print(f\u0026#34;Extras : {extras}\u0026#34;) register(\u0026#34;Aarav\u0026#34;, \u0026#34;Delhi\u0026#34;, 88, 92, 85, role=\u0026#34;Student\u0026#34;, batch=\u0026#34;2024\u0026#34;, section=\u0026#34;A\u0026#34;) # Name : Aarav # City : Delhi # Scores : (88, 92, 85) # Role : Student # Extras : {\u0026#39;batch\u0026#39;: \u0026#39;2024\u0026#39;, \u0026#39;section\u0026#39;: \u0026#39;A\u0026#39;} Keyword-Only Arguments Any parameter after *args (or a bare *) can only be passed as a keyword argument.\ndef send_email(to, subject, *, cc=None, bcc=None): print(f\u0026#34;To: {to}, Subject: {subject}, CC: {cc}, BCC: {bcc}\u0026#34;) send_email(\u0026#34;aarav@example.com\u0026#34;, \u0026#34;Hello\u0026#34;, cc=\u0026#34;priya@example.com\u0026#34;) # send_email(\u0026#34;a@b.com\u0026#34;, \u0026#34;Hi\u0026#34;, \u0026#34;c@d.com\u0026#34;) --\u0026gt; TypeError: too many positional args Positional-Only Arguments (Python 3.8+) Parameters before / can only be passed positionally, not by keyword.\ndef add(a, b, /): return a + b add(3, 5) # 8 -- fine add(a=3, b=5) # TypeError -- cannot use keyword for positional-only params 8. Return Values The return statement sends a value back to the caller and exits the function immediately.\nReturning Nothing A function with no return statement (or bare return) returns None.\ndef say_hi(): print(\u0026#34;Hi!\u0026#34;) result = say_hi() print(result) # None Returning Multiple Values Python returns multiple values as a tuple.\ndef score_stats(scores): return min(scores), max(scores), sum(scores) / len(scores) low, high, avg = score_stats([45, 88, 72, 91, 60]) print(f\u0026#34;Low: {low}, High: {high}, Avg: {avg}\u0026#34;) # Low: 45, High: 91, Avg: 71.2 Early Return return exits the function immediately. Useful to handle edge cases first.\ndef safe_divide(a, b): if b == 0: return None # exit early return a / b print(safe_divide(10, 2)) # 5.0 print(safe_divide(10, 0)) # None Returning Different Types Based on Condition def find_player(name, players): for i, player in enumerate(players): if player[\u0026#34;name\u0026#34;] == name: return player # return dict if found return None # return None if not found squad = [ {\u0026#34;name\u0026#34;: \u0026#34;Rohit\u0026#34;, \u0026#34;role\u0026#34;: \u0026#34;Batsman\u0026#34;}, {\u0026#34;name\u0026#34;: \u0026#34;Bumrah\u0026#34;, \u0026#34;role\u0026#34;: \u0026#34;Bowler\u0026#34;}, ] result = find_player(\u0026#34;Bumrah\u0026#34;, squad) if result: print(result[\u0026#34;role\u0026#34;]) # Bowler 9. Scope and the LEGB Rule Scope defines where a variable is accessible. Python looks up variables in this order:\nL -- Local : inside the current function E -- Enclosing : inside any enclosing function (for nested functions) G -- Global : at the module (file) level B -- Built-in : Python\u0026#39;s built-in names (len, print, range, etc.) This is called the LEGB rule.\nx = \u0026#34;global\u0026#34; # Global scope def outer(): x = \u0026#34;enclosing\u0026#34; # Enclosing scope def inner(): x = \u0026#34;local\u0026#34; # Local scope print(x) # \u0026#34;local\u0026#34; -- finds it in Local first inner() print(x) # \u0026#34;enclosing\u0026#34; -- Local is gone after inner() ends outer() print(x) # \u0026#34;global\u0026#34; -- Enclosing is gone after outer() ends Local vs Global A variable created inside a function is local \u0026ndash; it cannot be seen outside.\ndef calculate(): result = 100 # local variable print(result) calculate() # 100 print(result) # NameError: name \u0026#39;result\u0026#39; is not defined A function can READ a global variable without any special syntax.\ntax_rate = 0.18 # global def calculate_tax(price): return price * tax_rate # reads global -- this is fine print(calculate_tax(1000)) # 180.0 10. global and nonlocal Keywords global To MODIFY a global variable inside a function, declare it with global.\ncounter = 0 def increment(): global counter # tell Python: use the global variable counter += 1 increment() increment() increment() print(counter) # 3 Without global, Python creates a new local variable instead of modifying the global one.\ncounter = 0 def increment(): counter += 1 # UnboundLocalError -- Python sees assignment and treats counter as local # but it hasn\u0026#39;t been assigned locally yet increment() Use global sparingly. Modifying global state from inside functions makes code hard to reason about. Prefer returning a new value and reassigning it at the call site.\nnonlocal nonlocal is used in nested functions to modify a variable from the enclosing (but not global) scope.\ndef make_counter(): count = 0 # enclosing scope variable def increment(): nonlocal count # refer to enclosing scope\u0026#39;s count count += 1 return count return increment counter = make_counter() print(counter()) # 1 print(counter()) # 2 print(counter()) # 3 11. Docstrings and Function Annotations Docstrings A docstring is a string literal placed immediately after the def line. It documents what the function does.\ndef add_gst(price, rate=0.18): \u0026#34;\u0026#34;\u0026#34; Calculate price after adding GST. Parameters ---------- price : float The base price of the product. rate : float, optional GST rate as a decimal. Default is 0.18 (18%). Returns ------- float The price including GST, rounded to 2 decimal places. Example ------- \u0026gt;\u0026gt;\u0026gt; add_gst(1000) 1180.0 \u0026gt;\u0026gt;\u0026gt; add_gst(1000, rate=0.05) 1050.0 \u0026#34;\u0026#34;\u0026#34; return round(price * (1 + rate), 2) # Access the docstring print(add_gst.__doc__) help(add_gst) Function Annotations (Type Hints) Annotations document the expected types of parameters and return values. Python does not enforce them at runtime \u0026ndash; they are hints for developers and tools.\ndef calculate_emi(principal: float, rate: float, months: int) -\u0026gt; float: \u0026#34;\u0026#34;\u0026#34;Calculate monthly EMI for a loan.\u0026#34;\u0026#34;\u0026#34; monthly_rate = rate / 12 / 100 emi = principal * monthly_rate / (1 - (1 + monthly_rate) ** -months) return round(emi, 2) print(calculate_emi(500000, 8.5, 60)) # Monthly EMI for 5 lakh at 8.5% for 5 years # Access annotations print(calculate_emi.__annotations__) # {\u0026#39;principal\u0026#39;: \u0026lt;class \u0026#39;float\u0026#39;\u0026gt;, \u0026#39;rate\u0026#39;: \u0026lt;class \u0026#39;float\u0026#39;\u0026gt;, \u0026#39;months\u0026#39;: \u0026lt;class \u0026#39;int\u0026#39;\u0026gt;, \u0026#39;return\u0026#39;: \u0026lt;class \u0026#39;float\u0026#39;\u0026gt;} 12. Lambda Functions A lambda is a small, anonymous function defined in a single expression. It is not a replacement for def \u0026ndash; it is useful for short throwaway functions, especially when passing a function as an argument.\nSyntax lambda parameters: expression # Regular function def square(x): return x ** 2 # Equivalent lambda square = lambda x: x ** 2 print(square(5)) # 25 # Lambda with multiple parameters add = lambda a, b: a + b print(add(10, 20)) # 30 # Lambda with condition classify = lambda score: \u0026#34;Pass\u0026#34; if score \u0026gt;= 60 else \u0026#34;Fail\u0026#34; print(classify(75)) # Pass print(classify(45)) # Fail Where Lambdas Shine \u0026ndash; as Arguments players = [ {\u0026#34;name\u0026#34;: \u0026#34;Rohit\u0026#34;, \u0026#34;runs\u0026#34;: 440}, {\u0026#34;name\u0026#34;: \u0026#34;Kohli\u0026#34;, \u0026#34;runs\u0026#34;: 639}, {\u0026#34;name\u0026#34;: \u0026#34;Shubman\u0026#34;, \u0026#34;runs\u0026#34;: 520}, {\u0026#34;name\u0026#34;: \u0026#34;Jadeja\u0026#34;, \u0026#34;runs\u0026#34;: 180}, ] # Sort by runs using a lambda as the key sorted_players = sorted(players, key=lambda p: p[\u0026#34;runs\u0026#34;], reverse=True) for p in sorted_players: print(f\u0026#34;{p[\u0026#39;name\u0026#39;]}: {p[\u0026#39;runs\u0026#39;]}\u0026#34;) # Kohli: 639 # Shubman: 520 # Rohit: 440 # Jadeja: 180 # Sort a list of tuples by second element scores = [(\u0026#34;Aarav\u0026#34;, 85), (\u0026#34;Priya\u0026#34;, 92), (\u0026#34;Rohan\u0026#34;, 78)] scores.sort(key=lambda x: x[1]) print(scores) # [(\u0026#39;Rohan\u0026#39;, 78), (\u0026#39;Aarav\u0026#39;, 85), (\u0026#39;Priya\u0026#39;, 92)] Lambda Limitations Only one expression (no statements, no if blocks, no loops) No docstring Harder to debug (shows up as \u0026lt;lambda\u0026gt; in tracebacks) When the logic is more than one expression, use a regular def function.\n13. Higher Order Functions A higher order function is a function that either:\nTakes another function as an argument, or Returns a function as its result This is possible because Python treats functions as first-class objects.\nFunctions that Take Functions as Arguments def apply_twice(func, value): \u0026#34;\u0026#34;\u0026#34;Apply a function to a value, then apply it again to the result.\u0026#34;\u0026#34;\u0026#34; return func(func(value)) def double(x): return x * 2 print(apply_twice(double, 3)) # double(double(3)) = double(6) = 12 print(apply_twice(double, 10)) # double(double(10)) = double(20) = 40 # A flexible discount applier def apply_discount(prices, discount_fn): return [discount_fn(p) for p in prices] def festival_discount(price): return price * 0.80 # 20% off def loyalty_discount(price): return price * 0.90 # 10% off prices = [1000, 2500, 4800, 750] print(apply_discount(prices, festival_discount)) # [800.0, 2000.0, 3840.0, 600.0] print(apply_discount(prices, loyalty_discount)) # [900.0, 2250.0, 4320.0, 675.0] # Pass a lambda directly -- no need to define a named function print(apply_discount(prices, lambda p: p * 0.85)) Functions that Return Functions def make_multiplier(n): \u0026#34;\u0026#34;\u0026#34;Returns a function that multiplies its input by n.\u0026#34;\u0026#34;\u0026#34; def multiplier(x): return x * n return multiplier # returning the function, not calling it double = make_multiplier(2) triple = make_multiplier(3) tenfold = make_multiplier(10) print(double(5)) # 10 print(triple(5)) # 15 print(tenfold(5)) # 50 # A function factory for greetings in different languages def make_greeter(language): greetings = { \u0026#34;hindi\u0026#34;: \u0026#34;Namaste\u0026#34;, \u0026#34;tamil\u0026#34;: \u0026#34;Vanakkam\u0026#34;, \u0026#34;bengali\u0026#34;: \u0026#34;Namaskar\u0026#34;, \u0026#34;english\u0026#34;: \u0026#34;Hello\u0026#34;, } greeting = greetings.get(language, \u0026#34;Hello\u0026#34;) def greet(name): return f\u0026#34;{greeting}, {name}!\u0026#34; return greet hindi_greet = make_greeter(\u0026#34;hindi\u0026#34;) tamil_greet = make_greeter(\u0026#34;tamil\u0026#34;) print(hindi_greet(\u0026#34;Aarav\u0026#34;)) # Namaste, Aarav! print(tamil_greet(\u0026#34;Priya\u0026#34;)) # Vanakkam, Priya! 14. map(), filter(), and reduce() These are Python\u0026rsquo;s built-in higher order functions for working with iterables.\nmap() \u0026ndash; Apply a Function to Every Item map(function, iterable) returns a map object (lazy iterator). Convert with list() to see results.\nprices = [299, 499, 999, 1499, 2999] # Add 18% GST to every price with_gst = list(map(lambda p: round(p * 1.18, 2), prices)) print(with_gst) # [352.82, 588.82, 1178.82, 1768.82, 3538.82] # Equivalent list comprehension with_gst = [round(p * 1.18, 2) for p in prices] # map() with a named function def to_title(name): return name.strip().title() raw_names = [\u0026#34; aarav sharma\u0026#34;, \u0026#34;PRIYA patel \u0026#34;, \u0026#34;rohan VERMA\u0026#34;] clean_names = list(map(to_title, raw_names)) print(clean_names) # [\u0026#39;Aarav Sharma\u0026#39;, \u0026#39;Priya Patel\u0026#39;, \u0026#39;Rohan Verma\u0026#39;] filter() \u0026ndash; Keep Only Items That Pass a Test filter(function, iterable) keeps items where the function returns True.\nscores = [45, 82, 58, 91, 38, 74, 66, 49, 88] # Keep only passing scores passed = list(filter(lambda s: s \u0026gt;= 60, scores)) print(passed) # [82, 91, 74, 66, 88] # Keep only cities with more than 5 characters cities = [\u0026#34;Delhi\u0026#34;, \u0026#34;Mumbai\u0026#34;, \u0026#34;Pune\u0026#34;, \u0026#34;Bangalore\u0026#34;, \u0026#34;Surat\u0026#34;, \u0026#34;Hyderabad\u0026#34;] long_cities = list(filter(lambda c: len(c) \u0026gt; 5, cities)) print(long_cities) # [\u0026#39;Mumbai\u0026#39;, \u0026#39;Bangalore\u0026#39;, \u0026#39;Hyderabad\u0026#39;] reduce() \u0026ndash; Reduce a Sequence to a Single Value reduce is in the functools module. It applies a function cumulatively to all items.\nfrom functools import reduce numbers = [1, 2, 3, 4, 5] # Sum: ((((1+2)+3)+4)+5) = 15 total = reduce(lambda a, b: a + b, numbers) print(total) # 15 # Product: 1*2*3*4*5 = 120 product = reduce(lambda a, b: a * b, numbers) print(product) # 120 # Find maximum scores = [45, 88, 72, 91, 60] maximum = reduce(lambda a, b: a if a \u0026gt; b else b, scores) print(maximum) # 91 map + filter + reduce Together # IPL scenario: from a list of player scores, # keep scores above 50, double them (bonus runs), then find the total raw_scores = [25, 80, 45, 120, 15, 95, 60, 30] result = reduce( lambda a, b: a + b, map( lambda s: s * 2, filter(lambda s: s \u0026gt; 50, raw_scores) ) ) print(result) # (80+120+95+60)*2 = 710 15. Closures A closure is a function that remembers the variables from its enclosing scope even after that scope has finished executing.\nThree conditions for a closure:\nThere is a nested function (function inside a function) The nested function refers to a variable in the enclosing scope The enclosing function returns the nested function def make_counter(start=0): count = start # enclosing scope variable def increment(): nonlocal count count += 1 return count return increment # return the inner function counter1 = make_counter() counter2 = make_counter(10) # starts from 10 print(counter1()) # 1 print(counter1()) # 2 print(counter1()) # 3 print(counter2()) # 11 print(counter2()) # 12 # counter1 and counter2 each have their own independent \u0026#39;count\u0026#39; variable Practical Closure \u0026ndash; Rate Limiter / Multiplier Factory def make_gst_calculator(rate): \u0026#34;\u0026#34;\u0026#34;Returns a function that adds a specific GST rate.\u0026#34;\u0026#34;\u0026#34; def calculate(price): return round(price * (1 + rate), 2) return calculate gst_5 = make_gst_calculator(0.05) gst_12 = make_gst_calculator(0.12) gst_18 = make_gst_calculator(0.18) print(gst_5(1000)) # 1050.0 print(gst_12(1000)) # 1120.0 print(gst_18(1000)) # 1180.0 Inspecting a Closure def outer(x): def inner(y): return x + y return inner add_10 = outer(10) print(add_10(5)) # 15 print(add_10.__closure__) # closure cells exist print(add_10.__closure__[0].cell_contents) # 10 -- the remembered value of x 16. Decorators A decorator is a function that wraps another function to extend or modify its behavior \u0026ndash; without changing the original function\u0026rsquo;s code.\nThis is one of Python\u0026rsquo;s most powerful patterns. It relies on higher order functions and closures.\nThe Problem Decorators Solve Suppose you want to log every time a function is called. Without decorators:\ndef add(a, b): print(f\u0026#34;Calling add with {a}, {b}\u0026#34;) result = a + b print(f\u0026#34;add returned {result}\u0026#34;) return result Now imagine doing this for 20 functions. You would copy the logging lines 20 times. If the logging format changes, you update 20 places.\nWith a decorator, you write the logging once:\ndef logger(func): def wrapper(*args, **kwargs): print(f\u0026#34;Calling {func.__name__} with args={args}, kwargs={kwargs}\u0026#34;) result = func(*args, **kwargs) print(f\u0026#34;{func.__name__} returned {result}\u0026#34;) return result return wrapper def add(a, b): return a + b # Apply the decorator manually add = logger(add) add(3, 5) # Calling add with args=(3, 5), kwargs={} # add returned 8 The @ Syntax (Syntactic Sugar) The @decorator syntax does exactly the same thing as func = decorator(func), but more cleanly.\ndef logger(func): def wrapper(*args, **kwargs): print(f\u0026#34;Calling {func.__name__}\u0026#34;) result = func(*args, **kwargs) print(f\u0026#34;{func.__name__} done\u0026#34;) return result return wrapper @logger def greet(name): return f\u0026#34;Hello, {name}!\u0026#34; @logger def add(a, b): return a + b greet(\u0026#34;Aarav\u0026#34;) # Calling greet # greet done add(10, 20) # Calling add # add done Preserving the Original Function\u0026rsquo;s Identity with functools.wraps Without functools.wraps, the wrapped function loses its name and docstring.\nfrom functools import wraps def logger(func): @wraps(func) # copies name, docstring, etc. from func to wrapper def wrapper(*args, **kwargs): print(f\u0026#34;Calling {func.__name__}\u0026#34;) return func(*args, **kwargs) return wrapper @logger def add(a, b): \u0026#34;\u0026#34;\u0026#34;Add two numbers.\u0026#34;\u0026#34;\u0026#34; return a + b print(add.__name__) # add (not \u0026#39;wrapper\u0026#39;) print(add.__doc__) # Add two numbers. A Practical Decorator \u0026ndash; Timer import time from functools import wraps def timer(func): @wraps(func) def wrapper(*args, **kwargs): start = time.time() result = func(*args, **kwargs) end = time.time() print(f\u0026#34;{func.__name__} took {end - start:.4f} seconds\u0026#34;) return result return wrapper @timer def compute_sum(n): return sum(range(n)) compute_sum(10_000_000) # compute_sum took 0.3821 seconds A Practical Decorator \u0026ndash; Input Validator from functools import wraps def validate_positive(func): @wraps(func) def wrapper(*args, **kwargs): for arg in args: if isinstance(arg, (int, float)) and arg \u0026lt; 0: raise ValueError(f\u0026#34;All arguments must be positive. Got: {arg}\u0026#34;) return func(*args, **kwargs) return wrapper @validate_positive def calculate_emi(principal, rate, months): monthly_rate = rate / 12 / 100 return round(principal * monthly_rate / (1 - (1 + monthly_rate) ** -months), 2) print(calculate_emi(500000, 8.5, 60)) # works fine calculate_emi(-500000, 8.5, 60) # ValueError: All arguments must be positive 17. Decorators with Arguments Sometimes you want to pass arguments to the decorator itself \u0026ndash; for example, specifying how many times to retry or what role is required.\nTo do this, you add another layer of wrapping: a function that takes the decorator arguments and returns the actual decorator.\nfrom functools import wraps def repeat(n): \u0026#34;\u0026#34;\u0026#34;Decorator that runs the function n times.\u0026#34;\u0026#34;\u0026#34; def decorator(func): @wraps(func) def wrapper(*args, **kwargs): for _ in range(n): result = func(*args, **kwargs) return result return wrapper return decorator @repeat(3) def announce(message): print(message) announce(\u0026#34;Doors open for boarding!\u0026#34;) # Doors open for boarding! # Doors open for boarding! # Doors open for boarding! # Role-based access control decorator def require_role(role): def decorator(func): @wraps(func) def wrapper(user, *args, **kwargs): if user.get(\u0026#34;role\u0026#34;) != role: raise PermissionError(f\u0026#34;Access denied. Required role: {role}\u0026#34;) return func(user, *args, **kwargs) return wrapper return decorator @require_role(\u0026#34;admin\u0026#34;) def delete_record(user, record_id): print(f\u0026#34;Record {record_id} deleted by {user[\u0026#39;name\u0026#39;]}\u0026#34;) admin = {\u0026#34;name\u0026#34;: \u0026#34;Aarav\u0026#34;, \u0026#34;role\u0026#34;: \u0026#34;admin\u0026#34;} guest = {\u0026#34;name\u0026#34;: \u0026#34;Priya\u0026#34;, \u0026#34;role\u0026#34;: \u0026#34;viewer\u0026#34;} delete_record(admin, 42) # Record 42 deleted by Aarav delete_record(guest, 42) # PermissionError: Access denied. Required role: admin 18. Chaining Multiple Decorators You can stack multiple decorators on a single function. They are applied bottom-up (closest to the function first).\nfrom functools import wraps def bold(func): @wraps(func) def wrapper(*args, **kwargs): return \u0026#34;**\u0026#34; + func(*args, **kwargs) + \u0026#34;**\u0026#34; return wrapper def uppercase(func): @wraps(func) def wrapper(*args, **kwargs): return func(*args, **kwargs).upper() return wrapper @bold @uppercase def greet(name): return f\u0026#34;hello, {name}\u0026#34; print(greet(\u0026#34;Aarav\u0026#34;)) # uppercase is applied first: \u0026#34;HELLO, AARAV\u0026#34; # bold is applied second: \u0026#34;**HELLO, AARAV**\u0026#34; The execution order is:\n@bold --\u0026gt; applied second (outermost wrapper) @uppercase --\u0026gt; applied first (innermost wrapper) def greet --\u0026gt; the original function # Practical stacking: timer + logger @timer @logger def fetch_data(city): # simulate some work time.sleep(0.1) return f\u0026#34;Data for {city}\u0026#34; fetch_data(\u0026#34;Mumbai\u0026#34;) # logger runs first (inner), timer wraps that (outer) 19. Recursion A recursive function is one that calls itself. Every recursive function needs:\nA base case \u0026ndash; a condition under which it stops and returns directly A recursive case \u0026ndash; where it calls itself with a smaller input def factorial(n): if n == 0 or n == 1: # base case return 1 return n * factorial(n - 1) # recursive case print(factorial(5)) # 5 * 4 * 3 * 2 * 1 = 120 print(factorial(0)) # 1 How it unwinds:\nfactorial(5) = 5 * factorial(4) = 5 * 4 * factorial(3) = 5 * 4 * 3 * factorial(2) = 5 * 4 * 3 * 2 * factorial(1) = 5 * 4 * 3 * 2 * 1 = 120 Fibonacci with Recursion def fibonacci(n): if n \u0026lt;= 1: # base cases return n return fibonacci(n - 1) + fibonacci(n - 2) for i in range(8): print(fibonacci(i), end=\u0026#34; \u0026#34;) # 0 1 1 2 3 5 8 13 Recursive Sum of a List def recursive_sum(lst): if not lst: # base case: empty list return 0 return lst[0] + recursive_sum(lst[1:]) # first element + sum of rest print(recursive_sum([10, 20, 30, 40])) # 100 Recursion Limit Python has a default recursion limit of 1000. For deeply recursive problems, use iteration or increase the limit carefully.\nimport sys print(sys.getrecursionlimit()) # 1000 # You can increase it, but this is a sign you should rethink your approach sys.setrecursionlimit(5000) 20. Generator Functions and yield A generator function uses yield instead of return. It produces values one at a time, on demand, rather than computing and storing all values at once. This saves memory for large sequences.\n# Regular function -- builds entire list in memory def squares_list(n): result = [] for i in range(1, n + 1): result.append(i ** 2) return result # Generator function -- produces one value at a time def squares_gen(n): for i in range(1, n + 1): yield i ** 2 # pauses here, returns value, resumes on next call gen = squares_gen(5) print(next(gen)) # 1 print(next(gen)) # 4 print(next(gen)) # 9 # Or iterate with a for loop for square in squares_gen(5): print(square) # 1 4 9 16 25 yield pauses execution Each time yield is hit, the function pauses and saves its state. The next call to next() resumes from where it left off.\ndef countdown(n): print(\u0026#34;Starting countdown\u0026#34;) while n \u0026gt; 0: yield n # pause here and return n n -= 1 # resume here on next call print(\u0026#34;Done!\u0026#34;) gen = countdown(3) print(next(gen)) # Starting countdown \\n 3 print(next(gen)) # 2 print(next(gen)) # 1 print(next(gen)) # Done! \\n StopIteration Generator Expression (like list comprehension, but lazy) # List comprehension -- all 1 million values in memory squares = [x**2 for x in range(1_000_000)] # Generator expression -- only one value in memory at a time squares = (x**2 for x in range(1_000_000)) # Sum a billion numbers without storing them all total = sum(x**2 for x in range(1_000_000)) print(total) Practical Generator \u0026ndash; Reading Large Files def read_large_file(filepath): \u0026#34;\u0026#34;\u0026#34;Yields one line at a time -- never loads entire file.\u0026#34;\u0026#34;\u0026#34; with open(filepath) as f: for line in f: yield line.strip() # Usage -- memory-efficient regardless of file size for line in read_large_file(\u0026#34;huge_dataset.csv\u0026#34;): process(line) 21. Practice Questions Try each question on your own before looking at the solutions in Section 22. The questions are ordered: Easy (1-6), Medium (7-13), Hard (14-20).\nEasy Q1 \u0026ndash; Basic Function Write a function celsius_to_fahrenheit(c) that converts a Celsius temperature to Fahrenheit using the formula F = (C * 9/5) + 32. Test it with the temperatures below.\ntemps_c = [0, 100, 37, -40, 25] Expected output:\n0 C = 32.0 F 100 C = 212.0 F 37 C = 98.6 F -40 C = -40.0 F 25 C = 77.0 F Q2 \u0026ndash; Default Parameters Write a function book_ticket(passenger, train=\u0026quot;Rajdhani\u0026quot;, class_type=\u0026quot;Sleeper\u0026quot;, meals=False) that prints a booking summary. Call it four times:\nOnly the passenger name Passenger name and train All arguments Using keyword arguments in a different order Expected output:\nPassenger : Aarav Train : Rajdhani Class : Sleeper Meals : No ---------- Passenger : Priya Train : Shatabdi Class : Sleeper Meals : No ... (and so on) Q3 \u0026ndash; Return Multiple Values Write a function string_stats(text) that returns the number of words, number of characters (excluding spaces), and the longest word \u0026ndash; all in one return statement. Unpack the result when calling.\ntext = \u0026#34;The quick brown fox jumps over the lazy dog\u0026#34; Expected output:\nWords : 9 Characters: 35 Longest : jumps *Q4 \u0026ndash; args Write a function total_bill(*items) where each item is a tuple of (name, price). The function should print each item and its price, then print the total.\ntotal_bill((\u0026#34;Masala Dosa\u0026#34;, 80), (\u0026#34;Filter Coffee\u0026#34;, 30), (\u0026#34;Idli Sambar\u0026#34;, 60), (\u0026#34;Gulab Jamun\u0026#34;, 40)) Expected output:\nMasala Dosa : Rs.80 Filter Coffee : Rs.30 Idli Sambar : Rs.60 Gulab Jamun : Rs.40 --------------------- Total : Rs.210 **Q5 \u0026ndash; kwargs Write a function create_employee(**details) that accepts any number of keyword arguments and prints them in a formatted table. Then call it twice with different sets of fields.\nExpected output:\n---- Employee Profile ---- name : Vikram Nair department : Engineering salary : 85000 city : Bangalore experience_yrs : 4 Q6 \u0026ndash; Lambda and sorted() You have a list of student tuples (name, marks). Sort them:\nBy marks ascending By marks descending Alphabetically by name Use lambda functions as the sort key.\nstudents = [(\u0026#34;Rohan\u0026#34;, 78), (\u0026#34;Aarav\u0026#34;, 85), (\u0026#34;Sneha\u0026#34;, 92), (\u0026#34;Priya\u0026#34;, 78), (\u0026#34;Karan\u0026#34;, 65)] Medium Q7 \u0026ndash; Scope What will the following code print? Write your prediction before running it. Then explain why.\nx = 10 def outer(): x = 20 def inner(): x = 30 print(\u0026#34;inner:\u0026#34;, x) inner() print(\u0026#34;outer:\u0026#34;, x) outer() print(\u0026#34;global:\u0026#34;, x) Then modify the code so that:\ninner() modifies outer\u0026rsquo;s x using nonlocal outer() modifies the global x using global Q8 \u0026ndash; Higher Order Functions Write a function apply_all(value, *functions) that takes a starting value and any number of functions, applies them one after another (the output of each becomes the input of the next), and returns the final result.\ndef double(x): return x * 2 def add_10(x): return x + 10 def square(x): return x ** 2 print(apply_all(3, double, add_10, square)) # double(3) = 6, add_10(6) = 16, square(16) = 256 Q9 \u0026ndash; Closures Write a function make_validator(min_val, max_val) that returns a function. The returned function should take a number and return True if it falls within the range, and False otherwise.\nis_valid_age = make_validator(18, 60) is_valid_score = make_validator(0, 100) is_valid_pin = make_validator(1000, 9999) print(is_valid_age(25)) # True print(is_valid_age(15)) # False print(is_valid_score(105)) # False print(is_valid_pin(4521)) # True Q10 \u0026ndash; map() and filter() Given the list of orders below:\nUse map() to apply a 10% discount to all amounts Use filter() to keep only orders above Rs.500 (after discount) Use map() again to round all final amounts to 2 decimal places Do all three steps using map and filter only (no list comprehensions).\norder_amounts = [250, 1200, 480, 3500, 150, 800, 2200, 95, 600] Q11 \u0026ndash; Decorator Write a decorator validate_non_negative that raises a ValueError if any positional numeric argument passed to a function is negative. Apply it to a function calculate_rectangle_area(length, width).\ncalculate_rectangle_area(5, 3) # 15 calculate_rectangle_area(-2, 4) # ValueError calculate_rectangle_area(0, 7) # 0 Q12 \u0026ndash; Recursion Write a recursive function power(base, exp) that computes base raised to the power exp without using the ** operator or pow(). Handle the case where exp is 0.\nprint(power(2, 10)) # 1024 print(power(3, 4)) # 81 print(power(5, 0)) # 1 Q13 \u0026ndash; Generator Write a generator function fibonacci_gen(n) that yields the first n Fibonacci numbers. Then use it to:\nPrint the first 10 Fibonacci numbers Find the first Fibonacci number greater than 100 Hard Q14 \u0026ndash; Decorator with Arguments Write a decorator retry(max_attempts, delay=0) that retries a function up to max_attempts times if it raises an exception. After all attempts are exhausted, re-raise the last exception. Use time.sleep(delay) between attempts.\nTest it with a function that randomly fails.\nimport random import time @retry(max_attempts=4, delay=0.1) def unstable_api_call(city): if random.random() \u0026lt; 0.6: # 60% chance of failure raise ConnectionError(f\u0026#34;API call to {city} failed\u0026#34;) return f\u0026#34;Data received for {city}\u0026#34; Q15 \u0026ndash; Closure with State Write a function make_account(owner, initial_balance=0) that returns three functions as a tuple: deposit, withdraw, and get_balance. All three functions should share access to the same internal balance variable using closures.\ndeposit, withdraw, get_balance = make_account(\u0026#34;Aarav\u0026#34;, 1000) deposit(500) print(get_balance()) # 1500 withdraw(200) print(get_balance()) # 1300 withdraw(2000) # Should print: \u0026#34;Insufficient funds\u0026#34; Q16 \u0026ndash; Higher Order + map/filter/reduce Pipeline You have a list of employee dictionaries. Using only map(), filter(), and reduce() (no for loops, no list comprehensions), write a pipeline that:\nFilters out employees with less than 3 years of experience Applies a 15% salary hike to the remaining employees Computes the total salary payout after hike from functools import reduce employees = [ {\u0026#34;name\u0026#34;: \u0026#34;Aarav\u0026#34;, \u0026#34;salary\u0026#34;: 60000, \u0026#34;experience\u0026#34;: 4}, {\u0026#34;name\u0026#34;: \u0026#34;Priya\u0026#34;, \u0026#34;salary\u0026#34;: 45000, \u0026#34;experience\u0026#34;: 2}, {\u0026#34;name\u0026#34;: \u0026#34;Rohan\u0026#34;, \u0026#34;salary\u0026#34;: 75000, \u0026#34;experience\u0026#34;: 6}, {\u0026#34;name\u0026#34;: \u0026#34;Sneha\u0026#34;, \u0026#34;salary\u0026#34;: 52000, \u0026#34;experience\u0026#34;: 1}, {\u0026#34;name\u0026#34;: \u0026#34;Karan\u0026#34;, \u0026#34;salary\u0026#34;: 80000, \u0026#34;experience\u0026#34;: 8}, {\u0026#34;name\u0026#34;: \u0026#34;Meera\u0026#34;, \u0026#34;salary\u0026#34;: 55000, \u0026#34;experience\u0026#34;: 3}, ] Q17 \u0026ndash; Recursive Flatten Write a recursive function flatten(lst) that takes a deeply nested list and returns a flat list of all values. Do not use any built-in flatten utilities.\nnested = [1, [2, 3], [4, [5, 6]], [[7], [8, [9, 10]]]] print(flatten(nested)) # [1, 2, 3, 4, 5, 6, 7, 8, 9, 10] Q18 \u0026ndash; Memoization Decorator Write a decorator memoize that caches the results of a function so that repeated calls with the same arguments return instantly from the cache instead of recomputing.\nApply it to the recursive Fibonacci function and compare speed with and without caching.\n@memoize def fib(n): if n \u0026lt;= 1: return n return fib(n - 1) + fib(n - 2) print(fib(35)) # Should be fast with memoization Q19 \u0026ndash; Function Composition Write a function compose(*functions) that takes any number of functions and returns a new function that applies them from right to left (standard mathematical composition).\ndef add_5(x): return x + 5 def triple(x): return x * 3 def negate(x): return -x # compose(negate, triple, add_5) means: negate(triple(add_5(x))) transform = compose(negate, triple, add_5) print(transform(2)) # negate(triple(add_5(2))) = negate(triple(7)) = negate(21) = -21 print(transform(10)) # negate(triple(15)) = negate(45) = -45 Q20 \u0026ndash; Full Pipeline Build a data processing pipeline using functions. You are given raw employee data as a list of strings (CSV format). Write the following functions and chain them together:\nparse_employees(raw) \u0026ndash; parse each string into a dict clean_employees(employees) \u0026ndash; strip whitespace from all string fields, convert salary and experience to correct types filter_eligible(employees) \u0026ndash; keep only employees with experience \u0026gt;= 3 and salary \u0026gt;= 50000 apply_raise(employees, pct) \u0026ndash; apply a salary raise by pct percent to all remaining employees generate_report(employees) \u0026ndash; print a formatted report with rank, name, department, old salary, new salary Chain them into a single run_pipeline(raw, raise_pct) function.\nraw_data = [ \u0026#34;Aarav Sharma, Engineering, 75000, 5\u0026#34;, \u0026#34;Priya Patel, HR, 45000, 2\u0026#34;, \u0026#34;Rohan Verma, Sales, 52000, 4\u0026#34;, \u0026#34;Sneha Iyer, Engineering, 88000, 7\u0026#34;, \u0026#34;Karan Singh, Marketing, 48000, 1\u0026#34;, \u0026#34;Meera Nair, Engineering, 91000, 9\u0026#34;, \u0026#34;Arjun Kumar, Sales, 55000, 3\u0026#34;, ] run_pipeline(raw_data, raise_pct=12) Expected output:\n===== SALARY REVISION REPORT (12% Raise) ===== Rank Name Department Old Salary New Salary 1 Aarav Sharma Engineering Rs.75,000 Rs.84,000 2 Rohan Verma Sales Rs.52,000 Rs.58,240 3 Sneha Iyer Engineering Rs.88,000 Rs.98,560 4 Meera Nair Engineering Rs.91,000 Rs.1,01,920 5 Arjun Kumar Sales Rs.55,000 Rs.61,600 22. Solutions Q1 \u0026ndash; Basic Function def celsius_to_fahrenheit(c): return (c * 9/5) + 32 temps_c = [0, 100, 37, -40, 25] for c in temps_c: f = celsius_to_fahrenheit(c) print(f\u0026#34;{c} C\\t= {f} F\u0026#34;) Q2 \u0026ndash; Default Parameters def book_ticket(passenger, train=\u0026#34;Rajdhani\u0026#34;, class_type=\u0026#34;Sleeper\u0026#34;, meals=False): print(f\u0026#34;Passenger : {passenger}\u0026#34;) print(f\u0026#34;Train : {train}\u0026#34;) print(f\u0026#34;Class : {class_type}\u0026#34;) print(f\u0026#34;Meals : {\u0026#39;Yes\u0026#39; if meals else \u0026#39;No\u0026#39;}\u0026#34;) print(\u0026#34;-\u0026#34; * 10) book_ticket(\u0026#34;Aarav\u0026#34;) book_ticket(\u0026#34;Priya\u0026#34;, \u0026#34;Shatabdi\u0026#34;) book_ticket(\u0026#34;Rohan\u0026#34;, \u0026#34;Vande Bharat\u0026#34;, \u0026#34;AC 2-Tier\u0026#34;, True) book_ticket(\u0026#34;Sneha\u0026#34;, meals=True, class_type=\u0026#34;AC 1-Tier\u0026#34;, train=\u0026#34;Duronto\u0026#34;) Q3 \u0026ndash; Return Multiple Values def string_stats(text): words = text.split() chars = len(text.replace(\u0026#34; \u0026#34;, \u0026#34;\u0026#34;)) longest = max(words, key=len) return len(words), chars, longest word_count, char_count, longest_word = string_stats( \u0026#34;The quick brown fox jumps over the lazy dog\u0026#34; ) print(f\u0026#34;Words : {word_count}\u0026#34;) print(f\u0026#34;Characters: {char_count}\u0026#34;) print(f\u0026#34;Longest : {longest_word}\u0026#34;) Q4 \u0026ndash; *args def total_bill(*items): total = 0 for name, price in items: print(f\u0026#34;{name:\u0026lt;18}: Rs.{price}\u0026#34;) total += price print(\u0026#34;-\u0026#34; * 25) print(f\u0026#34;{\u0026#39;Total\u0026#39;:\u0026lt;18}: Rs.{total}\u0026#34;) total_bill( (\u0026#34;Masala Dosa\u0026#34;, 80), (\u0026#34;Filter Coffee\u0026#34;, 30), (\u0026#34;Idli Sambar\u0026#34;, 60), (\u0026#34;Gulab Jamun\u0026#34;, 40) ) Q5 \u0026ndash; **kwargs def create_employee(**details): print(\u0026#34;---- Employee Profile ----\u0026#34;) for key, value in details.items(): print(f\u0026#34;{key:\u0026lt;15}: {value}\u0026#34;) create_employee( name=\u0026#34;Vikram Nair\u0026#34;, department=\u0026#34;Engineering\u0026#34;, salary=85000, city=\u0026#34;Bangalore\u0026#34;, experience_yrs=4 ) print() create_employee( name=\u0026#34;Priya Shah\u0026#34;, role=\u0026#34;HR Manager\u0026#34;, email=\u0026#34;priya@company.com\u0026#34; ) Q6 \u0026ndash; Lambda and sorted() students = [(\u0026#34;Rohan\u0026#34;, 78), (\u0026#34;Aarav\u0026#34;, 85), (\u0026#34;Sneha\u0026#34;, 92), (\u0026#34;Priya\u0026#34;, 78), (\u0026#34;Karan\u0026#34;, 65)] by_marks_asc = sorted(students, key=lambda s: s[1]) by_marks_desc = sorted(students, key=lambda s: s[1], reverse=True) by_name = sorted(students, key=lambda s: s[0]) print(\u0026#34;By marks ascending :\u0026#34;, by_marks_asc) print(\u0026#34;By marks descending:\u0026#34;, by_marks_desc) print(\u0026#34;Alphabetically :\u0026#34;, by_name) Q7 \u0026ndash; Scope Prediction and explanation:\nx = 10 def outer(): x = 20 def inner(): x = 30 print(\u0026#34;inner:\u0026#34;, x) # 30 -- local to inner inner() print(\u0026#34;outer:\u0026#34;, x) # 20 -- local to outer, unaffected by inner\u0026#39;s x outer() print(\u0026#34;global:\u0026#34;, x) # 10 -- global, unaffected by either function Output:\ninner: 30 outer: 20 global: 10 Each x = ... inside a function creates a new local variable. They do not affect each other unless you use nonlocal or global.\nModified version:\nx = 10 def outer(): global x x = 20 # now modifies the global x def inner(): nonlocal x # ERROR: x is global here, not enclosing # nonlocal only works for enclosing function scope, not global # To demonstrate nonlocal properly, we need a variable that is truly enclosing: pass # Correct nonlocal demonstration: def outer2(): y = 20 def inner2(): nonlocal y y = 30 print(\u0026#34;inner2:\u0026#34;, y) # 30 inner2() print(\u0026#34;outer2:\u0026#34;, y) # 30 -- changed by nonlocal outer2() Q8 \u0026ndash; Higher Order Functions def apply_all(value, *functions): result = value for func in functions: result = func(result) return result def double(x): return x * 2 def add_10(x): return x + 10 def square(x): return x ** 2 print(apply_all(3, double, add_10, square)) # 256 print(apply_all(5, double, double, double)) # 40 Q9 \u0026ndash; Closures def make_validator(min_val, max_val): def validate(number): return min_val \u0026lt;= number \u0026lt;= max_val return validate is_valid_age = make_validator(18, 60) is_valid_score = make_validator(0, 100) is_valid_pin = make_validator(1000, 9999) print(is_valid_age(25)) # True print(is_valid_age(15)) # False print(is_valid_score(105)) # False print(is_valid_pin(4521)) # True Q10 \u0026ndash; map() and filter() order_amounts = [250, 1200, 480, 3500, 150, 800, 2200, 95, 600] # Step 1: apply 10% discount after_discount = map(lambda x: x * 0.90, order_amounts) # Step 2: keep only amounts above 500 above_500 = filter(lambda x: x \u0026gt; 500, after_discount) # Step 3: round to 2 decimal places final = list(map(lambda x: round(x, 2), above_500)) print(final) # [1080.0, 3150.0, 720.0, 1980.0, 540.0] Q11 \u0026ndash; Decorator from functools import wraps def validate_non_negative(func): @wraps(func) def wrapper(*args, **kwargs): for arg in args: if isinstance(arg, (int, float)) and arg \u0026lt; 0: raise ValueError(f\u0026#34;Negative value not allowed: {arg}\u0026#34;) return func(*args, **kwargs) return wrapper @validate_non_negative def calculate_rectangle_area(length, width): return length * width print(calculate_rectangle_area(5, 3)) # 15 print(calculate_rectangle_area(0, 7)) # 0 calculate_rectangle_area(-2, 4) # ValueError: Negative value not allowed: -2 Q12 \u0026ndash; Recursion def power(base, exp): if exp == 0: # base case: anything to power 0 is 1 return 1 return base * power(base, exp - 1) # recursive case print(power(2, 10)) # 1024 print(power(3, 4)) # 81 print(power(5, 0)) # 1 Q13 \u0026ndash; Generator def fibonacci_gen(n): a, b = 0, 1 count = 0 while count \u0026lt; n: yield a a, b = b, a + b count += 1 # First 10 Fibonacci numbers print(\u0026#34;First 10:\u0026#34;, list(fibonacci_gen(10))) # [0, 1, 1, 2, 3, 5, 8, 13, 21, 34] # First Fibonacci number greater than 100 gen = fibonacci_gen(1000) for num in gen: if num \u0026gt; 100: print(\u0026#34;First Fibonacci \u0026gt; 100:\u0026#34;, num) # 144 break Q14 \u0026ndash; Decorator with Arguments import random import time from functools import wraps def retry(max_attempts, delay=0): def decorator(func): @wraps(func) def wrapper(*args, **kwargs): last_error = None for attempt in range(1, max_attempts + 1): try: result = func(*args, **kwargs) print(f\u0026#34; Succeeded on attempt {attempt}\u0026#34;) return result except Exception as e: last_error = e print(f\u0026#34; Attempt {attempt} failed: {e}\u0026#34;) if delay \u0026gt; 0: time.sleep(delay) raise last_error return wrapper return decorator random.seed(42) @retry(max_attempts=4, delay=0) def unstable_api_call(city): if random.random() \u0026lt; 0.6: raise ConnectionError(f\u0026#34;API call to {city} failed\u0026#34;) return f\u0026#34;Data received for {city}\u0026#34; try: result = unstable_api_call(\u0026#34;Mumbai\u0026#34;) print(result) except ConnectionError as e: print(f\u0026#34;All attempts failed: {e}\u0026#34;) Q15 \u0026ndash; Closure with State def make_account(owner, initial_balance=0): balance = initial_balance def deposit(amount): nonlocal balance balance += amount print(f\u0026#34;Deposited Rs.{amount}. Balance: Rs.{balance}\u0026#34;) def withdraw(amount): nonlocal balance if amount \u0026gt; balance: print(\u0026#34;Insufficient funds\u0026#34;) else: balance -= amount print(f\u0026#34;Withdrew Rs.{amount}. Balance: Rs.{balance}\u0026#34;) def get_balance(): return balance return deposit, withdraw, get_balance deposit, withdraw, get_balance = make_account(\u0026#34;Aarav\u0026#34;, 1000) deposit(500) print(get_balance()) # 1500 withdraw(200) print(get_balance()) # 1300 withdraw(2000) # Insufficient funds Q16 \u0026ndash; Higher Order + map/filter/reduce Pipeline from functools import reduce employees = [ {\u0026#34;name\u0026#34;: \u0026#34;Aarav\u0026#34;, \u0026#34;salary\u0026#34;: 60000, \u0026#34;experience\u0026#34;: 4}, {\u0026#34;name\u0026#34;: \u0026#34;Priya\u0026#34;, \u0026#34;salary\u0026#34;: 45000, \u0026#34;experience\u0026#34;: 2}, {\u0026#34;name\u0026#34;: \u0026#34;Rohan\u0026#34;, \u0026#34;salary\u0026#34;: 75000, \u0026#34;experience\u0026#34;: 6}, {\u0026#34;name\u0026#34;: \u0026#34;Sneha\u0026#34;, \u0026#34;salary\u0026#34;: 52000, \u0026#34;experience\u0026#34;: 1}, {\u0026#34;name\u0026#34;: \u0026#34;Karan\u0026#34;, \u0026#34;salary\u0026#34;: 80000, \u0026#34;experience\u0026#34;: 8}, {\u0026#34;name\u0026#34;: \u0026#34;Meera\u0026#34;, \u0026#34;salary\u0026#34;: 55000, \u0026#34;experience\u0026#34;: 3}, ] # Step 1: filter -- keep experience \u0026gt;= 3 eligible = filter(lambda e: e[\u0026#34;experience\u0026#34;] \u0026gt;= 3, employees) # Step 2: map -- apply 15% hike with_hike = map( lambda e: {**e, \u0026#34;salary\u0026#34;: round(e[\u0026#34;salary\u0026#34;] * 1.15)}, eligible ) # Step 3: reduce -- sum all salaries total_payout = reduce( lambda acc, e: acc + e[\u0026#34;salary\u0026#34;], with_hike, 0 ) print(f\u0026#34;Total salary payout after hike: Rs.{total_payout:,}\u0026#34;) # Eligible: Aarav, Rohan, Karan, Meera # After 15% hike: 69000 + 86250 + 92000 + 63250 = Rs.3,10,500 Q17 \u0026ndash; Recursive Flatten def flatten(lst): result = [] for item in lst: if isinstance(item, list): result.extend(flatten(item)) # recurse into sublists else: result.append(item) return result nested = [1, [2, 3], [4, [5, 6]], [[7], [8, [9, 10]]]] print(flatten(nested)) # [1, 2, 3, 4, 5, 6, 7, 8, 9, 10] Q18 \u0026ndash; Memoization Decorator import time from functools import wraps def memoize(func): cache = {} @wraps(func) def wrapper(*args): if args not in cache: cache[args] = func(*args) return cache[args] return wrapper # Without memoization -- exponentially slow for large n def fib_slow(n): if n \u0026lt;= 1: return n return fib_slow(n - 1) + fib_slow(n - 2) @memoize def fib(n): if n \u0026lt;= 1: return n return fib(n - 1) + fib(n - 2) start = time.time() print(fib(35)) print(f\u0026#34;With memoize : {time.time() - start:.6f}s\u0026#34;) start = time.time() print(fib_slow(35)) print(f\u0026#34;Without memoize: {time.time() - start:.6f}s\u0026#34;) Q19 \u0026ndash; Function Composition from functools import reduce def compose(*functions): def composed(x): # apply functions right to left return reduce(lambda v, f: f(v), reversed(functions), x) return composed def add_5(x): return x + 5 def triple(x): return x * 3 def negate(x): return -x transform = compose(negate, triple, add_5) print(transform(2)) # negate(triple(add_5(2))) = negate(triple(7)) = negate(21) = -21 print(transform(10)) # negate(triple(15)) = negate(45) = -45 Q20 \u0026ndash; Full Pipeline def parse_employees(raw): employees = [] for line in raw: parts = [p.strip() for p in line.split(\u0026#34;,\u0026#34;)] employees.append({ \u0026#34;name\u0026#34;: parts[0], \u0026#34;department\u0026#34;: parts[1], \u0026#34;salary\u0026#34;: int(parts[2]), \u0026#34;experience\u0026#34;: int(parts[3]), }) return employees def clean_employees(employees): for emp in employees: emp[\u0026#34;name\u0026#34;] = emp[\u0026#34;name\u0026#34;].strip().title() emp[\u0026#34;department\u0026#34;] = emp[\u0026#34;department\u0026#34;].strip().title() return employees def filter_eligible(employees): return [e for e in employees if e[\u0026#34;experience\u0026#34;] \u0026gt;= 3 and e[\u0026#34;salary\u0026#34;] \u0026gt;= 50000] def apply_raise(employees, pct): factor = 1 + pct / 100 for emp in employees: emp[\u0026#34;old_salary\u0026#34;] = emp[\u0026#34;salary\u0026#34;] emp[\u0026#34;salary\u0026#34;] = round(emp[\u0026#34;salary\u0026#34;] * factor) return employees def generate_report(employees, raise_pct): print(f\u0026#34;\\n===== SALARY REVISION REPORT ({raise_pct}% Raise) =====\u0026#34;) print(f\u0026#34;{\u0026#39;Rank\u0026#39;:\u0026lt;5} {\u0026#39;Name\u0026#39;:\u0026lt;16} {\u0026#39;Department\u0026#39;:\u0026lt;14} {\u0026#39;Old Salary\u0026#39;:\u0026gt;12} {\u0026#39;New Salary\u0026#39;:\u0026gt;12}\u0026#34;) print(\u0026#34;-\u0026#34; * 60) for rank, emp in enumerate(employees, start=1): print( f\u0026#34;{rank:\u0026lt;5} {emp[\u0026#39;name\u0026#39;]:\u0026lt;16} {emp[\u0026#39;department\u0026#39;]:\u0026lt;14}\u0026#34; f\u0026#34; Rs.{emp[\u0026#39;old_salary\u0026#39;]:\u0026gt;9,} Rs.{emp[\u0026#39;salary\u0026#39;]:\u0026gt;9,}\u0026#34; ) def run_pipeline(raw, raise_pct): employees = parse_employees(raw) employees = clean_employees(employees) employees = filter_eligible(employees) employees = apply_raise(employees, raise_pct) generate_report(employees, raise_pct) raw_data = [ \u0026#34;Aarav Sharma, Engineering, 75000, 5\u0026#34;, \u0026#34;Priya Patel, HR, 45000, 2\u0026#34;, \u0026#34;Rohan Verma, Sales, 52000, 4\u0026#34;, \u0026#34;Sneha Iyer, Engineering, 88000, 7\u0026#34;, \u0026#34;Karan Singh, Marketing, 48000, 1\u0026#34;, \u0026#34;Meera Nair, Engineering, 91000, 9\u0026#34;, \u0026#34;Arjun Kumar, Sales, 55000, 3\u0026#34;, ] run_pipeline(raw_data, raise_pct=12) 23. What We Covered and What is Next What This Masterclass Covered Section Topics Basics def, calling, parameters vs arguments Argument types Positional, keyword, default, *args, **kwargs Advanced params Keyword-only, positional-only (3.8+), combining all types Return values Single, multiple, early return, returning None Scope LEGB rule, local vs global, reading globals global / nonlocal Modifying outer scope variables Docstrings Writing and accessing documentation Annotations Type hints for parameters and return values Lambda Syntax, use cases, sort keys, limitations Higher order functions Functions as arguments, functions returning functions map / filter / reduce Built-in functional tools Closures Remembered enclosing state, function factories Decorators Wrapping functions, @ syntax, functools.wraps Decorators with args Three-layer decorator pattern Chaining decorators Stacking and execution order Recursion Base case, recursive case, call stack, recursion limit Generators yield, lazy evaluation, generator expressions What to Explore Next 1. functools module Beyond wraps and reduce, functools has partial (pre-fill function arguments), lru_cache (built-in memoization), cache, and total_ordering. These are production-grade tools built around the same concepts covered here.\n2. itertools module Pairs naturally with generators. Gives you chain, product, combinations, permutations, groupby, islice, and more. Essential for advanced iteration.\n3. Object-Oriented Programming Functions become methods. Closures become classes. The concepts you have learned here are the functional programming side of Python \u0026ndash; OOP is the other side, and understanding both makes you a much more complete Python developer.\n4. Async Functions (asyncio) Python has async def and await for writing asynchronous code. Generators are the foundation that async/await is built on.\n5. Type Annotations at Scale Using typing module: List[int], Dict[str, float], Optional[str], Callable, TypeVar, Generic. Tools like mypy and pyright use these to catch bugs before runtime.\nFinal thought: Decorators, closures, and higher order functions are the concepts that separate someone who writes Python from someone who thinks in Python. The practice questions \u0026ndash; especially Q14 through Q20 \u0026ndash; are the ones worth spending real time on. Every one of them mirrors a pattern you will encounter in real codebases.\nMade with care for Codeverra learners | codeverra.com\n","permalink":"https://blogs.codeverra.com/blog/python-07-functions/","summary":"\u003ch1 id=\"python-functions-masterclass\"\u003ePython Functions Masterclass\u003c/h1\u003e\n\u003ch3 id=\"a-complete-guide-from-basics-to-advanced-concepts\"\u003eA Complete Guide from Basics to Advanced Concepts\u003c/h3\u003e\n\u003chr\u003e\n\u003cblockquote\u003e\n\u003cp\u003eWho is this for?\nYou know Python basics and have some experience with loops and collections.\nThis guide covers every important concept around functions in Python \u0026ndash; from defining your first function\nall the way through decorators and closures. Every concept is explained with a simple example first.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003chr\u003e\n\u003ch2 id=\"table-of-contents\"\u003eTable of Contents\u003c/h2\u003e\n\u003col\u003e\n\u003cli\u003e\u003ca href=\"#1-what-is-a-function-and-why-use-one\"\u003eWhat is a Function and Why Use One?\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#2-defining-and-calling-functions\"\u003eDefining and Calling Functions\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#3-parameters-and-arguments\"\u003eParameters and Arguments\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#4-default-parameters\"\u003eDefault Parameters\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#5-args----variable-positional-arguments\"\u003e*args \u0026ndash; Variable Positional Arguments\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#6-kwargs----variable-keyword-arguments\"\u003e**kwargs \u0026ndash; Variable Keyword Arguments\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#7-combining-all-argument-types\"\u003eCombining All Argument Types\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#8-return-values\"\u003eReturn Values\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#9-scope-and-the-legb-rule\"\u003eScope and the LEGB Rule\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#10-global-and-nonlocal-keywords\"\u003eglobal and nonlocal Keywords\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#11-docstrings-and-function-annotations\"\u003eDocstrings and Function Annotations\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#12-lambda-functions\"\u003eLambda Functions\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#13-higher-order-functions\"\u003eHigher Order Functions\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#14-map-filter-and-reduce\"\u003emap(), filter(), and reduce()\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#15-closures\"\u003eClosures\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#16-decorators\"\u003eDecorators\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#17-decorators-with-arguments\"\u003eDecorators with Arguments\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#18-chaining-multiple-decorators\"\u003eChaining Multiple Decorators\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#19-recursion\"\u003eRecursion\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#20-generator-functions-and-yield\"\u003eGenerator Functions and yield\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#21-practice-questions\"\u003ePractice Questions\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#22-solutions\"\u003eSolutions\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#23-what-we-covered-and-what-is-next\"\u003eWhat We Covered and What is Next\u003c/a\u003e\u003c/li\u003e\n\u003c/ol\u003e\n\u003chr\u003e\n\u003ch2 id=\"1-what-is-a-function-and-why-use-one\"\u003e1. What is a Function and Why Use One?\u003c/h2\u003e\n\u003cp\u003eImagine you need to calculate GST on product prices in three different parts of your program. Without a function, you write the same calculation three times. If the GST rate changes, you update it in three places \u0026ndash; and likely miss one.\u003c/p\u003e","tags":["python"],"title":"Functions in Python - Complete Guide"},{"content":"Lesson 3 - Aggregations \u0026amp; Grouping Theory + Practice | CricketDB + ShopDB Before You Start This lesson uses both databases you have already built:\nCricketDB - players, matches, performances (Lesson 2.1) ShopDB - customers, products, orders, order_items (Lesson 2.2) Run the setup_check.sql from this lesson folder to confirm both are ready. Each section tells you which database to connect to in pgAdmin.\nWhat you will learn in this lesson:\nConcept What it does Multi-column GROUP BY Group by two or more columns simultaneously GROUP BY with JOIN Aggregate across related tables HAVING with multiple conditions Filter groups on complex criteria FILTER clause Conditional aggregation without CASE WHEN ROLLUP Automatically add subtotals and grand total Window functions - ROW_NUMBER Assign a rank within a group Window functions - RANK / DENSE_RANK Rank with tie-handling Window functions - SUM OVER Running totals without collapsing rows Part 1 - Recap: Single-Column GROUP BY (CricketDB - connect to cricketdb in pgAdmin)\nYou already know the basics. Let\u0026rsquo;s start with a quick recap and build from there.\n-- Total runs per player (you wrote this in 2.1) SELECT player_id, SUM(runs) AS total_runs, SUM(wickets) AS total_wickets, COUNT(*) AS innings_played FROM performances GROUP BY player_id ORDER BY total_runs DESC; The problem: player_id = 1 means nothing to a reader. The fix: JOIN before grouping.\n-- Same query - but with actual player names SELECT pl.name, SUM(p.runs) AS total_runs, SUM(p.wickets) AS total_wickets, COUNT(*) AS innings_played FROM performances AS p JOIN players AS pl ON p.player_id = pl.id GROUP BY pl.id, pl.name ORDER BY total_runs DESC; Rule: When you JOIN and then GROUP BY, include the JOIN key (pl.id) in GROUP BY along with any display column (pl.name). This prevents grouping errors when two players share the same name.\nPart 2 - Multi-Column GROUP BY Group by more than one column to get finer breakdowns.\n(CricketDB)\n-- Runs per player per match format (Test / ODI / T20) SELECT pl.name, m.format, SUM(p.runs) AS total_runs, COUNT(*) AS innings FROM performances AS p JOIN players AS pl ON p.player_id = pl.id JOIN matches AS m ON p.match_id = m.id GROUP BY pl.id, pl.name, m.format ORDER BY pl.name, m.format; Each row now represents one player in one format - a finer grain than before.\n-- Which format does each player perform best in? -- (highest average runs) SELECT pl.name, m.format, ROUND(AVG(p.runs), 1) AS avg_runs, MAX(p.runs) AS best_score FROM performances AS p JOIN players AS pl ON p.player_id = pl.id JOIN matches AS m ON p.match_id = m.id GROUP BY pl.id, pl.name, m.format ORDER BY pl.name, avg_runs DESC; (Switch to ShopDB)\n-- Number of orders per month per payment method SELECT EXTRACT(MONTH FROM order_date) AS month, payment_method, COUNT(*) AS order_count FROM orders GROUP BY month, payment_method ORDER BY month, order_count DESC; -- Total revenue per category per month -- Revenue = SUM of (quantity × unit_price) per order item SELECT EXTRACT(MONTH FROM o.order_date) AS month, c.name AS category, SUM(oi.quantity * oi.unit_price) AS revenue_inr FROM order_items AS oi JOIN orders AS o ON oi.order_id = o.id JOIN products AS p ON oi.product_id = p.id JOIN categories AS c ON p.category_id = c.id WHERE o.status != \u0026#39;Cancelled\u0026#39; GROUP BY month, c.id, c.name ORDER BY month, revenue_inr DESC; ✏️ Practice Set 1 - Multi-column GROUP BY Q1. (CricketDB) For each match result (Won / Lost / Draw), show:\nTotal runs scored by Indian players Total wickets taken Number of innings played Q2. (CricketDB) For each player, show their performance broken down by match result (Won / Lost / Draw). Show player name, result, total_runs, and total_wickets.\nQ3. (ShopDB) How many customers joined per state per month? Show state, month number, and customer_count. Only show combinations with at least 1 customer.\nQ4. (ShopDB) Show total quantity sold per category. (Hint: JOIN order_items → products → categories. Exclude cancelled orders.)\nPart 3 - HAVING With Multiple Conditions HAVING filters groups after aggregation. You can combine conditions in HAVING just like in WHERE.\n(CricketDB)\n-- Players who have batted in at least 3 innings -- AND averaged more than 40 runs per innings SELECT pl.name, COUNT(*) AS innings, ROUND(AVG(p.runs), 1) AS batting_average FROM performances AS p JOIN players AS pl ON p.player_id = pl.id GROUP BY pl.id, pl.name HAVING COUNT(*) \u0026gt;= 3 AND AVG(p.runs) \u0026gt; 40 ORDER BY batting_average DESC; -- Formats where India\u0026#39;s average score exceeds 50 runs per innings SELECT m.format, ROUND(AVG(p.runs), 1) AS avg_runs, COUNT(*) AS innings_count FROM performances AS p JOIN matches AS m ON p.match_id = m.id GROUP BY m.format HAVING AVG(p.runs) \u0026gt; 50 ORDER BY avg_runs DESC; (Switch to ShopDB)\n-- Customers who have placed more than 1 order SELECT c.name, c.city, COUNT(o.id) AS order_count FROM customers AS c JOIN orders AS o ON o.customer_id = c.id GROUP BY c.id, c.name, c.city HAVING COUNT(o.id) \u0026gt; 1 ORDER BY order_count DESC; -- Categories where total revenue (non-cancelled) exceeds ₹10,000 SELECT cat.name AS category, SUM(oi.quantity * oi.unit_price) AS total_revenue FROM order_items AS oi JOIN orders AS o ON oi.order_id = o.id JOIN products AS p ON oi.product_id = p.id JOIN categories AS cat ON p.category_id = cat.id WHERE o.status != \u0026#39;Cancelled\u0026#39; GROUP BY cat.id, cat.name HAVING SUM(oi.quantity * oi.unit_price) \u0026gt; 10000 ORDER BY total_revenue DESC; ✏️ Practice Set 2 - HAVING with multiple conditions Q5. (CricketDB) Find players who have taken wickets in at least 2 innings AND have a total wicket count of 4 or more. Show player name, innings with wickets, and total wickets.\nQ6. (CricketDB) Which players scored a total of more than 100 runs AND played in both \u0026lsquo;Won\u0026rsquo; and \u0026lsquo;Lost\u0026rsquo; matches? (Hint: use COUNT(DISTINCT m.result) \u0026gt;= 2 in HAVING.)\nQ7. (ShopDB) Find states where the total number of orders placed by customers from that state is 3 or more. Show state and order_count.\nQ8. (ShopDB) Which products have been ordered more than once across all orders? Show product name, times ordered, and total quantity sold.\nPart 4 - The FILTER Clause FILTER (WHERE ...) is a cleaner alternative to CASE WHEN inside aggregates. It is PostgreSQL-specific but very readable.\n(CricketDB)\n-- Runs scored in Won matches vs Lost/Draw matches -- Using CASE WHEN (you already know this) SELECT pl.name, SUM(CASE WHEN m.result = \u0026#39;Won\u0026#39; THEN p.runs ELSE 0 END) AS runs_in_wins, SUM(CASE WHEN m.result != \u0026#39;Won\u0026#39; THEN p.runs ELSE 0 END) AS runs_in_losses FROM performances AS p JOIN players AS pl ON p.player_id = pl.id JOIN matches AS m ON p.match_id = m.id GROUP BY pl.id, pl.name ORDER BY runs_in_wins DESC; -- Exact same result - using FILTER (much more readable) SELECT pl.name, SUM(p.runs) FILTER (WHERE m.result = \u0026#39;Won\u0026#39;) AS runs_in_wins, SUM(p.runs) FILTER (WHERE m.result != \u0026#39;Won\u0026#39;) AS runs_in_losses, COUNT(*) FILTER (WHERE m.result = \u0026#39;Won\u0026#39;) AS innings_in_wins FROM performances AS p JOIN players AS pl ON p.player_id = pl.id JOIN matches AS m ON p.match_id = m.id GROUP BY pl.id, pl.name ORDER BY runs_in_wins DESC; (Switch to ShopDB)\n-- Order count and revenue, split by delivered vs other statuses SELECT COUNT(*) AS total_orders, COUNT(*) FILTER (WHERE status = \u0026#39;Delivered\u0026#39;) AS delivered_count, COUNT(*) FILTER (WHERE status = \u0026#39;Cancelled\u0026#39;) AS cancelled_count, COUNT(*) FILTER (WHERE status = \u0026#39;Pending\u0026#39;) AS pending_count FROM orders; -- Per customer: how many delivered orders and how many cancelled? SELECT c.name, COUNT(o.id) FILTER (WHERE o.status = \u0026#39;Delivered\u0026#39;) AS delivered, COUNT(o.id) FILTER (WHERE o.status = \u0026#39;Cancelled\u0026#39;) AS cancelled, COUNT(o.id) FILTER (WHERE o.status = \u0026#39;Pending\u0026#39;) AS pending FROM customers AS c JOIN orders AS o ON o.customer_id = c.id GROUP BY c.id, c.name ORDER BY delivered DESC; CASE WHEN vs FILTER: Both produce the same result. FILTER is cleaner and more expressive when you are conditionally aggregating the same column multiple times. CASE WHEN is standard SQL and works in all databases. FILTER is PostgreSQL (and SQLite 3.30+) only.\nPart 5 - ROLLUP: Subtotals and Grand Totals ROLLUP automatically adds subtotal and grand total rows to a GROUP BY.\n(ShopDB)\n-- Orders per status - with a grand total row added automatically SELECT COALESCE(status, \u0026#39;GRAND TOTAL\u0026#39;) AS status, COUNT(*) AS order_count FROM orders GROUP BY ROLLUP(status) ORDER BY status NULLS LAST; The ROLLUP(status) adds one extra row where status is NULL - that is the grand total row. COALESCE replaces the NULL with a readable label.\n-- Revenue by category - with category subtotals SELECT COALESCE(cat.name, \u0026#39;── TOTAL\u0026#39;) AS category, SUM(oi.quantity * oi.unit_price) AS revenue_inr FROM order_items AS oi JOIN orders AS o ON oi.order_id = o.id JOIN products AS p ON oi.product_id = p.id JOIN categories AS cat ON p.category_id = cat.id WHERE o.status != \u0026#39;Cancelled\u0026#39; GROUP BY ROLLUP(cat.name) ORDER BY revenue_inr DESC NULLS LAST; -- Multi-level ROLLUP: month → payment method → grand total SELECT COALESCE(CAST(EXTRACT(MONTH FROM order_date) AS TEXT), \u0026#39;All Months\u0026#39;) AS month, COALESCE(payment_method, \u0026#39;All Methods\u0026#39;) AS payment_method, COUNT(*) AS orders FROM orders GROUP BY ROLLUP(EXTRACT(MONTH FROM order_date), payment_method) ORDER BY month, payment_method NULLS LAST; ROLLUP(a, b) produces groups for:\n(a, b) - each unique combination (a) - subtotal per a across all b () - grand total Part 6 - Window Functions This is the most powerful concept in this lesson.\nThe key difference:\nGROUP BY collapses many rows into one row per group Window functions compute values across rows but keep every row Think of it like: GROUP BY destroys rows. Window functions don\u0026rsquo;t.\n6.1 - ROW_NUMBER: Number Rows Within a Group (CricketDB)\n-- Rank each player\u0026#39;s innings by runs scored, within each match SELECT pl.name, m.opponent, m.format, p.runs, ROW_NUMBER() OVER ( PARTITION BY p.match_id -- restart numbering per match ORDER BY p.runs DESC -- highest runs gets number 1 ) AS rank_in_match FROM performances AS p JOIN players AS pl ON p.player_id = pl.id JOIN matches AS m ON p.match_id = m.id ORDER BY m.id, rank_in_match; OVER (PARTITION BY ... ORDER BY ...) is the window specification:\nPARTITION BY match_id - restart the counter for each match ORDER BY runs DESC - assign 1 to the highest scorer in that match -- Top scorer in each match (using ROW_NUMBER in a subquery) SELECT name, opponent, format, runs FROM ( SELECT pl.name, m.opponent, m.format, p.runs, ROW_NUMBER() OVER ( PARTITION BY p.match_id ORDER BY p.runs DESC ) AS rn FROM performances AS p JOIN players AS pl ON p.player_id = pl.id JOIN matches AS m ON p.match_id = m.id ) AS ranked WHERE rn = 1 ORDER BY runs DESC; This pattern - compute a window function in a subquery, then filter on it in the outer query - is one of the most useful SQL patterns you will use in real work.\n6.2 - RANK and DENSE_RANK: Handle Ties ROW_NUMBER gives unique numbers even to tied rows (arbitrary tiebreak). RANK gives tied rows the same number, then skips (1, 1, 3, 4). DENSE_RANK gives tied rows the same number, no skipping (1, 1, 2, 3).\n(CricketDB)\n-- Overall ranking of players by total runs -- Shows the difference between RANK and DENSE_RANK SELECT pl.name, SUM(p.runs) AS total_runs, RANK() OVER (ORDER BY SUM(p.runs) DESC) AS rank, DENSE_RANK() OVER (ORDER BY SUM(p.runs) DESC) AS dense_rank FROM performances AS p JOIN players AS pl ON p.player_id = pl.id GROUP BY pl.id, pl.name ORDER BY total_runs DESC; (Switch to ShopDB)\n-- Rank products by price within each category SELECT cat.name AS category, p.name AS product, p.price_inr, RANK() OVER ( PARTITION BY p.category_id ORDER BY p.price_inr DESC ) AS price_rank FROM products AS p JOIN categories AS cat ON p.category_id = cat.id ORDER BY cat.name, price_rank; 6.3 - SUM OVER: Running Totals A running total accumulates as you move down the rows.\n(ShopDB)\n-- Running total of orders by date SELECT order_date, COUNT(*) AS orders_that_day, SUM(COUNT(*)) OVER (ORDER BY order_date) AS running_total_orders FROM orders GROUP BY order_date ORDER BY order_date; SUM(COUNT(*)) OVER (ORDER BY order_date) - the inner COUNT(*) runs per group (per day), then SUM OVER accumulates those counts in date order.\n-- Running revenue total across months (non-cancelled orders) SELECT EXTRACT(MONTH FROM o.order_date) AS month, SUM(oi.quantity * oi.unit_price) AS monthly_revenue, SUM(SUM(oi.quantity * oi.unit_price)) OVER (ORDER BY EXTRACT(MONTH FROM o.order_date)) AS running_revenue FROM order_items AS oi JOIN orders AS o ON oi.order_id = o.id WHERE o.status != \u0026#39;Cancelled\u0026#39; GROUP BY month ORDER BY month; -- Each product ranked by price within its category, -- plus the most expensive product\u0026#39;s price in that category SELECT cat.name AS category, p.name AS product, p.price_inr, MAX(p.price_inr) OVER (PARTITION BY p.category_id) AS category_max_price, ROUND( p.price_inr * 100.0 / MAX(p.price_inr) OVER (PARTITION BY p.category_id), 1 ) AS pct_of_max FROM products AS p JOIN categories AS cat ON p.category_id = cat.id ORDER BY cat.name, p.price_inr DESC; ✏️ Practice Set 3 - Window Functions Q9. (CricketDB) Assign a rank to each player\u0026rsquo;s performance (innings) within each match format (Test / ODI / T20), based on runs scored. Show player name, format, runs, and their rank within that format.\nQ10. (CricketDB) Find the top wicket-taker in each match. Show player name, opponent, and wickets. (Hint: Use ROW_NUMBER OVER PARTITION BY match_id ORDER BY wickets DESC, then filter on rn = 1 in a subquery.)\nQ11. (ShopDB) Show each order with:\nThe order date The revenue for that order (SUM of quantity × unit_price for its items) A running total of revenue ordered by order_date Q12. (ShopDB) For each product, show its price and the average price of all products in the same category. Use AVG(price_inr) OVER (PARTITION BY category_id) as a window function. Label this column category_avg_price.\nPart 7 - Putting It All Together Two realistic \u0026ldquo;analyst-level\u0026rdquo; queries that combine everything from this lesson.\n(CricketDB)\n-- Player report card: -- Total runs, batting average, best score, wickets, -- their rank by total runs, and runs in Won vs other matches SELECT pl.name, COUNT(*) AS innings, SUM(p.runs) AS total_runs, ROUND(AVG(p.runs), 1) AS batting_avg, MAX(p.runs) AS best_score, SUM(p.wickets) AS total_wickets, RANK() OVER (ORDER BY SUM(p.runs) DESC) AS run_rank, SUM(p.runs) FILTER (WHERE m.result = \u0026#39;Won\u0026#39;) AS runs_in_wins, SUM(p.runs) FILTER (WHERE m.result != \u0026#39;Won\u0026#39;) AS runs_in_losses FROM performances AS p JOIN players AS pl ON p.player_id = pl.id JOIN matches AS m ON p.match_id = m.id GROUP BY pl.id, pl.name ORDER BY total_runs DESC; (ShopDB)\n-- Monthly business dashboard: -- Orders, revenue, average order value, and running revenue total SELECT EXTRACT(MONTH FROM o.order_date) AS month, COUNT(DISTINCT o.id) AS total_orders, COUNT(DISTINCT o.id) FILTER (WHERE o.status = \u0026#39;Delivered\u0026#39;) AS delivered_orders, SUM(oi.quantity * oi.unit_price) AS gross_revenue, ROUND( SUM(oi.quantity * oi.unit_price) / COUNT(DISTINCT o.id), 2 ) AS avg_order_value, SUM(SUM(oi.quantity * oi.unit_price)) OVER (ORDER BY EXTRACT(MONTH FROM o.order_date)) AS running_revenue FROM orders AS o JOIN order_items AS oi ON oi.order_id = o.id WHERE o.status != \u0026#39;Cancelled\u0026#39; GROUP BY month ORDER BY month; Study both queries carefully. Every clause in them was introduced in this lesson or earlier ones. This is the kind of query you will write regularly when working with real data.\nPart 8 - Practice Set Answers Answers: Practice Set 1 Q1. Performance by match result:\nSELECT m.result, SUM(p.runs) AS total_runs, SUM(p.wickets) AS total_wickets, COUNT(*) AS innings FROM performances AS p JOIN matches AS m ON p.match_id = m.id GROUP BY m.result ORDER BY total_runs DESC; Q2. Per player per result:\nSELECT pl.name, m.result, SUM(p.runs) AS total_runs, SUM(p.wickets) AS total_wickets FROM performances AS p JOIN players AS pl ON p.player_id = pl.id JOIN matches AS m ON p.match_id = m.id GROUP BY pl.id, pl.name, m.result ORDER BY pl.name, m.result; Q3. Customers per state per month:\nSELECT state, EXTRACT(MONTH FROM joined_on) AS month, COUNT(*) AS customer_count FROM customers GROUP BY state, month ORDER BY state, month; Q4. Total quantity sold per category:\nSELECT cat.name AS category, SUM(oi.quantity) AS total_units_sold FROM order_items AS oi JOIN orders AS o ON oi.order_id = o.id JOIN products AS p ON oi.product_id = p.id JOIN categories AS cat ON p.category_id = cat.id WHERE o.status != \u0026#39;Cancelled\u0026#39; GROUP BY cat.id, cat.name ORDER BY total_units_sold DESC; Answers: Practice Set 2 Q5. Wicket-takers in 2+ innings with 4+ total wickets:\nSELECT pl.name, COUNT(*) FILTER (WHERE p.wickets \u0026gt; 0) AS innings_with_wickets, SUM(p.wickets) AS total_wickets FROM performances AS p JOIN players AS pl ON p.player_id = pl.id GROUP BY pl.id, pl.name HAVING COUNT(*) FILTER (WHERE p.wickets \u0026gt; 0) \u0026gt;= 2 AND SUM(p.wickets) \u0026gt;= 4 ORDER BY total_wickets DESC; Q6. 100+ runs AND played in both Won and Lost matches:\nSELECT pl.name, SUM(p.runs) AS total_runs, COUNT(DISTINCT m.result) AS distinct_results FROM performances AS p JOIN players AS pl ON p.player_id = pl.id JOIN matches AS m ON p.match_id = m.id GROUP BY pl.id, pl.name HAVING SUM(p.runs) \u0026gt; 100 AND COUNT(DISTINCT m.result) \u0026gt;= 2 ORDER BY total_runs DESC; Q7. States with 3+ orders:\nSELECT c.state, COUNT(o.id) AS order_count FROM customers AS c JOIN orders AS o ON o.customer_id = c.id GROUP BY c.state HAVING COUNT(o.id) \u0026gt;= 3 ORDER BY order_count DESC; Q8. Products ordered more than once:\nSELECT p.name, COUNT(oi.id) AS times_ordered, SUM(oi.quantity) AS total_quantity_sold FROM order_items AS oi JOIN products AS p ON oi.product_id = p.id GROUP BY p.id, p.name HAVING COUNT(oi.id) \u0026gt; 1 ORDER BY times_ordered DESC; Answers: Practice Set 3 Q9. Player rank within each format:\nSELECT pl.name, m.format, p.runs, RANK() OVER ( PARTITION BY m.format ORDER BY p.runs DESC ) AS rank_in_format FROM performances AS p JOIN players AS pl ON p.player_id = pl.id JOIN matches AS m ON p.match_id = m.id ORDER BY m.format, rank_in_format; Q10. Top wicket-taker per match:\nSELECT name, opponent, wickets FROM ( SELECT pl.name, m.opponent, p.wickets, ROW_NUMBER() OVER ( PARTITION BY p.match_id ORDER BY p.wickets DESC ) AS rn FROM performances AS p JOIN players AS pl ON p.player_id = pl.id JOIN matches AS m ON p.match_id = m.id ) AS ranked WHERE rn = 1 ORDER BY wickets DESC; Q11. Order revenue with running total:\nSELECT o.id AS order_id, o.order_date, SUM(oi.quantity * oi.unit_price) AS order_revenue, SUM(SUM(oi.quantity * oi.unit_price)) OVER (ORDER BY o.order_date) AS running_revenue FROM orders AS o JOIN order_items AS oi ON oi.order_id = o.id GROUP BY o.id, o.order_date ORDER BY o.order_date; Q12. Product price vs category average:\nSELECT cat.name AS category, p.name AS product, p.price_inr, ROUND( AVG(p.price_inr) OVER (PARTITION BY p.category_id), 2 ) AS category_avg_price FROM products AS p JOIN categories AS cat ON p.category_id = cat.id ORDER BY cat.name, p.price_inr DESC; What\u0026rsquo;s Next You have covered:\n✅ Multi-column GROUP BY - group by two or more columns ✅ GROUP BY with JOIN - aggregate across related tables ✅ HAVING with multiple conditions - filter groups on complex criteria ✅ FILTER clause - clean conditional aggregation ✅ ROLLUP - automatic subtotals and grand totals ✅ ROW_NUMBER - unique row numbering within partitions ✅ RANK / DENSE_RANK - ranking with tie-handling ✅ SUM OVER - running totals without collapsing rows ✅ The subquery + window function pattern - filter on a computed rank In Lesson 2.4 we go deep on JOINs \u0026amp; Relationships:\nINNER, LEFT, RIGHT, and FULL OUTER JOIN compared side by side Self-joins - a table joined to itself JOINs across 4 tables Common JOIN mistakes and how to spot them When to use a JOIN vs a subquery All on ShopDB with a full 4-table query at the end ","permalink":"https://blogs.codeverra.com/blog/sql-part3/","summary":"\u003ch1 id=\"lesson-3---aggregations--grouping\"\u003eLesson 3 - Aggregations \u0026amp; Grouping\u003c/h1\u003e\n\u003ch3 id=\"theory--practice--cricketdb--shopdb\"\u003eTheory + Practice | CricketDB + ShopDB\u003c/h3\u003e\n\u003chr\u003e\n\u003ch2 id=\"before-you-start\"\u003eBefore You Start\u003c/h2\u003e\n\u003cp\u003eThis lesson uses both databases you have already built:\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003e\u003cstrong\u003eCricketDB\u003c/strong\u003e  -  players, matches, performances (Lesson 2.1)\u003c/li\u003e\n\u003cli\u003e\u003cstrong\u003eShopDB\u003c/strong\u003e  -  customers, products, orders, order_items (Lesson 2.2)\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003eRun the \u003ccode\u003esetup_check.sql\u003c/code\u003e from this lesson folder to confirm both are ready.\nEach section tells you which database to connect to in pgAdmin.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eWhat you will learn in this lesson:\u003c/strong\u003e\u003c/p\u003e","tags":["sql","data-analysis"],"title":"Group by and Aggregations"},{"content":"Python Higher-Order Functions \u0026amp; Decorators A Complete Reference Guide Table of Contents The Big Picture Functions as First-Class Objects Higher-Order Functions Closures - The Foundation of Decorators Decorators - Core Concept The @ Syntax Preserving Metadata with functools.wraps Decorators with Arguments Class-Based Decorators Stacking Multiple Decorators Built-in Decorators Practical Decorator Patterns Common Mistakes to Avoid Practice Questions \u0026amp; Answers Quick Reference Cheat Sheet The Big Picture ┌─────────────────────────────────────────────────────────────────────┐ │ THE DECORATOR MENTAL MODEL │ │ │ │ A decorator is a function that WRAPS another function │ │ to add behaviour before, after, or around it. │ │ │ │ Original Function │ │ ┌────────────────┐ │ │ │ do_thing() │ │ │ └────────────────┘ │ │ │ decorator applied │ │ ▼ │ │ ┌────────────────────────────────────┐ │ │ │ wrapper() │ │ │ │ ┌──────────────────────────────┐ │ │ │ │ │ before logic (logging etc) │ │ │ │ │ ├──────────────────────────────┤ │ │ │ │ │ do_thing() ← called │ │ │ │ │ ├──────────────────────────────┤ │ │ │ │ │ after logic (timing etc) │ │ │ │ │ └──────────────────────────────┘ │ │ │ └────────────────────────────────────┘ │ │ │ │ @decorator is sugar for: func = decorator(func) │ └─────────────────────────────────────────────────────────────────────┘ Core Idea: A decorator is just a callable that takes a function and returns a new callable. Everything else - the @ syntax, functools.wraps, parameterised decorators - builds on top of this single idea.\nFunctions as First-Class Objects In Python, functions are first-class objects. This means a function can be:\nAssigned to a variable Passed as an argument to another function Returned from a function Stored in data structures This is the foundational property that makes higher-order functions and decorators possible.\n# 1. Assigned to a variable def greet(name): return f\u0026#34;Hello, {name}!\u0026#34; say_hello = greet # no parentheses - we assign the function itself print(say_hello(\u0026#34;Priya\u0026#34;)) # Hello, Priya! print(greet is say_hello) # True - same object # 2. Passed as an argument def apply(func, value): return func(value) print(apply(str.upper, \u0026#34;hello\u0026#34;)) # HELLO print(apply(len, [1, 2, 3, 4])) # 4 # 3. Returned from a function def make_multiplier(n): def multiplier(x): return x * n return multiplier # returning the inner function itself triple = make_multiplier(3) print(triple(10)) # 30 print(triple(7)) # 21 # 4. Stored in a data structure operations = { \u0026#34;double\u0026#34;: lambda x: x * 2, \u0026#34;square\u0026#34;: lambda x: x ** 2, \u0026#34;negate\u0026#34;: lambda x: -x, } print(operations[\u0026#34;square\u0026#34;](5)) # 25 Higher-Order Functions A Higher-Order Function (HOF) is a function that either:\nTakes one or more functions as arguments, OR Returns a function as its result (or both) Python\u0026rsquo;s Built-in HOFs map(func, iterable) - Apply a function to every element numbers = [1, 2, 3, 4, 5] # map returns a lazy iterator squares = list(map(lambda x: x**2, numbers)) print(squares) # [1, 4, 9, 16, 25] # With a named function def to_rupees(usd): return round(usd * 83.5, 2) prices_usd = [10.0, 25.5, 4.99, 100.0] prices_inr = list(map(to_rupees, prices_usd)) print(prices_inr) # [835.0, 2129.25, 416.67, 8350.0] # map with multiple iterables a = [1, 2, 3] b = [10, 20, 30] sums = list(map(lambda x, y: x + y, a, b)) print(sums) # [11, 22, 33] filter(func, iterable) - Keep elements where func returns True numbers = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10] evens = list(filter(lambda x: x % 2 == 0, numbers)) print(evens) # [2, 4, 6, 8, 10] # Filter with None - removes falsy values mixed = [0, 1, \u0026#34;\u0026#34;, \u0026#34;hello\u0026#34;, None, [1, 2], [], False, True] truthy_only = list(filter(None, mixed)) print(truthy_only) # [1, \u0026#39;hello\u0026#39;, [1, 2], True] # Real example: filter active users users = [ {\u0026#34;name\u0026#34;: \u0026#34;Ravi\u0026#34;, \u0026#34;active\u0026#34;: True}, {\u0026#34;name\u0026#34;: \u0026#34;Priya\u0026#34;, \u0026#34;active\u0026#34;: False}, {\u0026#34;name\u0026#34;: \u0026#34;Arjun\u0026#34;, \u0026#34;active\u0026#34;: True}, ] active_users = list(filter(lambda u: u[\u0026#34;active\u0026#34;], users)) print([u[\u0026#34;name\u0026#34;] for u in active_users]) # [\u0026#39;Ravi\u0026#39;, \u0026#39;Arjun\u0026#39;] reduce(func, iterable) - Fold a sequence into a single value from functools import reduce numbers = [1, 2, 3, 4, 5] total = reduce(lambda acc, x: acc + x, numbers) print(total) # 15 product = reduce(lambda acc, x: acc * x, numbers) print(product) # 120 # With an initial value total_with_offset = reduce(lambda acc, x: acc + x, numbers, 100) print(total_with_offset) # 115 # Step by step - seeing how reduce works: # Step 1: acc=1, x=2 → 3 # Step 2: acc=3, x=3 → 6 # Step 3: acc=6, x=4 → 10 # Step 4: acc=10, x=5 → 15 # Real example: building a dict from a list pairs = [(\u0026#34;name\u0026#34;, \u0026#34;Ravi\u0026#34;), (\u0026#34;age\u0026#34;, 28), (\u0026#34;city\u0026#34;, \u0026#34;Hyderabad\u0026#34;)] record = reduce(lambda d, kv: {**d, kv[0]: kv[1]}, pairs, {}) print(record) # {\u0026#39;name\u0026#39;: \u0026#39;Ravi\u0026#39;, \u0026#39;age\u0026#39;: 28, \u0026#39;city\u0026#39;: \u0026#39;Hyderabad\u0026#39;} sorted() with a key function players = [ {\u0026#34;name\u0026#34;: \u0026#34;Rohit\u0026#34;, \u0026#34;runs\u0026#34;: 320}, {\u0026#34;name\u0026#34;: \u0026#34;Virat\u0026#34;, \u0026#34;runs\u0026#34;: 450}, {\u0026#34;name\u0026#34;: \u0026#34;Dhoni\u0026#34;, \u0026#34;runs\u0026#34;: 280}, {\u0026#34;name\u0026#34;: \u0026#34;Hardik\u0026#34;, \u0026#34;runs\u0026#34;: 190}, ] # Sort by runs descending by_runs = sorted(players, key=lambda p: p[\u0026#34;runs\u0026#34;], reverse=True) for p in by_runs: print(f\u0026#34;{p[\u0026#39;name\u0026#39;]}: {p[\u0026#39;runs\u0026#39;]}\u0026#34;) # Virat: 450 → Rohit: 320 → Dhoni: 280 → Hardik: 190 # Sort by string length words = [\u0026#34;banana\u0026#34;, \u0026#34;fig\u0026#34;, \u0026#34;kiwi\u0026#34;, \u0026#34;apple\u0026#34;, \u0026#34;mango\u0026#34;] by_length = sorted(words, key=len) print(by_length) # [\u0026#39;fig\u0026#39;, \u0026#39;kiwi\u0026#39;, \u0026#39;apple\u0026#39;, \u0026#39;mango\u0026#39;, \u0026#39;banana\u0026#39;] # Multi-key sort using operator.attrgetter / itemgetter from operator import itemgetter data = [(\u0026#34;Alice\u0026#34;, 30), (\u0026#34;Bob\u0026#34;, 25), (\u0026#34;Charlie\u0026#34;, 30), (\u0026#34;Dave\u0026#34;, 25)] sorted_data = sorted(data, key=itemgetter(1, 0)) # sort by age, then name print(sorted_data) # [(\u0026#39;Bob\u0026#39;, 25), (\u0026#39;Dave\u0026#39;, 25), (\u0026#39;Alice\u0026#39;, 30), (\u0026#39;Charlie\u0026#39;, 30)] Writing Your Own HOFs # HOF that returns a function - a \u0026#34;factory\u0026#34; def make_validator(min_val, max_val): \u0026#34;\u0026#34;\u0026#34;Returns a function that checks if a value is in range.\u0026#34;\u0026#34;\u0026#34; def validator(value): return min_val \u0026lt;= value \u0026lt;= max_val return validator is_valid_age = make_validator(0, 120) is_valid_score = make_validator(0, 100) is_valid_pin = make_validator(1000, 9999) print(is_valid_age(25)) # True print(is_valid_age(200)) # False print(is_valid_score(95.5)) # True print(is_valid_pin(9999)) # True # HOF that takes a function - a \u0026#34;pipeline\u0026#34; builder def pipeline(*funcs): \u0026#34;\u0026#34;\u0026#34;Returns a function that passes data through all funcs in sequence.\u0026#34;\u0026#34;\u0026#34; def execute(data): result = data for func in funcs: result = func(result) return result return execute process = pipeline( str.strip, str.lower, lambda s: s.replace(\u0026#34; \u0026#34;, \u0026#34;_\u0026#34;), ) print(process(\u0026#34; Hello World \u0026#34;)) # hello_world print(process(\u0026#34; PYTHON Rocks \u0026#34;)) # python_rocks Closures - The Foundation of Decorators A closure is a function that remembers the variables from its enclosing scope even after that scope has finished executing. Decorators are closures.\ndef outer(x): # x is a \u0026#34;free variable\u0026#34; - it lives in outer\u0026#39;s scope def inner(y): return x + y # inner \u0026#34;closes over\u0026#34; x return inner add_five = outer(5) add_ten = outer(10) print(add_five(3)) # 8 - x=5 is remembered print(add_ten(3)) # 13 - x=10 is remembered print(add_five(3)) # 8 - still 8, x=5 is not gone Inspecting a Closure def make_counter(start=0): count = start def counter(): nonlocal count count += 1 return count return counter c = make_counter() print(c()) # 1 print(c()) # 2 print(c()) # 3 # Inspect the closure\u0026#39;s free variables print(c.__code__.co_freevars) # (\u0026#39;count\u0026#39;,) print(c.__closure__[0].cell_contents) # 3 ← current value of count nonlocal - Modifying an Enclosing Variable def make_accumulator(): total = 0 def add(value): nonlocal total # tells Python: total is NOT local to add() total += value return total return add acc = make_accumulator() print(acc(10)) # 10 print(acc(20)) # 30 print(acc(5)) # 35 Decorators - Core Concept A decorator is a function that wraps another function, adding behaviour without modifying the original function\u0026rsquo;s source code. It follows this exact pattern:\ndef my_decorator(func): # 1. Takes a function def wrapper(*args, **kwargs): # 2. Defines a wrapper # before logic result = func(*args, **kwargs) # 3. Calls the original # after logic return result return wrapper # 4. Returns the wrapper Manual Application (Without @) def shout(func): def wrapper(*args, **kwargs): result = func(*args, **kwargs) return result.upper() return wrapper def greet(name): return f\u0026#34;hello, {name}!\u0026#34; # Manually decorating greet = shout(greet) print(greet(\u0026#34;ravi\u0026#34;)) # HELLO, RAVI! The @ Syntax The @decorator syntax is syntactic sugar for func = decorator(func). Both are exactly equivalent.\ndef shout(func): def wrapper(*args, **kwargs): result = func(*args, **kwargs) return result.upper() return wrapper # Using @ - clean, readable @shout def greet(name): return f\u0026#34;hello, {name}!\u0026#34; print(greet(\u0026#34;priya\u0026#34;)) # HELLO, PRIYA! # This is EXACTLY the same as: # greet = shout(greet) A Timing Decorator - Classic Example import time def timer(func): \u0026#34;\u0026#34;\u0026#34;Prints how long a function took to execute.\u0026#34;\u0026#34;\u0026#34; def wrapper(*args, **kwargs): start = time.perf_counter() result = func(*args, **kwargs) end = time.perf_counter() print(f\u0026#34;{func.__name__} took {end - start:.6f}s\u0026#34;) return result return wrapper @timer def slow_sum(n): return sum(range(n)) result = slow_sum(10_000_000) # slow_sum took 0.341203s A Logging Decorator import functools def log_calls(func): \u0026#34;\u0026#34;\u0026#34;Logs every call to the decorated function.\u0026#34;\u0026#34;\u0026#34; @functools.wraps(func) # more on this next section def wrapper(*args, **kwargs): args_repr = [repr(a) for a in args] kwargs_repr = [f\u0026#34;{k}={v!r}\u0026#34; for k, v in kwargs.items()] signature = \u0026#34;, \u0026#34;.join(args_repr + kwargs_repr) print(f\u0026#34;Calling {func.__name__}({signature})\u0026#34;) result = func(*args, **kwargs) print(f\u0026#34;{func.__name__} returned {result!r}\u0026#34;) return result return wrapper @log_calls def add(a, b): return a + b add(3, 5) # Calling add(3, 5) # add returned 8 add(a=10, b=20) # Calling add(a=10, b=20) # add returned 30 Preserving Metadata with functools.wraps Without functools.wraps, decorating a function replaces its identity - its name, docstring, and other metadata are overwritten by the wrapper.\nimport functools # WITHOUT functools.wraps - identity is lost def bad_decorator(func): def wrapper(*args, **kwargs): return func(*args, **kwargs) return wrapper @bad_decorator def important_function(): \u0026#34;\u0026#34;\u0026#34;This function does something important.\u0026#34;\u0026#34;\u0026#34; pass print(important_function.__name__) # \u0026#39;wrapper\u0026#39; ← WRONG print(important_function.__doc__) # None ← WRONG # WITH functools.wraps - identity is preserved def good_decorator(func): @functools.wraps(func) # copies __name__, __doc__, __module__, __qualname__, __annotations__ def wrapper(*args, **kwargs): return func(*args, **kwargs) return wrapper @good_decorator def important_function(): \u0026#34;\u0026#34;\u0026#34;This function does something important.\u0026#34;\u0026#34;\u0026#34; pass print(important_function.__name__) # \u0026#39;important_function\u0026#39; ← correct print(important_function.__doc__) # \u0026#39;This function does something important.\u0026#39; print(important_function.__wrapped__) # \u0026lt;function important_function ...\u0026gt; ← original Rule: Always use @functools.wraps(func) inside your decorator\u0026rsquo;s wrapper. No exceptions.\nDecorators with Arguments Sometimes you want to parameterize a decorator: @retry(times=3). This requires one more layer of nesting - a factory function that returns the actual decorator.\n# Structure: factory → decorator → wrapper def repeat(times): # ← factory: takes the argument def decorator(func): # ← actual decorator: takes the function @functools.wraps(func) def wrapper(*args, **kwargs): # ← wrapper: called each time for _ in range(times): result = func(*args, **kwargs) return result return wrapper return decorator @repeat(times=3) def say_hi(name): print(f\u0026#34;Hi, {name}!\u0026#34;) say_hi(\u0026#34;Arjun\u0026#34;) # Hi, Arjun! # Hi, Arjun! # Hi, Arjun! Parameterized Timer import time import functools def timer(unit=\u0026#34;s\u0026#34;): \u0026#34;\u0026#34;\u0026#34;Decorator factory: measures time in \u0026#39;s\u0026#39; or \u0026#39;ms\u0026#39;.\u0026#34;\u0026#34;\u0026#34; def decorator(func): @functools.wraps(func) def wrapper(*args, **kwargs): start = time.perf_counter() result = func(*args, **kwargs) elapsed = time.perf_counter() - start if unit == \u0026#34;ms\u0026#34;: print(f\u0026#34;{func.__name__}: {elapsed * 1000:.2f}ms\u0026#34;) else: print(f\u0026#34;{func.__name__}: {elapsed:.6f}s\u0026#34;) return result return wrapper return decorator @timer(unit=\u0026#34;ms\u0026#34;) def fast_function(): return sum(range(100_000)) @timer(unit=\u0026#34;s\u0026#34;) def slow_function(): return sum(range(10_000_000)) fast_function() # fast_function: 4.21ms slow_function() # slow_function: 0.341203s Retry Decorator with Parameters import functools import time def retry(max_attempts=3, delay=1.0, exceptions=(Exception,)): \u0026#34;\u0026#34;\u0026#34; Retry a function on failure. max_attempts: total tries before giving up delay: seconds to wait between attempts exceptions: tuple of exception types to retry on \u0026#34;\u0026#34;\u0026#34; def decorator(func): @functools.wraps(func) def wrapper(*args, **kwargs): last_err = None for attempt in range(1, max_attempts + 1): try: return func(*args, **kwargs) except exceptions as e: last_err = e print(f\u0026#34;[retry] {func.__name__}: attempt {attempt} failed: {e}\u0026#34;) if attempt \u0026lt; max_attempts: time.sleep(delay) raise last_err return wrapper return decorator count = 0 @retry(max_attempts=3, delay=0, exceptions=(ValueError,)) def flaky(): global count count += 1 if count \u0026lt; 3: raise ValueError(\u0026#34;not ready\u0026#34;) return \u0026#34;done\u0026#34; print(flaky()) # [retry] flaky: attempt 1 failed: not ready # [retry] flaky: attempt 2 failed: not ready # done Class-Based Decorators You can implement a decorator as a class by implementing __call__. This is useful when the decorator needs to maintain state across calls.\nimport functools class CountCalls: \u0026#34;\u0026#34;\u0026#34;Counts how many times the decorated function has been called.\u0026#34;\u0026#34;\u0026#34; def __init__(self, func): functools.update_wrapper(self, func) # preserves metadata self.func = func self.count = 0 def __call__(self, *args, **kwargs): self.count += 1 print(f\u0026#34;{self.func.__name__} has been called {self.count} time(s)\u0026#34;) return self.func(*args, **kwargs) @CountCalls def process(x): return x * 2 process(5) # process has been called 1 time(s) process(10) # process has been called 2 time(s) process(15) # process has been called 3 time(s) print(process.count) # 3 Class-Based Decorator with Arguments class RateLimit: \u0026#34;\u0026#34;\u0026#34;Limits a function to `calls_per_second` calls per second.\u0026#34;\u0026#34;\u0026#34; def __init__(self, calls_per_second): self.min_interval = 1.0 / calls_per_second self.last_called = 0.0 def __call__(self, func): @functools.wraps(func) def wrapper(*args, **kwargs): now = time.time() elapsed = now - self.last_called if elapsed \u0026lt; self.min_interval: time.sleep(self.min_interval - elapsed) self.last_called = time.time() return func(*args, **kwargs) return wrapper @RateLimit(calls_per_second=2) def api_call(endpoint): print(f\u0026#34;Calling {endpoint}\u0026#34;) api_call(\u0026#34;/users\u0026#34;) api_call(\u0026#34;/orders\u0026#34;) # automatically waits 0.5s before calling Stacking Multiple Decorators Decorators stack from bottom to top (innermost first). The bottom decorator wraps the function first, then the one above wraps that result.\nimport functools def bold(func): @functools.wraps(func) def wrapper(*args, **kwargs): return f\u0026#34;\u0026lt;b\u0026gt;{func(*args, **kwargs)}\u0026lt;/b\u0026gt;\u0026#34; return wrapper def italic(func): @functools.wraps(func) def wrapper(*args, **kwargs): return f\u0026#34;\u0026lt;i\u0026gt;{func(*args, **kwargs)}\u0026lt;/i\u0026gt;\u0026#34; return wrapper def underline(func): @functools.wraps(func) def wrapper(*args, **kwargs): return f\u0026#34;\u0026lt;u\u0026gt;{func(*args, **kwargs)}\u0026lt;/u\u0026gt;\u0026#34; return wrapper @bold @italic @underline def text(content): return content print(text(\u0026#34;Hello\u0026#34;)) # \u0026lt;b\u0026gt;\u0026lt;i\u0026gt;\u0026lt;u\u0026gt;Hello\u0026lt;/u\u0026gt;\u0026lt;/i\u0026gt;\u0026lt;/b\u0026gt; # Execution order explained: # @bold → bold(italic(underline(text))) # @italic → italic(underline(text)) # @underline → underline(text) ← applied first (innermost) # # When called: bold wrapper runs → italic wrapper runs → underline wrapper runs → text() Practical Stack: Auth + Logging + Timing import functools, time def require_auth(func): @functools.wraps(func) def wrapper(*args, **kwargs): user = kwargs.get(\u0026#34;user\u0026#34;) if not user or not user.get(\u0026#34;authenticated\u0026#34;): raise PermissionError(f\u0026#34;Authentication required to call {func.__name__}\u0026#34;) return func(*args, **kwargs) return wrapper def log_call(func): @functools.wraps(func) def wrapper(*args, **kwargs): print(f\u0026#34;[LOG] {func.__name__} called\u0026#34;) result = func(*args, **kwargs) print(f\u0026#34;[LOG] {func.__name__} completed\u0026#34;) return result return wrapper def measure_time(func): @functools.wraps(func) def wrapper(*args, **kwargs): start = time.perf_counter() result = func(*args, **kwargs) print(f\u0026#34;[TIME] {func.__name__}: {time.perf_counter() - start:.4f}s\u0026#34;) return result return wrapper @measure_time @log_call @require_auth def get_dashboard(user=None): return {\u0026#34;data\u0026#34;: \u0026#34;dashboard content\u0026#34;} # Test it get_dashboard(user={\u0026#34;name\u0026#34;: \u0026#34;Ravi\u0026#34;, \u0026#34;authenticated\u0026#34;: True}) # [LOG] get_dashboard called # [LOG] get_dashboard completed # [TIME] get_dashboard: 0.0001s Built-in Decorators Python ships with several powerful built-in decorators.\n@staticmethod - No implicit first argument class MathHelper: @staticmethod def add(a, b): return a + b @staticmethod def is_even(n): return n % 2 == 0 # Can be called on the class OR an instance print(MathHelper.add(3, 5)) # 8 print(MathHelper().is_even(4)) # True # No `self` or `cls` passed @classmethod - Receives the class as first argument class Employee: company = \u0026#34;Codeverra\u0026#34; def __init__(self, name, salary): self.name = name self.salary = salary @classmethod def from_dict(cls, data): \u0026#34;\u0026#34;\u0026#34;Alternative constructor - creates instance from a dict.\u0026#34;\u0026#34;\u0026#34; return cls(data[\u0026#34;name\u0026#34;], data[\u0026#34;salary\u0026#34;]) @classmethod def change_company(cls, new_name): cls.company = new_name def __repr__(self): return f\u0026#34;Employee({self.name}, {self.salary}, {self.company})\u0026#34; emp = Employee.from_dict({\u0026#34;name\u0026#34;: \u0026#34;Priya\u0026#34;, \u0026#34;salary\u0026#34;: 75000}) print(emp) # Employee(Priya, 75000, Codeverra) Employee.change_company(\u0026#34;Codeverra Pro\u0026#34;) print(emp) # Employee(Priya, 75000, Codeverra Pro) @property - Computed attributes class Circle: def __init__(self, radius): self._radius = radius @property def radius(self): return self._radius @radius.setter def radius(self, value): if value \u0026lt; 0: raise ValueError(\u0026#34;Radius cannot be negative\u0026#34;) self._radius = value @radius.deleter def radius(self): del self._radius @property def area(self): import math return math.pi * self._radius ** 2 @property def circumference(self): import math return 2 * math.pi * self._radius c = Circle(5) print(c.radius) # 5 print(f\u0026#34;{c.area:.2f}\u0026#34;) # 78.54 c.radius = 10 # calls the setter print(c.area) # 314.159... c.radius = -1 # ValueError: Radius cannot be negative @functools.lru_cache - Memoization import functools # Without cache - exponential time def fib_slow(n): if n \u0026lt; 2: return n return fib_slow(n - 1) + fib_slow(n - 2) # With lru_cache - linear time (results cached in memory) @functools.lru_cache(maxsize=None) def fib(n): if n \u0026lt; 2: return n return fib(n - 1) + fib(n - 2) print(fib(50)) # 12586269025 - instant print(fib.cache_info()) # CacheInfo(hits=48, misses=51, maxsize=None, currsize=51) fib.cache_clear() # clear the cache # Python 3.9+: @functools.cache (unbounded, simpler) @functools.cache def factorial(n): return 1 if n \u0026lt;= 1 else n * factorial(n - 1) @functools.cached_property - Property computed once per instance import functools class DataSet: def __init__(self, data): self.data = data @functools.cached_property def stats(self): print(\u0026#34;Computing stats...\u0026#34;) # only runs once! import statistics return { \u0026#34;mean\u0026#34;: statistics.mean(self.data), \u0026#34;stdev\u0026#34;: statistics.stdev(self.data), \u0026#34;median\u0026#34;: statistics.median(self.data), } ds = DataSet([10, 20, 30, 40, 50]) print(ds.stats) # Computing stats... → {\u0026#39;mean\u0026#39;: 30, ...} print(ds.stats) # Cached - does NOT print \u0026#34;Computing stats...\u0026#34; again Practical Decorator Patterns Pattern 1 - Access Control / Authorization import functools def requires_role(*allowed_roles): \u0026#34;\u0026#34;\u0026#34;Decorator factory that restricts access by user role.\u0026#34;\u0026#34;\u0026#34; def decorator(func): @functools.wraps(func) def wrapper(*args, **kwargs): current_user = kwargs.get(\u0026#34;user\u0026#34;) or (args[0] if args else None) if not current_user: raise PermissionError(\u0026#34;No user provided\u0026#34;) if current_user.get(\u0026#34;role\u0026#34;) not in allowed_roles: raise PermissionError( f\u0026#34;User role \u0026#39;{current_user.get(\u0026#39;role\u0026#39;)}\u0026#39; not in {allowed_roles}\u0026#34; ) return func(*args, **kwargs) return wrapper return decorator @requires_role(\u0026#34;admin\u0026#34;, \u0026#34;superuser\u0026#34;) def delete_record(user=None, record_id=None): return f\u0026#34;Record {record_id} deleted\u0026#34; @requires_role(\u0026#34;admin\u0026#34;, \u0026#34;editor\u0026#34;, \u0026#34;viewer\u0026#34;) def view_report(user=None): return \u0026#34;Here is the report\u0026#34; # Test admin = {\u0026#34;name\u0026#34;: \u0026#34;Ravi\u0026#34;, \u0026#34;role\u0026#34;: \u0026#34;admin\u0026#34;} viewer = {\u0026#34;name\u0026#34;: \u0026#34;Priya\u0026#34;, \u0026#34;role\u0026#34;: \u0026#34;viewer\u0026#34;} print(delete_record(user=admin, record_id=42)) # Record 42 deleted print(view_report(user=viewer)) # Here is the report try: delete_record(user=viewer, record_id=99) # PermissionError except PermissionError as e: print(e) # User role \u0026#39;viewer\u0026#39; not in (\u0026#39;admin\u0026#39;, \u0026#39;superuser\u0026#39;) Pattern 2 - Caching / Memoization (Custom) import functools import time def timed_cache(seconds): \u0026#34;\u0026#34;\u0026#34;Cache results for a given number of seconds.\u0026#34;\u0026#34;\u0026#34; def decorator(func): cache = {} expiry = {} @functools.wraps(func) def wrapper(*args): now = time.time() if args in cache and now \u0026lt; expiry[args]: print(f\u0026#34;[cache hit] {func.__name__}{args}\u0026#34;) return cache[args] result = func(*args) cache[args] = result expiry[args] = now + seconds return result return wrapper return decorator @timed_cache(seconds=5) def get_exchange_rate(currency): print(f\u0026#34;[API call] fetching rate for {currency}\u0026#34;) return {\u0026#34;USD\u0026#34;: 83.5, \u0026#34;EUR\u0026#34;: 90.2}.get(currency, 1.0) print(get_exchange_rate(\u0026#34;USD\u0026#34;)) # [API call] → 83.5 print(get_exchange_rate(\u0026#34;USD\u0026#34;)) # [cache hit] → 83.5 time.sleep(0.1) print(get_exchange_rate(\u0026#34;USD\u0026#34;)) # [cache hit] → 83.5 Pattern 3 - Input Validation import functools def validate_types(**type_map): \u0026#34;\u0026#34;\u0026#34; Validates argument types at call time. Usage: @validate_types(name=str, age=int, score=float) \u0026#34;\u0026#34;\u0026#34; def decorator(func): @functools.wraps(func) def wrapper(*args, **kwargs): # Bind arguments to parameter names import inspect bound = inspect.signature(func).bind(*args, **kwargs) bound.apply_defaults() for param, expected_type in type_map.items(): if param in bound.arguments: value = bound.arguments[param] if not isinstance(value, expected_type): raise TypeError( f\u0026#34;Argument \u0026#39;{param}\u0026#39; must be {expected_type.__name__}, \u0026#34; f\u0026#34;got {type(value).__name__}\u0026#34; ) return func(*args, **kwargs) return wrapper return decorator @validate_types(name=str, age=int, score=float) def register_student(name, age, score): return f\u0026#34;Registered: {name}, age={age}, score={score}\u0026#34; print(register_student(\u0026#34;Arjun\u0026#34;, 22, 95.5)) # OK register_student(123, 22, 95.5) # TypeError: \u0026#39;name\u0026#39; must be str Pattern 4 - Deprecation Warning import functools import warnings def deprecated(reason=\u0026#34;\u0026#34;): \u0026#34;\u0026#34;\u0026#34;Marks a function as deprecated and warns when it is called.\u0026#34;\u0026#34;\u0026#34; def decorator(func): @functools.wraps(func) def wrapper(*args, **kwargs): msg = f\u0026#34;{func.__name__} is deprecated.\u0026#34; if reason: msg += f\u0026#34; {reason}\u0026#34; warnings.warn(msg, DeprecationWarning, stacklevel=2) return func(*args, **kwargs) return wrapper return decorator @deprecated(reason=\u0026#34;Use calculate_v2() instead.\u0026#34;) def calculate(x, y): return x + y result = calculate(3, 4) # DeprecationWarning: calculate is deprecated. Use calculate_v2() instead. Pattern 5 - Singleton Pattern import functools def singleton(cls): \u0026#34;\u0026#34;\u0026#34;Ensures only one instance of a class is ever created.\u0026#34;\u0026#34;\u0026#34; instances = {} @functools.wraps(cls) def get_instance(*args, **kwargs): if cls not in instances: instances[cls] = cls(*args, **kwargs) return instances[cls] return get_instance @singleton class DatabaseConnection: def __init__(self, host=\u0026#34;localhost\u0026#34;): self.host = host print(f\u0026#34;Connecting to {host}...\u0026#34;) db1 = DatabaseConnection(\u0026#34;prod-server\u0026#34;) # Connecting to prod-server... db2 = DatabaseConnection(\u0026#34;other-server\u0026#34;) # NOT printed - returns existing instance print(db1 is db2) # True - same object print(db2.host) # prod-server Common Mistakes to Avoid Mistake 1 - Forgetting functools.wraps # BAD def my_decorator(func): def wrapper(*args, **kwargs): return func(*args, **kwargs) return wrapper # wrapper\u0026#39;s name and docstring replace func\u0026#39;s # GOOD import functools def my_decorator(func): @functools.wraps(func) def wrapper(*args, **kwargs): return func(*args, **kwargs) return wrapper Mistake 2 - Calling the Function When Applying the Decorator def greet(): return \u0026#34;Hello!\u0026#34; def my_decorator(func): @functools.wraps(func) def wrapper(): return func().upper() return wrapper # BAD - calling greet() at decoration time, not at call time @my_decorator def greet(): # This is fine, but... return \u0026#34;Hello!\u0026#34; greet_result = my_decorator(greet()) # WRONG - passing the result \u0026#34;Hello!\u0026#34;, not the function # GOOD @my_decorator # pass the function object, without () def greet(): return \u0026#34;Hello!\u0026#34; Mistake 3 - Forgetting *args, **kwargs in the Wrapper # BAD - wrapper only works for functions with no arguments def timer(func): def wrapper(): # only works for zero-argument functions start = time.time() result = func() print(f\u0026#34;took {time.time() - start:.3f}s\u0026#34;) return result return wrapper @timer def greet(name): # This will break - wrapper() takes 0 args return f\u0026#34;Hi {name}\u0026#34; # GOOD - wrapper accepts any arguments and passes them through def timer(func): @functools.wraps(func) def wrapper(*args, **kwargs): # accepts any signature start = time.time() result = func(*args, **kwargs) print(f\u0026#34;took {time.time() - start:.3f}s\u0026#34;) return result return wrapper Mistake 4 - Decorator Applied at Import Time Has Side Effects # BAD - executes code at class/module definition time import time def timer(func): start = time.time() # runs when @timer is applied, NOT when func is called! @functools.wraps(func) def wrapper(*args, **kwargs): return func(*args, **kwargs) return wrapper # GOOD - timing must be INSIDE wrapper def timer(func): @functools.wraps(func) def wrapper(*args, **kwargs): start = time.time() # runs each time the function is called result = func(*args, **kwargs) print(f\u0026#34;{time.time() - start:.3f}s\u0026#34;) return result return wrapper Mistake 5 - Not Returning the Function\u0026rsquo;s Result # BAD - silently drops the return value def my_decorator(func): @functools.wraps(func) def wrapper(*args, **kwargs): func(*args, **kwargs) # result is discarded! return wrapper @my_decorator def add(a, b): return a + b print(add(3, 4)) # None - not 7! # GOOD def my_decorator(func): @functools.wraps(func) def wrapper(*args, **kwargs): return func(*args, **kwargs) # always return the result return wrapper Mistake 6 - Stacking Order Confusion @decorator_A @decorator_B def func(): pass # This is: func = decorator_A(decorator_B(func)) # decorator_B wraps func FIRST, then decorator_A wraps THAT. # When called: A\u0026#39;s logic → B\u0026#39;s logic → original func → B\u0026#39;s after → A\u0026#39;s after # If A checks authentication and B logs timing, # and you want auth checked BEFORE logging starts: @auth_check # outer - runs first @log_timing # inner - runs second def endpoint(): pass Practice Questions \u0026amp; Answers Question 1 - HOF Basics Q: Using map and filter, transform a list of temperatures from Celsius to Fahrenheit and keep only temperatures above 100°F.\nA:\ncelsius = [0, 20, 37, 45, 60, 100] # Convert: F = C * 9/5 + 32 fahrenheit = list(map(lambda c: c * 9/5 + 32, celsius)) print(fahrenheit) # [32.0, 68.0, 98.6, 113.0, 140.0, 212.0] hot_temps = list(filter(lambda f: f \u0026gt; 100, fahrenheit)) print(hot_temps) # [113.0, 140.0, 212.0] # One-liner using a generator expression hot = [c * 9/5 + 32 for c in celsius if c * 9/5 + 32 \u0026gt; 100] print(hot) # [113.0, 140.0, 212.0] Question 2 - Write a Closure Q: Write a closure make_power(n) that returns a function which raises its argument to the power n.\nA:\ndef make_power(n): def power(x): return x ** n return power square = make_power(2) cube = make_power(3) print(square(4)) # 16 print(cube(3)) # 27 print(make_power(10)(2)) # 1024 # The free variable `n` is captured in the closure: print(square.__code__.co_freevars) # (\u0026#39;n\u0026#39;,) Question 3 - Basic Decorator Q: Write a decorator @uppercase that converts a function\u0026rsquo;s string return value to uppercase.\nA:\nimport functools def uppercase(func): @functools.wraps(func) def wrapper(*args, **kwargs): result = func(*args, **kwargs) if isinstance(result, str): return result.upper() return result return wrapper @uppercase def greet(name): return f\u0026#34;hello, {name}!\u0026#34; @uppercase def get_city(): return \u0026#34;hyderabad\u0026#34; print(greet(\u0026#34;ravi\u0026#34;)) # HELLO, RAVI! print(get_city()) # HYDERABAD Question 4 - Decorator with Arguments Q: Write a decorator @truncate(max_len) that truncates a function\u0026rsquo;s string return value to max_len characters, appending \u0026quot;...\u0026quot; if truncated.\nA:\nimport functools def truncate(max_len): def decorator(func): @functools.wraps(func) def wrapper(*args, **kwargs): result = func(*args, **kwargs) if isinstance(result, str) and len(result) \u0026gt; max_len: return result[:max_len] + \u0026#34;...\u0026#34; return result return wrapper return decorator @truncate(max_len=10) def get_description(): return \u0026#34;This is a very long description that should be cut off\u0026#34; @truncate(max_len=20) def get_title(): return \u0026#34;Introduction to Python Decorators\u0026#34; print(get_description()) # This is a... print(get_title()) # Introduction to Pyt... Question 5 - Class-Based Decorator Q: Write a class-based decorator Memoize that caches the return value of a function based on its arguments.\nA:\nimport functools class Memoize: def __init__(self, func): functools.update_wrapper(self, func) self.func = func self.cache = {} def __call__(self, *args): if args not in self.cache: self.cache[args] = self.func(*args) return self.cache[args] def cache_info(self): return f\u0026#34;Cached {len(self.cache)} result(s): {list(self.cache.keys())}\u0026#34; @Memoize def expensive(n): print(f\u0026#34;Computing for {n}...\u0026#34;) return n ** 3 print(expensive(5)) # Computing for 5... → 125 print(expensive(5)) # (no print) → 125 (cached) print(expensive(10)) # Computing for 10... → 1000 print(expensive.cache_info()) # Cached 2 result(s): [(5,), (10,)] Question 6 - Stacking Decorators Q: Write two decorators: @add_html_tag(tag) that wraps output in an HTML tag. Then apply them to produce \u0026lt;p\u0026gt;\u0026lt;em\u0026gt;Hello\u0026lt;/em\u0026gt;\u0026lt;/p\u0026gt;.\nA:\nimport functools def add_html_tag(tag): def decorator(func): @functools.wraps(func) def wrapper(*args, **kwargs): return f\u0026#34;\u0026lt;{tag}\u0026gt;{func(*args, **kwargs)}\u0026lt;/{tag}\u0026gt;\u0026#34; return wrapper return decorator @add_html_tag(\u0026#34;p\u0026#34;) @add_html_tag(\u0026#34;em\u0026#34;) def hello(): return \u0026#34;Hello\u0026#34; print(hello()) # \u0026lt;p\u0026gt;\u0026lt;em\u0026gt;Hello\u0026lt;/em\u0026gt;\u0026lt;/p\u0026gt; # Stacking order: # @add_html_tag(\u0026#34;em\u0026#34;) applied first → wraps hello() in \u0026lt;em\u0026gt; # @add_html_tag(\u0026#34;p\u0026#34;) applied second → wraps that result in \u0026lt;p\u0026gt; Question 7 - Real World: Rate Limiter Q: Write a decorator @rate_limit(calls, period) that raises a RuntimeError if a function is called more than calls times within period seconds.\nA:\nimport functools import time from collections import deque def rate_limit(calls, period): \u0026#34;\u0026#34;\u0026#34;Allow at most `calls` calls in any `period`-second window.\u0026#34;\u0026#34;\u0026#34; def decorator(func): timestamps = deque() @functools.wraps(func) def wrapper(*args, **kwargs): now = time.time() # Remove timestamps outside the current window while timestamps and now - timestamps[0] \u0026gt; period: timestamps.popleft() if len(timestamps) \u0026gt;= calls: raise RuntimeError( f\u0026#34;{func.__name__} rate limit exceeded: \u0026#34; f\u0026#34;max {calls} calls per {period}s\u0026#34; ) timestamps.append(now) return func(*args, **kwargs) return wrapper return decorator @rate_limit(calls=3, period=1.0) def send_message(text): print(f\u0026#34;Sending: {text}\u0026#34;) send_message(\u0026#34;msg 1\u0026#34;) # OK send_message(\u0026#34;msg 2\u0026#34;) # OK send_message(\u0026#34;msg 3\u0026#34;) # OK try: send_message(\u0026#34;msg 4\u0026#34;) # RuntimeError: rate limit exceeded except RuntimeError as e: print(e) Question 8 - Predict the Output Q: What is printed?\nimport functools def deco(func): @functools.wraps(func) def wrapper(*args, **kwargs): print(f\u0026#34;Before {func.__name__}\u0026#34;) result = func(*args, **kwargs) print(f\u0026#34;After {func.__name__}\u0026#34;) return result return wrapper @deco def A(): print(\u0026#34;In A\u0026#34;) B() @deco def B(): print(\u0026#34;In B\u0026#34;) A() A:\nBefore A In A Before B In B After B After A A() is called → wrapper(A) runs → prints \u0026ldquo;Before A\u0026rdquo; → calls original A → prints \u0026ldquo;In A\u0026rdquo; → calls B() which is decorated → wrapper(B) runs → prints \u0026ldquo;Before B\u0026rdquo; → calls original B → prints \u0026ldquo;In B\u0026rdquo; → returns → prints \u0026ldquo;After B\u0026rdquo; → B\u0026rsquo;s wrapper returns → A\u0026rsquo;s original finishes → prints \u0026ldquo;After A\u0026rdquo;.\nQuick Reference Cheat Sheet ┌──────────────────────────────────────────────────────────────────────┐ │ DECORATORS QUICK REFERENCE │ ├──────────────────────────────────────────────────────────────────────┤ │ ANATOMY OF A DECORATOR │ │ │ │ import functools │ │ │ │ def my_decorator(func): # takes the function │ │ @functools.wraps(func) # preserves metadata │ │ def wrapper(*args, **kwargs): # accepts any signature │ │ # before logic │ │ result = func(*args, **kwargs) # call original │ │ # after logic │ │ return result # return the result! │ │ return wrapper # return wrapper, NOT wrapper() │ │ │ │ PARAMETERIZED DECORATOR (extra nesting layer) │ │ │ │ def my_decorator(param): # factory: takes the param │ │ def decorator(func): # real decorator │ │ @functools.wraps(func) │ │ def wrapper(*args, **kwargs): │ │ ... │ │ return wrapper │ │ return decorator │ │ │ ├──────────────────────┬───────────────────────────────────────────── │ │ @ SYNTAX │ EQUIVALENT TO │ ├──────────────────────┼───────────────────────────────────────────── │ │ @deco │ func = deco(func) │ │ @deco(arg) │ func = deco(arg)(func) │ │ @A │ func = A(B(func)) ← B applied first │ │ @B │ │ ├──────────────────────┼───────────────────────────────────────────── │ │ BUILT-IN DECORATORS │ PURPOSE │ ├──────────────────────┼───────────────────────────────────────────── │ │ @staticmethod │ No implicit argument │ │ @classmethod │ cls as first argument │ │ @property │ Attribute-style access with getter/setter │ │ @functools.wraps │ Copy metadata from wrapped function │ │ @functools.lru_cache │ Memoize with LRU eviction │ │ @functools.cache │ Unbounded memoize (Python 3.9+) │ │ @cached_property │ Computed once per instance │ ├──────────────────────┼───────────────────────────────────────────── │ │ CHECKLIST │ │ ├──────────────────────┼───────────────────────────────────────────── │ │ ALWAYS │ Use @functools.wraps(func) │ │ ALWAYS │ Use *args, **kwargs in wrapper │ │ ALWAYS │ return func(*args, **kwargs) │ │ NEVER │ Call func() when applying @decorator │ │ NEVER │ Put side-effectful code outside wrapper │ └──────────────────────┴───────────────────────────────────────────── │ HOF vs DECORATOR Higher-Order Function: takes/returns functions (general concept) Decorator: a HOF specifically designed to wrap another function and add behaviour via the @ syntax End of Lesson - Higher-Order Functions \u0026amp; Decorators in Python Codeverra - codeverra.com\n","permalink":"https://blogs.codeverra.com/blog/python-13-decorators/","summary":"\u003ch1 id=\"python-higher-order-functions--decorators\"\u003ePython Higher-Order Functions \u0026amp; Decorators\u003c/h1\u003e\n\u003ch3 id=\"a-complete-reference-guide\"\u003eA Complete Reference Guide\u003c/h3\u003e\n\u003chr\u003e\n\u003ch2 id=\"table-of-contents\"\u003eTable of Contents\u003c/h2\u003e\n\u003col\u003e\n\u003cli\u003e\u003ca href=\"#the-big-picture\"\u003eThe Big Picture\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#functions-as-first-class-objects\"\u003eFunctions as First-Class Objects\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#higher-order-functions\"\u003eHigher-Order Functions\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#closures--the-foundation-of-decorators\"\u003eClosures  -  The Foundation of Decorators\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#decorators--core-concept\"\u003eDecorators  -  Core Concept\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#the--syntax\"\u003eThe \u003ccode\u003e@\u003c/code\u003e Syntax\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#preserving-metadata-with-functoolswraps\"\u003ePreserving Metadata with \u003ccode\u003efunctools.wraps\u003c/code\u003e\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#decorators-with-arguments\"\u003eDecorators with Arguments\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#class-based-decorators\"\u003eClass-Based Decorators\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#stacking-multiple-decorators\"\u003eStacking Multiple Decorators\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#built-in-decorators\"\u003eBuilt-in Decorators\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#practical-decorator-patterns\"\u003ePractical Decorator Patterns\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#common-mistakes-to-avoid\"\u003eCommon Mistakes to Avoid\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#practice-questions--answers\"\u003ePractice Questions \u0026amp; Answers\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#quick-reference-cheat-sheet\"\u003eQuick Reference Cheat Sheet\u003c/a\u003e\u003c/li\u003e\n\u003c/ol\u003e\n\u003chr\u003e\n\u003ch2 id=\"the-big-picture\"\u003eThe Big Picture\u003c/h2\u003e\n\u003cpre tabindex=\"0\"\u003e\u003ccode\u003e┌─────────────────────────────────────────────────────────────────────┐\n│              THE DECORATOR MENTAL MODEL                             │\n│                                                                     │\n│  A decorator is a function that WRAPS another function             │\n│  to add behaviour before, after, or around it.                     │\n│                                                                     │\n│    Original Function                                                │\n│   ┌────────────────┐                                                │\n│   │   do_thing()   │                                                │\n│   └────────────────┘                                                │\n│           │  decorator applied                                      │\n│           ▼                                                         │\n│   ┌────────────────────────────────────┐                            │\n│   │           wrapper()                │                            │\n│   │  ┌──────────────────────────────┐  │                            │\n│   │  │  before logic (logging etc)  │  │                            │\n│   │  ├──────────────────────────────┤  │                            │\n│   │  │      do_thing() ← called     │  │                            │\n│   │  ├──────────────────────────────┤  │                            │\n│   │  │  after logic (timing etc)    │  │                            │\n│   │  └──────────────────────────────┘  │                            │\n│   └────────────────────────────────────┘                            │\n│                                                                     │\n│  @decorator        is sugar for:   func = decorator(func)          │\n└─────────────────────────────────────────────────────────────────────┘\n\u003c/code\u003e\u003c/pre\u003e\u003cblockquote\u003e\n\u003cp\u003e\u003cstrong\u003eCore Idea:\u003c/strong\u003e A decorator is just a callable that takes a function and returns a new callable. Everything else  -  the \u003ccode\u003e@\u003c/code\u003e syntax, \u003ccode\u003efunctools.wraps\u003c/code\u003e, parameterised decorators  -  builds on top of this single idea.\u003c/p\u003e","tags":["python"],"title":"Higher Order Functions and Decorators in Python"},{"content":"Lesson 2.7 - Indexes, Constraints \u0026amp; Schema Design Theory + Practice | All Three Databases Before You Start This lesson uses all three databases. Each section tells you which one to connect to.\nQuick check - run this in each database before starting:\n-- In cricketdb: SELECT COUNT(*) FROM performances; -- expect 30 -- In shopdb: SELECT COUNT(*) FROM order_items; -- expect 30+ -- In streamdb: SELECT COUNT(*) FROM watch_history; -- expect 50 What you will learn:\nConcept What it does Sequential scan vs index scan How PostgreSQL finds rows CREATE INDEX Speed up queries on large tables Composite index Index on two or more columns Partial index Index only a subset of rows Unique index Enforce uniqueness via an index EXPLAIN / EXPLAIN ANALYZE Read a query execution plan FK behaviours CASCADE, SET NULL, RESTRICT, SET DEFAULT Normalization 1NF, 2NF, 3NF - design rules with examples Common schema mistakes Real anti-patterns and how to fix them Part 1 - How PostgreSQL Finds Rows Before learning about indexes, you need to understand what problem they solve.\nSequential Scan Without an index, PostgreSQL reads every row in a table from start to finish to find the ones that match your WHERE clause. This is called a sequential scan.\nTable: products (25 rows) Query: WHERE name = \u0026#39;Samsung Galaxy S23\u0026#39; PostgreSQL reads: Row 1 → \u0026#39;Samsung Galaxy S23\u0026#39; ← Lenovo IdeaPad? No. Row 2 → \u0026#39;Samsung Galaxy S23\u0026#39; ← Apple iPhone? No. Row 3 → \u0026#39;Samsung Galaxy S23\u0026#39; ← OnePlus Nord? No. ... Row 1 → \u0026#39;Samsung Galaxy S23\u0026#39; ← Samsung Galaxy S23? YES. ✓ ... continues to end of table (must check all rows) For 25 rows - no problem. For 25 million rows - very slow.\nIndex Scan An index is a separate data structure (a B-tree by default) that stores column values in sorted order, each pointing to the row\u0026rsquo;s physical location.\nIndex on products.name (sorted): \u0026#39;Allen Solly Formal Shirt\u0026#39; → row 10 \u0026#39;Apple iPhone 15\u0026#39; → row 2 \u0026#39;Bajaj Mixer Grinder 500W\u0026#39; → row 16 \u0026#39;Biba Anarkali Dress\u0026#39; → row 9 ... \u0026#39;Samsung Galaxy S23\u0026#39; → row 1 ← found in O(log n) steps ... PostgreSQL jumps directly to the matching location - like looking up a word in a dictionary rather than reading every page.\nCost comparison:\nSequential scan: O(n) - reads every row Index scan: O(log n) - follows the tree to the answer For 1 million rows: sequential = 1,000,000 reads. Index = ~20 reads.\nPart 2 - CREATE INDEX (Connect to shopdb)\nBasic index -- Check if an index already exists on a column SELECT indexname, indexdef FROM pg_indexes WHERE tablename = \u0026#39;products\u0026#39;; -- Create an index on a frequently filtered column CREATE INDEX idx_products_category_id ON products (category_id); -- Now queries like this use the index instead of a full scan: SELECT * FROM products WHERE category_id = 1; -- Index on order status - very common filter CREATE INDEX idx_orders_status ON orders (status); -- Speeds up: SELECT * FROM orders WHERE status = \u0026#39;Pending\u0026#39;; SELECT * FROM orders WHERE status IN (\u0026#39;Pending\u0026#39;, \u0026#39;Shipped\u0026#39;); -- Index on a date column - speeds up range queries CREATE INDEX idx_orders_order_date ON orders (order_date); -- Speeds up: SELECT * FROM orders WHERE order_date \u0026gt;= \u0026#39;2024-03-01\u0026#39;; SELECT * FROM orders WHERE order_date BETWEEN \u0026#39;2024-01-01\u0026#39; AND \u0026#39;2024-03-31\u0026#39;; When to create an index Create an index when a column is:\nFrequently used in WHERE clauses Used in JOIN conditions (ON table.col = other.col) Used in ORDER BY on large result sets A foreign key column (PostgreSQL does NOT auto-index FKs) When NOT to create an index Do not index:\nColumns you never filter or sort by Very small tables (sequential scan is faster for \u0026lt;1000 rows) Columns that change very frequently (index must be updated on every write) Boolean columns with low selectivity - an index on is_active where 95% of rows are TRUE is nearly useless (PostgreSQL will ignore it) The trade-off: Indexes speed up reads but slow down writes. Every INSERT, UPDATE, and DELETE must also update all relevant indexes. Do not index everything - index the columns your slow queries filter on.\nPart 3 - Composite Index A composite index covers two or more columns together. It is useful when you commonly filter by multiple columns at the same time.\n-- Frequently run: orders by a specific customer with a specific status SELECT * FROM orders WHERE customer_id = 1 AND status = \u0026#39;Delivered\u0026#39;; -- A composite index on both columns together CREATE INDEX idx_orders_customer_status ON orders (customer_id, status); Column order matters in a composite index. The index is sorted by the first column, then the second within that.\nidx_orders_customer_status: customer_id=1, status=\u0026#39;Cancelled\u0026#39; → row 9 customer_id=1, status=\u0026#39;Delivered\u0026#39; → row 1 customer_id=1, status=\u0026#39;Delivered\u0026#39; → row 14 customer_id=3, status=\u0026#39;Delivered\u0026#39; → row 2 customer_id=4, status=\u0026#39;Delivered\u0026#39; → row 6 ... This index helps:\nWHERE customer_id = 1 AND status = 'Delivered' ✅ (both columns) WHERE customer_id = 1 ✅ (leading column only) WHERE status = 'Delivered' ❌ (non-leading column - index NOT used) Rule: A composite index can be used if your query filters on the leading column(s) of the index. Filtering on a non-leading column alone does not benefit from the composite index.\n-- Composite index on watch_history for common access pattern -- Connect to streamdb for this one CREATE INDEX idx_watch_history_user_show ON watch_history (user_id, show_id); -- Helps: SELECT * FROM watch_history WHERE user_id = 1; SELECT * FROM watch_history WHERE user_id = 1 AND show_id = 3; -- Does not help: SELECT * FROM watch_history WHERE show_id = 3; -- non-leading Part 4 - Partial Index A partial index only indexes rows that match a condition. It is smaller and faster than a full index when you mostly query a subset of rows.\n(Connect to shopdb)\n-- Most queries about orders care about Pending and Shipped orders -- Delivered and Cancelled orders are rarely queried -- Index only the \u0026#34;active\u0026#34; orders CREATE INDEX idx_orders_active ON orders (customer_id, order_date) WHERE status IN (\u0026#39;Pending\u0026#39;, \u0026#39;Shipped\u0026#39;); -- This index is used when: SELECT * FROM orders WHERE status IN (\u0026#39;Pending\u0026#39;, \u0026#39;Shipped\u0026#39;) AND customer_id = 5; -- ✅ matches the partial index condition -- Index only premium shows for StreamDB -- Connect to streamdb CREATE INDEX idx_shows_premium ON shows (rating, language) WHERE is_premium = TRUE; -- Helps queries like: SELECT title, rating FROM shows WHERE is_premium = TRUE AND language = \u0026#39;Hindi\u0026#39; ORDER BY rating DESC; Benefits of partial indexes:\nSmaller than full index - less memory, faster to scan Faster to update - only rebuilt when indexed rows change Very effective when a small fraction of rows are \u0026ldquo;hot\u0026rdquo; (frequently queried) Part 5 - Unique Index A UNIQUE constraint automatically creates a unique index. You can also create one explicitly for fine-grained control.\n-- Ensure no two products in the same category have the same name -- (Composite unique constraint) ALTER TABLE products ADD CONSTRAINT uq_product_name_per_category UNIQUE (name, category_id); -- Partial unique index: email must be unique among active customers only -- (hypothetical - useful when soft-deleting records) CREATE UNIQUE INDEX uq_active_customer_email ON customers (email) WHERE is_active = TRUE; -- if customers had an is_active column Part 6 - EXPLAIN: Reading a Query Plan EXPLAIN shows you how PostgreSQL plans to execute a query - without actually running it. EXPLAIN ANALYZE runs the query and shows actual timing.\n(Connect to shopdb)\n-- See the plan for a simple query EXPLAIN SELECT * FROM products WHERE category_id = 1; Output will look something like:\nSeq Scan on products (cost=0.00..1.31 rows=6 width=...) Filter: (category_id = 1) Seq Scan = sequential scan (no index used). After creating the index, you will see Index Scan instead.\n-- Create the index CREATE INDEX idx_products_category_id ON products (category_id); -- Now check the plan again EXPLAIN SELECT * FROM products WHERE category_id = 1; Index Scan using idx_products_category_id on products (cost=0.14..8.16 rows=6 width=...) Index Cond: (category_id = 1) -- EXPLAIN ANALYZE: run the query and show real timing EXPLAIN ANALYZE SELECT p.name, SUM(oi.quantity * oi.unit_price) AS revenue FROM products AS p JOIN order_items AS oi ON oi.product_id = p.id GROUP BY p.id, p.name ORDER BY revenue DESC; Key terms in EXPLAIN output:\nTerm Meaning Seq Scan Read every row in the table Index Scan Use an index to find rows Bitmap Heap Scan Use index to find row locations, then read from table Hash Join Join method using a hash table - good for larger joins Nested Loop Join method - one table iterated per row of the other cost=X..Y X = startup cost, Y = total cost (arbitrary units) rows=N Estimated number of rows returned actual time=X..Y Real time in ms (EXPLAIN ANALYZE only) You do not need to be an expert at reading EXPLAIN output yet. The key skill: recognise Seq Scan on large tables as a signal that an index might help, and verify with EXPLAIN ANALYZE.\n✏️ Practice Set 1 - Indexes and EXPLAIN Q1. (shopdb) Create an index that would speed up this query:\nSELECT * FROM order_items WHERE product_id = 5; Name it idx_order_items_product_id. Then run EXPLAIN before and after to see the difference.\nQ2. (shopdb) Create a composite index on watch_history (user_id, watched_on) in streamdb that would help the following query:\nSELECT * FROM watch_history WHERE user_id = 3 AND watched_on \u0026gt;= \u0026#39;2024-02-01\u0026#39;; Which column should be first in the index, and why?\nQ3. (shopdb) Create a partial index on orders (customer_id) only for orders where status = 'Pending'. What is the advantage of this over a full index on customer_id?\nQ4. (shopdb) List all indexes currently on the orders table using the pg_indexes system view. What indexes exist?\nPart 7 - Foreign Key Behaviours You have used REFERENCES table(col) as a foreign key. But you can control what happens when the parent row is deleted.\n-- The four ON DELETE behaviours: -- 1. RESTRICT (default): prevent deletion if child rows exist -- Error: \u0026#34;update or delete on table violates foreign key constraint\u0026#34; FOREIGN KEY (category_id) REFERENCES categories(id) ON DELETE RESTRICT -- 2. CASCADE: delete child rows automatically when parent is deleted FOREIGN KEY (order_id) REFERENCES orders(id) ON DELETE CASCADE -- 3. SET NULL: set the FK column to NULL when parent is deleted FOREIGN KEY (category_id) REFERENCES categories(id) ON DELETE SET NULL -- 4. SET DEFAULT: set the FK column to its DEFAULT value FOREIGN KEY (category_id) REFERENCES categories(id) ON DELETE SET DEFAULT When to use each Behaviour Use when RESTRICT Child rows should never be orphaned - deletion must be manual CASCADE Child rows are meaningless without the parent (order items without an order) SET NULL Child rows remain valid without the parent (a product can be uncategorised) SET DEFAULT Child rows fall back to a \u0026ldquo;general\u0026rdquo; parent (e.g. default category) Demonstrating CASCADE -- In shopdb, order_items reference orders -- Let\u0026#39;s see what happens with CASCADE -- First, check current FK definition SELECT conname, confdeltype FROM pg_constraint WHERE conrelid = \u0026#39;order_items\u0026#39;::regclass AND contype = \u0026#39;f\u0026#39;; -- confdeltype: \u0026#39;a\u0026#39; = NO ACTION (default), \u0026#39;c\u0026#39; = CASCADE, -- \u0026#39;n\u0026#39; = SET NULL, \u0026#39;d\u0026#39; = SET DEFAULT, \u0026#39;r\u0026#39; = RESTRICT -- Demonstration: add a test order and items, then delete the order BEGIN; INSERT INTO orders (customer_id, order_date, status, payment_method) VALUES (1, CURRENT_DATE, \u0026#39;Pending\u0026#39;, \u0026#39;UPI\u0026#39;) RETURNING id; -- Suppose id = 17 INSERT INTO order_items (order_id, product_id, quantity, unit_price) VALUES (17, 1, 1, 74999.00); -- Without CASCADE on order_items, this would FAIL: -- DELETE FROM orders WHERE id = 17; -- ERROR: violates foreign key constraint -- With CASCADE it would work and auto-delete the item too -- Our current FK has no CASCADE, so we delete items first: DELETE FROM order_items WHERE order_id = 17; DELETE FROM orders WHERE id = 17; ROLLBACK; -- undo the demo Part 8 - Normalization: Designing Good Schemas Normalization is a set of rules for organizing data in a relational database to reduce redundancy and prevent update anomalies.\nThink of it as: store each fact in exactly one place.\nThe Problem: Unnormalized Data Imagine storing order data in a single table:\norders_flat (BAD DESIGN) ┌──────────┬───────────────┬──────────────────┬───────────────┬────────┬───────────┬──────────┐ │ order_id │ customer_name │ customer_email │ customer_city │ product│ category │ price │ ├──────────┼───────────────┼──────────────────┼───────────────┼────────┼───────────┼──────────┤ │ 1 │ Aarav Mehta │ aarav@gmail.com │ Mumbai │ iPhone │Electronics│ 79999.00 │ │ 1 │ Aarav Mehta │ aarav@gmail.com │ Mumbai │ Headph.│Electronics│ 1499.00 │ │ 2 │ Rohan Iyer │ rohan@gmail.com │ Bengaluru │ WhiteT.│ Books │ 299.00 │ └──────────┴───────────────┴──────────────────┴───────────────┴────────┴───────────┴──────────┘ Problems:\nAarav\u0026rsquo;s email appears twice - update one, miss the other → inconsistency Delete the iPhone row → lose the fact that Aarav is from Mumbai Can\u0026rsquo;t add a product without an associated order These are called update anomalies, delete anomalies, and insert anomalies.\nFirst Normal Form (1NF) Rule: Each cell contains one atomic (indivisible) value. No repeating groups.\nViolation example:\nproducts ┌────┬─────────────┬──────────────────────────────────┐ │ id │ name │ genres │ ├────┼─────────────┼──────────────────────────────────┤ │ 1 │ Panchayat │ Drama, Comedy │ ← NOT 1NF │ 2 │ Mirzapur │ Crime, Action, Drama │ ← NOT 1NF └────┴─────────────┴──────────────────────────────────┘ Fix: Split into two tables with a junction table - exactly what show_genres is in StreamDB.\nshows show_genres genres ┌────┬───────────┐ ┌─────────┬──────────┐ ┌────┬────────┐ │ id │ title │ │ show_id │ genre_id │ │ id │ name │ ├────┼───────────┤ ───► ├─────────┼──────────┤ ├────┼────────┤ │ 1 │ Panchayat │ │ 1 │ 1 │ │ 1 │ Drama │ │ 2 │ Mirzapur │ │ 1 │ 3 │ │ 3 │ Comedy │ └────┴───────────┘ │ 2 │ 6 │ │ 6 │ Crime │ └─────────┴──────────┘ └────┴────────┘ 1NF checklist:\nEvery column has exactly one value per row No comma-separated lists in a single column No repeating column groups (phone1, phone2, phone3\u0026hellip;) Each row is uniquely identifiable (has a primary key) Second Normal Form (2NF) Rule: Must be in 1NF, AND every non-key column must depend on the whole primary key (not just part of it).\nThis only applies to tables with composite primary keys.\nViolation example:\norder_items_bad (composite PK: order_id + product_id) ┌──────────┬────────────┬──────────┬───────────────┬──────────────────┐ │ order_id │ product_id │ quantity │ product_name │ product_category │ ├──────────┼────────────┼──────────┼───────────────┼──────────────────┤ │ 1 │ 1 │ 1 │ Samsung Gal.. │ Electronics │ ← NOT 2NF │ 1 │ 4 │ 2 │ boAt Rocker.. │ Electronics │ └──────────┴────────────┴──────────┴───────────────┴──────────────────┘ product_name and product_category depend only on product_id - not the full composite key (order_id, product_id).\nFix: Move product details to their own table.\norder_items products ┌──────────┬────────────┬──────┐ ┌────┬───────────────┬──────────────┐ │ order_id │ product_id │ qty │ │ id │ name │ category │ ├──────────┼────────────┼──────┤ ├────┼───────────────┼──────────────┤ │ 1 │ 1 │ 1 │ │ 1 │ Samsung Gal.. │ Electronics │ │ 1 │ 4 │ 2 │ │ 4 │ boAt Rocker.. │ Electronics │ └──────────┴────────────┴──────┘ └────┴───────────────┴──────────────┘ quantity depends on the full key (how many of THIS product in THIS order). name and category depend only on product_id → they belong in products.\n2NF checklist:\nTable is in 1NF Every non-key column depends on the entire primary key No partial dependencies - each fact is stored in the right table Third Normal Form (3NF) Rule: Must be in 2NF, AND every non-key column must depend directly on the primary key, not on another non-key column.\nIn other words: no transitive dependencies - A → B → C (where B is not the key).\nViolation example:\norders_bad ┌────┬─────────────┬──────────────────┬───────────┬─────────┐ │ id │ customer_id │ customer_city │ state │ status │ ├────┼─────────────┼──────────────────┼───────────┼─────────┤ │ 1 │ 1 │ Mumbai │Maharashtra│Delivered│ ← NOT 3NF │ 2 │ 3 │ Bengaluru │Karnataka │Delivered│ └────┴─────────────┴──────────────────┴───────────┴─────────┘ customer_city and state depend on customer_id, not on id (the order\u0026rsquo;s PK).\nThe dependency chain is: order.id → customer_id → city → state\nstate depends on city (a non-key column) - transitive dependency.\nFix: Move customer details to the customers table where they belong.\norders customers ┌────┬─────────────┬──────────┐ ┌────┬──────────────┬─────────┬────────────┐ │ id │ customer_id │ status │ │ id │ name │ city │ state │ ├────┼─────────────┼──────────┤ ├────┼──────────────┼─────────┼────────────┤ │ 1 │ 1 │Delivered │ │ 1 │ Aarav Mehta │ Mumbai │Maharashtra │ │ 2 │ 3 │Delivered │ │ 3 │ Rohan Iyer │Bengaluru│Karnataka │ └────┴─────────────┴──────────┘ └────┴──────────────┴─────────┴────────────┘ ShopDB already follows 3NF - this is exactly how it is designed.\n3NF checklist:\nTable is in 2NF No non-key column determines another non-key column Each fact lives in exactly one table When a fact changes, you update exactly one row in exactly one table Normalization Summary 1NF → Each cell has one value. Table has a primary key. ↓ 2NF → Every non-key column depends on the WHOLE primary key. (Relevant for composite PKs) ↓ 3NF → Every non-key column depends DIRECTLY on the primary key. No A → B → C chains where B is not the key. In practice: Most well-designed databases are in 3NF by default if you follow the \u0026ldquo;one table per thing\u0026rdquo; principle from the start. The normalization forms are a diagnostic tool - use them to evaluate and fix existing schemas, not as a step-by-step process for new ones.\n✏️ Practice Set 2 - Normalization For each scenario below, identify the normalization violation and write the corrected table structure (you do not need to write SQL - table diagrams are fine).\nQ5. This table violates 1NF. Identify why and propose a fix:\nstudents ┌────┬──────────────┬────────────────────────────────┐ │ id │ name │ courses_enrolled │ ├────┼──────────────┼────────────────────────────────┤ │ 1 │ Rohit Sharma │ Maths, Physics, Chemistry │ │ 2 │ Priya Patel │ Biology, Chemistry │ └────┴──────────────┴────────────────────────────────┘ Q6. This table violates 2NF. The composite PK is (match_id, player_id). Identify the partial dependency and fix it:\nmatch_performances ┌──────────┬───────────┬──────┬─────────┬──────────────┬────────────┐ │ match_id │ player_id │ runs │ wickets │ player_name │ player_role│ ├──────────┼───────────┼──────┼─────────┼──────────────┼────────────┤ │ 1 │ 1 │ 121 │ 0 │ Virat Kohli │ Batsman │ │ 1 │ 3 │ 8 │ 4 │ J. Bumrah │ Bowler │ └──────────┴───────────┴──────┴─────────┴──────────────┴────────────┘ Q7. This table violates 3NF. Identify the transitive dependency and fix it:\nemployees ┌────┬──────────────┬────────────┬──────────────────┬─────────────┐ │ id │ name │ dept_id │ dept_name │ dept_budget │ ├────┼──────────────┼────────────┼──────────────────┼─────────────┤ │ 1 │ Aarav Mehta │ 3 │ Engineering │ 5000000 │ │ 2 │ Diya Sharma │ 3 │ Engineering │ 5000000 │ │ 3 │ Kabir Singh │ 1 │ Sales │ 2000000 │ └────┴──────────────┴────────────┴──────────────────┴─────────────┘ Q8. Design a normalized schema (table names + columns) for this scenario:\nA cricket coaching academy tracks coaches, students, and sessions. Each coach can coach many students. Each student can have sessions with multiple coaches on different dates. A session has a duration and a topic (e.g. \u0026ldquo;Batting techniques\u0026rdquo;, \u0026ldquo;Fielding drills\u0026rdquo;).\nPart 9 - Common Schema Mistakes These are real patterns that cause problems in production databases.\nMistake 1: Using VARCHAR for everything -- BAD: price stored as text CREATE TABLE products_bad ( price VARCHAR(20) -- \u0026#39;₹1,499.00\u0026#39; stored as text ); -- Cannot do: SUM(price), AVG(price), price \u0026gt; 1000 -- Sorting gives: \u0026#39;1000\u0026#39; \u0026lt; \u0026#39;200\u0026#39; (alphabetical, not numeric!) -- GOOD: use the right type CREATE TABLE products_good ( price_inr NUMERIC(10,2) ); Mistake 2: Storing calculated values -- BAD: storing total when it can be calculated CREATE TABLE orders_bad ( subtotal NUMERIC(10,2), tax NUMERIC(10,2), total NUMERIC(10,2) -- subtotal + tax - can go stale! ); -- GOOD: calculate at query time SELECT subtotal + tax AS total FROM orders; -- Or use a generated column if you really need to store it Mistake 3: Using NULL to mean something specific -- BAD: NULL used to mean \u0026#34;not yet shipped\u0026#34; UPDATE orders SET shipped_at = NULL WHERE status = \u0026#39;Pending\u0026#39;; -- But NULL also means \u0026#34;we don\u0026#39;t know when it shipped\u0026#34; -- These are different concepts - you can\u0026#39;t tell them apart -- GOOD: use a proper status column + nullable timestamp CREATE TABLE orders ( status VARCHAR(20) NOT NULL DEFAULT \u0026#39;Pending\u0026#39;, shipped_at TIMESTAMP -- NULL = not shipped yet (clear meaning) ); Mistake 4: No indexes on foreign key columns -- PostgreSQL automatically indexes PRIMARY KEY columns -- but NOT foreign key columns -- If you run this query frequently: SELECT * FROM order_items WHERE order_id = 5; -- And order_items has no index on order_id, every query is a full table scan -- Fix: index every FK column CREATE INDEX idx_order_items_order_id ON order_items (order_id); CREATE INDEX idx_order_items_product_id ON order_items (product_id); CREATE INDEX idx_performances_player_id ON performances (player_id); CREATE INDEX idx_performances_match_id ON performances (match_id); Mistake 5: Storing lists in a single column -- BAD: comma-separated values CREATE TABLE users_bad ( favourite_genres VARCHAR(200) -- \u0026#39;Drama,Action,Comedy\u0026#39; ); -- Cannot query: WHERE \u0026#39;Drama\u0026#39; IN (favourite_genres) - doesn\u0026#39;t work -- Cannot JOIN to genres table -- Searching requires LIKE \u0026#39;%Drama%\u0026#39; - slow and fragile -- GOOD: junction table CREATE TABLE user_genres ( user_id INT REFERENCES users(id), genre_id INT REFERENCES genres(id), PRIMARY KEY (user_id, genre_id) ); Mistake 6: Not enforcing constraints in the database -- BAD: relying only on application code to validate CREATE TABLE products_bad ( price_inr NUMERIC -- no constraint ); -- A bug in your Python code can insert price = -500 -- GOOD: enforce at the database level too CREATE TABLE products_good ( price_inr NUMERIC(10,2) NOT NULL CHECK (price_inr \u0026gt; 0) ); -- Even if Python has a bug, the database rejects invalid data ✏️ Practice Set 3 - Schema Mistakes and Indexes Q9. The following table is poorly designed. List every problem you can find and write the corrected CREATE TABLE statement:\nCREATE TABLE orders_bad ( id VARCHAR(50), customer VARCHAR(200), -- stores \u0026#34;Aarav Mehta, Mumbai, Maharashtra\u0026#34; items TEXT, -- stores \u0026#34;Samsung Galaxy S23 x1, boAt x2\u0026#34; total VARCHAR(20), -- stores \u0026#34;₹77,997\u0026#34; date VARCHAR(20), -- stores \u0026#34;15 Jan 2024\u0026#34; paid VARCHAR(5) -- stores \u0026#34;yes\u0026#34; or \u0026#34;no\u0026#34; ); Q10. (shopdb) Run EXPLAIN on this query before and after creating an appropriate index. Write the index creation statement and describe what changed in the EXPLAIN output:\nSELECT * FROM order_items WHERE order_id = 3; Q11. (shopdb) Write the SQL to add indexes on ALL foreign key columns across the entire ShopDB schema. List every FK column that needs indexing.\nQ12. (streamdb) The following query is slow on a large version of StreamDB. Propose one or two indexes that would help it, and explain why:\nSELECT u.name, s.title, wh.minutes_watched FROM watch_history AS wh JOIN users AS u ON wh.user_id = u.id JOIN shows AS s ON wh.show_id = s.id WHERE wh.completed = TRUE AND u.plan = \u0026#39;Premium\u0026#39; ORDER BY wh.minutes_watched DESC; Part 10 - Practice Set Answers Answers: Practice Set 1 Q1. Index for order_items.product_id:\n-- Before (run EXPLAIN to see Seq Scan) EXPLAIN SELECT * FROM order_items WHERE product_id = 5; -- Create the index CREATE INDEX idx_order_items_product_id ON order_items (product_id); -- After (run EXPLAIN again to see Index Scan) EXPLAIN SELECT * FROM order_items WHERE product_id = 5; With only 30 rows PostgreSQL may still choose a sequential scan - it is faster for tiny tables. The index becomes meaningful at 1000+ rows.\nQ2. Composite index for watch_history:\n-- Connect to streamdb first CREATE INDEX idx_watch_history_user_date ON watch_history (user_id, watched_on); user_id should be first because it is the equality filter (user_id = 3). Date is a range filter - ranges work best as the trailing column in a composite index. Equality conditions first, range conditions last.\nQ3. Partial index on pending orders:\nCREATE INDEX idx_orders_pending_customer ON orders (customer_id) WHERE status = \u0026#39;Pending\u0026#39;; Advantage: the index is much smaller - it only contains Pending orders, not all 15 orders. On a real system with millions of orders, most are Delivered. A partial index covers only the fraction you actually query frequently, using less memory and updating faster.\nQ4. List indexes on orders:\nSELECT indexname, indexdef FROM pg_indexes WHERE tablename = \u0026#39;orders\u0026#39; ORDER BY indexname; Answers: Practice Set 2 Q5. 1NF violation - courses_enrolled contains a comma-separated list. Fix: split into two tables with a junction:\nstudents: student_courses: courses: id | name student_id | course_id id | name 1 | Rohit Sharma 1 | 1 1 | Maths 2 | Priya Patel 1 | 2 2 | Physics 1 | 3 3 | Chemistry 2 | 3 4 | Biology 2 | 4 Q6. 2NF violation - player_name and player_role depend only on player_id, not the full composite key (match_id, player_id). Fix: move player details to a separate players table.\nmatch_performances: players: match_id | player_id | runs | wickets id | name | role 1 | 1 | 121 | 0 1 | Virat Kohli | Batsman 1 | 3 | 8 | 4 3 | J. Bumrah | Bowler This is exactly how CricketDB is designed.\nQ7. 3NF violation - dept_name and dept_budget depend on dept_id, not directly on employee.id. Transitive chain: id → dept_id → dept_name → dept_budget. Fix: create a departments table.\nemployees: departments: id | name | dept_id id | name | budget 1 | Aarav Mehta | 3 1 | Sales | 2000000 2 | Diya Sharma | 3 3 | Engineering | 5000000 3 | Kabir Singh | 1 Q8. Coaching academy normalized schema:\ncoaches: students: sessions: id | name id | name id | coach_id | student_id | date | duration_mins | topic PK FK→coaches FK→students Three tables. Many-to-many between coaches and students expressed through the sessions table (which is both a junction and an entity with its own attributes).\nAnswers: Practice Set 3 Q9. Problems with orders_bad:\nid VARCHAR - should be SERIAL PRIMARY KEY (integer, auto-increment) customer VARCHAR(200) - stores multiple facts (name, city, state) - violates 1NF and 3NF. Should be customer_id INT REFERENCES customers(id) items TEXT - stores a list in one column - violates 1NF. Should be a separate order_items table total VARCHAR(20) - stores a calculated value as text with currency symbol. Should be computed or NUMERIC(10,2) date VARCHAR(20) - should be DATE type paid VARCHAR(5) - should be BOOLEAN or a status enum -- Corrected version: CREATE TABLE orders_good ( id SERIAL PRIMARY KEY, customer_id INT NOT NULL REFERENCES customers(id), order_date DATE NOT NULL DEFAULT CURRENT_DATE, status VARCHAR(20) NOT NULL DEFAULT \u0026#39;Pending\u0026#39; CHECK (status IN (\u0026#39;Pending\u0026#39;,\u0026#39;Shipped\u0026#39;,\u0026#39;Delivered\u0026#39;,\u0026#39;Cancelled\u0026#39;)), payment_method VARCHAR(20) NOT NULL ); -- order items go in a separate order_items table -- total is computed: SUM(quantity * unit_price) Q10. Index for order_items.order_id:\nEXPLAIN SELECT * FROM order_items WHERE order_id = 3; -- Shows: Seq Scan on order_items CREATE INDEX idx_order_items_order_id ON order_items (order_id); EXPLAIN SELECT * FROM order_items WHERE order_id = 3; -- Shows: Index Scan using idx_order_items_order_id Q11. All FK columns in ShopDB needing indexes:\n-- products → categories CREATE INDEX idx_products_category_id ON products (category_id); -- orders → customers CREATE INDEX idx_orders_customer_id ON orders (customer_id); -- order_items → orders CREATE INDEX idx_order_items_order_id ON order_items (order_id); -- order_items → products CREATE INDEX idx_order_items_product_id ON order_items (product_id); Q12. Indexes for the slow StreamDB query:\n-- Index 1: filter on watch_history.completed (partial index) CREATE INDEX idx_watch_history_completed ON watch_history (user_id, show_id, minutes_watched) WHERE completed = TRUE; -- Index 2: filter on users.plan (if not already indexed) CREATE INDEX idx_users_plan ON users (plan); Explanation: The WHERE clause filters wh.completed = TRUE and u.plan = 'Premium'. A partial index on watch_history covering only completed=TRUE rows is smaller and faster. An index on users.plan speeds up the join filter. The ORDER BY on minutes_watched can also benefit from including it in the composite index.\nWhat\u0026rsquo;s Next You have covered:\n✅ Sequential scan vs index scan - the fundamental read performance difference ✅ CREATE INDEX - when, why, and what to index ✅ Composite indexes - column order and the leading-column rule ✅ Partial indexes - indexing a subset of rows ✅ Unique indexes - enforcing uniqueness ✅ EXPLAIN / EXPLAIN ANALYZE - reading query plans ✅ Foreign key behaviours - CASCADE, SET NULL, RESTRICT, SET DEFAULT ✅ 1NF, 2NF, 3NF - normalization rules with real examples ✅ Six common schema mistakes and their fixes ","permalink":"https://blogs.codeverra.com/blog/sql-part7/","summary":"\u003ch1 id=\"lesson-27-----indexes-constraints--schema-design\"\u003eLesson 2.7  -  Indexes, Constraints \u0026amp; Schema Design\u003c/h1\u003e\n\u003ch3 id=\"theory--practice--all-three-databases\"\u003eTheory + Practice | All Three Databases\u003c/h3\u003e\n\u003chr\u003e\n\u003ch2 id=\"before-you-start\"\u003eBefore You Start\u003c/h2\u003e\n\u003cp\u003eThis lesson uses all three databases.\nEach section tells you which one to connect to.\u003c/p\u003e\n\u003cp\u003eQuick check  -  run this in each database before starting:\u003c/p\u003e\n\u003cdiv class=\"highlight\"\u003e\u003cpre tabindex=\"0\" class=\"chroma\"\u003e\u003ccode class=\"language-sql\" data-lang=\"sql\"\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"c1\"\u003e-- In cricketdb:\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"k\"\u003eSELECT\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"k\"\u003eCOUNT\u003c/span\u003e\u003cspan class=\"p\"\u003e(\u003c/span\u003e\u003cspan class=\"o\"\u003e*\u003c/span\u003e\u003cspan class=\"p\"\u003e)\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"k\"\u003eFROM\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"n\"\u003eperformances\u003c/span\u003e\u003cspan class=\"p\"\u003e;\u003c/span\u003e\u003cspan class=\"w\"\u003e   \u003c/span\u003e\u003cspan class=\"c1\"\u003e-- expect 30\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"c1\"\u003e-- In shopdb:\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"k\"\u003eSELECT\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"k\"\u003eCOUNT\u003c/span\u003e\u003cspan class=\"p\"\u003e(\u003c/span\u003e\u003cspan class=\"o\"\u003e*\u003c/span\u003e\u003cspan class=\"p\"\u003e)\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"k\"\u003eFROM\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"n\"\u003eorder_items\u003c/span\u003e\u003cspan class=\"p\"\u003e;\u003c/span\u003e\u003cspan class=\"w\"\u003e    \u003c/span\u003e\u003cspan class=\"c1\"\u003e-- expect 30+\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"c1\"\u003e-- In streamdb:\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"k\"\u003eSELECT\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"k\"\u003eCOUNT\u003c/span\u003e\u003cspan class=\"p\"\u003e(\u003c/span\u003e\u003cspan class=\"o\"\u003e*\u003c/span\u003e\u003cspan class=\"p\"\u003e)\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"k\"\u003eFROM\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"n\"\u003ewatch_history\u003c/span\u003e\u003cspan class=\"p\"\u003e;\u003c/span\u003e\u003cspan class=\"w\"\u003e  \u003c/span\u003e\u003cspan class=\"c1\"\u003e-- expect 50\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/code\u003e\u003c/pre\u003e\u003c/div\u003e\u003cp\u003e\u003cstrong\u003eWhat you will learn:\u003c/strong\u003e\u003c/p\u003e","tags":["sql","data-analysis"],"title":"Indexes and Schema Design"},{"content":"SQL Lecture 01 - Introduction to SQL \u0026amp; Databases PostgreSQL | Beginner Reference Sheet | Use this during class 🐘\nTable of Contents What is a Database? Types of Databases What is SQL? How SQL Commands Are Grouped Creating a Table - Full Breakdown Data Types in PostgreSQL Constraints - Rules for Your Data Primary Key \u0026amp; Foreign Key Common SQL Commands with Examples What Happens When You Violate a Constraint? Practice Table \u0026amp; 10 Beginner Questions 1. What is a Database? A database is an organised collection of data stored so it can be easily accessed, managed, and updated.\nThink of it like this:\nReal World Analogy: ┌─────────────────────────────────────────────────┐ │ A school keeps student records │ │ → Filing cabinet = Database │ │ → Each drawer = Table │ │ → Each folder = Row (one student) │ │ → Labels on folder = Columns (name, age, etc) │ └─────────────────────────────────────────────────┘ Why not just use Excel?\nExcel Database Good for small data Built for millions of rows One user at a time Many users simultaneously No rules enforced Strict rules (constraints) Easy to corrupt Transactional, safe No relationships Tables linked together 2. Types of Databases Relational Databases (SQL) Data is stored in tables with rows and columns. Tables can be related to each other.\nExamples: PostgreSQL, MySQL, SQLite, SQL Server, Oracle Use when: Structured data, relationships matter, consistency is critical Non-Relational Databases (NoSQL) Data is stored in other formats - documents, key-value pairs, graphs, etc.\nExamples: MongoDB (documents), Redis (key-value), Neo4j (graph) Use when: Unstructured data, high speed, flexible schema For this course → We use PostgreSQL, a powerful open-source relational database. 3. What is SQL? SQL = Structured Query Language (pronounced \u0026ldquo;sequel\u0026rdquo; or \u0026ldquo;S-Q-L\u0026rdquo;)\nSQL is the language you use to talk to a relational database. You use it to:\nCreate tables and structure Insert, update, delete data Ask questions of your data (queries) Control who can access what -- This is a SQL query. It asks: \u0026#34;Show me all students.\u0026#34; SELECT * FROM students; The -- means it\u0026rsquo;s a comment - ignored by the database, just for humans reading the code.\n4. How SQL Commands Are Grouped SQL commands are organised into 5 categories:\n┌─────────────────────────────────────────────────────────────────┐ │ SQL COMMAND GROUPS │ ├──────────┬──────────────────────────────┬───────────────────────┤ │ Category │ Full Name │ Commands │ ├──────────┼──────────────────────────────┼───────────────────────┤ │ DDL │ Data Definition Language │ CREATE, ALTER, DROP, │ │ │ (structure of the database) │ TRUNCATE, RENAME │ ├──────────┼──────────────────────────────┼───────────────────────┤ │ DML │ Data Manipulation Language │ INSERT, UPDATE, │ │ │ (the actual data) │ DELETE, MERGE │ ├──────────┼──────────────────────────────┼───────────────────────┤ │ DQL │ Data Query Language │ SELECT │ │ │ (reading/fetching data) │ │ ├──────────┼──────────────────────────────┼───────────────────────┤ │ DCL │ Data Control Language │ GRANT, REVOKE │ │ │ (permissions \u0026amp; access) │ │ ├──────────┼──────────────────────────────┼───────────────────────┤ │ TCL │ Transaction Control Language │ BEGIN, COMMIT, │ │ │ (grouping operations safely) │ ROLLBACK, SAVEPOINT │ └──────────┴──────────────────────────────┴───────────────────────┘ For beginners: Focus on DDL + DML + DQL - that\u0026rsquo;s 90% of what you\u0026rsquo;ll write day to day.\n5. Creating a Table - Full Breakdown Basic Syntax CREATE TABLE table_name ( column_name data_type constraints, column_name data_type constraints, ... ); Real Example - Step by Step CREATE TABLE students ( student_id SERIAL PRIMARY KEY, first_name VARCHAR(50) NOT NULL, last_name VARCHAR(50) NOT NULL, email VARCHAR(100) UNIQUE NOT NULL, age INT CHECK (age \u0026gt;= 5 AND age \u0026lt;= 100), enrolled_on DATE DEFAULT CURRENT_DATE, grade CHAR(1) CHECK (grade IN (\u0026#39;A\u0026#39;,\u0026#39;B\u0026#39;,\u0026#39;C\u0026#39;,\u0026#39;D\u0026#39;,\u0026#39;F\u0026#39;)), is_active BOOLEAN DEFAULT TRUE ); Let\u0026rsquo;s break down each part:\n┌────────────────┬──────────────┬────────────────────────────────┐ │ Part │ Example │ What it means │ ├────────────────┼──────────────┼────────────────────────────────┤ │ Column name │ student_id │ Name of the column │ │ Data type │ SERIAL │ What kind of data it holds │ │ Constraint │ PRIMARY KEY │ Rule that applies to the data │ │ Default value │ DEFAULT TRUE │ Value used if none given │ └────────────────┴──────────────┴────────────────────────────────┘ 6. Data Types in PostgreSQL Most Commonly Used ┌──────────────┬────────────────────────────────────────────────────┐ │ Data Type │ Description \u0026amp; Example │ ├──────────────┼────────────────────────────────────────────────────┤ │ INT │ Whole numbers: 1, 42, -7 │ │ BIGINT │ Very large whole numbers │ │ SERIAL │ Auto-incrementing integer (1, 2, 3…) - use for IDs │ │ NUMERIC(p,s) │ Exact decimal: NUMERIC(10,2) → 99999999.99 │ │ FLOAT │ Approximate decimal (for science, not money) │ │ VARCHAR(n) │ Text up to n characters: VARCHAR(50) │ │ TEXT │ Unlimited length text │ │ CHAR(n) │ Fixed-length text - always exactly n chars │ │ BOOLEAN │ TRUE or FALSE │ │ DATE │ Date only: \u0026#39;2024-01-15\u0026#39; │ │ TIMESTAMP │ Date + time: \u0026#39;2024-01-15 09:30:00\u0026#39; │ │ TIME │ Time only: \u0026#39;14:30:00\u0026#39; │ └──────────────┴────────────────────────────────────────────────────┘ Choosing the Right Type -- ✅ Good choices salary NUMERIC(12, 2) -- money: exact, 2 decimal places phone VARCHAR(20) -- phone numbers vary in length is_member BOOLEAN -- yes/no flag birth_date DATE -- just a date, no time needed created_at TIMESTAMP -- when a record was created -- ❌ Common mistakes salary FLOAT -- BAD: floating point rounds money! age VARCHAR(10) -- BAD: age is a number, not text 7. Constraints - Rules for Your Data Constraints are rules enforced by the database. If you try to break them, the database refuses and gives an error.\nOverview of All Constraints ┌─────────────────┬───────────────────────────────────────────────────────┐ │ Constraint │ What it does │ ├─────────────────┼───────────────────────────────────────────────────────┤ │ PRIMARY KEY │ Uniquely identifies each row. Cannot be NULL. │ │ NOT NULL │ This column must always have a value. │ │ UNIQUE │ No two rows can have the same value in this column. │ │ CHECK │ Value must pass a custom condition. │ │ DEFAULT │ Uses a fallback value if none is provided. │ │ FOREIGN KEY │ Links to a PRIMARY KEY in another table. │ └─────────────────┴───────────────────────────────────────────────────────┘ Examples of Each CREATE TABLE employees ( -- PRIMARY KEY: every employee needs a unique ID emp_id SERIAL PRIMARY KEY, -- NOT NULL: name is required, cannot be left empty full_name VARCHAR(100) NOT NULL, -- UNIQUE: no two employees can share the same email email VARCHAR(100) UNIQUE, -- CHECK: salary must be a positive number salary NUMERIC(10,2) CHECK (salary \u0026gt; 0), -- DEFAULT: if no join date is given, use today\u0026#39;s date joined_on DATE DEFAULT CURRENT_DATE, -- FOREIGN KEY: dept_id must exist in the departments table dept_id INT REFERENCES departments(dept_id) ); 8. Primary Key \u0026amp; Foreign Key Primary Key A Primary Key is the unique identifier for each row in a table. Think of it like a fingerprint - no two rows can have the same primary key, and it can never be NULL.\n-- When you use SERIAL PRIMARY KEY: -- PostgreSQL automatically generates 1, 2, 3, 4... -- You never have to provide it manually CREATE TABLE customers ( customer_id SERIAL PRIMARY KEY, -- auto-generated: 1, 2, 3... name VARCHAR(100) NOT NULL ); INSERT INTO customers (name) VALUES (\u0026#39;Alice\u0026#39;); -- customer_id = 1 INSERT INTO customers (name) VALUES (\u0026#39;Bob\u0026#39;); -- customer_id = 2 INSERT INTO customers (name) VALUES (\u0026#39;Charlie\u0026#39;); -- customer_id = 3 Foreign Key A Foreign Key is a column in one table that refers to the Primary Key of another table. It creates a link (relationship) between two tables.\n-- Table 1: departments (the \u0026#34;parent\u0026#34; table) CREATE TABLE departments ( dept_id SERIAL PRIMARY KEY, dept_name VARCHAR(100) NOT NULL ); -- Table 2: employees (the \u0026#34;child\u0026#34; table) CREATE TABLE employees ( emp_id SERIAL PRIMARY KEY, name VARCHAR(100) NOT NULL, dept_id INT REFERENCES departments(dept_id) -- ↑ This is the FOREIGN KEY -- It MUST match a dept_id that exists in departments ); How They Work Together departments table employees table ┌─────────┬──────────────┐ ┌────────┬────────┬─────────┐ │ dept_id │ dept_name │ │ emp_id │ name │ dept_id │ ├─────────┼──────────────┤ ├────────┼────────┼─────────┤ │ 1 │ Engineering │◄────│ 1 │ Alice │ 1 │ │ 2 │ Marketing │◄────│ 2 │ Bob │ 2 │ │ 3 │ HR │ │ 3 │ Carol │ 1 │ └─────────┴──────────────┘ └────────┴────────┴─────────┘ ↑ Alice and Carol both work in Engineering (dept_id=1) Bob works in Marketing (dept_id=2) The rule: You cannot put a dept_id in the employees table that doesn\u0026rsquo;t exist in the departments table. If you try, the database will refuse.\n9. Common SQL Commands with Examples We\u0026rsquo;ll use this simple table for all examples:\nCREATE TABLE products ( product_id SERIAL PRIMARY KEY, name VARCHAR(100) NOT NULL, category VARCHAR(50), price NUMERIC(8,2) CHECK (price \u0026gt;= 0), stock INT DEFAULT 0, is_available BOOLEAN DEFAULT TRUE ); CREATE - Make a new table or database -- Create a database CREATE DATABASE school; -- Create a table (already shown above) CREATE TABLE products ( ... ); Think of it as: Building the empty filing cabinet drawer before you put any files in it.\nINSERT - Add new rows of data -- Insert one row INSERT INTO products (name, category, price, stock) VALUES (\u0026#39;Apple MacBook Pro\u0026#39;, \u0026#39;Laptops\u0026#39;, 1999.99, 15); -- Insert multiple rows at once INSERT INTO products (name, category, price, stock) VALUES (\u0026#39;Samsung TV 55\u0026#34;\u0026#39;, \u0026#39;Electronics\u0026#39;, 799.99, 30), (\u0026#39;Nike Running Shoes\u0026#39;,\u0026#39;Clothing\u0026#39;, 129.99, 50), (\u0026#39;Python Cookbook\u0026#39;, \u0026#39;Books\u0026#39;, 49.99, 100), (\u0026#39;USB-C Cable\u0026#39;, \u0026#39;Accessories\u0026#39;, 9.99, 200); Note: You don\u0026rsquo;t provide product_id - SERIAL generates it automatically. Note: You don\u0026rsquo;t provide is_available - it uses the DEFAULT TRUE.\nSELECT - Read / query data -- Get all columns, all rows SELECT * FROM products; -- Get specific columns only SELECT name, price FROM products; -- Filter rows with WHERE SELECT name, price FROM products WHERE category = \u0026#39;Electronics\u0026#39;; -- Sort results SELECT name, price FROM products ORDER BY price DESC; -- DESC = highest first, ASC = lowest first -- Limit how many rows you get back SELECT name, price FROM products ORDER BY price DESC LIMIT 3; -- Filter with multiple conditions SELECT name, price FROM products WHERE category = \u0026#39;Laptops\u0026#39; AND price \u0026lt; 2000; SELECT never changes your data. It is always safe to run.\nUPDATE - Modify existing rows -- Update one column for one specific row UPDATE products SET price = 1899.99 WHERE product_id = 1; -- Update multiple columns at once UPDATE products SET price = 89.99, stock = 75 WHERE name = \u0026#39;Nike Running Shoes\u0026#39;; -- Update all rows in a category UPDATE products SET is_available = FALSE WHERE stock = 0; ⚠️ Always use WHERE with UPDATE. Without it, you update EVERY row in the table!\n-- DANGEROUS - updates ALL products: UPDATE products SET price = 0; -- ❌ Don\u0026#39;t do this by accident! DELETE - Remove rows from a table -- Delete one specific row DELETE FROM products WHERE product_id = 3; -- Delete all rows matching a condition DELETE FROM products WHERE is_available = FALSE; -- Delete all rows (keeps the table structure, removes all data) DELETE FROM products; ⚠️ Always use WHERE with DELETE. Without it, you delete ALL rows!\nALTER - Change the structure of an existing table -- Add a new column ALTER TABLE products ADD COLUMN discount_pct NUMERIC(5,2) DEFAULT 0; -- Remove a column ALTER TABLE products DROP COLUMN discount_pct; -- Rename a column ALTER TABLE products RENAME COLUMN name TO product_name; -- Change a column\u0026#39;s data type ALTER TABLE products ALTER COLUMN stock TYPE BIGINT; -- Add a constraint after the fact ALTER TABLE products ADD CONSTRAINT chk_stock CHECK (stock \u0026gt;= 0); -- Rename the table itself ALTER TABLE products RENAME TO inventory; DROP - Permanently delete a table or database -- Delete the entire table (structure AND all data) DROP TABLE products; -- Safe version - won\u0026#39;t error if table doesn\u0026#39;t exist DROP TABLE IF EXISTS products; -- Delete a database entirely DROP DATABASE school; ⚠️ DROP is permanent and irreversible. There is no undo. Always double-check before running.\nTRUNCATE - Remove all data but keep the table -- Empties the table completely, but the table itself still exists TRUNCATE TABLE products; -- Also resets the auto-increment counter back to 1 TRUNCATE TABLE products RESTART IDENTITY; Quick Comparison: DELETE vs DROP vs TRUNCATE ┌──────────┬────────────────────────────┬─────────────┬────────────────┐ │ Command │ What it removes │ Table stays?│ Can filter? │ ├──────────┼────────────────────────────┼─────────────┼────────────────┤ │ DELETE │ Rows (can be filtered) │ Yes ✅ │ Yes (WHERE) │ │ TRUNCATE │ All rows (no filter) │ Yes ✅ │ No │ │ DROP │ Everything (table + data) │ No ❌ │ No │ └──────────┴────────────────────────────┴─────────────┴────────────────┘ 10. What Happens When You Violate a Constraint? The database will reject the operation and give you an error. Your data is NOT changed. Let\u0026rsquo;s see each one.\nNOT NULL Violation -- ❌ Trying to insert a row without the required \u0026#39;name\u0026#39; column INSERT INTO products (category, price) VALUES (\u0026#39;Electronics\u0026#39;, 999.99); ERROR: null value in column \u0026#34;name\u0026#34; of relation \u0026#34;products\u0026#34; violates not-null constraint DETAIL: Failing row contains (6, null, Electronics, 999.99, 0, true). Fix: Always provide a value for NOT NULL columns.\nUNIQUE Violation -- Setup: email must be unique CREATE TABLE users ( user_id SERIAL PRIMARY KEY, email VARCHAR(100) UNIQUE NOT NULL ); INSERT INTO users (email) VALUES (\u0026#39;alice@gmail.com\u0026#39;); -- ✅ Works INSERT INTO users (email) VALUES (\u0026#39;alice@gmail.com\u0026#39;); -- ❌ FAILS ERROR: duplicate key value violates unique constraint \u0026#34;users_email_key\u0026#34; DETAIL: Key (email)=(alice@gmail.com) already exists. Fix: Each email must be different.\nCHECK Constraint Violation -- ❌ Price cannot be negative INSERT INTO products (name, price) VALUES (\u0026#39;Mystery Item\u0026#39;, -50.00); ERROR: new row for relation \u0026#34;products\u0026#34; violates check constraint \u0026#34;products_price_check\u0026#34; DETAIL: Failing row contains (7, Mystery Item, null, -50.00, 0, true). Fix: Price must be \u0026gt;= 0.\nPRIMARY KEY Violation -- ❌ Trying to manually insert a duplicate primary key INSERT INTO products (product_id, name, price) VALUES (1, \u0026#39;Duplicate Product\u0026#39;, 99.99); ERROR: duplicate key value violates unique constraint \u0026#34;products_pkey\u0026#34; DETAIL: Key (product_id)=(1) already exists. Fix: Use SERIAL and don\u0026rsquo;t specify product_id - let the database generate it.\nFOREIGN KEY Violation -- Setup CREATE TABLE departments ( dept_id SERIAL PRIMARY KEY, dept_name VARCHAR(100) ); CREATE TABLE employees ( emp_id SERIAL PRIMARY KEY, name VARCHAR(100), dept_id INT REFERENCES departments(dept_id) ); INSERT INTO departments VALUES (1, \u0026#39;Engineering\u0026#39;); -- ❌ dept_id = 999 doesn\u0026#39;t exist in departments! INSERT INTO employees (name, dept_id) VALUES (\u0026#39;Alice\u0026#39;, 999); ERROR: insert or update on table \u0026#34;employees\u0026#34; violates foreign key constraint \u0026#34;employees_dept_id_fkey\u0026#34; DETAIL: Key (dept_id)=(999) is not present in table \u0026#34;departments\u0026#34;. Fix: Insert the department first, then insert the employee with a valid dept_id.\nTrying to DROP a Referenced Table -- ❌ Can\u0026#39;t delete departments while employees depend on it DROP TABLE departments; ERROR: cannot drop table departments because other objects depend on it DETAIL: constraint employees_dept_id_fkey on table employees depends on table departments HINT: Use DROP ... CASCADE to drop the dependent objects too. Fix: Drop the child table (employees) first, then drop the parent (departments). Or use CASCADE.\n11. Practice Table \u0026amp; 10 Beginner Questions The Table Run this to set up your practice data:\n-- Create the table CREATE TABLE books ( book_id SERIAL PRIMARY KEY, title VARCHAR(200) NOT NULL, author VARCHAR(100) NOT NULL, genre VARCHAR(50), price NUMERIC(6,2) CHECK (price \u0026gt;= 0), pages INT CHECK (pages \u0026gt; 0), published INT, -- year e.g. 2020 in_stock BOOLEAN DEFAULT TRUE ); -- Insert sample data INSERT INTO books (title, author, genre, price, pages, published, in_stock) VALUES (\u0026#39;The Alchemist\u0026#39;, \u0026#39;Paulo Coelho\u0026#39;, \u0026#39;Fiction\u0026#39;, 14.99, 208, 1988, TRUE), (\u0026#39;Atomic Habits\u0026#39;, \u0026#39;James Clear\u0026#39;, \u0026#39;Self-Help\u0026#39;, 16.99, 320, 2018, TRUE), (\u0026#39;Sapiens\u0026#39;, \u0026#39;Yuval Noah Harari\u0026#39;, \u0026#39;History\u0026#39;, 18.99, 443, 2011, TRUE), (\u0026#39;The Great Gatsby\u0026#39;, \u0026#39;F. Scott Fitzgerald\u0026#39;,\u0026#39;Fiction\u0026#39;, 9.99, 180, 1925, TRUE), (\u0026#39;Thinking, Fast and Slow\u0026#39;, \u0026#39;Daniel Kahneman\u0026#39;, \u0026#39;Psychology\u0026#39;, 17.99, 499, 2011, FALSE), (\u0026#39;1984\u0026#39;, \u0026#39;George Orwell\u0026#39;, \u0026#39;Fiction\u0026#39;, 10.99, 328, 1949, TRUE), (\u0026#39;Educated\u0026#39;, \u0026#39;Tara Westover\u0026#39;, \u0026#39;Memoir\u0026#39;, 15.99, 352, 2018, TRUE), (\u0026#39;The Lean Startup\u0026#39;, \u0026#39;Eric Ries\u0026#39;, \u0026#39;Business\u0026#39;, 19.99, 336, 2011, TRUE), (\u0026#39;Dune\u0026#39;, \u0026#39;Frank Herbert\u0026#39;, \u0026#39;Sci-Fi\u0026#39;, 13.99, 412, 1965, FALSE), (\u0026#39;To Kill a Mockingbird\u0026#39;, \u0026#39;Harper Lee\u0026#39;, \u0026#39;Fiction\u0026#39;, 11.99, 281, 1960, TRUE); The Table Contents book_id | title | author | genre | price | pages | published | in_stock --------+----------------------------+---------------------+------------+-------+-------+-----------+--------- 1 | The Alchemist | Paulo Coelho | Fiction | 14.99 | 208 | 1988 | true 2 | Atomic Habits | James Clear | Self-Help | 16.99 | 320 | 2018 | true 3 | Sapiens | Yuval Noah Harari | History | 18.99 | 443 | 2011 | true 4 | The Great Gatsby | F. Scott Fitzgerald | Fiction | 9.99 | 180 | 1925 | true 5 | Thinking, Fast and Slow | Daniel Kahneman | Psychology | 17.99 | 499 | 2011 | false 6 | 1984 | George Orwell | Fiction | 10.99 | 328 | 1949 | true 7 | Educated | Tara Westover | Memoir | 15.99 | 352 | 2018 | true 8 | The Lean Startup | Eric Ries | Business | 19.99 | 336 | 2011 | true 9 | Dune | Frank Herbert | Sci-Fi | 13.99 | 412 | 1965 | false 10 | To Kill a Mockingbird | Harper Lee | Fiction | 11.99 | 281 | 1960 | true 10 Practice Questions Try to write the query yourself before looking at the answer!\nQ1. Show all books in the table - every column, every row.\nAnswer SELECT * FROM books; Q2. Show only the title and author of every book.\nAnswer SELECT title, author FROM books; Q3. Show all books that belong to the 'Fiction' genre.\nAnswer SELECT * FROM books WHERE genre = \u0026#39;Fiction\u0026#39;; Q4. Show all books where the price is less than 15.00. Display only title, author, and price.\nAnswer SELECT title, author, price FROM books WHERE price \u0026lt; 15.00; Q5. Show all books that are currently in stock (in_stock = TRUE), sorted by price from lowest to highest.\nAnswer SELECT title, price FROM books WHERE in_stock = TRUE ORDER BY price ASC; Q6. How many books are in the table in total?\nAnswer SELECT COUNT(*) FROM books; -- Returns: 10 Q7. What is the most expensive book? Show its title and price.\nAnswer SELECT title, price FROM books ORDER BY price DESC LIMIT 1; Q8. A new book has arrived. Insert it into the table:\nTitle: 'Deep Work' Author: 'Cal Newport' Genre: 'Self-Help' Price: 15.49 Pages: 304 Published: 2016 In Stock: TRUE Answer INSERT INTO books (title, author, genre, price, pages, published, in_stock) VALUES (\u0026#39;Deep Work\u0026#39;, \u0026#39;Cal Newport\u0026#39;, \u0026#39;Self-Help\u0026#39;, 15.49, 304, 2016, TRUE); Q9. The price of 'Sapiens' has been reduced. Update its price to 14.99.\nAnswer UPDATE books SET price = 14.99 WHERE title = \u0026#39;Sapiens\u0026#39;; Q10. 'Dune' has just come back in stock. Update the in_stock column to TRUE for that book.\nAnswer UPDATE books SET in_stock = TRUE WHERE title = \u0026#39;Dune\u0026#39;; Quick Reference Card ┌───────────────────────────────────────────────────────────────┐ │ SQL QUICK REFERENCE │ ├──────────────────────┬────────────────────────────────────────┤ │ CREATE TABLE │ Build the table structure │ │ INSERT INTO │ Add new rows │ │ SELECT ... FROM │ Read / query data │ │ WHERE │ Filter rows │ │ ORDER BY │ Sort results (ASC / DESC) │ │ LIMIT │ Return only N rows │ │ UPDATE ... SET │ Change existing data │ │ DELETE FROM │ Remove rows │ │ ALTER TABLE │ Change table structure │ │ DROP TABLE │ Delete table permanently │ ├──────────────────────┴────────────────────────────────────────┤ │ GOLDEN RULES: │ │ ✅ Always use WHERE with UPDATE and DELETE │ │ ✅ DROP is permanent - double-check before running │ │ ✅ SERIAL handles your ID column automatically │ │ ✅ Constraints protect your data quality │ │ ✅ Foreign Key = link between two tables │ └───────────────────────────────────────────────────────────────┘ End of Lecture 01 - Happy querying! 🐘\n","permalink":"https://blogs.codeverra.com/blog/sql-part1/","summary":"\u003ch1 id=\"sql-lecture-01---introduction-to-sql--databases\"\u003eSQL Lecture 01 - Introduction to SQL \u0026amp; Databases\u003c/h1\u003e\n\u003cblockquote\u003e\n\u003cp\u003e\u003cstrong\u003ePostgreSQL\u003c/strong\u003e | Beginner Reference Sheet | Use this during class 🐘\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003chr\u003e\n\u003ch2 id=\"table-of-contents\"\u003eTable of Contents\u003c/h2\u003e\n\u003col\u003e\n\u003cli\u003e\u003ca href=\"#1-what-is-a-database\"\u003eWhat is a Database?\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#2-types-of-databases\"\u003eTypes of Databases\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#3-what-is-sql\"\u003eWhat is SQL?\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#4-how-sql-commands-are-grouped\"\u003eHow SQL Commands Are Grouped\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#5-creating-a-table--full-breakdown\"\u003eCreating a Table  -  Full Breakdown\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#6-data-types-in-postgresql\"\u003eData Types in PostgreSQL\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#7-constraints--rules-for-your-data\"\u003eConstraints  -  Rules for Your Data\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#8-primary-key--foreign-key\"\u003ePrimary Key \u0026amp; Foreign Key\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#9-common-sql-commands-with-examples\"\u003eCommon SQL Commands with Examples\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#10-what-happens-when-you-violate-a-constraint\"\u003eWhat Happens When You Violate a Constraint?\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#11-practice-table--10-beginner-questions\"\u003ePractice Table \u0026amp; 10 Beginner Questions\u003c/a\u003e\u003c/li\u003e\n\u003c/ol\u003e\n\u003chr\u003e\n\u003ch2 id=\"1-what-is-a-database\"\u003e1. What is a Database?\u003c/h2\u003e\n\u003cp\u003eA \u003cstrong\u003edatabase\u003c/strong\u003e is an organised collection of data stored so it can be easily accessed, managed, and updated.\u003c/p\u003e","tags":["sql","data-analysis"],"title":"Introduction to SQL and Databases"},{"content":"Python Iterables, Iterators \u0026amp; Generators A Complete Lesson Plan \u0026amp; Reference Guide Table of Contents The Big Picture Iterables Iterators The Iterator Protocol Building Custom Iterators Generators Generator Expressions Advanced Generator Features Common Pitfalls \u0026amp; Things to Keep in Mind Real-Life Applications Practice Questions \u0026amp; Answers Quick Reference Cheat Sheet The Big Picture Before diving in, here\u0026rsquo;s the conceptual map you need to hold in your head:\n┌──────────────────────────────────────────────────┐ │ ITERABLE │ │ Any object you can loop over with a for loop │ │ Examples: list, tuple, str, dict, set, file │ │ │ │ Has __iter__() → returns an Iterator │ └──────────────────────┬───────────────────────────┘ │ __iter__() called ▼ ┌──────────────────────────────────────────────────┐ │ ITERATOR │ │ An object that knows where it is in a sequence │ │ and can produce the next value on demand │ │ │ │ Has __iter__() → returns self │ │ Has __next__() → returns next value or raises │ │ StopIteration │ └──────────────────────┬───────────────────────────┘ │ special case of ▼ ┌──────────────────────────────────────────────────┐ │ GENERATOR │ │ A convenient way to create iterators using │ │ the `yield` keyword - Python auto-builds the │ │ iterator protocol for you │ └──────────────────────────────────────────────────┘ Key Insight: Every iterator is an iterable, but not every iterable is an iterator. A list is an iterable but NOT an iterator. iter(list) gives you an iterator.\nIterables What is an Iterable? An iterable is any Python object that can return an iterator when iter() is called on it. In practical terms: if you can use it in a for loop, it\u0026rsquo;s an iterable.\n# All of these are iterables my_list = [1, 2, 3] my_tuple = (4, 5, 6) my_string = \u0026#34;hello\u0026#34; my_dict = {\u0026#34;a\u0026#34;: 1, \u0026#34;b\u0026#34;: 2} my_set = {10, 20, 30} my_range = range(5) for item in my_list: print(item) # works for char in my_string: print(char) # works for key in my_dict: print(key) # works How Does Python Know Something is Iterable? Python looks for the __iter__ dunder method. If an object has it, Python calls it to get an iterator.\n# Checking for iterability print(hasattr([1, 2, 3], \u0026#39;__iter__\u0026#39;)) # True print(hasattr(42, \u0026#39;__iter__\u0026#39;)) # False print(hasattr(\u0026#34;hello\u0026#34;, \u0026#39;__iter__\u0026#39;)) # True # Using isinstance with collections.abc from collections.abc import Iterable print(isinstance([1, 2, 3], Iterable)) # True print(isinstance(42, Iterable)) # False What Happens Under the Hood of a for Loop? numbers = [10, 20, 30] for num in numbers: print(num) # Python internally does EXACTLY this: _iterator = iter(numbers) # calls numbers.__iter__() while True: try: num = next(_iterator) # calls _iterator.__next__() print(num) except StopIteration: # iterator is exhausted break This is the magic behind every for loop in Python.\nIterators What is an Iterator? An iterator is an object that:\nHas a __iter__() method that returns self Has a __next__() method that returns the next value, or raises StopIteration when done my_list = [1, 2, 3] # Get an iterator from the list it = iter(my_list) print(type(it)) # \u0026lt;class \u0026#39;list_iterator\u0026#39;\u0026gt; # Call next() manually print(next(it)) # 1 print(next(it)) # 2 print(next(it)) # 3 # One more call raises StopIteration next(it) # StopIteration ← the iterator is exhausted Iterators are Stateful and One-Way Unlike a list, an iterator remembers its position. It can only move forward, never backward.\nmy_list = [1, 2, 3] it = iter(my_list) print(next(it)) # 1 print(next(it)) # 2 # You cannot go back to 1 # You cannot reset (unless you call iter() again on the original list) # The iterator and the list are SEPARATE objects: print(id(my_list) == id(it)) # False - different objects Key Difference: Iterable vs Iterator my_list = [1, 2, 3] it = iter(my_list) # A list is NOT an iterator - it has no __next__ print(hasattr(my_list, \u0026#39;__next__\u0026#39;)) # False print(hasattr(it, \u0026#39;__next__\u0026#39;)) # True # An iterator IS an iterable - iter(it) returns itself print(iter(it) is it) # True print(iter(my_list) is my_list) # False - a new iterator is created The Iterator Protocol The iterator protocol is the contract every iterator must follow:\nMethod Purpose Must Do __iter__() Make the object usable in a for loop Return self __next__() Get the next value Return next item OR raise StopIteration # Verifying any object follows the protocol from collections.abc import Iterator print(isinstance(iter([1, 2, 3]), Iterator)) # True print(isinstance([1, 2, 3], Iterator)) # False print(isinstance(iter(\u0026#34;hello\u0026#34;), Iterator)) # True Building Custom Iterators Example 1 - A Simple Counter class Counter: \u0026#34;\u0026#34;\u0026#34;An iterator that counts from `start` up to (but not including) `stop`.\u0026#34;\u0026#34;\u0026#34; def __init__(self, start, stop): self.current = start self.stop = stop def __iter__(self): \u0026#34;\u0026#34;\u0026#34;Return self - this object IS the iterator.\u0026#34;\u0026#34;\u0026#34; return self def __next__(self): \u0026#34;\u0026#34;\u0026#34;Return the next value, or raise StopIteration.\u0026#34;\u0026#34;\u0026#34; if self.current \u0026gt;= self.stop: raise StopIteration value = self.current self.current += 1 return value counter = Counter(1, 5) for num in counter: print(num) # Output: 1 2 3 4 # Because Counter is an iterator, it\u0026#39;s also an iterable: print(list(Counter(10, 14))) # [10, 11, 12, 13] Example 2 - Infinite Iterator (Fibonacci) Iterators don\u0026rsquo;t have to end! This is one of their most powerful features.\nclass FibonacciIterator: \u0026#34;\u0026#34;\u0026#34;Generates Fibonacci numbers indefinitely.\u0026#34;\u0026#34;\u0026#34; def __init__(self): self.a = 0 self.b = 1 def __iter__(self): return self def __next__(self): value = self.a self.a, self.b = self.b, self.a + self.b return value fib = FibonacciIterator() # Take the first 10 Fibonacci numbers safely import itertools first_10 = list(itertools.islice(fib, 10)) print(first_10) # [0, 1, 1, 2, 3, 5, 8, 13, 21, 34] Example 3 - Separating Iterable from Iterator The best practice for containers (like lists) is to keep the iterable (the data holder) separate from the iterator (the traversal state):\nclass NumberRange: \u0026#34;\u0026#34;\u0026#34;An iterable that creates a fresh iterator each time.\u0026#34;\u0026#34;\u0026#34; def __init__(self, start, stop): self.start = start self.stop = stop def __iter__(self): # Return a NEW iterator object each time return NumberRangeIterator(self.start, self.stop) class NumberRangeIterator: \u0026#34;\u0026#34;\u0026#34;The actual iterator - holds traversal state.\u0026#34;\u0026#34;\u0026#34; def __init__(self, start, stop): self.current = start self.stop = stop def __iter__(self): return self def __next__(self): if self.current \u0026gt;= self.stop: raise StopIteration value = self.current self.current += 1 return value r = NumberRange(1, 4) # Because __iter__ returns a NEW iterator, you can loop multiple times! print(list(r)) # [1, 2, 3] print(list(r)) # [1, 2, 3] ← still works! # Compare with an iterator (single-use): it = iter(r) print(list(it)) # [1, 2, 3] print(list(it)) # [] ← exhausted! Generators What is a Generator? A generator is a special kind of function that uses the yield keyword to produce values one at a time, pausing execution between each yield. Python automatically creates an iterator object for you - you get all the benefits of writing an iterator without the boilerplate.\ndef simple_generator(): print(\u0026#34;Step 1\u0026#34;) yield 10 print(\u0026#34;Step 2\u0026#34;) yield 20 print(\u0026#34;Step 3\u0026#34;) yield 30 print(\u0026#34;Done!\u0026#34;) gen = simple_generator() print(type(gen)) # \u0026lt;class \u0026#39;generator\u0026#39;\u0026gt; print(next(gen)) # prints \u0026#34;Step 1\u0026#34;, then returns 10 print(next(gen)) # prints \u0026#34;Step 2\u0026#34;, then returns 20 print(next(gen)) # prints \u0026#34;Step 3\u0026#34;, then returns 30 next(gen) # prints \u0026#34;Done!\u0026#34;, then raises StopIteration The magic of yield: When yield is hit, the function\u0026rsquo;s entire state (local variables, position in code) is frozen. On the next next() call, execution resumes from exactly where it left off.\nGenerator vs Regular Function # Regular function - computes everything, returns all at once def get_squares_list(n): result = [] for i in range(n): result.append(i * i) return result # Returns a list of ALL values # Generator function - produces one value at a time, on demand def get_squares_gen(n): for i in range(n): yield i * i # Yields one value, then pauses # Both can be iterated: for val in get_squares_list(5): print(val) # works for val in get_squares_gen(5): print(val) # works # But memory usage is very different: import sys big_list = get_squares_list(1_000_000) big_gen = get_squares_gen(1_000_000) print(sys.getsizeof(big_list)) # ~8,000,056 bytes (~8 MB) print(sys.getsizeof(big_gen)) # ~104 bytes ← negligible! Rewriting the Custom Iterator as a Generator Compare the verbosity:\n# ----- Iterator approach: ~15 lines ----- class CounterIterator: def __init__(self, start, stop): self.current = start self.stop = stop def __iter__(self): return self def __next__(self): if self.current \u0026gt;= self.stop: raise StopIteration value = self.current self.current += 1 return value # ----- Generator approach: 3 lines ----- def counter_gen(start, stop): while start \u0026lt; stop: yield start start += 1 # Both do the same thing: print(list(CounterIterator(1, 5))) # [1, 2, 3, 4] print(list(counter_gen(1, 5))) # [1, 2, 3, 4] yield from - Delegating to Sub-Generators yield from lets a generator delegate to another iterable or generator:\ndef inner(): yield 1 yield 2 yield 3 def outer(): yield 0 yield from inner() # delegates to inner() yield 4 print(list(outer())) # [0, 1, 2, 3, 4] # Useful for flattening nested structures def flatten(nested): for item in nested: if isinstance(item, list): yield from flatten(item) # recursive delegation else: yield item data = [1, [2, [3, 4], 5], [6, 7]] print(list(flatten(data))) # [1, 2, 3, 4, 5, 6, 7] Generator Expressions Just like list comprehensions, Python has generator expressions - a compact syntax for simple generators.\n# List comprehension - creates the full list in memory squares_list = [x**2 for x in range(10)] # Generator expression - lazy, no memory allocated upfront squares_gen = (x**2 for x in range(10)) print(type(squares_list)) # \u0026lt;class \u0026#39;list\u0026#39;\u0026gt; print(type(squares_gen)) # \u0026lt;class \u0026#39;generator\u0026#39;\u0026gt; # They produce the same values: print(list(squares_gen)) # [0, 1, 4, 9, 16, 25, 36, 49, 64, 81] # Generator expressions are great inside function calls: total = sum(x**2 for x in range(1000)) # No extra list created first_even = next(x for x in range(100) if x % 2 == 0) # 0 # Chaining with conditions big_even_squares = (x**2 for x in range(100) if x % 2 == 0 if x**2 \u0026gt; 100) print(list(big_even_squares)) # [144, 196, 256, ..., 9604] Advanced Generator Features send() - Two-Way Communication Generators can receive values back from the caller using send(). This transforms them from simple producers into coroutines.\ndef accumulator(): \u0026#34;\u0026#34;\u0026#34;Receives numbers and yields the running total.\u0026#34;\u0026#34;\u0026#34; total = 0 while True: value = yield total # yield sends total OUT, receives value IN if value is None: break total += value gen = accumulator() next(gen) # Must call next() once to start the generator (advances to first yield) print(gen.send(10)) # 10 print(gen.send(20)) # 30 print(gen.send(5)) # 35 throw() - Injecting Exceptions def safe_generator(): try: while True: yield \u0026#34;running\u0026#34; except ValueError as e: yield f\u0026#34;caught error: {e}\u0026#34; yield \u0026#34;recovered\u0026#34; gen = safe_generator() print(next(gen)) # \u0026#39;running\u0026#39; print(gen.throw(ValueError, \u0026#34;oops!\u0026#34;)) # \u0026#39;caught error: oops!\u0026#39; print(next(gen)) # \u0026#39;recovered\u0026#39; close() - Shutting Down a Generator def countdown(): n = 10 while n \u0026gt; 0: try: yield n n -= 1 except GeneratorExit: print(\u0026#34;Generator shutting down cleanly.\u0026#34;) return # MUST return or re-raise GeneratorExit gen = countdown() print(next(gen)) # 10 print(next(gen)) # 9 gen.close() # \u0026#34;Generator shutting down cleanly.\u0026#34; Generator return Value A generator can return a value. It\u0026rsquo;s captured via StopIteration:\ndef gen_with_return(): yield 1 yield 2 return \u0026#34;finished\u0026#34; # This becomes StopIteration.value gen = gen_with_return() print(next(gen)) # 1 print(next(gen)) # 2 try: next(gen) except StopIteration as e: print(e.value) # \u0026#34;finished\u0026#34; # Or, using yield from: def wrapper(): result = yield from gen_with_return() print(f\u0026#34;Sub-generator returned: {result}\u0026#34;) Common Pitfalls \u0026amp; Things to Keep in Mind ⚠️ 1. Iterators are Single-Use nums = [1, 2, 3] it = iter(nums) print(list(it)) # [1, 2, 3] print(list(it)) # [] ← exhausted! No error, just empty. # Fix: Call iter() again, or use the original iterable print(list(iter(nums))) # [1, 2, 3] ✓ ⚠️ 2. Generators are Single-Use Too def gen(): yield 1 yield 2 g = gen() print(list(g)) # [1, 2] print(list(g)) # [] ← exhausted! # Fix: Call the generator function again to get a fresh generator g = gen() print(list(g)) # [1, 2] ✓ ⚠️ 3. Don\u0026rsquo;t Mix next() and for on the Same Iterator it = iter([1, 2, 3, 4, 5]) print(next(it)) # 1 ← consumed manually print(next(it)) # 2 ← consumed manually for val in it: # continues from 3, not 1! print(val) # 3, 4, 5 ⚠️ 4. The Generator Must Be Started with next() Before send() def my_gen(): val = yield gen = my_gen() # gen.send(10) ← TypeError! Can\u0026#39;t send non-None to just-started generator next(gen) # Advance to first yield first gen.send(10) # Now this works ⚠️ 5. return Inside a Generator Raises StopIteration, Not Returns a Value def gen(): yield 1 return # This raises StopIteration, NOT a normal return! yield 2 # Unreachable g = gen() print(next(g)) # 1 next(g) # StopIteration - the return triggered it ⚠️ 6. Lazy Evaluation Can Surprise You Generator expressions capture variables by reference, not by value:\n# BUG: All generators share the same variable `i` funcs = [lambda: i for i in range(3)] # list comprehension gens = [(lambda: i)() for i in range(3)] # captured correctly in list comp # With generators, be careful with closures: multipliers = [lambda x, n=n: x * n for n in range(5)] print(multipliers[3](10)) # 30 - n=3 captured correctly with default arg ⚠️ 7. Infinite Generators Need a Break Condition def infinite_counter(): n = 0 while True: yield n n += 1 # NEVER do this - it will run forever: # for x in infinite_counter(): print(x) # Always pair infinite generators with a limit: import itertools for x in itertools.islice(infinite_counter(), 10): print(x) # 0 through 9, then stops ⚠️ 8. Exception Handling Inside Generators def safe_divide_gen(numbers, divisor): for n in numbers: try: yield n / divisor except ZeroDivisionError: yield float(\u0026#39;inf\u0026#39;) # yield a sentinel instead of crashing results = list(safe_divide_gen([10, 20, 0, 30], 0)) print(results) # [inf, inf, inf, inf] Real-Life Applications Application 1 - Reading Large Files Line by Line When processing log files or CSVs that are gigabytes large, loading the entire file into memory is not feasible.\ndef read_large_file(filepath): \u0026#34;\u0026#34;\u0026#34;Memory-efficient file reader using a generator.\u0026#34;\u0026#34;\u0026#34; with open(filepath, \u0026#39;r\u0026#39;, encoding=\u0026#39;utf-8\u0026#39;) as f: for line in f: yield line.rstrip(\u0026#39;\\n\u0026#39;) def count_error_lines(filepath): \u0026#34;\u0026#34;\u0026#34;Count lines containing \u0026#39;ERROR\u0026#39; without loading the whole file.\u0026#34;\u0026#34;\u0026#34; return sum(1 for line in read_large_file(filepath) if \u0026#39;ERROR\u0026#39; in line) # Pipeline: read → filter → process def parse_csv_generator(filepath): reader = read_large_file(filepath) headers = next(reader).split(\u0026#39;,\u0026#39;) for line in reader: values = line.split(\u0026#39;,\u0026#39;) yield dict(zip(headers, values)) # Usage # for row in parse_csv_generator(\u0026#39;huge_data.csv\u0026#39;): # process(row) # only ONE row in memory at a time Application 2 - Database Query Streaming import sqlite3 def stream_query_results(db_path, query, params=()): \u0026#34;\u0026#34;\u0026#34; Execute a SQL query and yield rows one at a time. Great for large result sets that don\u0026#39;t fit in RAM. \u0026#34;\u0026#34;\u0026#34; conn = sqlite3.connect(db_path) cursor = conn.cursor() cursor.execute(query, params) try: for row in cursor: yield row finally: cursor.close() conn.close() # Usage - stream 1 million rows without loading them all into memory # for player_row in stream_query_results(\u0026#39;cricket.db\u0026#39;, \u0026#39;SELECT * FROM matches\u0026#39;): # analyze(player_row) Application 3 - Infinite Data Pipelines import time def sensor_data_stream(): \u0026#34;\u0026#34;\u0026#34;Simulates an IoT sensor producing readings indefinitely.\u0026#34;\u0026#34;\u0026#34; import random while True: yield { \u0026#34;timestamp\u0026#34;: time.time(), \u0026#34;temperature\u0026#34;: 20 + random.uniform(-2, 2), \u0026#34;humidity\u0026#34;: 60 + random.uniform(-5, 5), } def filter_anomalies(stream, threshold=22.0): \u0026#34;\u0026#34;\u0026#34;Filter only anomalous readings from a stream.\u0026#34;\u0026#34;\u0026#34; for reading in stream: if reading[\u0026#34;temperature\u0026#34;] \u0026gt; threshold: yield reading def enrich_readings(stream): \u0026#34;\u0026#34;\u0026#34;Add a human-readable timestamp to each reading.\u0026#34;\u0026#34;\u0026#34; for reading in stream: reading[\u0026#34;time_str\u0026#34;] = time.strftime(\u0026#39;%H:%M:%S\u0026#39;, time.localtime(reading[\u0026#34;timestamp\u0026#34;])) yield reading # Build a pipeline by chaining generators raw_stream = sensor_data_stream() anomaly_stream = filter_anomalies(raw_stream, threshold=21.5) enriched = enrich_readings(anomaly_stream) # Read from the pipeline - processing is LAZY, only happens when pulled import itertools for reading in itertools.islice(enriched, 3): print(reading) Application 4 - Pagination / API Scrolling import time def paginate_api(fetch_page_func, start_page=1): \u0026#34;\u0026#34;\u0026#34; Generic generator for paginated APIs. `fetch_page_func(page_num)` should return (data_list, has_more_bool). \u0026#34;\u0026#34;\u0026#34; page = start_page while True: data, has_more = fetch_page_func(page) yield from data # yield each item in the page if not has_more: break page += 1 time.sleep(0.1) # polite rate limiting # Simulated API def fake_api(page): all_data = list(range(1, 26)) # 25 items total page_size = 5 start = (page - 1) * page_size end = start + page_size chunk = all_data[start:end] has_more = end \u0026lt; len(all_data) return chunk, has_more # Usage for item in paginate_api(fake_api): print(item, end=\u0026#34; \u0026#34;) # 1 2 3 4 5 6 7 8 9 10 ... 25 Application 5 - Tree/Graph Traversal class TreeNode: def __init__(self, val, left=None, right=None): self.val = val self.left = left self.right = right def inorder_traversal(node): \u0026#34;\u0026#34;\u0026#34;In-order traversal of a BST as a generator.\u0026#34;\u0026#34;\u0026#34; if node is None: return yield from inorder_traversal(node.left) yield node.val yield from inorder_traversal(node.right) def bfs_traversal(root): \u0026#34;\u0026#34;\u0026#34;Breadth-first traversal using a generator.\u0026#34;\u0026#34;\u0026#34; from collections import deque queue = deque([root]) while queue: node = queue.popleft() if node: yield node.val queue.append(node.left) queue.append(node.right) # Build a small tree: # 4 # / \\ # 2 6 # / \\ / \\ # 1 3 5 7 root = TreeNode(4, TreeNode(2, TreeNode(1), TreeNode(3)), TreeNode(6, TreeNode(5), TreeNode(7)) ) print(list(inorder_traversal(root))) # [1, 2, 3, 4, 5, 6, 7] print(list(bfs_traversal(root))) # [4, 2, 6, 1, 3, 5, 7] Application 6 - Data Transformation Pipelines (ETL) def extract(raw_records): \u0026#34;\u0026#34;\u0026#34;Yield raw records from a source.\u0026#34;\u0026#34;\u0026#34; for record in raw_records: yield record def transform(records): \u0026#34;\u0026#34;\u0026#34;Clean and transform each record.\u0026#34;\u0026#34;\u0026#34; for record in records: if record.get(\u0026#39;salary\u0026#39;) and record.get(\u0026#39;name\u0026#39;): yield { \u0026#39;name\u0026#39;: record[\u0026#39;name\u0026#39;].strip().title(), \u0026#39;salary\u0026#39;: float(record[\u0026#39;salary\u0026#39;]), \u0026#39;department\u0026#39;: record.get(\u0026#39;dept\u0026#39;, \u0026#39;Unknown\u0026#39;).upper(), } def filter_high_earners(records, threshold=50000): \u0026#34;\u0026#34;\u0026#34;Keep only high earners.\u0026#34;\u0026#34;\u0026#34; for record in records: if record[\u0026#39;salary\u0026#39;] \u0026gt;= threshold: yield record def load(records, output_file): \u0026#34;\u0026#34;\u0026#34;Write records to output.\u0026#34;\u0026#34;\u0026#34; import json with open(output_file, \u0026#39;w\u0026#39;) as f: for record in records: f.write(json.dumps(record) + \u0026#39;\\n\u0026#39;) # ETL pipeline - each stage is a generator, completely lazy raw_data = [ {\u0026#39;name\u0026#39;: \u0026#39; ravi kumar \u0026#39;, \u0026#39;salary\u0026#39;: \u0026#39;75000\u0026#39;, \u0026#39;dept\u0026#39;: \u0026#39;engineering\u0026#39;}, {\u0026#39;name\u0026#39;: \u0026#39;Priya Sharma\u0026#39;, \u0026#39;salary\u0026#39;: \u0026#39;45000\u0026#39;, \u0026#39;dept\u0026#39;: \u0026#39;marketing\u0026#39;}, {\u0026#39;name\u0026#39;: \u0026#39;Arjun Patel \u0026#39;, \u0026#39;salary\u0026#39;: \u0026#39;92000\u0026#39;, \u0026#39;dept\u0026#39;: \u0026#39;data science\u0026#39;}, ] pipeline = filter_high_earners(transform(extract(raw_data)), threshold=60000) for result in pipeline: print(result) # {\u0026#39;name\u0026#39;: \u0026#39;Ravi Kumar\u0026#39;, \u0026#39;salary\u0026#39;: 75000.0, \u0026#39;department\u0026#39;: \u0026#39;ENGINEERING\u0026#39;} # {\u0026#39;name\u0026#39;: \u0026#39;Arjun Patel\u0026#39;, \u0026#39;salary\u0026#39;: 92000.0, \u0026#39;department\u0026#39;: \u0026#39;DATA SCIENCE\u0026#39;} Practice Questions \u0026amp; Answers ✅ Question 1 - Basics Q: What is the difference between an iterable and an iterator? Give two examples of each.\nA:\nAn iterable is any object you can loop over - it has a __iter__() method that returns an iterator when called. Examples: list, str, dict, tuple, set, range.\nAn iterator is an object that maintains traversal state - it has both __iter__() (returns self) and __next__() (returns the next value or raises StopIteration). Examples: the object returned by iter([1, 2, 3]), or a generator object.\nlst = [1, 2, 3] # iterable, NOT an iterator it = iter(lst) # iterator print(hasattr(lst, \u0026#39;__next__\u0026#39;)) # False print(hasattr(it, \u0026#39;__next__\u0026#39;)) # True print(iter(it) is it) # True - iterator is its own iterable ✅ Question 2 - Custom Iterator Q: Implement a class Squares that works as an iterator, yielding the squares of integers from 1 up to n.\n# Expected: sq = Squares(5) print(list(sq)) # [1, 4, 9, 16, 25] A:\nclass Squares: def __init__(self, n): self.n = n self.current = 1 def __iter__(self): return self def __next__(self): if self.current \u0026gt; self.n: raise StopIteration value = self.current ** 2 self.current += 1 return value sq = Squares(5) print(list(sq)) # [1, 4, 9, 16, 25] # Also works in a for loop: for val in Squares(4): print(val) # 1, 4, 9, 16 ✅ Question 3 - Generator Function Q: Write a generator function even_numbers(limit) that yields all even numbers from 0 up to limit (inclusive).\nA:\ndef even_numbers(limit): for i in range(0, limit + 1, 2): yield i print(list(even_numbers(10))) # [0, 2, 4, 6, 8, 10] print(list(even_numbers(7))) # [0, 2, 4, 6] # Can also be written as a generator expression: gen = (x for x in range(0, 11, 2)) print(list(gen)) # [0, 2, 4, 6, 8, 10] ✅ Question 4 - Infinite Generator + itertools.islice Q: Write a generator powers_of_two() that yields 1, 2, 4, 8, 16, ... indefinitely. Then use itertools.islice to print the first 8 values.\nA:\nimport itertools def powers_of_two(): value = 1 while True: yield value value *= 2 result = list(itertools.islice(powers_of_two(), 8)) print(result) # [1, 2, 4, 8, 16, 32, 64, 128] ✅ Question 5 - yield from Q: Write a generator chain_iterables(*iterables) that yields all items from multiple iterables one after another (similar to itertools.chain).\nA:\ndef chain_iterables(*iterables): for it in iterables: yield from it result = list(chain_iterables([1, 2], (3, 4), \u0026#34;56\u0026#34;)) print(result) # [1, 2, 3, 4, \u0026#39;5\u0026#39;, \u0026#39;6\u0026#39;] ✅ Question 6 - Memory Efficiency Q: You have a list of 10 million numbers. Using a generator, compute the sum of squares of all even numbers. Explain why this is more memory efficient.\nA:\nimport sys n = 10_000_000 # List approach - creates 3 large lists in memory even_list = [x for x in range(n) if x % 2 == 0] squares_list = [x**2 for x in even_list] result_list = sum(squares_list) # Generator approach - processes one number at a time result_gen = sum(x**2 for x in range(n) if x % 2 == 0) # Both give the same result: print(result_list == result_gen) # True # Memory comparison: print(sys.getsizeof(even_list)) # ~40 MB print(sys.getsizeof( (x**2 for x in range(n) if x % 2 == 0) )) # ~104 bytes (generator object only) The generator is more memory efficient because it never stores the intermediate lists. It computes each value on demand and immediately passes it to sum(). The generator object itself is a tiny state machine (~104 bytes), regardless of how large n is.\n✅ Question 7 - Tricky Exhaustion Q: What is the output of the following code? Explain why.\ngen = (x for x in range(5)) a = list(gen) b = list(gen) print(a, b) A:\n[0, 1, 2, 3, 4] [] list(gen) the first time exhausts the generator completely. Calling list(gen) a second time returns an empty list because the generator is already at StopIteration and there is no way to reset it. To get two copies, you need to call the generator function twice or convert to a list once and reuse it.\n✅ Question 8 - Pipeline Design Q: Write a generator pipeline to:\nTake a list of strings Strip whitespace Convert to uppercase Keep only strings longer than 3 characters Collect results A:\ndef strip_strings(strings): for s in strings: yield s.strip() def to_uppercase(strings): for s in strings: yield s.upper() def filter_by_length(strings, min_len): for s in strings: if len(s) \u0026gt; min_len: yield s raw = [\u0026#34; hello \u0026#34;, \u0026#34; hi \u0026#34;, \u0026#34; python \u0026#34;, \u0026#34; ok \u0026#34;, \u0026#34; world \u0026#34;, \u0026#34; go \u0026#34;] pipeline = filter_by_length( to_uppercase( strip_strings(raw) ), min_len=3 ) result = list(pipeline) print(result) # [\u0026#39;HELLO\u0026#39;, \u0026#39;PYTHON\u0026#39;, \u0026#39;WORLD\u0026#39;] ✅ Question 9 - send() Coroutine Q: Write a generator function running_average() that receives numbers via send() and yields the current running average after each number.\nA:\ndef running_average(): total = 0 count = 0 avg = None while True: value = yield avg if value is None: return total += value count += 1 avg = total / count gen = running_average() next(gen) # prime the generator print(gen.send(10)) # 10.0 print(gen.send(20)) # 15.0 print(gen.send(30)) # 20.0 print(gen.send(40)) # 25.0 ✅ Question 10 - Real World: Chunked File Processing Q: Write a generator that reads a file and yields it in chunks of n lines (useful for batch-processing large files).\nA:\ndef read_in_chunks(filepath, chunk_size=100): \u0026#34;\u0026#34;\u0026#34;Yield lists of `chunk_size` lines from a large file.\u0026#34;\u0026#34;\u0026#34; chunk = [] with open(filepath, \u0026#39;r\u0026#39;, encoding=\u0026#39;utf-8\u0026#39;) as f: for line in f: chunk.append(line.rstrip(\u0026#39;\\n\u0026#39;)) if len(chunk) == chunk_size: yield chunk chunk = [] # reset for next chunk if chunk: # yield remaining lines yield chunk # Usage: # for batch in read_in_chunks(\u0026#39;server_logs.txt\u0026#39;, chunk_size=500): # process_batch(batch) # handle 500 lines at a time Quick Reference Cheat Sheet ┌─────────────────────────────────────────────────────────────────────┐ │ QUICK REFERENCE │ ├──────────────────────┬──────────────────────────────────────────────┤ │ CONCEPT │ SUMMARY │ ├──────────────────────┼──────────────────────────────────────────────┤ │ Iterable │ Has __iter__() → returns iterator │ │ │ Examples: list, str, dict, set, range │ ├──────────────────────┼──────────────────────────────────────────────┤ │ Iterator │ Has __iter__() + __next__() │ │ │ Stateful, single-pass, raises StopIteration │ ├──────────────────────┼──────────────────────────────────────────────┤ │ Generator Function │ def f(): yield x → returns generator object │ ├──────────────────────┼──────────────────────────────────────────────┤ │ Generator Expression │ (x for x in iterable if condition) │ ├──────────────────────┼──────────────────────────────────────────────┤ │ yield │ Pauses function, returns value, resumes next │ ├──────────────────────┼──────────────────────────────────────────────┤ │ yield from │ Delegates to a sub-iterator/generator │ ├──────────────────────┼──────────────────────────────────────────────┤ │ send(val) │ Sends value INTO a generator (coroutine) │ ├──────────────────────┼──────────────────────────────────────────────┤ │ close() │ Throws GeneratorExit into generator │ ├──────────────────────┼──────────────────────────────────────────────┤ │ throw(exc) │ Injects an exception into the generator │ ├──────────────────────┼──────────────────────────────────────────────┤ │ iter(obj) │ Calls obj.__iter__() │ ├──────────────────────┼──────────────────────────────────────────────┤ │ next(it) │ Calls it.__next__(), returns next value │ └──────────────────────┴──────────────────────────────────────────────┘ WHEN TO USE WHAT ───────────────── Use a LIST when: • You need to access elements by index • You need to iterate multiple times • The dataset fits comfortably in memory Use an ITERATOR when: • You need custom traversal logic • You want to model a stateful sequence as a class Use a GENERATOR when: • Processing large/infinite data streams • Building lazy pipelines • Memory efficiency matters • Implementing coroutines with send() Use a GENERATOR EXPRESSION when: • Feeding data into sum(), max(), min(), list(), etc. • Simple filtering/mapping inline End of Lesson - Iterables, Iterators \u0026amp; Generators Codeverra - codeverra.com\n","permalink":"https://blogs.codeverra.com/blog/python-14-iterators-generators/","summary":"\u003ch1 id=\"python-iterables-iterators--generators\"\u003ePython Iterables, Iterators \u0026amp; Generators\u003c/h1\u003e\n\u003ch3 id=\"a-complete-lesson-plan--reference-guide\"\u003eA Complete Lesson Plan \u0026amp; Reference Guide\u003c/h3\u003e\n\u003chr\u003e\n\u003ch2 id=\"table-of-contents\"\u003eTable of Contents\u003c/h2\u003e\n\u003col\u003e\n\u003cli\u003e\u003ca href=\"#the-big-picture\"\u003eThe Big Picture\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#iterables\"\u003eIterables\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#iterators\"\u003eIterators\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#the-iterator-protocol\"\u003eThe Iterator Protocol\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#building-custom-iterators\"\u003eBuilding Custom Iterators\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#generators\"\u003eGenerators\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#generator-expressions\"\u003eGenerator Expressions\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#advanced-generator-features\"\u003eAdvanced Generator Features\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#common-pitfalls--things-to-keep-in-mind\"\u003eCommon Pitfalls \u0026amp; Things to Keep in Mind\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#real-life-applications\"\u003eReal-Life Applications\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#practice-questions--answers\"\u003ePractice Questions \u0026amp; Answers\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#quick-reference-cheat-sheet\"\u003eQuick Reference Cheat Sheet\u003c/a\u003e\u003c/li\u003e\n\u003c/ol\u003e\n\u003chr\u003e\n\u003ch2 id=\"the-big-picture\"\u003eThe Big Picture\u003c/h2\u003e\n\u003cp\u003eBefore diving in, here\u0026rsquo;s the conceptual map you need to hold in your head:\u003c/p\u003e\n\u003cpre tabindex=\"0\"\u003e\u003ccode\u003e┌──────────────────────────────────────────────────┐\n│                    ITERABLE                      │\n│  Any object you can loop over with a for loop    │\n│  Examples: list, tuple, str, dict, set, file     │\n│                                                  │\n│   Has __iter__() → returns an Iterator           │\n└──────────────────────┬───────────────────────────┘\n                       │ __iter__() called\n                       ▼\n┌──────────────────────────────────────────────────┐\n│                    ITERATOR                      │\n│  An object that knows where it is in a sequence │\n│  and can produce the next value on demand        │\n│                                                  │\n│   Has __iter__() → returns self                  │\n│   Has __next__() → returns next value or raises  │\n│                     StopIteration                │\n└──────────────────────┬───────────────────────────┘\n                       │ special case of\n                       ▼\n┌──────────────────────────────────────────────────┐\n│                   GENERATOR                      │\n│  A convenient way to create iterators using      │\n│  the `yield` keyword  -  Python auto-builds the   │\n│  iterator protocol for you                       │\n└──────────────────────────────────────────────────┘\n\u003c/code\u003e\u003c/pre\u003e\u003cblockquote\u003e\n\u003cp\u003e\u003cstrong\u003eKey Insight:\u003c/strong\u003e Every iterator is an iterable, but not every iterable is an iterator.\nA list is an iterable but NOT an iterator. \u003ccode\u003eiter(list)\u003c/code\u003e gives you an iterator.\u003c/p\u003e","tags":["python"],"title":"Iterators and Generators in Python - Complete Guide"},{"content":"Python\u0026rsquo;s itertools - The Missing Manual A deep dive into one of Python\u0026rsquo;s most underused standard library modules Wait, Why Should I Care About Another Module? You\u0026rsquo;ve written Python for a while. You know how to use for loops, list comprehensions, zip, and enumerate. Things work. So when someone says \u0026ldquo;hey, you should learn itertools\u0026rdquo;, it\u0026rsquo;s easy to nod politely and move on.\nHere\u0026rsquo;s what changes your mind. Imagine you\u0026rsquo;re asked to generate all possible batting orders for 4 players chosen from a squad of 11. Or you need to group a massive log file by date without loading it all into memory. Or you want to pair up every possible combination of feature columns for a machine learning experiment.\nWithout itertools, you end up writing nested loops, managing index variables, building intermediate lists, and generally fighting Python to do what should be simple. With itertools, each of those problems is one function call.\nThis guide is a complete reference for itertools - written not as a dry API listing, but as a walkthrough that builds your intuition for when to reach for it and why it works the way it does.\nWhat Exactly Is itertools? itertools is a module in Python\u0026rsquo;s standard library - no installation needed, just import itertools. It provides a collection of fast, memory-efficient tools for working with iterators.\nThe name is a compound of two words: iter (as in iterators) and tools (as in building blocks). That framing is intentional. The module\u0026rsquo;s author, Raymond Hettinger, designed it specifically to give you composable primitives - small, focused functions that you can chain together into powerful pipelines.\nEvery function in itertools returns an iterator, not a list. This is the fundamental design choice that makes the whole module useful:\nimport itertools # This does NOT build a list of 1 trillion numbers in memory counter = itertools.count(start=1) # It creates a lazy iterator - values computed one at a time, on demand print(next(counter)) # 1 print(next(counter)) # 2 print(next(counter)) # 3 # The other 999,999,999,997 values haven\u0026#39;t been computed yet If it returned a list, itertools.count() would be impossible to use. Because it returns an iterator, it\u0026rsquo;s perfectly fine - you pull values out one at a time.\nThe Problem It Solves Let\u0026rsquo;s be concrete. Here are three real situations where itertools replaces messy code.\nProblem 1: Combinations and Permutations You want to test every possible pair of columns in a dataset with 10 columns. How many pairs is that? 10 * 9 / 2 = 45. Writing nested loops is clunky and error-prone.\n# Without itertools - manual nested loop columns = [\u0026#39;age\u0026#39;, \u0026#39;salary\u0026#39;, \u0026#39;score\u0026#39;, \u0026#39;tenure\u0026#39;] pairs = [] for i in range(len(columns)): for j in range(i + 1, len(columns)): pairs.append((columns[i], columns[j])) print(pairs) # With itertools - one line import itertools pairs = list(itertools.combinations(columns, 2)) print(pairs) # [(\u0026#39;age\u0026#39;, \u0026#39;salary\u0026#39;), (\u0026#39;age\u0026#39;, \u0026#39;score\u0026#39;), (\u0026#39;age\u0026#39;, \u0026#39;tenure\u0026#39;), # (\u0026#39;salary\u0026#39;, \u0026#39;score\u0026#39;), (\u0026#39;salary\u0026#39;, \u0026#39;tenure\u0026#39;), (\u0026#39;score\u0026#39;, \u0026#39;tenure\u0026#39;)] Problem 2: Grouping Data You have a list of transactions sorted by category and want to total each category. Without itertools, you write a dict-accumulation loop. With it, you use groupby and the intent is obvious.\ntransactions = [ (\u0026#34;food\u0026#34;, 150), (\u0026#34;food\u0026#34;, 200), (\u0026#34;travel\u0026#34;, 800), (\u0026#34;travel\u0026#34;, 1200), (\u0026#34;food\u0026#34;, 75), (\u0026#34;utilities\u0026#34;, 900), ] # Without itertools totals = {} for category, amount in transactions: totals[category] = totals.get(category, 0) + amount # With itertools.groupby (after sorting) from itertools import groupby from operator import itemgetter sorted_txns = sorted(transactions, key=itemgetter(0)) for category, group in groupby(sorted_txns, key=itemgetter(0)): total = sum(amt for _, amt in group) print(f\u0026#34;{category}: Rs.{total}\u0026#34;) Problem 3: Infinite Sequences You need to cycle through a list of colours to assign alternating row colours in a report. You don\u0026rsquo;t know the number of rows ahead of time.\n# Without itertools - manual index tracking colours = [\u0026#39;white\u0026#39;, \u0026#39;lightgrey\u0026#39;] rows = get_report_rows() # could be any length for i, row in enumerate(rows): colour = colours[i % len(colours)] render_row(row, background=colour) # With itertools - the intent is crystal clear from itertools import cycle colours = cycle([\u0026#39;white\u0026#39;, \u0026#39;lightgrey\u0026#39;]) for row, colour in zip(get_report_rows(), colours): render_row(row, background=colour) In every case, itertools doesn\u0026rsquo;t just shorten the code - it makes the intent clearer. The function name tells you exactly what\u0026rsquo;s happening.\nHow to Think About itertools Before looking at individual functions, there\u0026rsquo;s a mental model that unifies the whole module. The functions fall into three categories:\n┌─────────────────────────────────────────────────────────────────────┐ │ ITERTOOLS FUNCTION FAMILIES │ ├──────────────────────────────┬──────────────────────────────────────┤ │ INFINITE ITERATORS │ Generate values without end │ │ count, cycle, repeat │ Always pair with islice or zip │ ├──────────────────────────────┼──────────────────────────────────────┤ │ FINITE ITERATORS │ Operate on finite input sequences │ │ chain, islice, zip_longest, │ Transform, filter, or combine │ │ starmap, takewhile, │ existing iterables │ │ dropwhile, groupby, │ │ │ compress, filterfalse, │ │ │ accumulate, pairwise, │ │ │ batched, product │ │ ├──────────────────────────────┼──────────────────────────────────────┤ │ COMBINATORIC ITERATORS │ Generate structured combinations │ │ combinations, │ Essential for search, testing, │ │ combinations_with_repl, │ and mathematical problems │ │ permutations, product │ │ └──────────────────────────────┴──────────────────────────────────────┘ One more thing to internalize: itertools functions are lazy. None of them do any work until you ask for the next value. This means you can chain ten itertools calls together and the total memory used is still proportional to one element at a time, not the full dataset.\nThings to Keep in Mind Before You Start These are the rules that trip people up most often. Read them once before diving into examples.\n1. All itertools functions return iterators, not lists. To see the values, wrap in list() or tuple(), or iterate in a loop. Don\u0026rsquo;t be surprised when print(itertools.chain([1,2],[3,4])) shows \u0026lt;itertools.chain object at 0x...\u0026gt;.\n2. Iterators are single-pass. Once exhausted, you can\u0026rsquo;t rewind. If you need to iterate twice, either call the function again or convert to a list.\nfrom itertools import chain it = chain([1, 2], [3, 4]) print(list(it)) # [1, 2, 3, 4] print(list(it)) # [] ← already exhausted! 3. groupby only groups consecutive elements. If your data isn\u0026rsquo;t sorted, groupby will produce multiple groups for the same key. Always sort first if you want all items of the same key together.\nfrom itertools import groupby data = [1, 1, 2, 1, 1] # 1s are NOT consecutive everywhere for key, group in groupby(data): print(key, list(group)) # 1 [1, 1] # 2 [2] # 1 [1, 1] ← a SECOND group of 1s, not merged with the first! 4. Always call islice to guard infinite iterators. itertools.count(), itertools.cycle(), and itertools.repeat() run forever. Passing them to list() will hang your program. Use islice to take only what you need.\n5. Combinatoric functions can produce astronomically large outputs. permutations(range(20)) produces 2,432,902,008,176,640,000 tuples. Use islice or consume lazily.\n6. chain takes multiple iterables as arguments; chain.from_iterable takes one iterable of iterables.\nfrom itertools import chain # chain(*args) - pass iterables directly list(chain([1,2], [3,4], [5,6])) # [1, 2, 3, 4, 5, 6] # chain.from_iterable - pass one iterable that contains iterables nested = [[1,2], [3,4], [5,6]] list(chain.from_iterable(nested)) # [1, 2, 3, 4, 5, 6] Part 1: Infinite Iterators These three functions generate values indefinitely. They\u0026rsquo;re the simplest functions in the module but among the most useful.\ncount(start=0, step=1) - An Infinite Counter count produces an infinite sequence of evenly-spaced numbers. Think of it as a programmable counter that never stops.\nfrom itertools import count, islice # Basic counting for n in islice(count(), 5): print(n, end=\u0026#34; \u0026#34;) # 0 1 2 3 4 # Start and step for n in islice(count(10, 5), 5): print(n, end=\u0026#34; \u0026#34;) # 10 15 20 25 30 # Counting backwards for n in islice(count(100, -10), 6): print(n, end=\u0026#34; \u0026#34;) # 100 90 80 70 60 50 # Floating point steps for n in islice(count(0.0, 0.5), 6): print(round(n, 1), end=\u0026#34; \u0026#34;) # 0.0 0.5 1.0 1.5 2.0 2.5 Real-world use case: Assigning auto-incrementing IDs to records coming in from a stream, without knowing how many records there will be.\nfrom itertools import count def assign_ids(records): \u0026#34;\u0026#34;\u0026#34;Attach a unique ID to each record in a stream.\u0026#34;\u0026#34;\u0026#34; id_counter = count(start=1001) for record in records: record[\u0026#34;id\u0026#34;] = next(id_counter) yield record players = [ {\u0026#34;name\u0026#34;: \u0026#34;Rohit Sharma\u0026#34;}, {\u0026#34;name\u0026#34;: \u0026#34;Virat Kohli\u0026#34;}, {\u0026#34;name\u0026#34;: \u0026#34;Jasprit Bumrah\u0026#34;}, ] for player in assign_ids(players): print(player) # {\u0026#39;name\u0026#39;: \u0026#39;Rohit Sharma\u0026#39;, \u0026#39;id\u0026#39;: 1001} # {\u0026#39;name\u0026#39;: \u0026#39;Virat Kohli\u0026#39;, \u0026#39;id\u0026#39;: 1002} # {\u0026#39;name\u0026#39;: \u0026#39;Jasprit Bumrah\u0026#39;, \u0026#39;id\u0026#39;: 1003} cycle(iterable) - Loop an Iterable Forever cycle takes any iterable and loops through it indefinitely: A, B, C, A, B, C, A, B, \u0026hellip;\nfrom itertools import cycle, islice colours = cycle([\u0026#39;red\u0026#39;, \u0026#39;green\u0026#39;, \u0026#39;blue\u0026#39;]) print(list(islice(colours, 8))) # [\u0026#39;red\u0026#39;, \u0026#39;green\u0026#39;, \u0026#39;blue\u0026#39;, \u0026#39;red\u0026#39;, \u0026#39;green\u0026#39;, \u0026#39;blue\u0026#39;, \u0026#39;red\u0026#39;, \u0026#39;green\u0026#39;] # Assign round-robin teams from itertools import cycle teams = cycle([\u0026#39;Team A\u0026#39;, \u0026#39;Team B\u0026#39;, \u0026#39;Team C\u0026#39;]) players = [\u0026#39;Ravi\u0026#39;, \u0026#39;Priya\u0026#39;, \u0026#39;Arjun\u0026#39;, \u0026#39;Meera\u0026#39;, \u0026#39;Suresh\u0026#39;, \u0026#39;Kavya\u0026#39;, \u0026#39;Dev\u0026#39;] assignments = list(zip(players, teams)) for player, team in assignments: print(f\u0026#34;{player} → {team}\u0026#34;) # Ravi → Team A # Priya → Team B # Arjun → Team C # Meera → Team A # Suresh → Team B # Kavya → Team C # Dev → Team A Real-world use case: Load balancing requests across multiple servers.\nfrom itertools import cycle servers = cycle([\u0026#39;server-1\u0026#39;, \u0026#39;server-2\u0026#39;, \u0026#39;server-3\u0026#39;]) def route_request(request): server = next(servers) print(f\u0026#34;Routing {request!r} → {server}\u0026#34;) route_request(\u0026#34;/api/users\u0026#34;) # → server-1 route_request(\u0026#34;/api/orders\u0026#34;) # → server-2 route_request(\u0026#34;/api/products\u0026#34;) # → server-3 route_request(\u0026#34;/api/users\u0026#34;) # → server-1 (cycles back) repeat(object, times=None) - Repeat a Value repeat produces the same value over and over, either a fixed number of times or infinitely.\nfrom itertools import repeat # Fixed repetition print(list(repeat(\u0026#34;hello\u0026#34;, 4))) # [\u0026#39;hello\u0026#39;, \u0026#39;hello\u0026#39;, \u0026#39;hello\u0026#39;, \u0026#39;hello\u0026#39;] # Often used with map to supply a constant second argument doubles = list(map(pow, range(1, 6), repeat(2))) print(doubles) # [1, 4, 9, 16, 25] - same as [x**2 for x in range(1,6)] # Filling a default grid grid = [list(repeat(0, 4)) for _ in repeat(None, 3)] print(grid) # [[0, 0, 0, 0], [0, 0, 0, 0], [0, 0, 0, 0]] Part 2: Finite Iterators These are the workhorses of the module. They transform, filter, slice, and combine iterables.\nchain(*iterables) - Flatten Multiple Iterables Into One chain takes multiple iterables and yields from each in sequence, as if they were one long iterable. It\u0026rsquo;s the lazy version of concatenation.\nfrom itertools import chain # Basic usage combined = list(chain([1, 2, 3], [4, 5], [6, 7, 8, 9])) print(combined) # [1, 2, 3, 4, 5, 6, 7, 8, 9] # Works with any iterables - lists, tuples, strings, generators result = list(chain(\u0026#34;IPL\u0026#34;, [2024], (\u0026#34;T20\u0026#34;, \u0026#34;Cricket\u0026#34;))) print(result) # [\u0026#39;I\u0026#39;, \u0026#39;P\u0026#39;, \u0026#39;L\u0026#39;, 2024, \u0026#39;T20\u0026#39;, \u0026#39;Cricket\u0026#39;] chain.from_iterable - when you have a list of lists (dynamic nesting):\nfrom itertools import chain # Flatten one level deep nested = [[1, 2, 3], [4, 5], [6, 7, 8]] flat = list(chain.from_iterable(nested)) print(flat) # [1, 2, 3, 4, 5, 6, 7, 8] # Real use: merging query results from multiple DB shards def get_orders_from_shards(shards): return chain.from_iterable(shard.get_orders() for shard in shards) # Real use: word tokenization across multiple sentences sentences = [\u0026#34;The quick brown fox\u0026#34;, \u0026#34;jumps over\u0026#34;, \u0026#34;the lazy dog\u0026#34;] words = list(chain.from_iterable(s.split() for s in sentences)) print(words) # [\u0026#39;The\u0026#39;, \u0026#39;quick\u0026#39;, \u0026#39;brown\u0026#39;, \u0026#39;fox\u0026#39;, \u0026#39;jumps\u0026#39;, \u0026#39;over\u0026#39;, \u0026#39;the\u0026#39;, \u0026#39;lazy\u0026#39;, \u0026#39;dog\u0026#39;] islice(iterable, stop) / islice(iterable, start, stop, step) - Lazy Slicing islice is like Python\u0026rsquo;s slice notation [start:stop:step] but for any iterator - including infinite ones. It does not support negative indices.\nfrom itertools import islice, count # Take first N elements first_five = list(islice(count(), 5)) print(first_five) # [0, 1, 2, 3, 4] # Skip first N, take next M result = list(islice(count(0), 5, 10)) print(result) # [5, 6, 7, 8, 9] # With a step result = list(islice(range(100), 0, 30, 5)) print(result) # [0, 5, 10, 15, 20, 25] # Paginate a large dataset - page 3, 10 items per page def get_page(iterable, page_number, page_size=10): start = (page_number - 1) * page_size return list(islice(iterable, start, start + page_size)) data = range(1, 101) print(get_page(data, page=3, page_size=10)) # [21, 22, ..., 30] zip_longest(*iterables, fillvalue=None) - Zip Without Truncating Python\u0026rsquo;s built-in zip stops at the shortest iterable. zip_longest continues until all iterables are exhausted, filling missing values with a default.\nfrom itertools import zip_longest names = [\u0026#39;Rohit\u0026#39;, \u0026#39;Virat\u0026#39;, \u0026#39;Dhoni\u0026#39;] scores = [89, 112] overs = [18.4, 20.0, 15.2, 19.1] result = list(zip_longest(names, scores, overs, fillvalue=\u0026#39;N/A\u0026#39;)) for row in result: print(row) # (\u0026#39;Rohit\u0026#39;, 89, 18.4) # (\u0026#39;Virat\u0026#39;, 112, 20.0) # (\u0026#39;Dhoni\u0026#39;, \u0026#39;N/A\u0026#39;, 15.2) # (\u0026#39;N/A\u0026#39;, \u0026#39;N/A\u0026#39;, 19.1) # Merging two ordered streams - useful in merge sort logic stream_a = [1, 3, 5, 7] stream_b = [2, 4, 6] pairs = list(zip_longest(stream_a, stream_b, fillvalue=0)) print(pairs) # [(1, 2), (3, 4), (5, 6), (7, 0)] accumulate(iterable, func=operator.add, *, initial=None) - Running Totals accumulate applies a function cumulatively - each output value is the result of applying the function to all elements seen so far. The default function is addition, giving running totals.\nfrom itertools import accumulate import operator # Running sum (default) sales = [120, 340, 210, 450, 180, 300] running_total = list(accumulate(sales)) print(running_total) # [120, 460, 670, 1120, 1300, 1600] # Running maximum - track best score seen so far match_scores = [67, 82, 45, 95, 71, 88, 100, 55] running_max = list(accumulate(match_scores, func=max)) print(running_max) # [67, 82, 82, 95, 95, 95, 100, 100] # Running product values = [1, 2, 3, 4, 5] running_product = list(accumulate(values, func=operator.mul)) print(running_product) # [1, 2, 6, 24, 120] # With initial value (Python 3.8+) running_with_start = list(accumulate([10, 20, 30], initial=100)) print(running_with_start) # [100, 110, 130, 160] # Real-world: cumulative rainfall over months rainfall_mm = [45, 62, 28, 90, 115, 70, 33, 80, 55, 40, 20, 10] months = [\u0026#39;Jan\u0026#39;,\u0026#39;Feb\u0026#39;,\u0026#39;Mar\u0026#39;,\u0026#39;Apr\u0026#39;,\u0026#39;May\u0026#39;,\u0026#39;Jun\u0026#39;,\u0026#39;Jul\u0026#39;,\u0026#39;Aug\u0026#39;,\u0026#39;Sep\u0026#39;,\u0026#39;Oct\u0026#39;,\u0026#39;Nov\u0026#39;,\u0026#39;Dec\u0026#39;] cumulative = list(accumulate(rainfall_mm)) for month, rain, total in zip(months, rainfall_mm, cumulative): print(f\u0026#34;{month}: {rain}mm (cumulative: {total}mm)\u0026#34;) takewhile(predicate, iterable) - Take While Condition Holds takewhile yields elements as long as the predicate returns True. It stops permanently at the first False - it does not resume.\nfrom itertools import takewhile numbers = [2, 4, 6, 8, 3, 10, 12] evens_until_odd = list(takewhile(lambda x: x % 2 == 0, numbers)) print(evens_until_odd) # [2, 4, 6, 8] ← stops at 3, does NOT include 10 or 12 # Reading a log file until a certain timestamp is reached log_lines = [ \u0026#34;2024-01-01 INFO Server started\u0026#34;, \u0026#34;2024-01-01 INFO Request received\u0026#34;, \u0026#34;2024-01-02 WARNING Disk space low\u0026#34;, \u0026#34;2024-01-03 ERROR Database timeout\u0026#34;, \u0026#34;2024-01-03 INFO Retry succeeded\u0026#34;, ] jan_first_logs = list(takewhile(lambda line: line.startswith(\u0026#34;2024-01-01\u0026#34;), log_lines)) print(jan_first_logs) # [\u0026#39;2024-01-01 INFO Server started\u0026#39;, \u0026#39;2024-01-01 INFO Request received\u0026#39;] # Streaming sensor data: read until temperature exceeds threshold def safe_readings(sensor_stream, max_temp=80): return takewhile(lambda reading: reading[\u0026#39;temp\u0026#39;] \u0026lt;= max_temp, sensor_stream) dropwhile(predicate, iterable) - Skip While Condition Holds dropwhile is the mirror of takewhile. It skips elements while the predicate is True, then yields everything from the first False onward.\nfrom itertools import dropwhile numbers = [2, 4, 6, 8, 3, 10, 12] after_first_odd = list(dropwhile(lambda x: x % 2 == 0, numbers)) print(after_first_odd) # [3, 10, 12] ← skipped 2,4,6,8; yields from 3 onward # Skip file header lines that start with \u0026#39;#\u0026#39; config_lines = [ \u0026#34;# Configuration file\u0026#34;, \u0026#34;# Generated automatically\u0026#34;, \u0026#34;host=localhost\u0026#34;, \u0026#34;port=5432\u0026#34;, \u0026#34;# This comment appears mid-file\u0026#34;, \u0026#34;db=mydb\u0026#34;, ] non_comment_lines = list(dropwhile(lambda l: l.startswith(\u0026#39;#\u0026#39;), config_lines)) print(non_comment_lines) # [\u0026#39;host=localhost\u0026#39;, \u0026#39;port=5432\u0026#39;, \u0026#39;# This comment appears mid-file\u0026#39;, \u0026#39;db=mydb\u0026#39;] # Note: the mid-file comment IS included because dropwhile only skips the leading ones filterfalse(predicate, iterable) - Keep What a Filter Would Reject While filter(func, iterable) keeps elements where func returns True, filterfalse keeps elements where it returns False.\nfrom itertools import filterfalse numbers = range(10) # filter keeps evens, filterfalse keeps odds evens = list(filter(lambda x: x % 2 == 0, numbers)) odds = list(filterfalse(lambda x: x % 2 == 0, numbers)) print(evens) # [0, 2, 4, 6, 8] print(odds) # [1, 3, 5, 7, 9] # Partition a list into two groups simultaneously def partition(predicate, iterable): \u0026#34;\u0026#34;\u0026#34;Split an iterable into (items where pred is True, items where False).\u0026#34;\u0026#34;\u0026#34; from itertools import filterfalse, tee t1, t2 = tee(iterable) return filter(predicate, t1), filterfalse(predicate, t2) students = [ {\u0026#34;name\u0026#34;: \u0026#34;Ravi\u0026#34;, \u0026#34;score\u0026#34;: 78}, {\u0026#34;name\u0026#34;: \u0026#34;Priya\u0026#34;, \u0026#34;score\u0026#34;: 55}, {\u0026#34;name\u0026#34;: \u0026#34;Arjun\u0026#34;, \u0026#34;score\u0026#34;: 92}, {\u0026#34;name\u0026#34;: \u0026#34;Meera\u0026#34;, \u0026#34;score\u0026#34;: 43}, ] passing, failing = partition(lambda s: s[\u0026#34;score\u0026#34;] \u0026gt;= 60, students) print(\u0026#34;Passed:\u0026#34;, [s[\u0026#34;name\u0026#34;] for s in passing]) # [\u0026#39;Ravi\u0026#39;, \u0026#39;Arjun\u0026#39;] print(\u0026#34;Failed:\u0026#34;, [s[\u0026#34;name\u0026#34;] for s in failing]) # [\u0026#39;Priya\u0026#39;, \u0026#39;Meera\u0026#39;] compress(data, selectors) - Boolean Masking compress takes two iterables: data and boolean selectors. It yields only the data items where the corresponding selector is True (or truthy). Think of it as numpy boolean indexing, but for any iterator.\nfrom itertools import compress data = [\u0026#39;A\u0026#39;, \u0026#39;B\u0026#39;, \u0026#39;C\u0026#39;, \u0026#39;D\u0026#39;, \u0026#39;E\u0026#39;, \u0026#39;F\u0026#39;] selectors = [True, False, True, False, True, True] selected = list(compress(data, selectors)) print(selected) # [\u0026#39;A\u0026#39;, \u0026#39;C\u0026#39;, \u0026#39;E\u0026#39;, \u0026#39;F\u0026#39;] # Real use: select columns from a CSV row based on a flag list columns = [\u0026#39;name\u0026#39;, \u0026#39;age\u0026#39;, \u0026#39;salary\u0026#39;, \u0026#39;city\u0026#39;, \u0026#39;phone\u0026#39;, \u0026#39;email\u0026#39;] include = [True, True, False, True, False, True ] selected_cols = list(compress(columns, include)) print(selected_cols) # [\u0026#39;name\u0026#39;, \u0026#39;age\u0026#39;, \u0026#39;city\u0026#39;, \u0026#39;email\u0026#39;] # Combine with a search result to mask items products = [\u0026#39;Laptop\u0026#39;, \u0026#39;Phone\u0026#39;, \u0026#39;Tablet\u0026#39;, \u0026#39;Watch\u0026#39;, \u0026#39;Earbuds\u0026#39;] in_stock = [True, False, True, True, False ] available = list(compress(products, in_stock)) print(available) # [\u0026#39;Laptop\u0026#39;, \u0026#39;Tablet\u0026#39;, \u0026#39;Watch\u0026#39;] groupby(iterable, key=None) - Group Consecutive Elements groupby scans through an iterable and groups consecutive elements that share the same key. It yields (key, group_iterator) pairs.\nCritical reminder: Only consecutive equal keys are grouped. Always sort first if you want all matching items together.\nfrom itertools import groupby from operator import itemgetter # Simple grouping data = [1, 1, 2, 2, 2, 3, 1, 1] for key, group in groupby(data): print(key, list(group)) # 1 [1, 1] # 2 [2, 2, 2] # 3 [3] # 1 [1, 1] ← separate group because not consecutive with the first # Real use: group sorted transactions by category transactions = [ (\u0026#34;food\u0026#34;, 120), (\u0026#34;food\u0026#34;, 340), (\u0026#34;travel\u0026#34;, 800), (\u0026#34;travel\u0026#34;, 1200), (\u0026#34;utilities\u0026#34;, 450), (\u0026#34;food\u0026#34;, 95), # ← would be a separate group if not pre-sorted ] sorted_txns = sorted(transactions, key=itemgetter(0)) print(\u0026#34;\\nCategory Summary:\u0026#34;) for category, group in groupby(sorted_txns, key=itemgetter(0)): amounts = [amt for _, amt in group] print(f\u0026#34; {category}: {len(amounts)} transactions, total Rs.{sum(amounts)}\u0026#34;) # food: 3 transactions, total Rs.555 # travel: 2 transactions, total Rs.2000 # utilities: 1 transactions, total Rs.450 Advanced groupby - group by computed property:\nfrom itertools import groupby ipl_players = [ {\u0026#34;name\u0026#34;: \u0026#34;Rohit Sharma\u0026#34;, \u0026#34;team\u0026#34;: \u0026#34;MI\u0026#34;}, {\u0026#34;name\u0026#34;: \u0026#34;Hardik Pandya\u0026#34;, \u0026#34;team\u0026#34;: \u0026#34;MI\u0026#34;}, {\u0026#34;name\u0026#34;: \u0026#34;Virat Kohli\u0026#34;, \u0026#34;team\u0026#34;: \u0026#34;RCB\u0026#34;}, {\u0026#34;name\u0026#34;: \u0026#34;Glenn Maxwell\u0026#34;, \u0026#34;team\u0026#34;: \u0026#34;RCB\u0026#34;}, {\u0026#34;name\u0026#34;: \u0026#34;MS Dhoni\u0026#34;, \u0026#34;team\u0026#34;: \u0026#34;CSK\u0026#34;}, {\u0026#34;name\u0026#34;: \u0026#34;Ruturaj Gaikwad\u0026#34;, \u0026#34;team\u0026#34;: \u0026#34;CSK\u0026#34;}, ] sorted_players = sorted(ipl_players, key=lambda p: p[\u0026#34;team\u0026#34;]) for team, members in groupby(sorted_players, key=lambda p: p[\u0026#34;team\u0026#34;]): names = [p[\u0026#34;name\u0026#34;] for p in members] print(f\u0026#34;{team}: {\u0026#39;, \u0026#39;.join(names)}\u0026#34;) # CSK: MS Dhoni, Ruturaj Gaikwad # MI: Rohit Sharma, Hardik Pandya # RCB: Virat Kohli, Glenn Maxwell starmap(func, iterable) - Map with Argument Unpacking starmap is like map, but it unpacks each element of the iterable as arguments to the function. Useful when your iterable contains tuples of arguments.\nfrom itertools import starmap # map requires separate iterables; starmap takes one iterable of tuples coordinates = [(2, 3), (4, 5), (1, 8), (3, 3)] distances = list(starmap(pow, [(2, 3), (3, 2), (10, 0.5)])) print(distances) # [8, 9, 3.1622...] (2^3, 3^2, 10^0.5) # Real use: apply functions to argument tuples import operator operations = [ (operator.add, 10, 5), (operator.mul, 4, 7), (operator.sub, 20, 3), ] results = [func(a, b) for func, a, b in operations] print(results) # [15, 28, 17] # starmap with a named function def compute_tax(price, rate_pct): return round(price * rate_pct / 100, 2) products = [ (\u0026#34;Laptop\u0026#34;, 75000, 18), (\u0026#34;Book\u0026#34;, 500, 5), (\u0026#34;Phone\u0026#34;, 25000, 12), ] taxes = list(starmap(compute_tax, [(p[1], p[2]) for p in products])) for (name, price, rate), tax in zip(products, taxes): print(f\u0026#34;{name}: Rs.{price} + {rate}% tax = Rs.{price + tax}\u0026#34;) pairwise(iterable) - Sliding Window of Size 2 (Python 3.10+) pairwise yields consecutive overlapping pairs from an iterable. This is the most common sliding window pattern.\nfrom itertools import pairwise data = [1, 2, 3, 4, 5] print(list(pairwise(data))) # [(1, 2), (2, 3), (3, 4), (4, 5)] # Compute differences between consecutive values prices = [100, 105, 98, 112, 107, 115] changes = [curr - prev for prev, curr in pairwise(prices)] print(changes) # [5, -7, 14, -5, 8] # Detect if sequence is sorted def is_sorted(iterable): return all(a \u0026lt;= b for a, b in pairwise(iterable)) print(is_sorted([1, 2, 3, 4, 5])) # True print(is_sorted([1, 3, 2, 4, 5])) # False # Count direction changes in a stock price series prices = [100, 105, 103, 108, 107, 112, 110, 115] directions = [\u0026#39;up\u0026#39; if b \u0026gt; a else \u0026#39;down\u0026#39; for a, b in pairwise(prices)] print(directions) # [\u0026#39;up\u0026#39;, \u0026#39;down\u0026#39;, \u0026#39;up\u0026#39;, \u0026#39;down\u0026#39;, \u0026#39;up\u0026#39;, \u0026#39;down\u0026#39;, \u0026#39;up\u0026#39;] For Python \u0026lt; 3.10, here is the equivalent:\nfrom itertools import tee def pairwise(iterable): a, b = tee(iterable) next(b, None) # advance b by one return zip(a, b) batched(iterable, n) - Split Into Fixed-Size Chunks (Python 3.12+) batched yields tuples of exactly n elements, with the last batch being smaller if the iterable doesn\u0026rsquo;t divide evenly.\nfrom itertools import batched data = range(1, 12) for batch in batched(data, 3): print(batch) # (1, 2, 3) # (4, 5, 6) # (7, 8, 9) # (10, 11) ← last batch is smaller # Real use: batch API calls to avoid rate limits player_ids = list(range(1, 26)) # 25 player IDs for batch in batched(player_ids, n=5): print(f\u0026#34;Fetching players: {batch}\u0026#34;) # fetch_players(batch) ← 5 at a time instead of 25 at once For Python \u0026lt; 3.12, the manual equivalent:\nfrom itertools import islice def batched(iterable, n): it = iter(iterable) while chunk := tuple(islice(it, n)): yield chunk tee(iterable, n=2) - Clone an Iterator tee creates n independent copies of a single iterator. This lets you iterate over the same sequence multiple times from different starting points.\nfrom itertools import tee it = iter([1, 2, 3, 4, 5]) a, b = tee(it) print(list(a)) # [1, 2, 3, 4, 5] print(list(b)) # [1, 2, 3, 4, 5] ← independent copy # IMPORTANT: After tee, don\u0026#39;t use the original iterator # Using `it` after tee() produces undefined behaviour # Use case: compute multiple statistics in one pass from itertools import tee def describe(iterable): a, b, c = tee(iterable, 3) values = list(a) n = len(values) total = sum(b) mx = max(c) return {\u0026#34;count\u0026#34;: n, \u0026#34;sum\u0026#34;: total, \u0026#34;max\u0026#34;: mx, \u0026#34;mean\u0026#34;: total / n} print(describe([67, 82, 45, 95, 71, 88])) # {\u0026#39;count\u0026#39;: 6, \u0026#39;sum\u0026#39;: 448, \u0026#39;max\u0026#39;: 95, \u0026#39;mean\u0026#39;: 74.67} Memory warning: tee stores values internally so that both copies can access them independently. If one copy advances far ahead of the other, the stored values accumulate in memory. For large datasets, prefer materializing to a list.\nPart 3: Combinatoric Iterators These are the functions that generate structured combinations of items. They\u0026rsquo;re invaluable for testing, optimization, search, and mathematical work.\nproduct(*iterables, repeat=1) - Cartesian Product product generates the Cartesian product of iterables - every possible combination of one element from each iterable. It\u0026rsquo;s the equivalent of nested for loops.\nfrom itertools import product # All combinations of colours and sizes colours = [\u0026#39;red\u0026#39;, \u0026#39;blue\u0026#39;, \u0026#39;green\u0026#39;] sizes = [\u0026#39;S\u0026#39;, \u0026#39;M\u0026#39;, \u0026#39;L\u0026#39;] for colour, size in product(colours, sizes): print(f\u0026#34;{colour}-{size}\u0026#34;, end=\u0026#34; \u0026#34;) # red-S red-M red-L blue-S blue-M blue-L green-S green-M green-L # The `repeat` argument - same iterable crossed with itself # All 2-digit combinations of 0 and 1 (binary numbers) binary_pairs = list(product([0, 1], repeat=2)) print(binary_pairs) # [(0, 0), (0, 1), (1, 0), (1, 1)] # Generate all possible test inputs def test_matrix(input_lists): return product(*input_lists) params = [ [\u0026#39;GET\u0026#39;, \u0026#39;POST\u0026#39;], # HTTP method [\u0026#39;/api/v1\u0026#39;, \u0026#39;/api/v2\u0026#39;],# endpoint version [True, False], # authenticated ] for method, version, auth in test_matrix(params): print(f\u0026#34;{method} {version} auth={auth}\u0026#34;) # GET /api/v1 auth=True # GET /api/v1 auth=False # GET /api/v2 auth=True # ... (8 total combinations) permutations(iterable, r=None) - Ordered Arrangements permutations yields all possible ordered arrangements of r items from the iterable. Order matters: (A, B) and (B, A) are different permutations.\nfrom itertools import permutations players = [\u0026#39;Rohit\u0026#39;, \u0026#39;Virat\u0026#39;, \u0026#39;Dhoni\u0026#39;] # All arrangements of all 3 players for perm in permutations(players): print(perm) # (\u0026#39;Rohit\u0026#39;, \u0026#39;Virat\u0026#39;, \u0026#39;Dhoni\u0026#39;) # (\u0026#39;Rohit\u0026#39;, \u0026#39;Dhoni\u0026#39;, \u0026#39;Virat\u0026#39;) # (\u0026#39;Virat\u0026#39;, \u0026#39;Rohit\u0026#39;, \u0026#39;Dhoni\u0026#39;) # (\u0026#39;Virat\u0026#39;, \u0026#39;Dhoni\u0026#39;, \u0026#39;Rohit\u0026#39;) # (\u0026#39;Dhoni\u0026#39;, \u0026#39;Rohit\u0026#39;, \u0026#39;Virat\u0026#39;) # (\u0026#39;Dhoni\u0026#39;, \u0026#39;Virat\u0026#39;, \u0026#39;Rohit\u0026#39;) # r=2 - all possible batting pairs (order matters: opening vs #3) batting_pairs = list(permutations(players, r=2)) print(batting_pairs) # [(\u0026#39;Rohit\u0026#39;,\u0026#39;Virat\u0026#39;), (\u0026#39;Rohit\u0026#39;,\u0026#39;Dhoni\u0026#39;), (\u0026#39;Virat\u0026#39;,\u0026#39;Rohit\u0026#39;), # (\u0026#39;Virat\u0026#39;,\u0026#39;Dhoni\u0026#39;), (\u0026#39;Dhoni\u0026#39;,\u0026#39;Rohit\u0026#39;), (\u0026#39;Dhoni\u0026#39;,\u0026#39;Virat\u0026#39;)] # Count: n! / (n-r)! # permutations(10 players, r=3): 10 * 9 * 8 = 720 arrangements print(f\u0026#34;Total arrangements: {len(list(permutations(\u0026#39;ABCD\u0026#39;, 2)))}\u0026#34;) # 12 Real use case: checking all possible evaluation orders:\nfrom itertools import permutations steps = [\u0026#39;load_data\u0026#39;, \u0026#39;validate\u0026#39;, \u0026#39;transform\u0026#39;, \u0026#39;save\u0026#39;] # Find which ordering doesn\u0026#39;t cause dependency errors for order in permutations(steps): if is_valid_order(order): # your dependency check print(f\u0026#34;Valid order: {order}\u0026#34;) break combinations(iterable, r) - Unordered Selections (No Repeats) combinations yields all possible selections of r items where order does NOT matter and each item can only be used once. (A, B) and (B, A) are the same combination - only (A, B) is produced.\nfrom itertools import combinations squad = [\u0026#39;Rohit\u0026#39;, \u0026#39;Virat\u0026#39;, \u0026#39;Dhoni\u0026#39;, \u0026#39;Jadeja\u0026#39;, \u0026#39;Bumrah\u0026#39;] # Choose 3 from 5 - order doesn\u0026#39;t matter playing_11_selections = list(combinations(squad, 3)) for combo in playing_11_selections: print(combo) # (\u0026#39;Rohit\u0026#39;, \u0026#39;Virat\u0026#39;, \u0026#39;Dhoni\u0026#39;) # (\u0026#39;Rohit\u0026#39;, \u0026#39;Virat\u0026#39;, \u0026#39;Jadeja\u0026#39;) # (\u0026#39;Rohit\u0026#39;, \u0026#39;Virat\u0026#39;, \u0026#39;Bumrah\u0026#39;) # ... (10 total) print(f\u0026#34;Ways to choose 3 from 5: {len(playing_11_selections)}\u0026#34;) # 10 # Feature selection in ML - test every pair of features features = [\u0026#39;age\u0026#39;, \u0026#39;salary\u0026#39;, \u0026#39;experience\u0026#39;, \u0026#39;score\u0026#39;] for f1, f2 in combinations(features, 2): print(f\u0026#34;Testing pair: ({f1}, {f2})\u0026#34;) # Testing pair: (age, salary) # Testing pair: (age, experience) # ... (6 pairs total) # Find all triangle sides from a list of lengths lengths = [3, 4, 5, 6, 7] triangles = [ combo for combo in combinations(lengths, 3) if combo[0] + combo[1] \u0026gt; combo[2] # triangle inequality ] print(f\u0026#34;Valid triangles: {triangles}\u0026#34;) combinations_with_replacement(iterable, r) - Unordered Selections (With Repeats) Like combinations, but each item can appear more than once in a selection.\nfrom itertools import combinations_with_replacement # Toss a coin 3 times - how many distinct outcomes by count? outcomes = list(combinations_with_replacement([\u0026#39;H\u0026#39;, \u0026#39;T\u0026#39;], 3)) print(outcomes) # [(\u0026#39;H\u0026#39;, \u0026#39;H\u0026#39;, \u0026#39;H\u0026#39;), (\u0026#39;H\u0026#39;, \u0026#39;H\u0026#39;, \u0026#39;T\u0026#39;), (\u0026#39;H\u0026#39;, \u0026#39;T\u0026#39;, \u0026#39;T\u0026#39;), (\u0026#39;T\u0026#39;, \u0026#39;T\u0026#39;, \u0026#39;T\u0026#39;)] # Only 4 distinct \u0026#34;types\u0026#34; of 3-toss outcome by composition (not sequence) # Compare with combinations (no repeat) - shows the difference from itertools import combinations print(list(combinations([\u0026#39;H\u0026#39;, \u0026#39;T\u0026#39;], 3))) # [(\u0026#39;H\u0026#39;, \u0026#39;T\u0026#39;)] ← only one selection of 3 from 2 without repetition? Actually 0 exist for r \u0026gt; n # Better example: choosing 2 from 3 with and without replacement items = [\u0026#39;A\u0026#39;, \u0026#39;B\u0026#39;, \u0026#39;C\u0026#39;] print(\u0026#34;Without replacement:\u0026#34;, list(combinations(items, 2))) # [(\u0026#39;A\u0026#39;, \u0026#39;B\u0026#39;), (\u0026#39;A\u0026#39;, \u0026#39;C\u0026#39;), (\u0026#39;B\u0026#39;, \u0026#39;C\u0026#39;)] print(\u0026#34;With replacement: \u0026#34;, list(combinations_with_replacement(items, 2))) # [(\u0026#39;A\u0026#39;, \u0026#39;A\u0026#39;), (\u0026#39;A\u0026#39;, \u0026#39;B\u0026#39;), (\u0026#39;A\u0026#39;, \u0026#39;C\u0026#39;), (\u0026#39;B\u0026#39;, \u0026#39;B\u0026#39;), (\u0026#39;B\u0026#39;, \u0026#39;C\u0026#39;), (\u0026#39;C\u0026#39;, \u0026#39;C\u0026#39;)] # Count: C(n+r-1, r) for with_replacement vs C(n, r) without Building Pipelines - Putting It All Together The real power of itertools shows up when you chain multiple functions together into a data pipeline. Because every function returns an iterator, you can compose them with essentially zero memory overhead.\nPipeline 1: Log File Analysis Parse a large log file, filter for errors, extract fields, group by hour.\nfrom itertools import groupby, islice import re def parse_log_line(line): \u0026#34;\u0026#34;\u0026#34;Parse a log line into a dict.\u0026#34;\u0026#34;\u0026#34; pattern = r\u0026#39;(\\d{4}-\\d{2}-\\d{2} \\d{2}:\\d{2}:\\d{2}) \\[(\\w+)\\] (.+)\u0026#39; match = re.match(pattern, line.strip()) if match: return {\u0026#34;timestamp\u0026#34;: match[1], \u0026#34;level\u0026#34;: match[2], \u0026#34;message\u0026#34;: match[3]} return None def stream_logs(filepath): \u0026#34;\u0026#34;\u0026#34;Lazy generator over log lines.\u0026#34;\u0026#34;\u0026#34; with open(filepath) as f: for line in f: record = parse_log_line(line) if record: yield record def only_errors(records): return filter(lambda r: r[\u0026#34;level\u0026#34;] == \u0026#34;ERROR\u0026#34;, records) def get_hour(record): return record[\u0026#34;timestamp\u0026#34;][:13] # \u0026#34;2024-01-15 14\u0026#34; def analyze_errors(filepath): records = stream_logs(filepath) errors = only_errors(records) sorted_errors = sorted(errors, key=get_hour) # groupby needs sorted input for hour, group in groupby(sorted_errors, key=get_hour): error_list = list(group) print(f\u0026#34;{hour}:xx - {len(error_list)} errors\u0026#34;) for e in islice(error_list, 3): # show first 3 per hour print(f\u0026#34; {e[\u0026#39;message\u0026#39;]}\u0026#34;) Pipeline 2: Cricket Stats Rolling Average Compute a 5-match rolling batting average using pairwise-style windowing:\nfrom itertools import islice, accumulate from collections import deque def rolling_average(scores, window=5): \u0026#34;\u0026#34;\u0026#34;Yield the rolling average over the last `window` scores.\u0026#34;\u0026#34;\u0026#34; buf = deque(maxlen=window) for score in scores: buf.append(score) if len(buf) == window: yield round(sum(buf) / window, 2) rohit_scores = [45, 67, 12, 89, 34, 76, 55, 43, 98, 21, 67, 82] averages = list(rolling_average(rohit_scores, window=3)) print(\u0026#34;3-match rolling avg:\u0026#34;, averages) # [41.33, 56.0, 45.0, 66.33, 55.0, 58.0, 65.33, 54.0, 62.0, 56.67] Pipeline 3: E-Commerce Product Report from itertools import chain, groupby, accumulate, islice from operator import itemgetter products = list(chain( [{\u0026#34;cat\u0026#34;: \u0026#34;Electronics\u0026#34;, \u0026#34;name\u0026#34;: \u0026#34;Laptop\u0026#34;, \u0026#34;price\u0026#34;: 75000, \u0026#34;sold\u0026#34;: 12}], [{\u0026#34;cat\u0026#34;: \u0026#34;Electronics\u0026#34;, \u0026#34;name\u0026#34;: \u0026#34;Phone\u0026#34;, \u0026#34;price\u0026#34;: 25000, \u0026#34;sold\u0026#34;: 45}], [{\u0026#34;cat\u0026#34;: \u0026#34;Clothing\u0026#34;, \u0026#34;name\u0026#34;: \u0026#34;T-Shirt\u0026#34;, \u0026#34;price\u0026#34;: 599, \u0026#34;sold\u0026#34;: 200}], [{\u0026#34;cat\u0026#34;: \u0026#34;Clothing\u0026#34;, \u0026#34;name\u0026#34;: \u0026#34;Jeans\u0026#34;, \u0026#34;price\u0026#34;: 1299, \u0026#34;sold\u0026#34;: 89}], [{\u0026#34;cat\u0026#34;: \u0026#34;Books\u0026#34;, \u0026#34;name\u0026#34;: \u0026#34;Python\u0026#34;, \u0026#34;price\u0026#34;: 499, \u0026#34;sold\u0026#34;: 150}], )) # Sort by category, then by revenue within category sorted_products = sorted(products, key=lambda p: (p[\u0026#34;cat\u0026#34;], -p[\u0026#34;price\u0026#34;] * p[\u0026#34;sold\u0026#34;])) # Group by category and summarize for category, group in groupby(sorted_products, key=itemgetter(\u0026#34;cat\u0026#34;)): items = list(group) revenues = [p[\u0026#34;price\u0026#34;] * p[\u0026#34;sold\u0026#34;] for p in items] total = sum(revenues) print(f\u0026#34;\\n{category}:\u0026#34;) for item, rev in zip(items, revenues): print(f\u0026#34; {item[\u0026#39;name\u0026#39;]:\u0026lt;12} Rs.{rev:\u0026gt;10,}\u0026#34;) print(f\u0026#34; {\u0026#39;TOTAL\u0026#39;:\u0026lt;12} Rs.{total:\u0026gt;10,}\u0026#34;) Practice Problems Work through these problems before looking at the solutions. Each one targets a specific part of the module.\nProblem 1 - Running Maximum Task: Given a list of daily temperatures, use accumulate to compute the running maximum temperature seen so far for each day.\nInput: [22, 28, 25, 31, 29, 35, 30, 33] Output: [22, 28, 28, 31, 31, 35, 35, 35] Solution:\nfrom itertools import accumulate temps = [22, 28, 25, 31, 29, 35, 30, 33] running_max = list(accumulate(temps, func=max)) print(running_max) # [22, 28, 28, 31, 31, 35, 35, 35] Problem 2 - Round-Robin Tournament Fixtures Task: Given a list of 4 cricket teams, generate all possible match fixtures using combinations. Each pair plays exactly once.\nTeams: [\u0026#39;MI\u0026#39;, \u0026#39;CSK\u0026#39;, \u0026#39;RCB\u0026#39;, \u0026#39;KKR\u0026#39;] Output: (\u0026#39;MI\u0026#39;, \u0026#39;CSK\u0026#39;), (\u0026#39;MI\u0026#39;, \u0026#39;RCB\u0026#39;), (\u0026#39;MI\u0026#39;, \u0026#39;KKR\u0026#39;), (\u0026#39;CSK\u0026#39;, \u0026#39;RCB\u0026#39;), ... Solution:\nfrom itertools import combinations teams = [\u0026#39;MI\u0026#39;, \u0026#39;CSK\u0026#39;, \u0026#39;RCB\u0026#39;, \u0026#39;KKR\u0026#39;] fixtures = list(combinations(teams, 2)) print(f\u0026#34;Total fixtures: {len(fixtures)}\u0026#34;) # 6 for match_num, (team_a, team_b) in enumerate(fixtures, start=1): print(f\u0026#34;Match {match_num}: {team_a} vs {team_b}\u0026#34;) # Match 1: MI vs CSK # Match 2: MI vs RCB # Match 3: MI vs KKR # Match 4: CSK vs RCB # Match 5: CSK vs KKR # Match 6: RCB vs KKR Problem 3 - Flatten a Nested List Task: Flatten a list of lists (one level deep) using chain.from_iterable.\nInput: [[1, 2, 3], [4, 5], [], [6, 7, 8, 9]] Output: [1, 2, 3, 4, 5, 6, 7, 8, 9] Solution:\nfrom itertools import chain nested = [[1, 2, 3], [4, 5], [], [6, 7, 8, 9]] flat = list(chain.from_iterable(nested)) print(flat) # [1, 2, 3, 4, 5, 6, 7, 8, 9] # Bonus: verify empty sublists are handled gracefully (they are) Problem 4 - Paginate a Large Dataset Task: Write a function paginate(data, page_size) using islice that returns a generator. Each call to next() on the generator should yield one page of data as a list.\nSolution:\nfrom itertools import islice def paginate(data, page_size): it = iter(data) while True: page = list(islice(it, page_size)) if not page: break yield page data = list(range(1, 23)) # 22 items pages = list(paginate(data, page_size=5)) for i, page in enumerate(pages, 1): print(f\u0026#34;Page {i}: {page}\u0026#34;) # Page 1: [1, 2, 3, 4, 5] # Page 2: [6, 7, 8, 9, 10] # Page 3: [11, 12, 13, 14, 15] # Page 4: [16, 17, 18, 19, 20] # Page 5: [21, 22] Problem 5 - Find First Duplicate in a Stream Task: Using pairwise (or tee), find the first value that appears consecutively in a list.\nInput: [1, 2, 3, 3, 4, 5, 5, 6] Output: 3 (first value that equals the next value) Solution:\nfrom itertools import pairwise # Python 3.10+ def first_consecutive_duplicate(seq): for a, b in pairwise(seq): if a == b: return a return None data = [1, 2, 3, 3, 4, 5, 5, 6] print(first_consecutive_duplicate(data)) # 3 # Python \u0026lt; 3.10 version using tee: from itertools import tee def first_consecutive_duplicate_v2(seq): a, b = tee(seq) next(b, None) for x, y in zip(a, b): if x == y: return x return None Problem 6 - Group and Summarize Orders Task: Given a list of order records (already sorted by status), use groupby to count and total the orders in each status group.\norders = [ {\u0026#34;id\u0026#34;: 1, \u0026#34;status\u0026#34;: \u0026#34;delivered\u0026#34;, \u0026#34;amount\u0026#34;: 1200}, {\u0026#34;id\u0026#34;: 2, \u0026#34;status\u0026#34;: \u0026#34;delivered\u0026#34;, \u0026#34;amount\u0026#34;: 850}, {\u0026#34;id\u0026#34;: 3, \u0026#34;status\u0026#34;: \u0026#34;delivered\u0026#34;, \u0026#34;amount\u0026#34;: 2100}, {\u0026#34;id\u0026#34;: 4, \u0026#34;status\u0026#34;: \u0026#34;pending\u0026#34;, \u0026#34;amount\u0026#34;: 500}, {\u0026#34;id\u0026#34;: 5, \u0026#34;status\u0026#34;: \u0026#34;pending\u0026#34;, \u0026#34;amount\u0026#34;: 750}, {\u0026#34;id\u0026#34;: 6, \u0026#34;status\u0026#34;: \u0026#34;returned\u0026#34;, \u0026#34;amount\u0026#34;: 1500}, ] Solution:\nfrom itertools import groupby from operator import itemgetter for status, group in groupby(orders, key=itemgetter(\u0026#34;status\u0026#34;)): order_list = list(group) count = len(order_list) total = sum(o[\u0026#34;amount\u0026#34;] for o in order_list) print(f\u0026#34;{status:12}: {count} orders, Rs.{total:,} total\u0026#34;) # delivered : 3 orders, Rs.4,150 total # pending : 2 orders, Rs.1,250 total # returned : 1 orders, Rs.1,500 total Problem 7 - Generate a Test Matrix Task: You are testing a login function. Using product, generate all combinations of these test parameters and count them.\nusernames = [\u0026#34;valid_user\u0026#34;, \u0026#34;\u0026#34;] # 2 values passwords = [\u0026#34;correct_pw\u0026#34;, \u0026#34;wrong_pw\u0026#34;, \u0026#34;\u0026#34;] # 3 values remember_me = [True, False] # 2 values # Expected: 2 * 3 * 2 = 12 test cases Solution:\nfrom itertools import product usernames = [\u0026#34;valid_user\u0026#34;, \u0026#34;\u0026#34;] passwords = [\u0026#34;correct_pw\u0026#34;, \u0026#34;wrong_pw\u0026#34;, \u0026#34;\u0026#34;] remember_me = [True, False] test_cases = list(product(usernames, passwords, remember_me)) print(f\u0026#34;Total test cases: {len(test_cases)}\u0026#34;) # 12 for i, (user, pw, rem) in enumerate(test_cases, 1): user_repr = repr(user) if not user else user pw_repr = repr(pw) if not pw else \u0026#34;***\u0026#34; print(f\u0026#34; TC{i:02d}: user={user_repr:\u0026lt;15} pw={pw_repr:\u0026lt;12} remember={rem}\u0026#34;) Problem 8 - Consecutive Price Changes Task: Given a list of daily stock closing prices, use pairwise to compute the percentage change between each consecutive pair. Then use takewhile to find how many consecutive days a stock was rising from the start.\nprices = [100, 105, 108, 107, 112, 115, 110, 118] Solution:\nfrom itertools import pairwise, takewhile prices = [100, 105, 108, 107, 112, 115, 110, 118] # Percentage changes changes = [round((b - a) / a * 100, 2) for a, b in pairwise(prices)] print(\u0026#34;Daily % changes:\u0026#34;, changes) # [5.0, 2.86, -0.93, 4.67, 2.68, -4.35, 7.27] # Days rising from start (takewhile positive change) rising_days = list(takewhile(lambda x: x \u0026gt; 0, changes)) print(f\u0026#34;Consecutive rising days from start: {len(rising_days)}\u0026#34;) # 2 print(f\u0026#34;Changes during rise: {rising_days}\u0026#34;) # [5.0, 2.86] Problem 9 - Interleave Two Lists Task: Without zip, use chain and zip_longest together to interleave two lists element by element, even if they have different lengths.\nInput: [1, 2, 3, 4], [\u0026#39;a\u0026#39;, \u0026#39;b\u0026#39;, \u0026#39;c\u0026#39;] Output: [1, \u0026#39;a\u0026#39;, 2, \u0026#39;b\u0026#39;, 3, \u0026#39;c\u0026#39;, 4] Solution:\nfrom itertools import chain, zip_longest def interleave(a, b, fillvalue=None): \u0026#34;\u0026#34;\u0026#34; Interleave two iterables. Extra elements from the longer one appear at the end. \u0026#34;\u0026#34;\u0026#34; zipped = zip_longest(a, b, fillvalue=fillvalue) return [x for pair in zipped for x in pair if x is not fillvalue] print(interleave([1, 2, 3, 4], [\u0026#39;a\u0026#39;, \u0026#39;b\u0026#39;, \u0026#39;c\u0026#39;])) # [1, \u0026#39;a\u0026#39;, 2, \u0026#39;b\u0026#39;, 3, \u0026#39;c\u0026#39;, 4] # Alternative: chain after zip def interleave_v2(a, b): pairs = zip_longest(a, b) return list(chain.from_iterable( (x for x in pair if x is not None) for pair in pairs )) print(interleave_v2([1, 2, 3, 4], [\u0026#39;a\u0026#39;, \u0026#39;b\u0026#39;, \u0026#39;c\u0026#39;])) # [1, \u0026#39;a\u0026#39;, 2, \u0026#39;b\u0026#39;, 3, \u0026#39;c\u0026#39;, 4] Summary - When to Use What ┌──────────────────────────────────────────────────────────────────────────┐ │ ITERTOOLS DECISION GUIDE │ ├────────────────────────────┬─────────────────────────────────────────────┤ │ I want to... │ Use... │ ├────────────────────────────┼─────────────────────────────────────────────┤ │ Count from N forever │ count(start, step) │ │ Loop a list forever │ cycle(iterable) │ │ Repeat a value N times │ repeat(value, times) │ ├────────────────────────────┼─────────────────────────────────────────────┤ │ Flatten nested iterables │ chain.from_iterable(nested) │ │ Concatenate iterables │ chain(a, b, c) │ │ Slice an iterator lazily │ islice(it, start, stop, step) │ │ Zip unequal iterables │ zip_longest(a, b, fillvalue=x) │ ├────────────────────────────┼─────────────────────────────────────────────┤ │ Running totals/max/etc │ accumulate(data, func=max) │ │ Take while condition holds │ takewhile(pred, it) │ │ Skip while condition holds │ dropwhile(pred, it) │ │ Keep falsy items │ filterfalse(pred, it) │ │ Boolean masking │ compress(data, selectors) │ │ Group by key │ groupby(sorted_data, key=fn) │ │ Apply func to arg tuples │ starmap(func, arg_tuples) │ │ Sliding pairs │ pairwise(iterable) │ │ Split into chunks │ batched(it, n) │ │ Clone an iterator │ tee(it, n) │ ├────────────────────────────┼─────────────────────────────────────────────┤ │ All ordered arrangements │ permutations(it, r) │ │ All unordered selections │ combinations(it, r) │ │ Selections with repeats │ combinations_with_replacement(it, r) │ │ Cartesian product │ product(a, b, repeat=n) │ └────────────────────────────┴─────────────────────────────────────────────┘ GOLDEN RULES ──────────── 1. Always sort before groupby 2. Always guard infinite iterators with islice 3. Never reuse an exhausted iterator 4. Don\u0026#39;t use the original iterator after tee() 5. Wrap in list() only when you need random access or multiple passes 6. Prefer generator pipelines for large/streaming data End of Guide - Python\u0026rsquo;s itertools Codeverra - codeverra.com\n","permalink":"https://blogs.codeverra.com/blog/python-15-itertools/","summary":"\u003ch1 id=\"pythons-itertools---the-missing-manual\"\u003ePython\u0026rsquo;s \u003ccode\u003eitertools\u003c/code\u003e - The Missing Manual\u003c/h1\u003e\n\u003ch3 id=\"a-deep-dive-into-one-of-pythons-most-underused-standard-library-modules\"\u003eA deep dive into one of Python\u0026rsquo;s most underused standard library modules\u003c/h3\u003e\n\u003chr\u003e\n\u003ch2 id=\"wait-why-should-i-care-about-another-module\"\u003eWait, Why Should I Care About Another Module?\u003c/h2\u003e\n\u003cp\u003eYou\u0026rsquo;ve written Python for a while. You know how to use \u003ccode\u003efor\u003c/code\u003e loops, list comprehensions, \u003ccode\u003ezip\u003c/code\u003e, and \u003ccode\u003eenumerate\u003c/code\u003e. Things work. So when someone says \u0026ldquo;hey, you should learn \u003ccode\u003eitertools\u003c/code\u003e\u0026rdquo;, it\u0026rsquo;s easy to nod politely and move on.\u003c/p\u003e\n\u003cp\u003eHere\u0026rsquo;s what changes your mind. Imagine you\u0026rsquo;re asked to generate all possible batting orders for 4 players chosen from a squad of 11. Or you need to group a massive log file by date without loading it all into memory. Or you want to pair up every possible combination of feature columns for a machine learning experiment.\u003c/p\u003e","tags":["python"],"title":"Itertools Module in Python - Complete Guide"},{"content":"Lesson 4 - JOINs \u0026amp; Relationships Theory + Practice | ShopDB Before You Start This lesson uses ShopDB exclusively. Connect to shopdb in pgAdmin before running any query.\nRun this to confirm your data is ready:\nSELECT \u0026#39;customers\u0026#39; AS t, COUNT(*) FROM customers UNION ALL SELECT \u0026#39;products\u0026#39;, COUNT(*) FROM products UNION ALL SELECT \u0026#39;orders\u0026#39;, COUNT(*) FROM orders UNION ALL SELECT \u0026#39;order_items\u0026#39;,COUNT(*) FROM order_items; -- Expected: 20, 25, 15, 30 What you will learn:\nConcept What it does INNER JOIN Only rows that match in both tables LEFT JOIN All left rows + matches from right (NULL if no match) RIGHT JOIN All right rows + matches from left (NULL if no match) FULL OUTER JOIN All rows from both tables Self-join A table joined to itself 4-table JOIN The full ShopDB chain Common mistakes Duplicate rows, wrong COUNT, missing ON clause JOIN vs subquery When to use which Part 1 - The Four JOIN Types, Side by Side The best way to understand JOINs is to see them compared on the same data.\nLet us use two small subsets to make the differences crystal clear.\n-- Our test: customers who have placed orders vs customers who haven\u0026#39;t -- All 20 customers -- Orders exist for customer ids: 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 12, 15, 17, 20 -- No orders for: 11 (Ravi), 13 (Kavya), 14 (Aditya), 16 (Nikhil), 18 (Rahul), 19 (Tanvi) INNER JOIN - the intersection Returns only rows that have a matching record in both tables. Customers with no orders are excluded. Orders with no customer are excluded (impossible here due to FK, but worth knowing).\nSELECT c.id, c.name, c.city, o.id AS order_id, o.order_date, o.status FROM customers AS c INNER JOIN orders AS o ON o.customer_id = c.id ORDER BY c.name; Result: 15 rows - one per order. The 6 customers who never ordered are absent.\nJOIN and INNER JOIN are identical. INNER is optional - most developers write just JOIN.\nLEFT JOIN - keep all from the left Returns all rows from the left table (customers), plus matching rows from the right (orders). Customers with no orders appear once, with NULL in all order columns.\nSELECT c.id, c.name, c.city, o.id AS order_id, o.order_date, o.status FROM customers AS c LEFT JOIN orders AS o ON o.customer_id = c.id ORDER BY c.name; Result: 20 rows - all customers. The 6 who never ordered show NULL for order columns.\n-- Find customers who have NEVER placed an order -- Pattern: LEFT JOIN + WHERE right_side IS NULL SELECT c.id, c.name, c.city, c.state FROM customers AS c LEFT JOIN orders AS o ON o.customer_id = c.id WHERE o.id IS NULL ORDER BY c.name; This is called the anti-join pattern - one of the most useful LEFT JOIN techniques. The WHERE clause filters to only the rows where no match was found on the right side.\nRIGHT JOIN - keep all from the right Returns all rows from the right table (orders), plus matching rows from the left (customers). In practice, RIGHT JOIN is rare - you can always rewrite it as a LEFT JOIN by swapping table order.\n-- RIGHT JOIN: keep all orders even if customer data is missing -- (In our dataset this won\u0026#39;t happen due to FK constraints, -- but in a messy real-world DB it might) SELECT c.name, c.city, o.id AS order_id, o.status FROM customers AS c RIGHT JOIN orders AS o ON o.customer_id = c.id ORDER BY o.id; Tip: Most developers avoid RIGHT JOIN entirely. A RIGHT JOIN B = B LEFT JOIN A - just swap the tables. Sticking to LEFT JOIN makes queries easier to read and reason about.\nFULL OUTER JOIN - keep everything Returns all rows from both tables. Where there is no match, NULLs fill the gaps.\n-- All customers and all orders - matched where possible SELECT c.name AS customer, c.city, o.id AS order_id, o.status FROM customers AS c FULL OUTER JOIN orders AS o ON o.customer_id = c.id ORDER BY c.name NULLS LAST; Result: 20+ rows. Customers without orders show NULL on the right. If there were orphan orders (no customer), they\u0026rsquo;d show NULL on the left.\nFULL OUTER JOIN is rare in day-to-day work. The most useful application is data reconciliation - comparing two lists to find what\u0026rsquo;s in A but not B, what\u0026rsquo;s in B but not A, and what\u0026rsquo;s in both.\nVisual Summary Tables: Customers (C) and Orders (O) INNER JOIN LEFT JOIN RIGHT JOIN FULL OUTER JOIN C ∩ O C + (C ∩ O) O + (C ∩ O) C ∪ O ┌──┬──┐ ┌──┬──┐ ┌──┬──┐ ┌──┬──┐ │ │██│ │██│██│ │ │██│ │██│██│ │ │██│ │██│██│ │ │██│ │██│██│ └──┴──┘ └──┴──┘ └──┴──┘ └──┴──┘ C O C O C O C O Rows returned: Rows returned: Rows returned: Rows returned: Only matches All C rows All O rows All rows + matches + matches from both ✏️ Practice Set 1 - JOIN types Q1. Write a query using LEFT JOIN to find all products that have never appeared in any order. Show product name, category_id, and price.\nQ2. Write an INNER JOIN query that shows every order alongside the customer\u0026rsquo;s name and city. Include order_id, order_date, status, customer name, and city. Order by order_date.\nQ3. Using LEFT JOIN, show all categories along with the count of products in each. Include categories that have zero products. Show category name and product_count.\nQ4. The anti-join pattern: find all customers who have placed at least one order that was Cancelled. Show customer name, city, and the number of cancelled orders.\nPart 2 - JOINs Across Multiple Tables ShopDB has 5 tables. A full order summary requires linking all of them:\ncustomers → orders → order_items → products → categories Each arrow is a foreign key relationship. Let\u0026rsquo;s build the full chain.\n-- Step 1: customers + orders (2 tables) SELECT c.name AS customer, o.id AS order_id, o.order_date, o.status FROM customers AS c JOIN orders AS o ON o.customer_id = c.id ORDER BY o.order_date; -- Step 2: add order_items (3 tables) SELECT c.name AS customer, o.id AS order_id, o.order_date, oi.product_id, oi.quantity, oi.unit_price FROM customers AS c JOIN orders AS o ON o.customer_id = c.id JOIN order_items AS oi ON oi.order_id = o.id ORDER BY o.order_date, oi.product_id; -- Step 3: add products (4 tables) SELECT c.name AS customer, o.id AS order_id, o.order_date, o.status, p.name AS product, oi.quantity, oi.unit_price, oi.quantity * oi.unit_price AS line_total FROM customers AS c JOIN orders AS o ON o.customer_id = c.id JOIN order_items AS oi ON oi.order_id = o.id JOIN products AS p ON oi.product_id = p.id ORDER BY o.order_date, p.name; -- Step 4: add categories (all 5 tables - the full chain) SELECT c.name AS customer, c.city, o.id AS order_id, o.order_date, o.status, o.payment_method, cat.name AS category, p.name AS product, oi.quantity, oi.unit_price, oi.quantity * oi.unit_price AS line_total FROM customers AS c JOIN orders AS o ON o.customer_id = c.id JOIN order_items AS oi ON oi.order_id = o.id JOIN products AS p ON oi.product_id = p.id JOIN categories AS cat ON p.category_id = cat.id ORDER BY o.order_date, cat.name, p.name; Study the JOIN chain: each table connects to the next via a foreign key. This 5-table query is the backbone of most e-commerce reports.\nPart 3 - JOIN + GROUP BY: Aggregating Across Relationships The most common real-world pattern: join to get context, then group to summarise.\n-- Total revenue per customer (non-cancelled orders) SELECT c.name AS customer, c.city, COUNT(DISTINCT o.id) AS orders_placed, SUM(oi.quantity * oi.unit_price) AS total_spent_inr FROM customers AS c JOIN orders AS o ON o.customer_id = c.id JOIN order_items AS oi ON oi.order_id = o.id WHERE o.status != \u0026#39;Cancelled\u0026#39; GROUP BY c.id, c.name, c.city ORDER BY total_spent_inr DESC; -- Revenue per category, with item count and average item price SELECT cat.name AS category, COUNT(oi.id) AS items_sold, SUM(oi.quantity) AS units_sold, SUM(oi.quantity * oi.unit_price) AS gross_revenue, ROUND(AVG(oi.unit_price), 2) AS avg_unit_price FROM categories AS cat JOIN products AS p ON p.category_id = cat.id JOIN order_items AS oi ON oi.product_id = p.id JOIN orders AS o ON oi.order_id = o.id WHERE o.status != \u0026#39;Cancelled\u0026#39; GROUP BY cat.id, cat.name ORDER BY gross_revenue DESC; -- Best-selling product per category -- (most units sold - uses the window function pattern from Lesson 2.3) SELECT category, product, units_sold FROM ( SELECT cat.name AS category, p.name AS product, SUM(oi.quantity) AS units_sold, RANK() OVER ( PARTITION BY cat.id ORDER BY SUM(oi.quantity) DESC ) AS rnk FROM categories AS cat JOIN products AS p ON p.category_id = cat.id JOIN order_items AS oi ON oi.product_id = p.id JOIN orders AS o ON oi.order_id = o.id WHERE o.status != \u0026#39;Cancelled\u0026#39; GROUP BY cat.id, cat.name, p.id, p.name ) AS ranked WHERE rnk = 1 ORDER BY category; ✏️ Practice Set 2 - Multi-table JOINs Q5. Write a 4-table JOIN query showing each order\u0026rsquo;s total value (sum of all its line items). Show order_id, customer name, order_date, status, and order_total. Order by order_total descending.\nQ6. Which city has the highest total spending? JOIN all necessary tables and GROUP BY city. Show city, number of orders, and total_spent. Exclude cancelled orders.\nQ7. Show each customer\u0026rsquo;s most recent order date and the status of that most recent order. Hint: Use MAX(order_date) grouped by customer, then join back to get the status.\nQ8. For each category, show:\nCategory name Number of distinct products that have been ordered at least once Total revenue from that category Order by revenue descending. Part 4 - Self-Join: A Table Joined to Itself A self-join is when a table references its own rows. It looks unusual at first but it is just a regular JOIN - both sides happen to be the same table.\nUse case: Find customers in the same city as another customer.\n-- Customers who share a city with at least one other customer SELECT a.name AS customer_1, b.name AS customer_2, a.city FROM customers AS a JOIN customers AS b ON a.city = b.city AND a.id \u0026lt;\u0026gt; b.id -- exclude self-match ORDER BY a.city, a.name, b.name; The key details:\nTwo aliases for the same table: a and b ON a.city = b.city - the join condition AND a.id \u0026lt;\u0026gt; b.id - prevents a row matching itself -- One row per city pair (avoid showing both A-B and B-A) SELECT a.name AS customer_1, b.name AS customer_2, a.city FROM customers AS a JOIN customers AS b ON a.city = b.city AND a.id \u0026lt; b.id -- \u0026lt; instead of \u0026lt;\u0026gt; avoids duplicates ORDER BY a.city; -- Products in the same category that are similarly priced (within ₹500 of each other) SELECT a.name AS product_1, b.name AS product_2, cat.name AS category, a.price_inr AS price_1, b.price_inr AS price_2, ABS(a.price_inr - b.price_inr) AS price_difference FROM products AS a JOIN products AS b ON a.category_id = b.category_id AND a.id \u0026lt; b.id AND ABS(a.price_inr - b.price_inr) \u0026lt;= 500 JOIN categories AS cat ON a.category_id = cat.id ORDER BY cat.name, price_difference; Part 5 - Common JOIN Mistakes These are the mistakes every SQL developer makes at least once. Reading about them now will save you significant debugging time.\nMistake 1: Forgetting the ON clause (Cartesian product) -- WRONG: no ON clause → every customer × every order = 20 × 15 = 300 rows! SELECT c.name, o.id FROM customers AS c JOIN orders AS o; -- ❌ Missing ON clause -- CORRECT SELECT c.name, o.id FROM customers AS c JOIN orders AS o ON o.customer_id = c.id; -- ✅ A missing ON clause creates a Cartesian product - every row in the left table matched with every row in the right table. With large tables this can produce billions of rows and crash your database session.\nMistake 2: Duplicate rows from one-to-many JOINs -- This looks like it counts customers - but it doesn\u0026#39;t -- Each customer with 2 orders is counted twice! SELECT COUNT(*) AS customer_count FROM customers AS c JOIN orders AS o ON o.customer_id = c.id; -- Returns 15 (number of orders) not 14 (customers who ordered) -- Fix: count distinct customer ids SELECT COUNT(DISTINCT c.id) AS customers_who_ordered FROM customers AS c JOIN orders AS o ON o.customer_id = c.id; -- Returns 14 ✅ When you JOIN a one-to-many relationship, the \u0026ldquo;one\u0026rdquo; side gets duplicated. Always think: which table am I really counting? If the answer is the \u0026ldquo;one\u0026rdquo; side, use COUNT(DISTINCT one_side.id).\nMistake 3: COUNT(*) vs COUNT(column) with LEFT JOIN -- WRONG: COUNT(*) counts the NULL row too SELECT cat.name, COUNT(*) AS product_count -- ❌ Will show 1 even for empty categories FROM categories AS cat LEFT JOIN products AS p ON p.category_id = cat.id GROUP BY cat.id, cat.name; -- CORRECT: COUNT(p.id) only counts non-NULL values SELECT cat.name, COUNT(p.id) AS product_count -- ✅ Shows 0 for categories with no products FROM categories AS cat LEFT JOIN products AS p ON p.category_id = cat.id GROUP BY cat.id, cat.name; With LEFT JOIN, unmatched rows have NULL for all right-side columns. COUNT(*) counts the row anyway (it counts rows, not values). COUNT(p.id) skips NULLs - which is what you almost always want.\nMistake 4: Filtering on a LEFT JOIN column in WHERE (converts to INNER JOIN) -- INTENDED: all customers, flag those who ordered in January -- ACTUAL: only customers who have January orders (LEFT JOIN becomes INNER JOIN!) SELECT c.name, o.order_date FROM customers AS c LEFT JOIN orders AS o ON o.customer_id = c.id WHERE o.order_date \u0026gt;= \u0026#39;2024-01-01\u0026#39; -- ❌ This kills the LEFT JOIN effect AND o.order_date \u0026lt; \u0026#39;2024-02-01\u0026#39;; -- FIX: move the date filter into the ON clause SELECT c.name, o.order_date FROM customers AS c LEFT JOIN orders AS o ON o.customer_id = c.id AND o.order_date \u0026gt;= \u0026#39;2024-01-01\u0026#39; -- ✅ Part of the JOIN AND o.order_date \u0026lt; \u0026#39;2024-02-01\u0026#39;; When you filter on a RIGHT-side column in WHERE, rows that had NULL (no match) are eliminated - turning your LEFT JOIN into an INNER JOIN silently. Move those filters into the ON clause instead.\n✏️ Practice Set 3 - JOIN mistakes and self-joins Q9. The following query has a bug - it returns the wrong count. Identify the bug and fix it:\nSELECT COUNT(*) AS customers_who_ordered FROM customers AS c JOIN orders AS o ON o.customer_id = c.id; Q10. Write a query that shows all categories with their product count, including categories that have no products (count should show 0, not be missing). Use the correct form of COUNT.\nQ11. Find pairs of customers from the same state. Show customer_1 name, customer_2 name, and state. Avoid showing the same pair twice (use a.id \u0026lt; b.id).\nQ12. Write a query to find all products that have never been ordered. Use the LEFT JOIN anti-join pattern (WHERE right_side IS NULL). Show product name, category name, and price.\nQ13. Fix this query so it correctly shows all customers, including those who have never ordered, with a 0 for their order count:\n-- Broken version SELECT c.name, COUNT(o.id) AS order_count FROM customers AS c JOIN orders AS o ON o.customer_id = c.id GROUP BY c.id, c.name ORDER BY order_count DESC; Part 6 - JOIN vs Subquery: When to Use Which Both JOINs and subqueries can answer the same questions in different ways. Knowing which to reach for makes you a better SQL writer.\n-- Question: which customers have placed at least one order? -- Option A: JOIN SELECT DISTINCT c.name, c.city FROM customers AS c JOIN orders AS o ON o.customer_id = c.id ORDER BY c.name; -- Option B: Subquery with IN SELECT name, city FROM customers WHERE id IN ( SELECT DISTINCT customer_id FROM orders ) ORDER BY name; -- Option C: EXISTS (most semantically clear) SELECT name, city FROM customers AS c WHERE EXISTS ( SELECT 1 FROM orders AS o WHERE o.customer_id = c.id ) ORDER BY name; All three return the same result. Which should you use?\nSituation Prefer You need columns from both tables JOIN You need to aggregate across the relationship JOIN + GROUP BY You only need to check existence EXISTS You are matching against a list of values IN (subquery) You want to exclude rows based on another table NOT EXISTS or LEFT JOIN + IS NULL The subquery returns many rows and is reused CTE (WITH ...) - covered in Lesson 2.5 Part 7 - Full Realistic Query: Customer Order Report This query combines everything from this lesson into a single, production-ready report.\n-- Complete customer order report -- Shows every customer, their order history, and spending summary -- Customers who never ordered appear with 0s SELECT c.name AS customer, c.city, c.state, COUNT(DISTINCT o.id) AS total_orders, COUNT(DISTINCT o.id) FILTER (WHERE o.status = \u0026#39;Delivered\u0026#39;) AS delivered, COUNT(DISTINCT o.id) FILTER (WHERE o.status = \u0026#39;Cancelled\u0026#39;) AS cancelled, COALESCE(SUM(oi.quantity * oi.unit_price) FILTER (WHERE o.status != \u0026#39;Cancelled\u0026#39;), 0) AS total_spent_inr, MAX(o.order_date) AS last_order_date FROM customers AS c LEFT JOIN orders AS o ON o.customer_id = c.id LEFT JOIN order_items AS oi ON oi.order_id = o.id GROUP BY c.id, c.name, c.city, c.state ORDER BY total_spent_inr DESC; Read this query carefully. Every clause has a reason:\nLEFT JOIN so customers with no orders appear (with 0s) LEFT JOIN order_items because we need item-level data for the sum COUNT(DISTINCT o.id) because each order has multiple items - without DISTINCT we\u0026rsquo;d count items, not orders FILTER for the delivered/cancelled breakdown COALESCE(..., 0) so customers with no orders show 0, not NULL GROUP BY c.id, c.name, c.city, c.state - all non-aggregate SELECT columns Part 8 - Practice Set Answers Answers: Practice Set 1 Q1. Products never ordered:\nSELECT p.name, p.category_id, p.price_inr FROM products AS p LEFT JOIN order_items AS oi ON oi.product_id = p.id WHERE oi.id IS NULL ORDER BY p.name; Q2. Orders with customer details:\nSELECT o.id AS order_id, o.order_date, o.status, c.name AS customer, c.city FROM orders AS o JOIN customers AS c ON o.customer_id = c.id ORDER BY o.order_date; Q3. Categories with product count (including 0):\nSELECT cat.name, COUNT(p.id) AS product_count FROM categories AS cat LEFT JOIN products AS p ON p.category_id = cat.id GROUP BY cat.id, cat.name ORDER BY product_count DESC; Q4. Customers with cancelled orders:\nSELECT c.name, c.city, COUNT(o.id) AS cancelled_orders FROM customers AS c JOIN orders AS o ON o.customer_id = c.id WHERE o.status = \u0026#39;Cancelled\u0026#39; GROUP BY c.id, c.name, c.city ORDER BY cancelled_orders DESC; Answers: Practice Set 2 Q5. Order total per order:\nSELECT o.id AS order_id, c.name AS customer, o.order_date, o.status, SUM(oi.quantity * oi.unit_price) AS order_total FROM orders AS o JOIN customers AS c ON o.customer_id = c.id JOIN order_items AS oi ON oi.order_id = o.id GROUP BY o.id, c.name, o.order_date, o.status ORDER BY order_total DESC; Q6. City with highest spending:\nSELECT c.city, COUNT(DISTINCT o.id) AS total_orders, SUM(oi.quantity * oi.unit_price) AS total_spent FROM customers AS c JOIN orders AS o ON o.customer_id = c.id JOIN order_items AS oi ON oi.order_id = o.id WHERE o.status != \u0026#39;Cancelled\u0026#39; GROUP BY c.city ORDER BY total_spent DESC; Q7. Each customer\u0026rsquo;s most recent order and its status:\nSELECT c.name, o.order_date AS latest_order_date, o.status FROM customers AS c JOIN orders AS o ON o.customer_id = c.id WHERE o.order_date = ( SELECT MAX(o2.order_date) FROM orders AS o2 WHERE o2.customer_id = c.id ) ORDER BY o.order_date DESC; Q8. Category revenue with distinct products ordered:\nSELECT cat.name AS category, COUNT(DISTINCT p.id) AS distinct_products_ordered, SUM(oi.quantity * oi.unit_price) AS total_revenue FROM categories AS cat JOIN products AS p ON p.category_id = cat.id JOIN order_items AS oi ON oi.product_id = p.id JOIN orders AS o ON oi.order_id = o.id WHERE o.status != \u0026#39;Cancelled\u0026#39; GROUP BY cat.id, cat.name ORDER BY total_revenue DESC; Answers: Practice Set 3 Q9. Fix the count bug:\n-- Bug: COUNT(*) counts order rows, not customers -- Fix: COUNT(DISTINCT c.id) SELECT COUNT(DISTINCT c.id) AS customers_who_ordered FROM customers AS c JOIN orders AS o ON o.customer_id = c.id; Q10. Categories with product count including 0:\nSELECT cat.name, COUNT(p.id) AS product_count -- NOT COUNT(*) FROM categories AS cat LEFT JOIN products AS p ON p.category_id = cat.id GROUP BY cat.id, cat.name ORDER BY product_count DESC; Q11. Customer pairs in same state:\nSELECT a.name AS customer_1, b.name AS customer_2, a.state FROM customers AS a JOIN customers AS b ON a.state = b.state AND a.id \u0026lt; b.id ORDER BY a.state, a.name; Q12. Products never ordered:\nSELECT p.name, cat.name AS category, p.price_inr FROM products AS p JOIN categories AS cat ON p.category_id = cat.id LEFT JOIN order_items AS oi ON oi.product_id = p.id WHERE oi.id IS NULL ORDER BY p.name; Q13. Fix the query - all customers including those with 0 orders:\nSELECT c.name, COUNT(o.id) AS order_count -- COUNT(o.id) not COUNT(*) - handles NULLs correctly FROM customers AS c LEFT JOIN orders AS o ON o.customer_id = c.id -- LEFT JOIN not JOIN GROUP BY c.id, c.name ORDER BY order_count DESC; What\u0026rsquo;s Next You have covered:\n✅ INNER JOIN - only matching rows ✅ LEFT JOIN - all left rows + matches ✅ RIGHT JOIN - all right rows + matches (and why to avoid it) ✅ FULL OUTER JOIN - all rows from both ✅ The anti-join pattern - LEFT JOIN + WHERE IS NULL ✅ 4-table JOIN - the full ShopDB chain ✅ JOIN + GROUP BY - aggregating across relationships ✅ Self-joins - a table joined to itself ✅ The 4 most common JOIN mistakes and how to fix them ✅ JOIN vs subquery vs EXISTS - when to use which In Lesson 2.5 we build StreamDB - a Hotstar-style streaming platform - and go deep on Subqueries \u0026amp; CTEs:\nSubqueries in WHERE, FROM, and SELECT IN, NOT IN, EXISTS, NOT EXISTS Correlated subqueries WITH (CTEs) - named, chainable subqueries When a CTE is cleaner than a nested subquery ","permalink":"https://blogs.codeverra.com/blog/sql-part4/","summary":"\u003ch1 id=\"lesson-4---joins--relationships\"\u003eLesson 4 - JOINs \u0026amp; Relationships\u003c/h1\u003e\n\u003ch3 id=\"theory--practice--shopdb\"\u003eTheory + Practice | ShopDB\u003c/h3\u003e\n\u003chr\u003e\n\u003ch2 id=\"before-you-start\"\u003eBefore You Start\u003c/h2\u003e\n\u003cp\u003eThis lesson uses \u003cstrong\u003eShopDB\u003c/strong\u003e exclusively.\nConnect to \u003ccode\u003eshopdb\u003c/code\u003e in pgAdmin before running any query.\u003c/p\u003e\n\u003cp\u003eRun this to confirm your data is ready:\u003c/p\u003e\n\u003cdiv class=\"highlight\"\u003e\u003cpre tabindex=\"0\" class=\"chroma\"\u003e\u003ccode class=\"language-sql\" data-lang=\"sql\"\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"k\"\u003eSELECT\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"s1\"\u003e\u0026#39;customers\u0026#39;\u003c/span\u003e\u003cspan class=\"w\"\u003e  \u003c/span\u003e\u003cspan class=\"k\"\u003eAS\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"n\"\u003et\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"k\"\u003eCOUNT\u003c/span\u003e\u003cspan class=\"p\"\u003e(\u003c/span\u003e\u003cspan class=\"o\"\u003e*\u003c/span\u003e\u003cspan class=\"p\"\u003e)\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"k\"\u003eFROM\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"n\"\u003ecustomers\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"k\"\u003eUNION\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"k\"\u003eALL\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"k\"\u003eSELECT\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"s1\"\u003e\u0026#39;products\u0026#39;\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e\u003cspan class=\"w\"\u003e   \u003c/span\u003e\u003cspan class=\"k\"\u003eCOUNT\u003c/span\u003e\u003cspan class=\"p\"\u003e(\u003c/span\u003e\u003cspan class=\"o\"\u003e*\u003c/span\u003e\u003cspan class=\"p\"\u003e)\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"k\"\u003eFROM\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"n\"\u003eproducts\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"k\"\u003eUNION\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"k\"\u003eALL\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"k\"\u003eSELECT\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"s1\"\u003e\u0026#39;orders\u0026#39;\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e\u003cspan class=\"w\"\u003e     \u003c/span\u003e\u003cspan class=\"k\"\u003eCOUNT\u003c/span\u003e\u003cspan class=\"p\"\u003e(\u003c/span\u003e\u003cspan class=\"o\"\u003e*\u003c/span\u003e\u003cspan class=\"p\"\u003e)\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"k\"\u003eFROM\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"n\"\u003eorders\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"k\"\u003eUNION\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"k\"\u003eALL\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"k\"\u003eSELECT\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"s1\"\u003e\u0026#39;order_items\u0026#39;\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e\u003cspan class=\"k\"\u003eCOUNT\u003c/span\u003e\u003cspan class=\"p\"\u003e(\u003c/span\u003e\u003cspan class=\"o\"\u003e*\u003c/span\u003e\u003cspan class=\"p\"\u003e)\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"k\"\u003eFROM\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"n\"\u003eorder_items\u003c/span\u003e\u003cspan class=\"p\"\u003e;\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"c1\"\u003e-- Expected: 20, 25, 15, 30\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/code\u003e\u003c/pre\u003e\u003c/div\u003e\u003cp\u003e\u003cstrong\u003eWhat you will learn:\u003c/strong\u003e\u003c/p\u003e","tags":["sql","data-analysis"],"title":"Joins and Relationships"},{"content":"Lesson 5 - Subqueries \u0026amp; CTEs Theory + Practice | StreamDB - Indian Streaming Platform Before You Start This lesson introduces StreamDB - a Hotstar-style streaming platform.\nRun setup.sql to create and populate StreamDB Verify you see 6 tables with the correct row counts Connect to streamdb in pgAdmin before running any query The StreamDB schema:\nusers shows genres ┌────┬───────┬──────┐ ┌────┬────────┬──────────┐ ┌────┬──────────┐ │ id │ name │ plan │ │ id │ title │is_premium│ │ id │ name │ └────┴───────┴──────┘ └────┴────────┴──────────┘ └────┴──────────┘ │ │ │ │ │ show_genres │ │ │ ┌──────────────┘ ┌────────────┘ │ │ show_id | genre_id │ │ └──────────────────────────┘ (many-to-many) │ subscriptions watch_history ┌────┬─────────┬──────────────┐ ┌────┬─────────┬─────────┬─────────┐ │ id │ user_id │ plan │ │ id │ user_id │ show_id │ minutes │ │ │ │ started_on │ │ │ │ watched │ watched │ │ │ │ expires_on │ └────┴─────────┴─────────┴─────────┘ └────┴─────────┴──────────────┘ What you will learn:\nConcept What it does Subquery in WHERE with IN Filter rows based on another query\u0026rsquo;s result Subquery in WHERE with NOT IN Exclude rows based on another query EXISTS / NOT EXISTS Check if any matching row exists Subquery in FROM Use a query result as a table Scalar subquery in SELECT One value computed per row Correlated subquery Subquery that references the outer query WITH - CTE Named, reusable query blocks Chained CTEs Multiple CTEs in sequence CTE vs subquery vs JOIN When to use which Part 1 - What Is a Subquery? A subquery is a query inside another query. The inner query runs first and its result is used by the outer query.\n-- Question: which shows have been watched by Premium users? -- Step 1: find all Premium user ids SELECT id FROM users WHERE plan = \u0026#39;Premium\u0026#39;; -- Returns: 1, 5, 12, 17 -- Step 2: find watch history for those users SELECT DISTINCT show_id FROM watch_history WHERE user_id IN (1, 5, 12, 17); -- Combined as a subquery (the inner query replaces the hardcoded list) SELECT DISTINCT show_id FROM watch_history WHERE user_id IN ( SELECT id FROM users WHERE plan = \u0026#39;Premium\u0026#39; ); The inner query - SELECT id FROM users WHERE plan = 'Premium' - runs first. Its result (a list of ids) is handed to the outer query\u0026rsquo;s IN clause.\nYou never have to know the actual ids. The query figures it out dynamically.\nPart 2 - Subquery in WHERE with IN IN (subquery) - include rows where the column matches any value in the subquery result.\n-- Shows that have at least one watch history entry -- (i.e. shows that have actually been watched) SELECT title, language, rating FROM shows WHERE id IN ( SELECT DISTINCT show_id FROM watch_history ) ORDER BY rating DESC; -- Users who have watched a Premium show -- (is_premium = TRUE in the shows table) SELECT name, city, plan FROM users WHERE id IN ( SELECT DISTINCT user_id FROM watch_history WHERE show_id IN ( SELECT id FROM shows WHERE is_premium = TRUE ) ) ORDER BY name; This is a nested subquery - a subquery inside a subquery. Read from the innermost outward:\nSELECT id FROM shows WHERE is_premium = TRUE → premium show ids SELECT DISTINCT user_id FROM watch_history WHERE show_id IN (...) → users who watched them Outer query filters users by those ids Nested subqueries work but can get hard to read quickly. CTEs (Part 6) solve the readability problem.\n-- Users from Maharashtra or Kerala who have an active subscription SELECT name, city, plan FROM users WHERE state IN (\u0026#39;Maharashtra\u0026#39;, \u0026#39;Kerala\u0026#39;) AND id IN ( SELECT user_id FROM subscriptions WHERE expires_on \u0026gt; CURRENT_DATE ) ORDER BY name; Part 3 - Subquery in WHERE with NOT IN NOT IN (subquery) - include rows where the column matches none of the subquery values.\n-- Shows that have NEVER been watched SELECT title, language, is_premium FROM shows WHERE id NOT IN ( SELECT DISTINCT show_id FROM watch_history ) ORDER BY title; -- Users who have NEVER watched anything SELECT name, city, plan FROM users WHERE id NOT IN ( SELECT DISTINCT user_id FROM watch_history ) ORDER BY name; -- Free users who have never had a subscription of any kind SELECT name, city, joined_on FROM users WHERE plan = \u0026#39;Free\u0026#39; AND id NOT IN ( SELECT DISTINCT user_id FROM subscriptions ) ORDER BY joined_on; ⚠️ The NOT IN + NULL Trap This is one of the most dangerous mistakes in SQL. Memorise it.\n-- Suppose watch_history had a NULL in the show_id column -- This query would return ZERO rows - not the shows that were never watched SELECT title FROM shows WHERE id NOT IN ( SELECT show_id FROM watch_history -- if any show_id is NULL here... ); -- the entire NOT IN returns nothing Why? NOT IN uses != comparisons internally. Comparing anything to NULL gives NULL (unknown), not FALSE. So the entire NOT IN condition becomes unknown → no rows pass.\nThe fix: always use IS NOT NULL inside a NOT IN subquery:\n-- Safe version SELECT title FROM shows WHERE id NOT IN ( SELECT show_id FROM watch_history WHERE show_id IS NOT NULL -- ✅ guard against NULLs ); Better fix: use NOT EXISTS instead (Part 4) - it handles NULLs correctly by design.\n✏️ Practice Set 1 - IN and NOT IN Q1. Find all shows (title and rating) that have been watched by users from Tamil Nadu or Kerala.\nQ2. Find all users who have never watched any show rated above 9.0. Show name, city, and plan.\nQ3. Find shows that are premium (is_premium = TRUE) but have never appeared in any watch history entry. These are premium shows nobody has watched yet.\nQ4. Find all users who joined in 2023 AND have at least one subscription record (in the subscriptions table). Show name, joined_on, and plan.\nPart 4 - EXISTS and NOT EXISTS EXISTS (subquery) returns TRUE if the subquery produces any rows at all. It does not care about the values - just whether a row exists.\nEXISTS is generally faster than IN for large datasets because it stops as soon as it finds the first match, rather than building the whole list. It also handles NULLs correctly - making it the preferred choice over NOT IN.\n-- Users who have watched at least one show -- EXISTS version (compare with IN version from Part 2) SELECT name, city, plan FROM users AS u WHERE EXISTS ( SELECT 1 FROM watch_history AS wh WHERE wh.user_id = u.id -- ← references outer query: this is a correlated subquery ) ORDER BY name; SELECT 1 - we don\u0026rsquo;t care what the subquery returns, just whether a row exists. The WHERE wh.user_id = u.id links the inner query to each row of the outer query. This makes it a correlated subquery - explained in detail in Part 5.\n-- Users who have NEVER watched anything (NOT EXISTS) SELECT name, city, plan FROM users AS u WHERE NOT EXISTS ( SELECT 1 FROM watch_history AS wh WHERE wh.user_id = u.id ) ORDER BY name; -- Shows that have at least one Crime genre tag SELECT title, language, rating FROM shows AS s WHERE EXISTS ( SELECT 1 FROM show_genres AS sg JOIN genres AS g ON sg.genre_id = g.id WHERE sg.show_id = s.id AND g.name = \u0026#39;Crime\u0026#39; ) ORDER BY rating DESC; -- Users who have an active subscription (expires_on in the future) SELECT name, plan, city FROM users AS u WHERE EXISTS ( SELECT 1 FROM subscriptions AS sub WHERE sub.user_id = u.id AND sub.expires_on \u0026gt; CURRENT_DATE ) ORDER BY name; IN vs EXISTS:\nUse IN when the subquery returns a small, static list of values Use EXISTS when checking for the existence of a related row Always use NOT EXISTS instead of NOT IN when NULLs might be present Both often produce the same result - EXISTS is usually safer and sometimes faster ✏️ Practice Set 2 - EXISTS and NOT EXISTS Q5. Using EXISTS, find all shows that have at least one Drama genre tag. Show title, language, and rating.\nQ6. Using NOT EXISTS, find all users who do not have any subscription record at all (past or present). Show name, city, and plan.\nQ7. Find all users who have watched every show with a rating above 9.0 that appears in the watch_history table. Hint: use a combination of IN and a subquery.\nQ8. Find shows that have been watched by at least one user on a Free plan. Use EXISTS. Show title and rating.\nPart 5 - Correlated Subqueries A correlated subquery references a column from the outer query. This means the inner query runs once per row of the outer query - making it powerful but potentially slow on large tables.\n-- For each user, show how many shows they have watched -- The inner query runs separately for each user row SELECT name, plan, ( SELECT COUNT(*) FROM watch_history AS wh WHERE wh.user_id = u.id -- references outer query\u0026#39;s u.id ) AS shows_watched FROM users AS u ORDER BY shows_watched DESC; -- For each show, show the total minutes watched across all users SELECT title, language, ( SELECT COALESCE(SUM(wh.minutes_watched), 0) FROM watch_history AS wh WHERE wh.show_id = s.id ) AS total_minutes_watched FROM shows AS s ORDER BY total_minutes_watched DESC; -- Users who watched more than the average number of minutes -- across all watch history entries SELECT u.name, u.plan, SUM(wh.minutes_watched) AS total_minutes FROM users AS u JOIN watch_history AS wh ON wh.user_id = u.id GROUP BY u.id, u.name, u.plan HAVING SUM(wh.minutes_watched) \u0026gt; ( SELECT AVG(minutes_watched) FROM watch_history ) ORDER BY total_minutes DESC; The subquery in HAVING - SELECT AVG(minutes_watched) FROM watch_history - computes one value (the overall average) and the HAVING clause compares each user\u0026rsquo;s total against it.\n-- Each show\u0026#39;s rating compared to the average rating of its language group SELECT title, language, rating, ROUND( (SELECT AVG(s2.rating) FROM shows AS s2 WHERE s2.language = s1.language), -- correlated: same language as outer row 2 ) AS lang_avg_rating, CASE WHEN rating \u0026gt; (SELECT AVG(s2.rating) FROM shows AS s2 WHERE s2.language = s1.language) THEN \u0026#39;Above average\u0026#39; ELSE \u0026#39;Below average\u0026#39; END AS vs_language_avg FROM shows AS s1 WHERE rating IS NOT NULL ORDER BY language, rating DESC; Performance note: Correlated subqueries can be slow because they run once per outer row. For large tables, a JOIN or CTE is usually faster. Use correlated subqueries when they make the query clearest - then optimise if needed.\nPart 6 - Subquery in FROM (Derived Tables) You can use a subquery as if it were a table in the FROM clause. This is called a derived table or inline view.\n-- Average minutes watched per plan type -- Step 1: get total minutes per user -- Step 2: average those totals by plan SELECT plan, ROUND(AVG(total_minutes), 1) AS avg_minutes_per_user FROM ( SELECT u.plan, u.id, SUM(wh.minutes_watched) AS total_minutes FROM users AS u JOIN watch_history AS wh ON wh.user_id = u.id GROUP BY u.plan, u.id ) AS user_totals -- ← the derived table needs an alias GROUP BY plan ORDER BY avg_minutes_per_user DESC; -- Top 3 most-watched shows (by total minutes) per language SELECT language, title, total_minutes FROM ( SELECT s.language, s.title, SUM(wh.minutes_watched) AS total_minutes, RANK() OVER ( PARTITION BY s.language ORDER BY SUM(wh.minutes_watched) DESC ) AS rnk FROM shows AS s JOIN watch_history AS wh ON wh.show_id = s.id GROUP BY s.id, s.language, s.title ) AS ranked WHERE rnk \u0026lt;= 3 ORDER BY language, total_minutes DESC; Derived tables work - but when a subquery is complex or reused in multiple places, a CTE is much cleaner. That is what Part 7 is for.\nPart 7 - WITH: Common Table Expressions (CTEs) A CTE names a subquery so you can:\nReference it by name instead of nesting it Use it multiple times in the same query Chain multiple CTEs together in sequence Syntax:\nWITH cte_name AS ( SELECT ... -- the subquery ) SELECT ... FROM cte_name; -- used like a regular table -- Rewrite the derived table example using a CTE -- Much more readable WITH user_totals AS ( SELECT u.id, u.plan, SUM(wh.minutes_watched) AS total_minutes FROM users AS u JOIN watch_history AS wh ON wh.user_id = u.id GROUP BY u.id, u.plan ) SELECT plan, ROUND(AVG(total_minutes), 1) AS avg_minutes_per_user, COUNT(*) AS user_count FROM user_totals GROUP BY plan ORDER BY avg_minutes_per_user DESC; -- Which genres are most popular? (by total watch minutes) WITH genre_watch AS ( SELECT g.name AS genre, SUM(wh.minutes_watched) AS total_minutes, COUNT(DISTINCT wh.user_id) AS unique_watchers FROM genres AS g JOIN show_genres AS sg ON sg.genre_id = g.id JOIN watch_history AS wh ON wh.show_id = sg.show_id GROUP BY g.id, g.name ) SELECT genre, total_minutes, unique_watchers, ROUND(total_minutes::NUMERIC / unique_watchers, 1) AS avg_minutes_per_watcher FROM genre_watch ORDER BY total_minutes DESC; Part 8 - Chained CTEs You can define multiple CTEs in one WITH block, each building on the previous.\n-- Multi-step analysis: plan upgrade candidates -- Users on Free or Mobile who watch a lot but don\u0026#39;t have Premium WITH -- Step 1: total watch minutes per user user_activity AS ( SELECT user_id, SUM(minutes_watched) AS total_minutes, COUNT(DISTINCT show_id) AS shows_watched, COUNT(*) FILTER (WHERE completed = TRUE) AS completed_shows FROM watch_history GROUP BY user_id ), -- Step 2: users with active subscriptions active_subs AS ( SELECT DISTINCT user_id FROM subscriptions WHERE expires_on \u0026gt; CURRENT_DATE ), -- Step 3: join everything and flag upgrade candidates user_summary AS ( SELECT u.name, u.plan, u.city, COALESCE(ua.total_minutes, 0) AS total_minutes, COALESCE(ua.shows_watched, 0) AS shows_watched, CASE WHEN a.user_id IS NOT NULL THEN \u0026#39;Active sub\u0026#39; ELSE \u0026#39;No active sub\u0026#39; END AS subscription_status FROM users AS u LEFT JOIN user_activity AS ua ON ua.user_id = u.id LEFT JOIN active_subs AS a ON a.user_id = u.id ) -- Final query: Free/Mobile users who watch a lot - candidates to upsell SELECT name, plan, city, total_minutes, shows_watched, subscription_status FROM user_summary WHERE plan IN (\u0026#39;Free\u0026#39;, \u0026#39;Mobile\u0026#39;) AND total_minutes \u0026gt; 100 ORDER BY total_minutes DESC; Read the CTEs from top to bottom - each one is a named, reusable step. The final SELECT reads cleanly because all the complexity is named above.\n-- Which shows should be recommended to Free users? -- (Popular shows that Free users haven\u0026#39;t watched yet) WITH free_users AS ( SELECT id FROM users WHERE plan = \u0026#39;Free\u0026#39; ), popular_shows AS ( SELECT show_id, COUNT(DISTINCT user_id) AS watcher_count FROM watch_history GROUP BY show_id HAVING COUNT(DISTINCT user_id) \u0026gt;= 3 -- watched by 3+ people ), free_user_watches AS ( SELECT DISTINCT show_id FROM watch_history WHERE user_id IN (SELECT id FROM free_users) ) SELECT s.title, s.language, s.rating, s.is_premium, ps.watcher_count FROM popular_shows AS ps JOIN shows AS s ON s.id = ps.show_id WHERE ps.show_id NOT IN (SELECT show_id FROM free_user_watches) AND s.is_premium = FALSE -- only recommend free shows to free users ORDER BY ps.watcher_count DESC, s.rating DESC; ✏️ Practice Set 3 - Correlated Subqueries, Derived Tables, CTEs Q9. Write a correlated subquery that shows each user\u0026rsquo;s name, plan, and the title of the last show they watched (most recent watched_on).\nQ10. Using a CTE, find the top 3 most-watched shows overall (by total minutes). Show title, language, and total_minutes.\nQ11. Write a chained CTE query that:\nStep 1: counts total watch minutes per user Step 2: labels users as \u0026lsquo;High\u0026rsquo; (\u0026gt;200 min), \u0026lsquo;Medium\u0026rsquo; (100–200), or \u0026lsquo;Low\u0026rsquo; (\u0026lt;100) Final: counts how many users fall into each engagement_level Q12. Using NOT EXISTS, find all Hindi or Tamil shows that no user has ever completed (i.e. no watch_history row where show_id matches AND completed = TRUE). Show title, language, and rating.\nQ13. Using a CTE and window function together, rank all shows by total minutes watched within each language group. Show language, title, total_minutes, and rank_in_language. Only include shows that appear in watch_history.\nPart 9 - CTE vs Subquery vs JOIN: Decision Guide ┌─────────────────────────────────────┐ │ Do you need columns from both sides? │ └──────────────┬──────────────────────┘ │ YES ────────── JOIN ──────────────────► │ NO │ ┌──────────────▼──────────────────────┐ │ Are you checking existence only? │ └──────────────┬──────────────────────┘ │ YES ────────── EXISTS / NOT EXISTS ───► │ NO │ ┌──────────────▼──────────────────────┐ │ Is the subquery reused or complex? │ └──────────────┬──────────────────────┘ │ YES ────────── CTE (WITH) ────────────► │ NO │ IN / subquery in WHERE Tool Best when JOIN You need columns from both tables, or aggregating across a relationship IN (subquery) Filtering against a small list from another table EXISTS Checking if any related row exists (especially with possible NULLs) NOT EXISTS Excluding rows where a related row exists - always prefer over NOT IN Subquery in FROM You need to pre-aggregate before the main query Scalar subquery in SELECT One computed value per row, infrequently CTE Subquery is reused, the query has multiple steps, or readability matters Part 10 - Practice Set Answers Answers: Practice Set 1 Q1. Shows watched by Tamil Nadu or Kerala users:\nSELECT DISTINCT s.title, s.rating FROM shows AS s WHERE s.id IN ( SELECT DISTINCT wh.show_id FROM watch_history AS wh WHERE wh.user_id IN ( SELECT id FROM users WHERE state IN (\u0026#39;Tamil Nadu\u0026#39;, \u0026#39;Kerala\u0026#39;) ) ) ORDER BY s.rating DESC; Q2. Users who never watched a show rated above 9.0:\nSELECT name, city, plan FROM users WHERE id NOT IN ( SELECT DISTINCT user_id FROM watch_history WHERE show_id IN ( SELECT id FROM shows WHERE rating \u0026gt; 9.0 ) AND user_id IS NOT NULL ) ORDER BY name; Q3. Premium shows never watched:\nSELECT title, language, rating FROM shows WHERE is_premium = TRUE AND id NOT IN ( SELECT DISTINCT show_id FROM watch_history WHERE show_id IS NOT NULL ) ORDER BY title; Q4. Users who joined in 2023 and have a subscription:\nSELECT name, joined_on, plan FROM users WHERE EXTRACT(YEAR FROM joined_on) = 2023 AND id IN ( SELECT DISTINCT user_id FROM subscriptions ) ORDER BY joined_on; Answers: Practice Set 2 Q5. Shows with Drama genre (EXISTS):\nSELECT title, language, rating FROM shows AS s WHERE EXISTS ( SELECT 1 FROM show_genres AS sg JOIN genres AS g ON sg.genre_id = g.id WHERE sg.show_id = s.id AND g.name = \u0026#39;Drama\u0026#39; ) ORDER BY rating DESC; Q6. Users with no subscription ever (NOT EXISTS):\nSELECT name, city, plan FROM users AS u WHERE NOT EXISTS ( SELECT 1 FROM subscriptions AS sub WHERE sub.user_id = u.id ) ORDER BY name; Q7. Users who watched every show rated above 9.0:\nSELECT name, city, plan FROM users WHERE id IN ( SELECT user_id FROM watch_history WHERE show_id IN ( SELECT id FROM shows WHERE rating \u0026gt; 9.0 ) GROUP BY user_id HAVING COUNT(DISTINCT show_id) = ( SELECT COUNT(*) FROM shows WHERE rating \u0026gt; 9.0 ) ) ORDER BY name; Q8. Shows watched by at least one Free user (EXISTS):\nSELECT s.title, s.rating FROM shows AS s WHERE EXISTS ( SELECT 1 FROM watch_history AS wh JOIN users AS u ON wh.user_id = u.id WHERE wh.show_id = s.id AND u.plan = \u0026#39;Free\u0026#39; ) ORDER BY s.rating DESC; Answers: Practice Set 3 Q9. Each user\u0026rsquo;s last watched show:\nSELECT u.name, u.plan, ( SELECT s.title FROM watch_history AS wh JOIN shows AS s ON s.id = wh.show_id WHERE wh.user_id = u.id ORDER BY wh.watched_on DESC LIMIT 1 ) AS last_show_watched FROM users AS u ORDER BY u.name; Q10. Top 3 shows by total minutes (CTE):\nWITH show_totals AS ( SELECT s.title, s.language, SUM(wh.minutes_watched) AS total_minutes FROM shows AS s JOIN watch_history AS wh ON wh.show_id = s.id GROUP BY s.id, s.title, s.language ) SELECT title, language, total_minutes FROM show_totals ORDER BY total_minutes DESC LIMIT 3; Q11. User engagement level breakdown (chained CTEs):\nWITH user_minutes AS ( SELECT user_id, SUM(minutes_watched) AS total_minutes FROM watch_history GROUP BY user_id ), user_levels AS ( SELECT u.id, CASE WHEN COALESCE(um.total_minutes, 0) \u0026gt; 200 THEN \u0026#39;High\u0026#39; WHEN COALESCE(um.total_minutes, 0) \u0026gt;= 100 THEN \u0026#39;Medium\u0026#39; ELSE \u0026#39;Low\u0026#39; END AS engagement_level FROM users AS u LEFT JOIN user_minutes AS um ON um.user_id = u.id ) SELECT engagement_level, COUNT(*) AS user_count FROM user_levels GROUP BY engagement_level ORDER BY user_count DESC; Q12. Hindi or Tamil shows no one completed (NOT EXISTS):\nSELECT title, language, rating FROM shows AS s WHERE language IN (\u0026#39;Hindi\u0026#39;, \u0026#39;Tamil\u0026#39;) AND NOT EXISTS ( SELECT 1 FROM watch_history AS wh WHERE wh.show_id = s.id AND wh.completed = TRUE ) ORDER BY language, rating DESC; Q13. Shows ranked by watch minutes within language (CTE + window):\nWITH show_minutes AS ( SELECT s.id, s.title, s.language, SUM(wh.minutes_watched) AS total_minutes FROM shows AS s JOIN watch_history AS wh ON wh.show_id = s.id GROUP BY s.id, s.title, s.language ) SELECT language, title, total_minutes, RANK() OVER ( PARTITION BY language ORDER BY total_minutes DESC ) AS rank_in_language FROM show_minutes ORDER BY language, rank_in_language; What\u0026rsquo;s Next You have covered:\n✅ Subquery in WHERE with IN and NOT IN ✅ The NULL trap with NOT IN and how to avoid it ✅ EXISTS and NOT EXISTS - safer and often faster ✅ Correlated subqueries - inner query references outer query ✅ Subquery in FROM - derived tables ✅ Scalar subquery in SELECT - one value per row ✅ WITH - CTE syntax and single-CTE usage ✅ Chained CTEs - multi-step analysis with named blocks ✅ CTE vs subquery vs JOIN decision guide In Lesson 2.6 we cover Modifying Data \u0026amp; Schema Changes back on ShopDB:\nINSERT variations - single row, multi-row, INSERT from SELECT UPDATE - single column, multiple columns, UPDATE with a subquery DELETE - safe deletion with WHERE, cascade effects TRUNCATE - fast full-table wipe ALTER TABLE - add/drop columns, change types, rename Transactions - BEGIN, COMMIT, ROLLBACK Savepoints - partial rollback within a transaction ","permalink":"https://blogs.codeverra.com/blog/sql-part5/","summary":"\u003ch1 id=\"lesson-5---subqueries--ctes\"\u003eLesson 5 - Subqueries \u0026amp; CTEs\u003c/h1\u003e\n\u003ch3 id=\"theory--practice--streamdb-----indian-streaming-platform\"\u003eTheory + Practice | StreamDB  -  Indian Streaming Platform\u003c/h3\u003e\n\u003chr\u003e\n\u003ch2 id=\"before-you-start\"\u003eBefore You Start\u003c/h2\u003e\n\u003cp\u003eThis lesson introduces \u003cstrong\u003eStreamDB\u003c/strong\u003e  -  a Hotstar-style streaming platform.\u003c/p\u003e\n\u003col\u003e\n\u003cli\u003eRun \u003ccode\u003esetup.sql\u003c/code\u003e to create and populate StreamDB\u003c/li\u003e\n\u003cli\u003eVerify you see 6 tables with the correct row counts\u003c/li\u003e\n\u003cli\u003eConnect to \u003ccode\u003estreamdb\u003c/code\u003e in pgAdmin before running any query\u003c/li\u003e\n\u003c/ol\u003e\n\u003cp\u003e\u003cstrong\u003eThe StreamDB schema:\u003c/strong\u003e\u003c/p\u003e\n\u003cpre tabindex=\"0\"\u003e\u003ccode\u003eusers               shows               genres\n┌────┬───────┬──────┐  ┌────┬────────┬──────────┐  ┌────┬──────────┐\n│ id │ name  │ plan │  │ id │ title  │is_premium│  │ id │ name     │\n└────┴───────┴──────┘  └────┴────────┴──────────┘  └────┴──────────┘\n       │                       │  │                        │\n       │              show_genres  │                        │\n       │           ┌──────────────┘           ┌────────────┘\n       │           │ show_id | genre_id        │\n       │           └──────────────────────────┘ (many-to-many)\n       │\nsubscriptions                watch_history\n┌────┬─────────┬──────────────┐  ┌────┬─────────┬─────────┬─────────┐\n│ id │ user_id │ plan         │  │ id │ user_id │ show_id │ minutes │\n│    │         │ started_on   │  │    │         │ watched │ watched │\n│    │         │ expires_on   │  └────┴─────────┴─────────┴─────────┘\n└────┴─────────┴──────────────┘\n\u003c/code\u003e\u003c/pre\u003e\u003cp\u003e\u003cstrong\u003eWhat you will learn:\u003c/strong\u003e\u003c/p\u003e","tags":["sql","data-analysis"],"title":"Joins and Relationships"},{"content":"Lesson 6 - Modifying Data \u0026amp; Schema Changes Theory + Practice | ShopDB Before You Start This lesson uses ShopDB. Connect to shopdb in pgAdmin before running any query.\nImportant: This lesson modifies real data. Some queries will permanently change your database. The section on transactions shows you how to do this safely - read Part 7 before running anything you are unsure about.\nQuick data check:\nSELECT \u0026#39;customers\u0026#39; AS t, COUNT(*) FROM customers UNION ALL SELECT \u0026#39;products\u0026#39;, COUNT(*) FROM products UNION ALL SELECT \u0026#39;orders\u0026#39;, COUNT(*) FROM orders UNION ALL SELECT \u0026#39;order_items\u0026#39;, COUNT(*) FROM order_items; -- Expected: 20, 25, 15, 30 What you will learn:\nConcept What it does INSERT variations Single row, multi-row, INSERT from SELECT UPDATE Change values in existing rows UPDATE with subquery Update based on data from another table DELETE Remove rows safely TRUNCATE Fast full-table wipe ALTER TABLE Add/drop columns, change types, rename DROP TABLE Remove a table entirely BEGIN / COMMIT / ROLLBACK Wrap changes in a transaction SAVEPOINT Partial rollback within a transaction Part 1 - INSERT: Adding New Rows You have already used basic INSERT. Here are the variations that matter.\nSingle row INSERT -- Add one new customer INSERT INTO customers (name, email, city, state, joined_on) VALUES (\u0026#39;Siddharth Rao\u0026#39;, \u0026#39;siddharth.rao@gmail.com\u0026#39;, \u0026#39;Bengaluru\u0026#39;, \u0026#39;Karnataka\u0026#39;, \u0026#39;2024-06-01\u0026#39;); -- Using DEFAULT for joined_on (will use CURRENT_DATE) INSERT INTO customers (name, email, city, state) VALUES (\u0026#39;Fatima Khan\u0026#39;, \u0026#39;fatima.khan@gmail.com\u0026#39;, \u0026#39;Hyderabad\u0026#39;, \u0026#39;Telangana\u0026#39;); Multi-row INSERT -- Insert multiple rows in one statement - much faster than separate INSERTs INSERT INTO products (name, category_id, price_inr, stock_quantity) VALUES (\u0026#39;Sony WH-1000XM5\u0026#39;, 1, 29999.00, 20), (\u0026#39;JBL Flip 6\u0026#39;, 1, 9999.00, 40), (\u0026#39;Woodland Sneakers\u0026#39;, 2, 3499.00, 60); INSERT and get the new id back -- RETURNING: get the auto-assigned id after INSERT -- Useful when you immediately need the new row\u0026#39;s id INSERT INTO customers (name, email, city, state) VALUES (\u0026#39;Preethi Menon\u0026#39;, \u0026#39;preethi.menon@gmail.com\u0026#39;, \u0026#39;Chennai\u0026#39;, \u0026#39;Tamil Nadu\u0026#39;) RETURNING id, name; RETURNING is PostgreSQL-specific. It returns the inserted row (or any columns you specify) - handy when you need the new id to insert into a child table next.\nINSERT from SELECT -- Create an archive table first CREATE TABLE customers_archive ( LIKE customers INCLUDING ALL -- copy the structure of customers exactly ); -- Copy all inactive-looking customers (joined before 2023) to the archive INSERT INTO customers_archive SELECT * FROM customers WHERE joined_on \u0026lt; \u0026#39;2023-03-01\u0026#39;; INSERT INTO ... SELECT ... - the SELECT result becomes the data to insert. No VALUES keyword. The column count and types must match.\n✏️ Practice Set 1 - INSERT Q1. Insert a new category called 'Grocery' with the description 'Daily essentials, packaged foods, beverages'. Use RETURNING id, name to see the new category\u0026rsquo;s id.\nQ2. Insert two new products in the Grocery category (use the id returned from Q1):\n'Tata Salt 1kg' - ₹25.00 - 500 units in stock 'Aashirvaad Atta 5kg' - ₹265.00 - 200 units in stock Q3. A new customer signs up today. Insert them using DEFAULT for joined_on:\nName: Kiran Bhat, Email: kiran.bhat@gmail.com City: Mysore, State: Karnataka Q4. Create a table called products_premium with the same structure as products, then insert all products priced above ₹20000 into it using INSERT from SELECT.\nPart 2 - UPDATE: Changing Existing Data UPDATE modifies values in existing rows. Always use WHERE unless you deliberately want to update every row.\nBasic UPDATE -- A product\u0026#39;s price has changed UPDATE products SET price_inr = 27999.00 WHERE name = \u0026#39;Samsung Galaxy S23\u0026#39;; -- Update multiple columns at once UPDATE products SET price_inr = 72999.00, stock_quantity = 35 WHERE name = \u0026#39;Samsung Galaxy S23\u0026#39;; -- Check how many rows were affected -- pgAdmin shows \u0026#34;UPDATE N\u0026#34; in the Messages panel -- N = number of rows changed UPDATE with expressions -- Apply a 10% discount to all Electronics products UPDATE products SET price_inr = ROUND(price_inr * 0.90, 2) WHERE category_id = ( SELECT id FROM categories WHERE name = \u0026#39;Electronics\u0026#39; ); -- Increase stock by 50 for all Sports products -- Using the current value in the expression UPDATE products SET stock_quantity = stock_quantity + 50 WHERE category_id = ( SELECT id FROM categories WHERE name = \u0026#39;Sports\u0026#39; ); UPDATE with a subquery in WHERE -- Mark all orders from customers in Delhi as high priority -- (We don\u0026#39;t have a priority column yet - we\u0026#39;ll add one in Part 5) -- For now: update status to \u0026#39;Shipped\u0026#39; for all Pending Delhi orders UPDATE orders SET status = \u0026#39;Shipped\u0026#39; WHERE status = \u0026#39;Pending\u0026#39; AND customer_id IN ( SELECT id FROM customers WHERE state = \u0026#39;Delhi\u0026#39; ); UPDATE with FROM (PostgreSQL-specific join-style update) -- Update order status using a JOIN-style UPDATE -- Set all orders from Karnataka customers to \u0026#39;Shipped\u0026#39; if still Pending UPDATE orders AS o SET status = \u0026#39;Shipped\u0026#39; FROM customers AS c WHERE o.customer_id = c.id AND c.state = \u0026#39;Karnataka\u0026#39; AND o.status = \u0026#39;Pending\u0026#39;; UPDATE ... FROM is PostgreSQL\u0026rsquo;s way of updating based on data from another table. It is cleaner than a subquery when the condition involves multiple columns.\nSafe UPDATE pattern - always preview first -- STEP 1: Run as SELECT first to see which rows would be affected SELECT id, name, price_inr FROM products WHERE category_id = 1; -- Electronics -- STEP 2: Only after confirming, run the UPDATE UPDATE products SET price_inr = ROUND(price_inr * 0.90, 2) WHERE category_id = 1; Golden rule: Before any UPDATE or DELETE, write the equivalent SELECT first. Confirm the rows, then change SELECT ... to UPDATE SET. This habit prevents accidental mass-updates.\n✏️ Practice Set 2 - UPDATE Q5. The iPhone 15\u0026rsquo;s price has dropped to ₹74,999. Update it in the products table.\nQ6. All products in the Beauty category are going out of stock. Set stock_quantity = 0 and is_available = FALSE for every product in the Beauty category. (Use a subquery to find the category id.)\nQ7. A customer changed their email address. Update Meera Gupta\u0026rsquo;s email to meera.gupta.new@gmail.com.\nQ8. Apply a 5% price increase to all products priced under ₹500. Round the result to 2 decimal places. Preview with a SELECT first, then run the UPDATE.\nPart 3 - DELETE: Removing Rows DELETE removes rows permanently. Like UPDATE, always use WHERE unless you mean to delete everything.\nBasic DELETE -- Delete a specific order item DELETE FROM order_items WHERE id = 1; -- Delete all cancelled orders\u0026#39; items first (FK constraint requires this) DELETE FROM order_items WHERE order_id IN ( SELECT id FROM orders WHERE status = \u0026#39;Cancelled\u0026#39; ); -- Then delete the cancelled orders themselves DELETE FROM orders WHERE status = \u0026#39;Cancelled\u0026#39;; Foreign key order matters: You cannot delete a parent row if child rows still reference it. Delete children first, then parents. (Unless the FK is set up with ON DELETE CASCADE - then the DB handles it.)\nDELETE with a subquery -- Remove all products that have never been ordered DELETE FROM products WHERE id NOT IN ( SELECT DISTINCT product_id FROM order_items WHERE product_id IS NOT NULL ); Preview before DELETE -- STEP 1: SELECT to preview what would be deleted SELECT id, name, price_inr FROM products WHERE stock_quantity = 0 AND is_available = FALSE; -- STEP 2: If the result looks right, delete DELETE FROM products WHERE stock_quantity = 0 AND is_available = FALSE; RETURNING with DELETE -- Delete and see exactly what was removed DELETE FROM customers WHERE email = \u0026#39;preethi.menon@gmail.com\u0026#39; RETURNING id, name, email; RETURNING works with DELETE too - useful for logging or confirming exactly which rows were removed.\nPart 4 - TRUNCATE: Fast Full-Table Wipe TRUNCATE removes all rows from a table instantly. It is much faster than DELETE FROM table (no WHERE) for large tables because it does not scan rows - it just drops and recreates the data pages.\n-- Remove all rows from the archive table we created TRUNCATE TABLE customers_archive; -- Truncate and reset the auto-increment counter TRUNCATE TABLE customers_archive RESTART IDENTITY; RESTART IDENTITY resets the SERIAL counter back to 1. Without it, the next inserted row picks up from where the old counter left off.\nTRUNCATE vs DELETE:\nDELETE TRUNCATE Removes all rows ✅ ✅ Can use WHERE ✅ ❌ Can ROLLBACK ✅ ✅ (in PostgreSQL) Fires triggers ✅ ❌ Resets SERIAL ❌ ✅ (with RESTART IDENTITY) Speed on large tables Slow Fast Use DELETE when you need WHERE. Use TRUNCATE when wiping a whole table.\n✏️ Practice Set 3 - DELETE and TRUNCATE Q9. Delete the two Grocery products you inserted in Practice Set 1 (Tata Salt and Aashirvaad Atta). Use a subquery to find the Grocery category id rather than hardcoding it.\nQ10. Preview then delete all products where is_available = FALSE AND stock_quantity = 0. How many rows are affected?\nQ11. Delete the customers_archive table\u0026rsquo;s data using TRUNCATE. Then confirm it is empty with a SELECT COUNT(*).\nQ12. A customer (Fatima Khan, inserted in Q3 of Practice Set 1) wants to delete their account. Delete her from the customers table using RETURNING to confirm which row was removed.\nPart 5 - ALTER TABLE: Changing the Schema ALTER TABLE changes the structure of a table - adding or removing columns, changing types, renaming things. It does not touch the data (unless the type change requires conversion).\nAdd a column -- Add a \u0026#39;phone\u0026#39; column to customers ALTER TABLE customers ADD COLUMN phone VARCHAR(15); -- New column is NULL for all existing rows (since no DEFAULT specified) -- Add a column with a default value ALTER TABLE orders ADD COLUMN is_priority BOOLEAN NOT NULL DEFAULT FALSE; -- All existing rows get FALSE automatically -- Add a column with a constraint ALTER TABLE products ADD COLUMN discount_pct NUMERIC(5,2) DEFAULT 0.00 CHECK (discount_pct BETWEEN 0 AND 100); Drop a column -- Remove a column entirely (data is gone permanently) ALTER TABLE customers DROP COLUMN phone; Rename a column ALTER TABLE customers RENAME COLUMN joined_on TO registration_date; Rename a table ALTER TABLE customers_archive RENAME TO customers_backup; Change a column\u0026rsquo;s data type -- Change stock_quantity from INT to BIGINT (safe - no data loss) ALTER TABLE products ALTER COLUMN stock_quantity TYPE BIGINT; -- Change a VARCHAR to TEXT (also safe - TEXT is just unlimited VARCHAR) ALTER TABLE categories ALTER COLUMN description TYPE TEXT; Changing to a narrower type (e.g. TEXT to VARCHAR(10)) may fail if existing data is longer than the new limit. Always check with a SELECT first.\nAdd a constraint to an existing column -- Add a NOT NULL constraint to an existing column -- (only works if no existing rows have NULL in that column) UPDATE customers SET phone = \u0026#39;Unknown\u0026#39; WHERE phone IS NULL; ALTER TABLE customers ALTER COLUMN phone SET NOT NULL; -- Add a UNIQUE constraint ALTER TABLE customers ADD CONSTRAINT customers_phone_unique UNIQUE (phone); Drop a constraint ALTER TABLE customers DROP CONSTRAINT customers_phone_unique; ✏️ Practice Set 4 - ALTER TABLE Q13. Add a loyalty_points column to the customers table. It should be an integer, NOT NULL, with a default of 0.\nQ14. Add a return_policy column to products of type TEXT, allowing NULL (some products may not have a return policy stated).\nQ15. Rename the is_priority column in orders (that you added in Part 5) to priority_order.\nQ16. Update all orders with total value above ₹50,000 to set priority_order = TRUE. (Hint: use UPDATE \u0026hellip; FROM with a subquery that sums order_items.)\nPart 6 - DROP TABLE: Removing a Table DROP TABLE removes the table and all its data permanently. Drop in reverse dependency order - children before parents.\n-- Drop the archive/backup table (no foreign key dependencies) DROP TABLE IF EXISTS customers_backup; DROP TABLE IF EXISTS products_premium; -- IF EXISTS prevents an error if the table does not exist -- Safe to run even if the table was already dropped DROP TABLE IF EXISTS some_table_that_might_not_exist; -- CASCADE: also drops any objects that depend on this table -- (e.g. views, foreign keys referencing it) -- Use with caution DROP TABLE categories CASCADE; -- This would also drop the FK constraint in products -- Do NOT run this on your live ShopDB Never drop a table you are not 100% sure about. Always use IF EXISTS. Always check dependencies first with:\nSELECT conname, conrelid::regclass FROM pg_constraint WHERE confrelid = \u0026#39;your_table\u0026#39;::regclass; Part 7 - Transactions: BEGIN, COMMIT, ROLLBACK A transaction groups multiple SQL statements into one atomic unit. Either all succeed together, or all fail together - there is no partial state.\nBEGIN ← start the transaction statement 1 statement 2 statement 3 COMMIT ← make all changes permanent -- or -- ROLLBACK ← undo everything back to BEGIN Why transactions matter -- Scenario: customer places an order -- We need to: -- 1. Create the order record -- 2. Insert order items -- 3. Decrease stock quantity -- If step 3 fails, steps 1 and 2 should also be undone. -- Without a transaction, you get a half-complete order. Basic transaction BEGIN; -- Step 1: create the order INSERT INTO orders (customer_id, order_date, status, payment_method) VALUES (1, CURRENT_DATE, \u0026#39;Pending\u0026#39;, \u0026#39;UPI\u0026#39;) RETURNING id; -- Suppose it returns id = 16 -- Step 2: add items (using the new order id) INSERT INTO order_items (order_id, product_id, quantity, unit_price) VALUES (16, 3, 1, 24999.00), -- OnePlus Nord (16, 4, 2, 1499.00); -- boAt Headphones x2 -- Step 3: reduce stock UPDATE products SET stock_quantity = stock_quantity - 1 WHERE id = 3; UPDATE products SET stock_quantity = stock_quantity - 2 WHERE id = 4; COMMIT; -- All 4 statements are now permanently saved together ROLLBACK: undoing a transaction BEGIN; -- Accidentally update ALL products (forgot the WHERE clause) UPDATE products SET price_inr = 0; -- Oh no. Let\u0026#39;s check the damage SELECT id, name, price_inr FROM products LIMIT 5; -- All prices are 0! -- Undo everything since BEGIN ROLLBACK; -- Prices are restored SELECT id, name, price_inr FROM products LIMIT 5; -- ✅ Back to original values In pgAdmin: If you accidentally run a destructive query, immediately type ROLLBACK; and run it. As long as you have not yet COMMIT-ted, your changes can be undone. This is why wrapping risky operations in BEGIN first is a good habit.\nSAVEPOINT: partial rollback A savepoint lets you roll back to a specific point within a transaction, without undoing everything from BEGIN.\nBEGIN; -- Safe operation - add a new product INSERT INTO products (name, category_id, price_inr, stock_quantity) VALUES (\u0026#39;Test Product A\u0026#39;, 1, 999.00, 10); SAVEPOINT after_insert; -- mark this point -- Risky operation - update with a mistake UPDATE products SET price_inr = 0 WHERE category_id = 1; -- Check the damage SELECT name, price_inr FROM products WHERE category_id = 1; -- All Electronics are ₹0 - not what we wanted -- Roll back only to the savepoint (INSERT is preserved) ROLLBACK TO SAVEPOINT after_insert; -- The INSERT is still there, the bad UPDATE is undone SELECT name, price_inr FROM products WHERE category_id = 1; -- ✅ Original Electronics prices restored -- Now do the right update UPDATE products SET price_inr = ROUND(price_inr * 0.95, 2) WHERE category_id = 1; COMMIT; -- commit the INSERT + correct UPDATE Transaction best practices -- Always wrap multi-step operations in a transaction BEGIN; -- all related changes COMMIT; -- For any risky single statement, use BEGIN first BEGIN; DELETE FROM orders WHERE status = \u0026#39;Cancelled\u0026#39;; -- review Messages panel: DELETE N -- If N looks right: COMMIT; -- If N looks wrong: ROLLBACK; ✏️ Practice Set 5 - Transactions Q17. Using a transaction, do the following as one atomic unit:\nInsert a new order for customer id 5 (Kabir Singh), dated today, status Pending, payment UPI Insert one order item for that order: product id 19 (SG Cricket Bat), quantity 1, unit_price 3499.00 Reduce the stock of product id 19 by 1 COMMIT if all three succeed Q18. Open a transaction, update ALL products to set is_available = FALSE (simulate a store shutdown), then ROLLBACK before it becomes permanent. Confirm the data is unchanged with a SELECT after the ROLLBACK.\nQ19. Using SAVEPOINT, do the following:\nBEGIN a transaction Insert a new customer (any name/email/city/state) Set a SAVEPOINT called after_customer Try to insert a product with price_inr = -100 (this should fail the CHECK constraint) ROLLBACK TO SAVEPOINT after_customer COMMIT (the customer insert should be saved, the failed product insert rolled back) Part 8 - Putting It All Together A realistic sequence of DML operations for a \u0026ldquo;new product launch\u0026rdquo; scenario:\n-- ============================================================ -- New product launch: add products, update stock, create promo -- ============================================================ BEGIN; -- 1. Add a new sub-category (if it doesn\u0026#39;t exist) INSERT INTO categories (name, description) VALUES (\u0026#39;Wearables\u0026#39;, \u0026#39;Smartwatches, fitness bands, earbuds\u0026#39;) ON CONFLICT (name) DO NOTHING; -- skip if category already exists -- ON CONFLICT is PostgreSQL\u0026#39;s UPSERT - covered more in advanced SQL -- 2. Add new products in that category INSERT INTO products (name, category_id, price_inr, stock_quantity) SELECT p.name, c.id, p.price, p.qty FROM ( VALUES (\u0026#39;boAt Wave Horizon Smartwatch\u0026#39;, 5999.00, 80), (\u0026#39;Noise ColorFit Pro 5\u0026#39;, 3499.00, 120), (\u0026#39;Realme TechLife Watch R100\u0026#39;, 1999.00, 200) ) AS p(name, price, qty) CROSS JOIN ( SELECT id FROM categories WHERE name = \u0026#39;Wearables\u0026#39; ) AS c; -- 3. Apply a launch discount to all Electronics (10% off) UPDATE products SET price_inr = ROUND(price_inr * 0.90, 2) WHERE category_id = (SELECT id FROM categories WHERE name = \u0026#39;Electronics\u0026#39;); -- 4. Verify before committing SELECT name, price_inr, stock_quantity FROM products WHERE category_id IN ( SELECT id FROM categories WHERE name IN (\u0026#39;Electronics\u0026#39;, \u0026#39;Wearables\u0026#39;) ) ORDER BY category_id, price_inr DESC; COMMIT; Part 9 - Practice Set Answers Answers: Practice Set 1 Q1. Insert Grocery category:\nINSERT INTO categories (name, description) VALUES (\u0026#39;Grocery\u0026#39;, \u0026#39;Daily essentials, packaged foods, beverages\u0026#39;) RETURNING id, name; Q2. Insert grocery products (assuming Grocery got id=7):\nINSERT INTO products (name, category_id, price_inr, stock_quantity) VALUES (\u0026#39;Tata Salt 1kg\u0026#39;, 7, 25.00, 500), (\u0026#39;Aashirvaad Atta 5kg\u0026#39;, 7, 265.00, 200); Q3. New customer with DEFAULT joined_on:\nINSERT INTO customers (name, email, city, state) VALUES (\u0026#39;Kiran Bhat\u0026#39;, \u0026#39;kiran.bhat@gmail.com\u0026#39;, \u0026#39;Mysore\u0026#39;, \u0026#39;Karnataka\u0026#39;); Q4. Premium products table:\nCREATE TABLE products_premium (LIKE products INCLUDING ALL); INSERT INTO products_premium SELECT * FROM products WHERE price_inr \u0026gt; 20000; Answers: Practice Set 2 Q5. Update iPhone price:\nUPDATE products SET price_inr = 74999.00 WHERE name = \u0026#39;Apple iPhone 15\u0026#39;; Q6. Beauty products out of stock:\nUPDATE products SET stock_quantity = 0, is_available = FALSE WHERE category_id = ( SELECT id FROM categories WHERE name = \u0026#39;Beauty\u0026#39; ); Q7. Update Meera\u0026rsquo;s email:\nUPDATE customers SET email = \u0026#39;meera.gupta.new@gmail.com\u0026#39; WHERE name = \u0026#39;Meera Gupta\u0026#39;; Q8. 5% price increase for sub-₹500 products:\n-- Preview first SELECT name, price_inr, ROUND(price_inr * 1.05, 2) AS new_price FROM products WHERE price_inr \u0026lt; 500; -- Then update UPDATE products SET price_inr = ROUND(price_inr * 1.05, 2) WHERE price_inr \u0026lt; 500; Answers: Practice Set 3 Q9. Delete grocery products:\nDELETE FROM products WHERE category_id = ( SELECT id FROM categories WHERE name = \u0026#39;Grocery\u0026#39; ); Q10. Delete unavailable, out-of-stock products:\n-- Preview SELECT id, name FROM products WHERE is_available = FALSE AND stock_quantity = 0; -- Delete DELETE FROM products WHERE is_available = FALSE AND stock_quantity = 0; Q11. Truncate archive:\nTRUNCATE TABLE customers_archive RESTART IDENTITY; SELECT COUNT(*) FROM customers_archive; -- should be 0 Q12. Delete Fatima Khan:\nDELETE FROM customers WHERE email = \u0026#39;fatima.khan@gmail.com\u0026#39; RETURNING id, name, email; Answers: Practice Set 4 Q13. Add loyalty_points:\nALTER TABLE customers ADD COLUMN loyalty_points INT NOT NULL DEFAULT 0; Q14. Add return_policy:\nALTER TABLE products ADD COLUMN return_policy TEXT; Q15. Rename is_priority:\nALTER TABLE orders RENAME COLUMN is_priority TO priority_order; Q16. Set priority for high-value orders:\nUPDATE orders AS o SET priority_order = TRUE FROM ( SELECT order_id, SUM(quantity * unit_price) AS total FROM order_items GROUP BY order_id HAVING SUM(quantity * unit_price) \u0026gt; 50000 ) AS big_orders WHERE o.id = big_orders.order_id; Answers: Practice Set 5 Q17. Atomic order placement:\nBEGIN; INSERT INTO orders (customer_id, order_date, status, payment_method) VALUES (5, CURRENT_DATE, \u0026#39;Pending\u0026#39;, \u0026#39;UPI\u0026#39;); -- Note the id returned - assume it is 16 INSERT INTO order_items (order_id, product_id, quantity, unit_price) VALUES (16, 19, 1, 3499.00); UPDATE products SET stock_quantity = stock_quantity - 1 WHERE id = 19; COMMIT; Q18. Store shutdown + ROLLBACK:\nBEGIN; UPDATE products SET is_available = FALSE; SELECT name, is_available FROM products LIMIT 5; -- All FALSE ROLLBACK; SELECT name, is_available FROM products LIMIT 5; -- Back to original values ✅ Q19. SAVEPOINT with constraint failure:\nBEGIN; INSERT INTO customers (name, email, city, state) VALUES (\u0026#39;Test User\u0026#39;, \u0026#39;test.user@gmail.com\u0026#39;, \u0026#39;Mumbai\u0026#39;, \u0026#39;Maharashtra\u0026#39;); SAVEPOINT after_customer; -- This will fail: CHECK (price_inr \u0026gt; 0) rejects -100 INSERT INTO products (name, category_id, price_inr, stock_quantity) VALUES (\u0026#39;Bad Product\u0026#39;, 1, -100, 10); -- ERROR: new row violates check constraint ROLLBACK TO SAVEPOINT after_customer; COMMIT; -- Customer is saved, failed product insert is not What\u0026rsquo;s Next You have covered:\n✅ INSERT - single row, multi-row, RETURNING, INSERT from SELECT ✅ UPDATE - basic, expressions, subquery, UPDATE FROM ✅ Safe preview pattern - SELECT before UPDATE/DELETE ✅ DELETE - with WHERE, with subquery, RETURNING ✅ TRUNCATE - fast wipe, RESTART IDENTITY ✅ ALTER TABLE - add/drop/rename columns, change types, add constraints ✅ DROP TABLE - IF EXISTS, CASCADE ✅ BEGIN / COMMIT / ROLLBACK - atomic transactions ✅ SAVEPOINT - partial rollback within a transaction In Lesson 2.7 we cover Indexes, Constraints \u0026amp; Schema Design:\nHow PostgreSQL finds rows (sequential scan vs index scan) CREATE INDEX - when and what to index Partial and composite indexes Foreign key behaviour - CASCADE, SET NULL, RESTRICT Normalization - 1NF, 2NF, 3NF with real examples Common schema mistakes and how to fix them EXPLAIN - reading a query execution plan ","permalink":"https://blogs.codeverra.com/blog/sql-part6/","summary":"\u003ch1 id=\"lesson-6---modifying-data--schema-changes\"\u003eLesson 6 - Modifying Data \u0026amp; Schema Changes\u003c/h1\u003e\n\u003ch3 id=\"theory--practice--shopdb\"\u003eTheory + Practice | ShopDB\u003c/h3\u003e\n\u003chr\u003e\n\u003ch2 id=\"before-you-start\"\u003eBefore You Start\u003c/h2\u003e\n\u003cp\u003eThis lesson uses \u003cstrong\u003eShopDB\u003c/strong\u003e.\nConnect to \u003ccode\u003eshopdb\u003c/code\u003e in pgAdmin before running any query.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eImportant:\u003c/strong\u003e This lesson modifies real data. Some queries will permanently\nchange your database. The section on transactions shows you how to do this\nsafely  -  read Part 7 before running anything you are unsure about.\u003c/p\u003e\n\u003cp\u003eQuick data check:\u003c/p\u003e\n\u003cdiv class=\"highlight\"\u003e\u003cpre tabindex=\"0\" class=\"chroma\"\u003e\u003ccode class=\"language-sql\" data-lang=\"sql\"\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"k\"\u003eSELECT\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"s1\"\u003e\u0026#39;customers\u0026#39;\u003c/span\u003e\u003cspan class=\"w\"\u003e  \u003c/span\u003e\u003cspan class=\"k\"\u003eAS\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"n\"\u003et\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"k\"\u003eCOUNT\u003c/span\u003e\u003cspan class=\"p\"\u003e(\u003c/span\u003e\u003cspan class=\"o\"\u003e*\u003c/span\u003e\u003cspan class=\"p\"\u003e)\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"k\"\u003eFROM\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"n\"\u003ecustomers\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"k\"\u003eUNION\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"k\"\u003eALL\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"k\"\u003eSELECT\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"s1\"\u003e\u0026#39;products\u0026#39;\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e\u003cspan class=\"w\"\u003e    \u003c/span\u003e\u003cspan class=\"k\"\u003eCOUNT\u003c/span\u003e\u003cspan class=\"p\"\u003e(\u003c/span\u003e\u003cspan class=\"o\"\u003e*\u003c/span\u003e\u003cspan class=\"p\"\u003e)\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"k\"\u003eFROM\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"n\"\u003eproducts\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"k\"\u003eUNION\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"k\"\u003eALL\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"k\"\u003eSELECT\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"s1\"\u003e\u0026#39;orders\u0026#39;\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e\u003cspan class=\"w\"\u003e      \u003c/span\u003e\u003cspan class=\"k\"\u003eCOUNT\u003c/span\u003e\u003cspan class=\"p\"\u003e(\u003c/span\u003e\u003cspan class=\"o\"\u003e*\u003c/span\u003e\u003cspan class=\"p\"\u003e)\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"k\"\u003eFROM\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"n\"\u003eorders\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"k\"\u003eUNION\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"k\"\u003eALL\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"k\"\u003eSELECT\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"s1\"\u003e\u0026#39;order_items\u0026#39;\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"k\"\u003eCOUNT\u003c/span\u003e\u003cspan class=\"p\"\u003e(\u003c/span\u003e\u003cspan class=\"o\"\u003e*\u003c/span\u003e\u003cspan class=\"p\"\u003e)\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"k\"\u003eFROM\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"n\"\u003eorder_items\u003c/span\u003e\u003cspan class=\"p\"\u003e;\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"c1\"\u003e-- Expected: 20, 25, 15, 30\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/code\u003e\u003c/pre\u003e\u003c/div\u003e\u003cp\u003e\u003cstrong\u003eWhat you will learn:\u003c/strong\u003e\u003c/p\u003e","tags":["sql","data-analysis"],"title":"Modifying data \u0026 Schema Changes"},{"content":"Pandas Practice Sheet - IPL Dataset A complete two-table exploration: from match summaries to ball-by-ball intelligence Why IPL Is the Best Dataset for Pandas After Zomato The Zomato dataset taught you how to clean a single messy table. The IPL dataset teaches you something more important: how to work with two related tables and merge them to answer questions neither table can answer alone.\nThis is how data exists in every real company - not as one giant flat file, but as a set of related tables that you must join, aggregate across levels, and reason about carefully.\nThe IPL dataset has two tables:\nmatches.csv - one row per match (meta-data: who played, who won, when, where) deliveries.csv - one row per ball bowled (~180,000 rows: the ball-by-ball record) Answering most interesting questions requires combining both. Who is the best death-overs bowler? You need the bowler\u0026rsquo;s name (deliveries), linked to the season (matches). What is a team\u0026rsquo;s win rate in knockout matches at Wankhede? You need venue and match type (matches) linked to team info (deliveries).\nThis practice sheet takes you from basic aggregation on a single table, through multi-table merges, through window functions and ranking, all the way to a full player performance scorecard. By the end, you will be comfortable working with any multi-table sports dataset - and that skill transfers directly to e-commerce order data, financial transaction data, and healthcare records.\nGetting the Datasets Both files are available as direct raw CSVs - no login required.\nimport pandas as pd import numpy as np # --- Matches: one row per IPL match (2008–2019, ~756 matches) --- matches_url = ( \u0026#34;https://raw.githubusercontent.com/ashutoshkrris/Data-Analysis-with-Python\u0026#34; \u0026#34;/master/zerotopandas-course-project-starter/dataset/matches.csv\u0026#34; ) # --- Deliveries: one row per ball bowled (~179,000 rows) --- deliveries_url = ( \u0026#34;https://raw.githubusercontent.com/ashutoshkrris/Data-Analysis-with-Python\u0026#34; \u0026#34;/master/zerotopandas-course-project-starter/dataset/deliveries.csv\u0026#34; ) matches = pd.read_csv(matches_url) deliveries = pd.read_csv(deliveries_url) print(\u0026#34;Matches shape :\u0026#34;, matches.shape) # (~756, 18) print(\u0026#34;Deliveries shape:\u0026#34;, deliveries.shape) # (~179000, 21) Alternative (Kaggle, newer seasons up to 2024):\nhttps://www.kaggle.com/datasets/patrickb1912/ipl-complete-dataset-20082020 Throughout this sheet we use matches and deliveries as the variable names. The id column in matches corresponds to match_id in deliveries - this is the join key between the two tables.\nColumn Dictionaries matches - One Row Per Match ┌──────────────────────┬──────────┬───────────────────────────────────────────────────────┐ │ Column │ Type │ Meaning │ ├──────────────────────┼──────────┼───────────────────────────────────────────────────────┤ │ id │ int │ Unique match identifier. JOIN KEY to deliveries. │ │ season │ int │ IPL season year (2008–2019). │ │ city │ string │ City where the match was played. │ │ date │ string │ Date of match (YYYY-MM-DD or DD-MM-YYYY format). │ │ team1 │ string │ First team listed in the fixture. │ │ team2 │ string │ Second team listed in the fixture. │ │ toss_winner │ string │ Team that won the toss. │ │ toss_decision │ string │ Toss winner\u0026#39;s decision: \u0026#34;bat\u0026#34; or \u0026#34;field\u0026#34;. │ │ result │ string │ How the match was decided: \u0026#34;normal\u0026#34;, \u0026#34;tie\u0026#34;, \u0026#34;no result\u0026#34;│ │ dl_applied │ int │ 1 if Duckworth-Lewis method was applied, else 0. │ │ winner │ string │ Name of the winning team. NaN for no-result matches. │ │ win_by_runs │ int │ Margin of victory in runs (if batting first won). │ │ win_by_wickets │ int │ Margin of victory in wickets (if chasing team won). │ │ player_of_match │ string │ Player of the match award winner. │ │ venue │ string │ Full name of the stadium. │ │ umpire1 │ string │ First on-field umpire. │ │ umpire2 │ string │ Second on-field umpire. │ │ umpire3 │ string │ Third umpire (TV umpire). Mostly NaN. │ └──────────────────────┴──────────┴───────────────────────────────────────────────────────┘ deliveries - One Row Per Ball ┌──────────────────────┬──────────┬───────────────────────────────────────────────────────┐ │ Column │ Type │ Meaning │ ├──────────────────────┼──────────┼───────────────────────────────────────────────────────┤ │ match_id │ int │ FK → matches.id │ │ inning │ int │ 1 = first innings, 2 = second innings. │ │ batting_team │ string │ Team currently batting. │ │ bowling_team │ string │ Team currently bowling. │ │ over │ int │ Over number (0-indexed in some versions, 1-indexed │ │ │ │ in others - always check!). │ │ ball │ int │ Ball number within the over (1–6, or more for extras). │ │ batsman │ string │ Batsman on strike. │ │ non_striker │ string │ Batsman at the non-striking end. │ │ bowler │ string │ Bowler delivering the ball. │ │ is_super_over │ int │ 1 if the ball was bowled in a Super Over. │ │ wide_runs │ int │ Runs from wide (0 or 1+). │ │ bye_runs │ int │ Runs from byes. │ │ legbye_runs │ int │ Runs from leg byes. │ │ noball_runs │ int │ Runs from no ball. │ │ penalty_runs │ int │ Penalty runs. │ │ batsman_runs │ int │ Runs credited to the batsman (0,1,2,3,4,6). │ │ extra_runs │ int │ Total extras on this delivery. │ │ total_runs │ int │ Total runs on this delivery (batsman + extras). │ │ player_dismissed │ string │ Name of dismissed batsman. NaN if no dismissal. │ │ dismissal_kind │ string │ How out: caught, bowled, run out, lbw, etc. │ │ fielder │ string │ Fielder involved in dismissal (for catches/run outs). │ └──────────────────────┴──────────┴───────────────────────────────────────────────────────┘ The most important columns for analysis:\nbatsman_runs → runs scored by the batsman (use for batting stats) total_runs → total runs on the delivery including extras (use for team totals) extra_runs → only extras (use to isolate economy rate without extras) player_dismissed → non-null means a wicket fell on this delivery over → over number (verify if 0-indexed or 1-indexed with deliveries[\u0026quot;over\u0026quot;].min()) Setup Block import pandas as pd import numpy as np matches = pd.read_csv(matches_url) deliveries = pd.read_csv(deliveries_url) # Orientation print(\u0026#34;=== MATCHES ===\u0026#34;) print(matches.shape) print(matches.dtypes) print(matches.isnull().sum()) print(matches.head(3)) print(\u0026#34;\\n=== DELIVERIES ===\u0026#34;) print(deliveries.shape) print(deliveries.dtypes) print(deliveries.isnull().sum()) print(deliveries.head(3)) # Verify the join key print(\u0026#34;\\nMatch IDs in matches :\u0026#34;, matches[\u0026#34;id\u0026#34;].nunique()) print(\u0026#34;Match IDs in deliveries:\u0026#34;, deliveries[\u0026#34;match_id\u0026#34;].nunique()) print(\u0026#34;Over numbering starts at:\u0026#34;, deliveries[\u0026#34;over\u0026#34;].min()) # 1 or 0? Always verify the join key before merging. If matches[\u0026quot;id\u0026quot;].nunique() ≠ deliveries[\u0026quot;match_id\u0026quot;].nunique(), some matches have no ball data (e.g., abandoned matches or missing data). Know this before you merge.\nSECTION 1 - Matches Table Basics (Questions 1–5) Learn the structure of the match-level data before touching deliveries Question 1 How many matches were played each season? Which season had the most matches? Which had the fewest, and why? Concepts: value_counts(), groupby().count(), sort_index(), domain knowledge about IPL format\nAnswer # Method 1 - value_counts sorted by season year matches_per_season = matches[\u0026#34;season\u0026#34;].value_counts().sort_index() print(matches_per_season) # 2008 58 # 2009 57 ← hosted in South Africa (security concerns in India) # 2010 60 # 2011 73 # 2012 76 # 2013 76 # 2014 60 ← partial hosting in UAE # 2015 59 # 2016 60 # 2017 59 # 2018 60 # 2019 60 # Method 2 - groupby for the same result print(matches.groupby(\u0026#34;season\u0026#34;).size().rename(\u0026#34;match_count\u0026#34;)) # Most and fewest print(f\u0026#34;\\nSeason with most matches : {matches_per_season.idxmax()} ({matches_per_season.max()} matches)\u0026#34;) print(f\u0026#34;Season with fewest matches: {matches_per_season.idxmin()} ({matches_per_season.min()} matches)\u0026#34;) # Total matches across all seasons print(f\u0026#34;\\nTotal matches in dataset: {len(matches)}\u0026#34;) Domain insight: 2009 had slightly fewer matches because the tournament was moved to South Africa due to general elections in India. 2011 onwards had more matches because the league expanded to 10 teams (adding Pune Warriors and Kochi Tuskers Kerala). Understanding why numbers fluctuate across time is as important as reading the numbers themselves.\nsort_index() vs sort_values(): value_counts() sorts by frequency (most common first) by default. For time series data like seasons, you almost always want chronological order - use .sort_index() to get that.\nQuestion 2 Which teams have played the most matches in IPL history? Calculate total matches played, total wins, and win percentage for each team. Concepts: Combining two columns (team1 + team2) with pd.concat(), value_counts(), separate groupby for wins, merging two aggregations\nAnswer # Total matches played = appearances as team1 + appearances as team2 # We need to count every time a team name appears in EITHER column # Step 1: Stack team1 and team2 into a single Series all_teams_played = pd.concat([matches[\u0026#34;team1\u0026#34;], matches[\u0026#34;team2\u0026#34;]], ignore_index=True) total_played = all_teams_played.value_counts().rename(\u0026#34;total_played\u0026#34;) # Step 2: Count wins total_wins = matches[\u0026#34;winner\u0026#34;].value_counts().rename(\u0026#34;total_wins\u0026#34;) # Step 3: Merge into one table team_record = pd.DataFrame({ \u0026#34;total_played\u0026#34;: total_played, \u0026#34;total_wins\u0026#34; : total_wins, }).dropna() # remove teams that never won (or never appeared) team_record[\u0026#34;win_pct\u0026#34;] = ( team_record[\u0026#34;total_wins\u0026#34;] / team_record[\u0026#34;total_played\u0026#34;] * 100 ).round(2) team_record = team_record.sort_values(\u0026#34;total_played\u0026#34;, ascending=False) print(team_record) # Quick sanity check: losses team_record[\u0026#34;total_losses\u0026#34;] = team_record[\u0026#34;total_played\u0026#34;] - team_record[\u0026#34;total_wins\u0026#34;] The pd.concat([team1, team2]) pattern is the standard way to count appearances across two columns. team1 and team2 are two perspectives on the same fact (\u0026ldquo;this team was in this match\u0026rdquo;), and concatenating them before value_counts() counts total appearances correctly. This pattern appears constantly in sports data - goals scored/conceded, home/away records, etc.\nWhy dropna() here? total_wins comes from matches[\u0026quot;winner\u0026quot;], which has NaN for no-result matches. When we merge, teams that never won or were never in a result have NaN wins. dropna() removes them - but in production you\u0026rsquo;d want to fillna(0) instead to keep all teams with 0 wins.\nQuestion 3 What percentage of toss winners also won the match? Break this down by toss decision (bat vs field). Concepts: Boolean column from comparison, groupby(), mean() on binary flag, crosstab()\nAnswer # Step 1: Flag whether the toss winner also won the match # Filter out no-result matches first valid = matches[matches[\u0026#34;result\u0026#34;] == \u0026#34;normal\u0026#34;].copy() valid[\u0026#34;toss_won_match\u0026#34;] = (valid[\u0026#34;toss_winner\u0026#34;] == valid[\u0026#34;winner\u0026#34;]).astype(int) # Overall toss-to-win conversion overall_rate = valid[\u0026#34;toss_won_match\u0026#34;].mean() print(f\u0026#34;Overall: toss winner won the match {overall_rate:.1%} of the time\u0026#34;) # Step 2: Break down by toss decision toss_decision_stats = ( valid.groupby(\u0026#34;toss_decision\u0026#34;)[\u0026#34;toss_won_match\u0026#34;] .agg( matches_count = \u0026#34;count\u0026#34;, won_match = \u0026#34;sum\u0026#34;, win_rate = \u0026#34;mean\u0026#34;, ) .round(3) ) print(\u0026#34;\\nBy toss decision:\u0026#34;) print(toss_decision_stats) # Step 3: Crosstab for a cleaner view ct = pd.crosstab( index = valid[\u0026#34;toss_decision\u0026#34;], columns = valid[\u0026#34;toss_won_match\u0026#34;].map({1: \u0026#34;Won Match\u0026#34;, 0: \u0026#34;Lost Match\u0026#34;}), margins = True, margins_name = \u0026#34;Total\u0026#34; ) print(\u0026#34;\\nCrosstab:\u0026#34;) print(ct) # Normalize by row - win rate per decision type ct_pct = pd.crosstab( index = valid[\u0026#34;toss_decision\u0026#34;], columns = valid[\u0026#34;toss_won_match\u0026#34;].map({1: \u0026#34;Won\u0026#34;, 0: \u0026#34;Lost\u0026#34;}), normalize = \u0026#34;index\u0026#34; ).round(3) print(\u0026#34;\\nNormalized:\u0026#34;) print(ct_pct) Insight: Across most IPL seasons, fielding first (chasing) has been slightly more advantageous. Teams that elected to field after winning the toss have a marginally higher win rate than those who batted. This trend has strengthened over the years as T20 teams got better at chasing. The toss is not destiny - but the decision matters more than people think.\nQuestion 4 Which venues have hosted the most matches? For the top 10 venues, what is the average win margin (in runs for first-innings wins, in wickets for second-innings wins)? Concepts: value_counts(), conditional aggregation, loc[] filtering, groupby() with separate filters\nAnswer # Top 10 venues by match count top10_venues = matches[\u0026#34;venue\u0026#34;].value_counts().head(10) print(\u0026#34;Top 10 venues:\u0026#34;) print(top10_venues) # Win margins - separated by type # Batting-first wins: win_by_runs \u0026gt; 0 # Chasing wins: win_by_wickets \u0026gt; 0 batting_first_wins = matches[matches[\u0026#34;win_by_runs\u0026#34;] \u0026gt; 0] chasing_wins = matches[matches[\u0026#34;win_by_wickets\u0026#34;] \u0026gt; 0] venue_bat_margin = ( batting_first_wins.groupby(\u0026#34;venue\u0026#34;)[\u0026#34;win_by_runs\u0026#34;] .agg(count=\u0026#34;count\u0026#34;, avg_margin=\u0026#34;mean\u0026#34;) .round(1) .rename(columns={\u0026#34;count\u0026#34;: \u0026#34;bat_first_wins\u0026#34;, \u0026#34;avg_margin\u0026#34;: \u0026#34;avg_runs_margin\u0026#34;}) ) venue_chase_margin = ( chasing_wins.groupby(\u0026#34;venue\u0026#34;)[\u0026#34;win_by_wickets\u0026#34;] .agg(count=\u0026#34;count\u0026#34;, avg_margin=\u0026#34;mean\u0026#34;) .round(1) .rename(columns={\u0026#34;count\u0026#34;: \u0026#34;chase_wins\u0026#34;, \u0026#34;avg_margin\u0026#34;: \u0026#34;avg_wkt_margin\u0026#34;}) ) # Combine into one venue table venue_stats = pd.DataFrame({\u0026#34;total_matches\u0026#34;: matches[\u0026#34;venue\u0026#34;].value_counts()}) venue_stats = venue_stats.join(venue_bat_margin).join(venue_chase_margin) print(\u0026#34;\\nVenue statistics for top 10:\u0026#34;) print(venue_stats.head(10).to_string()) Separate filtering before groupby is often cleaner than a conditional inside agg. Matches won by runs and matches won by wickets are mutually exclusive - filtering first, then aggregating, keeps the logic explicit and avoids the need for np.where() or lambda gymnastics.\nQuestion 5 Who are the top 10 \u0026ldquo;Player of the Match\u0026rdquo; winners all-time? Show their count and the teams they most often played for when winning the award. Concepts: value_counts(), groupby() with mode() inside lambda, merging two aggregations\nAnswer # Top 10 player of the match winners top_pom = ( matches[\u0026#34;player_of_match\u0026#34;] .value_counts() .head(10) .reset_index() ) top_pom.columns = [\u0026#34;player\u0026#34;, \u0026#34;pom_count\u0026#34;] print(top_pom) # For each player, which team were they playing for most often when they won? # This requires checking both team1 and team2 against the winner column # Approximation: use the winning team when the player won PoM # Build a player→winning_team mapping per match pom_with_winner = matches[[\u0026#34;player_of_match\u0026#34;, \u0026#34;winner\u0026#34;]].dropna() pom_team_mode = ( pom_with_winner .groupby(\u0026#34;player_of_match\u0026#34;)[\u0026#34;winner\u0026#34;] .agg(lambda x: x.mode().iloc[0]) # most common winning team for this player .reset_index() .rename(columns={\u0026#34;player_of_match\u0026#34;: \u0026#34;player\u0026#34;, \u0026#34;winner\u0026#34;: \u0026#34;primary_team\u0026#34;}) ) # Merge top_pom = top_pom.merge(pom_team_mode, on=\u0026#34;player\u0026#34;, how=\u0026#34;left\u0026#34;) print(\u0026#34;\\nTop 10 PoM winners with primary team:\u0026#34;) print(top_pom.to_string(index=False)) SECTION 2 - Deliveries Table Basics (Questions 6–9) Learn the ball-by-ball table before joining the two tables Question 6 What is the total number of runs scored in IPL history (from the deliveries table)? Break it down into batsman runs vs extra runs. What percentage of all runs came from extras? Concepts: sum() on multiple columns, percentage calculation, describe() for distribution\nAnswer # Total runs from each source total_batsman_runs = deliveries[\u0026#34;batsman_runs\u0026#34;].sum() total_extra_runs = deliveries[\u0026#34;extra_runs\u0026#34;].sum() total_runs_all = deliveries[\u0026#34;total_runs\u0026#34;].sum() print(f\u0026#34;Total batsman runs : {total_batsman_runs:,}\u0026#34;) print(f\u0026#34;Total extra runs : {total_extra_runs:,}\u0026#34;) print(f\u0026#34;Total runs (all) : {total_runs_all:,}\u0026#34;) print(f\u0026#34;Extras as % of all runs: {total_extra_runs / total_runs_all * 100:.2f}%\u0026#34;) # Break extras down further extra_types = { \u0026#34;wide_runs\u0026#34; : deliveries[\u0026#34;wide_runs\u0026#34;].sum(), \u0026#34;noball_runs\u0026#34; : deliveries[\u0026#34;noball_runs\u0026#34;].sum(), \u0026#34;bye_runs\u0026#34; : deliveries[\u0026#34;bye_runs\u0026#34;].sum(), \u0026#34;legbye_runs\u0026#34; : deliveries[\u0026#34;legbye_runs\u0026#34;].sum(), \u0026#34;penalty_runs\u0026#34;: deliveries[\u0026#34;penalty_runs\u0026#34;].sum(), } print(\u0026#34;\\nExtra run breakdown:\u0026#34;) for k, v in extra_types.items(): print(f\u0026#34; {k:\u0026lt;15}: {v:,} ({v/total_extra_runs*100:.1f}% of extras)\u0026#34;) # Distribution of total_runs per delivery print(\u0026#34;\\nDistribution of runs per ball:\u0026#34;) print(deliveries[\u0026#34;total_runs\u0026#34;].value_counts().sort_index()) # 0 ~55% (dot balls) # 1 ~24% # 2 ~4% # 4 ~9% # 6 ~5% total_runs vs batsman_runs: Always be deliberate about which column you use. For team scoring totals (match scores), use total_runs - it includes extras. For individual batting statistics (strike rate, batting average), use batsman_runs - extras don\u0026rsquo;t count toward a batsman\u0026rsquo;s personal tally. Using the wrong column is a very common mistake in cricket analytics.\nQuestion 7 Who are the top 10 run-scorers of all time in IPL? Show total runs, balls faced (legal deliveries only), boundaries (4s and 6s), and strike rate. Concepts: Multiple aggregations, groupby() on player, filtering wide balls for legal deliveries, strike rate calculation as a derived column\nAnswer # Legal deliveries = balls where wide_runs == 0 # (Wide balls don\u0026#39;t count as \u0026#34;balls faced\u0026#34; by the batsman) legal_deliveries = deliveries[deliveries[\u0026#34;wide_runs\u0026#34;] == 0] batting_stats = ( legal_deliveries.groupby(\u0026#34;batsman\u0026#34;) .agg( total_runs = (\u0026#34;batsman_runs\u0026#34;, \u0026#34;sum\u0026#34;), balls_faced = (\u0026#34;batsman_runs\u0026#34;, \u0026#34;count\u0026#34;), # count of legal balls fours = (\u0026#34;batsman_runs\u0026#34;, lambda x: (x == 4).sum()), sixes = (\u0026#34;batsman_runs\u0026#34;, lambda x: (x == 6).sum()), ) .reset_index() ) # Derived columns batting_stats[\u0026#34;strike_rate\u0026#34;] = ( batting_stats[\u0026#34;total_runs\u0026#34;] / batting_stats[\u0026#34;balls_faced\u0026#34;] * 100 ).round(2) batting_stats[\u0026#34;boundary_runs\u0026#34;] = ( batting_stats[\u0026#34;fours\u0026#34;] * 4 + batting_stats[\u0026#34;sixes\u0026#34;] * 6 ) batting_stats[\u0026#34;boundary_pct\u0026#34;] = ( batting_stats[\u0026#34;boundary_runs\u0026#34;] / batting_stats[\u0026#34;total_runs\u0026#34;] * 100 ).round(1) # Top 10 by runs top10_batsmen = ( batting_stats .nlargest(10, \u0026#34;total_runs\u0026#34;) .reset_index(drop=True) ) top10_batsmen.index += 1 # rank starts at 1 print(\u0026#34;Top 10 Run Scorers in IPL History:\u0026#34;) print(top10_batsmen[[\u0026#34;batsman\u0026#34;, \u0026#34;total_runs\u0026#34;, \u0026#34;balls_faced\u0026#34;, \u0026#34;fours\u0026#34;, \u0026#34;sixes\u0026#34;, \u0026#34;strike_rate\u0026#34;, \u0026#34;boundary_pct\u0026#34;]].to_string()) Filtering wides before counting balls faced is a non-negotiable step in cricket analytics. A wide ball is not counted as a ball faced by the batsman in the official scorecard. If you count all rows per batsman, you\u0026rsquo;ll overcount their balls faced (and therefore undercount their strike rate) by roughly 2–4%.\nLambda inside agg() is the correct way to count how many times a value equals 4 or 6: lambda x: (x == 4).sum(). This is readable and efficient for this dataset size. For very large datasets, consider doing it in a separate step using boolean masks.\nQuestion 8 Who are the top 10 wicket-takers of all time? Show wickets, overs bowled, runs conceded, economy rate, and bowling average. Concepts: Filtering dismissals, handling run-outs (not credited to bowler), groupby() with multiple derived stats\nAnswer # Wickets: player_dismissed is not NaN AND dismissal is NOT run_out or retired_hurt # (Run-outs are not credited to the bowler) bowler_wickets = deliveries[ deliveries[\u0026#34;player_dismissed\u0026#34;].notna() \u0026amp; ~deliveries[\u0026#34;dismissal_kind\u0026#34;].isin([\u0026#34;run out\u0026#34;, \u0026#34;retired hurt\u0026#34;, \u0026#34;obstructing the field\u0026#34;]) ] # Count wickets per bowler wickets_per_bowler = ( bowler_wickets.groupby(\u0026#34;bowler\u0026#34;)[\u0026#34;player_dismissed\u0026#34;] .count() .rename(\u0026#34;wickets\u0026#34;) ) # Runs conceded (exclude wide and no-ball runs for economy) # Economy = runs per over (6 legal balls = 1 over) # Balls bowled = deliveries where wide_runs == 0 legal_balls = deliveries[deliveries[\u0026#34;wide_runs\u0026#34;] == 0] bowling_stats = ( legal_balls.groupby(\u0026#34;bowler\u0026#34;) .agg( legal_balls_bowled = (\u0026#34;total_runs\u0026#34;, \u0026#34;count\u0026#34;), runs_conceded = (\u0026#34;total_runs\u0026#34;, \u0026#34;sum\u0026#34;), ) .reset_index() ) # Add wides and no-balls to runs conceded (they count toward economy) all_extras = ( deliveries.groupby(\u0026#34;bowler\u0026#34;)[[\u0026#34;wide_runs\u0026#34;, \u0026#34;noball_runs\u0026#34;]] .sum() .reset_index() ) all_extras[\u0026#34;extra_runs_bowler\u0026#34;] = all_extras[\u0026#34;wide_runs\u0026#34;] + all_extras[\u0026#34;noball_runs\u0026#34;] bowling_stats = bowling_stats.merge( all_extras[[\u0026#34;bowler\u0026#34;, \u0026#34;extra_runs_bowler\u0026#34;]], on=\u0026#34;bowler\u0026#34;, how=\u0026#34;left\u0026#34; ) bowling_stats[\u0026#34;total_runs_conceded\u0026#34;] = ( bowling_stats[\u0026#34;runs_conceded\u0026#34;] + bowling_stats[\u0026#34;extra_runs_bowler\u0026#34;].fillna(0) ) # Merge wickets in bowling_stats = bowling_stats.merge( wickets_per_bowler.reset_index(), on=\u0026#34;bowler\u0026#34;, how=\u0026#34;left\u0026#34; ) bowling_stats[\u0026#34;wickets\u0026#34;] = bowling_stats[\u0026#34;wickets\u0026#34;].fillna(0).astype(int) # Derived metrics bowling_stats[\u0026#34;overs_bowled\u0026#34;] = (bowling_stats[\u0026#34;legal_balls_bowled\u0026#34;] / 6).round(2) bowling_stats[\u0026#34;economy_rate\u0026#34;] = ( bowling_stats[\u0026#34;total_runs_conceded\u0026#34;] / bowling_stats[\u0026#34;overs_bowled\u0026#34;] ).round(2) bowling_stats[\u0026#34;bowling_avg\u0026#34;] = ( bowling_stats[\u0026#34;total_runs_conceded\u0026#34;] / bowling_stats[\u0026#34;wickets\u0026#34;].replace(0, np.nan) ).round(2) # Top 10 wicket-takers (min 50 overs bowled) top10_bowlers = ( bowling_stats[bowling_stats[\u0026#34;overs_bowled\u0026#34;] \u0026gt;= 50] .nlargest(10, \u0026#34;wickets\u0026#34;) .reset_index(drop=True) ) top10_bowlers.index += 1 print(\u0026#34;Top 10 Wicket-Takers in IPL History:\u0026#34;) print(top10_bowlers[[\u0026#34;bowler\u0026#34;, \u0026#34;wickets\u0026#34;, \u0026#34;overs_bowled\u0026#34;, \u0026#34;total_runs_conceded\u0026#34;, \u0026#34;economy_rate\u0026#34;, \u0026#34;bowling_avg\u0026#34;]].to_string()) Why exclude run-outs from bowler wickets? In cricket scoring rules, run-outs, retired hurts, and obstruction are not credited to the bowler. Counting them would inflate a bowler\u0026rsquo;s wicket tally incorrectly. This kind of domain knowledge is essential - pandas will happily count them if you don\u0026rsquo;t filter explicitly.\nEconomy rate uses TOTAL runs conceded (including wides and no-balls), but overs bowled counts only LEGAL deliveries. This is the official cricket convention. A bowler who bowls 3 wides in an over has a 7-ball over (those 3 wides are counted toward runs but the over length is still 6 legal balls).\nQuestion 9 What is the run-scoring pattern across overs? Calculate the average runs per over for each over number (1–20) across all matches. Identify the powerplay (overs 1–6), middle (7–15), and death (16–20) phase averages. Concepts: groupby() on over number, phase labeling with pd.cut(), transform() for phase averages\nAnswer # Average runs per over across all deliveries # total_runs per over = sum of all runs in that over number across all matches runs_per_over = ( deliveries.groupby(\u0026#34;over\u0026#34;)[\u0026#34;total_runs\u0026#34;] .agg( total_runs = \u0026#34;sum\u0026#34;, total_balls = \u0026#34;count\u0026#34;, avg_per_ball = \u0026#34;mean\u0026#34;, ) .reset_index() ) # Average runs per over = total_runs / number of times that over was bowled # \u0026#34;Number of times over X was bowled\u0026#34; = total_balls / ~6 (approx overs) # More precisely: group by (match_id, inning, over) to count actual overs overs_bowled_count = ( deliveries.groupby([\u0026#34;match_id\u0026#34;, \u0026#34;inning\u0026#34;, \u0026#34;over\u0026#34;])[\u0026#34;total_runs\u0026#34;] .sum() .reset_index() ) avg_runs_per_over = ( overs_bowled_count.groupby(\u0026#34;over\u0026#34;)[\u0026#34;total_runs\u0026#34;] .mean() .round(2) .reset_index() .rename(columns={\u0026#34;total_runs\u0026#34;: \u0026#34;avg_runs_per_over\u0026#34;}) ) print(\u0026#34;Average runs per over:\u0026#34;) print(avg_runs_per_over.to_string(index=False)) # Phase labeling avg_runs_per_over[\u0026#34;phase\u0026#34;] = pd.cut( avg_runs_per_over[\u0026#34;over\u0026#34;], bins = [0, 6, 15, 20], labels = [\u0026#34;Powerplay (1-6)\u0026#34;, \u0026#34;Middle (7-15)\u0026#34;, \u0026#34;Death (16-20)\u0026#34;], right = True ) phase_summary = ( avg_runs_per_over.groupby(\u0026#34;phase\u0026#34;, observed=True)[\u0026#34;avg_runs_per_over\u0026#34;] .mean() .round(2) ) print(\u0026#34;\\nPhase averages:\u0026#34;) print(phase_summary) # Powerplay: ~7.5–8 runs/over # Middle: ~7.0–7.5 runs/over # Death: ~9.5–10 runs/over ← highest scoring phase The two-step groupby pattern (first group by match+inning+over, then group by over) is how you correctly compute \u0026ldquo;average runs per over.\u0026rdquo; If you directly average total_runs across all rows with over=16, you\u0026rsquo;re averaging individual ball runs, not over totals. Always think about what the unit of your analysis is.\nSECTION 3 - Merging the Two Tables (Questions 10–13) The most important section - joining match-level and ball-level data Question 10 Add the season information to the deliveries table by merging. Then calculate the total runs scored per team per season. Which team had the highest scoring season in IPL history? Concepts: pd.merge(), left vs inner join, groupby() on merged columns, understanding join types\nAnswer # Step 1: Merge season info from matches into deliveries # We need: match_id → season (from matches table) deliveries_with_season = deliveries.merge( matches[[\u0026#34;id\u0026#34;, \u0026#34;season\u0026#34;]], # only bring the columns we need left_on = \u0026#34;match_id\u0026#34;, right_on = \u0026#34;id\u0026#34;, how = \u0026#34;left\u0026#34; # keep all deliveries even if match not found ) print(f\u0026#34;Deliveries before merge: {len(deliveries):,}\u0026#34;) print(f\u0026#34;Deliveries after merge : {len(deliveries_with_season):,}\u0026#34;) print(f\u0026#34;Season column added : {deliveries_with_season[\u0026#39;season\u0026#39;].notna().sum():,} non-null\u0026#34;) # Step 2: Total runs per team per season team_season_runs = ( deliveries_with_season.groupby([\u0026#34;season\u0026#34;, \u0026#34;batting_team\u0026#34;])[\u0026#34;total_runs\u0026#34;] .sum() .reset_index() .rename(columns={\u0026#34;total_runs\u0026#34;: \u0026#34;total_runs_scored\u0026#34;}) ) # Highest single-season run total by a team best_season = team_season_runs.nlargest(10, \u0026#34;total_runs_scored\u0026#34;) print(\u0026#34;\\nTop 10 highest-scoring team-seasons:\u0026#34;) print(best_season.to_string(index=False)) # Overall total runs per team across all seasons overall_runs = ( team_season_runs.groupby(\u0026#34;batting_team\u0026#34;)[\u0026#34;total_runs_scored\u0026#34;] .sum() .sort_values(ascending=False) .reset_index() .rename(columns={\u0026#34;total_runs_scored\u0026#34;: \u0026#34;all_time_runs\u0026#34;}) ) print(\u0026#34;\\nAll-time runs scored per team:\u0026#34;) print(overall_runs.to_string(index=False)) how='left' vs how='inner':\ninner → only keeps rows where the key exists in BOTH tables (safe, no NaN from join) left → keeps ALL rows from the left table; adds NaN for right columns when no match found For deliveries, use left join. If a match was abandoned mid-game, the deliveries still exist but the match might be marked \u0026ldquo;no result.\u0026rdquo; An inner join would silently drop those deliveries. Use left and check .isnull().sum() on the new column to see how many rows didn\u0026rsquo;t match.\nOnly bring columns you need from the right table. matches[[\u0026quot;id\u0026quot;, \u0026quot;season\u0026quot;]] instead of the full matches DataFrame. This avoids column name conflicts and keeps the merged table clean.\nQuestion 11 Which batsmen perform best in the death overs (overs 16–20)? Calculate death-over runs, death-over strike rate, and the percentage of team\u0026rsquo;s death-over runs each player contributes. Minimum 100 balls faced in death overs. Concepts: Multi-condition filtering, groupby() on filtered subset, transform() for within-group percentage\nAnswer # Step 1: Filter to death overs only (over numbers 16–20) # Check if overs are 0-indexed or 1-indexed print(\u0026#34;Over range:\u0026#34;, deliveries[\u0026#34;over\u0026#34;].min(), \u0026#34;to\u0026#34;, deliveries[\u0026#34;over\u0026#34;].max()) # If min is 0: overs 15-19 = death overs # If min is 1: overs 16-20 = death overs is_one_indexed = deliveries[\u0026#34;over\u0026#34;].min() == 1 death_over_start = 16 if is_one_indexed else 15 death_over_end = 20 if is_one_indexed else 19 death_overs = deliveries[ (deliveries[\u0026#34;over\u0026#34;] \u0026gt;= death_over_start) \u0026amp; (deliveries[\u0026#34;over\u0026#34;] \u0026lt;= death_over_end) \u0026amp; (deliveries[\u0026#34;wide_runs\u0026#34;] == 0) # legal balls only for balls faced ].copy() # Step 2: Death-over batting stats per player death_batting = ( death_overs.groupby(\u0026#34;batsman\u0026#34;) .agg( runs_scored = (\u0026#34;batsman_runs\u0026#34;, \u0026#34;sum\u0026#34;), balls_faced = (\u0026#34;batsman_runs\u0026#34;, \u0026#34;count\u0026#34;), fours = (\u0026#34;batsman_runs\u0026#34;, lambda x: (x == 4).sum()), sixes = (\u0026#34;batsman_runs\u0026#34;, lambda x: (x == 6).sum()), ) .reset_index() ) death_batting[\u0026#34;strike_rate\u0026#34;] = ( death_batting[\u0026#34;runs_scored\u0026#34;] / death_batting[\u0026#34;balls_faced\u0026#34;] * 100 ).round(2) # Step 3: Filter minimum 100 balls death_batting = death_batting[death_batting[\u0026#34;balls_faced\u0026#34;] \u0026gt;= 100] # Step 4: Rank by death-over strike rate top_death_batsmen = ( death_batting .sort_values(\u0026#34;strike_rate\u0026#34;, ascending=False) .head(15) .reset_index(drop=True) ) top_death_batsmen.index += 1 print(\u0026#34;Best Death-Over Batsmen (min 100 balls, sorted by SR):\u0026#34;) print(top_death_batsmen[[\u0026#34;batsman\u0026#34;, \u0026#34;runs_scored\u0026#34;, \u0026#34;balls_faced\u0026#34;, \u0026#34;fours\u0026#34;, \u0026#34;sixes\u0026#34;, \u0026#34;strike_rate\u0026#34;]].to_string()) Phase-based analysis (powerplay, middle, death) is the backbone of modern T20 analytics. Most batting and bowling records are meaningless without phase context - a batsman with SR 120 in the middle overs is average; the same SR in death overs is weak. Always segment by phase before comparing players.\nQuestion 12 Build a head-to-head record between any two teams: total matches, wins each, win rate, and average winning margin. Then show the all-time head-to-head matrix for all team pairs. Concepts: Multi-condition filtering, groupby() on two columns simultaneously, pivot_table() to create a matrix, crosstab()\nAnswer # Step 1: Head-to-head function for any two teams def head_to_head(team_a, team_b): h2h = matches[ ((matches[\u0026#34;team1\u0026#34;] == team_a) \u0026amp; (matches[\u0026#34;team2\u0026#34;] == team_b)) | ((matches[\u0026#34;team1\u0026#34;] == team_b) \u0026amp; (matches[\u0026#34;team2\u0026#34;] == team_a)) ].copy() h2h = h2h[h2h[\u0026#34;result\u0026#34;] == \u0026#34;normal\u0026#34;] total = len(h2h) wins_a = (h2h[\u0026#34;winner\u0026#34;] == team_a).sum() wins_b = (h2h[\u0026#34;winner\u0026#34;] == team_b).sum() print(f\u0026#34;\\n{team_a} vs {team_b}\u0026#34;) print(f\u0026#34; Total matches : {total}\u0026#34;) print(f\u0026#34; {team_a:\u0026lt;35}: {wins_a} wins ({wins_a/total*100:.1f}%)\u0026#34;) print(f\u0026#34; {team_b:\u0026lt;35}: {wins_b} wins ({wins_b/total*100:.1f}%)\u0026#34;) head_to_head(\u0026#34;Mumbai Indians\u0026#34;, \u0026#34;Chennai Super Kings\u0026#34;) head_to_head(\u0026#34;Royal Challengers Bangalore\u0026#34;, \u0026#34;Kolkata Knight Riders\u0026#34;) # Step 2: All-time win matrix (how many times team A beat team B) # Create a row per match with (winner, loser) results_df = matches[matches[\u0026#34;result\u0026#34;] == \u0026#34;normal\u0026#34;].copy() results_df[\u0026#34;loser\u0026#34;] = results_df.apply( lambda r: r[\u0026#34;team2\u0026#34;] if r[\u0026#34;winner\u0026#34;] == r[\u0026#34;team1\u0026#34;] else r[\u0026#34;team1\u0026#34;], axis=1 ) # Pivot: rows = winner, columns = loser, values = count win_matrix = pd.crosstab( index = results_df[\u0026#34;winner\u0026#34;], columns = results_df[\u0026#34;loser\u0026#34;] ) win_matrix.index.name = \u0026#34;Winner →\u0026#34; win_matrix.columns.name = \u0026#34;← Loser\u0026#34; print(\u0026#34;\\nHead-to-head win matrix (row team beat column team N times):\u0026#34;) print(win_matrix) Using apply(axis=1) with a lambda is the right tool for row-level logic that depends on multiple columns simultaneously. Here we need both winner and team1/team2 to determine the loser. This can\u0026rsquo;t be done with a simple vectorized operation - apply(axis=1) iterates row-by-row, which is slower but correct.\naxis=1 in apply vs axis=0:\naxis=0 (default) → function applied to each COLUMN axis=1 → function applied to each ROW When your lambda receives a full row and uses row[\u0026quot;col_name\u0026quot;], you need axis=1. Question 13 Merge the two tables to find: for each player, how many \u0026ldquo;Player of the Match\u0026rdquo; awards have they won, and what is their average runs scored in those specific matches (vs their overall average)? Concepts: Multi-step merge, filtering on merged data, comparing two aggregations with merge()\nAnswer # Step 1: Get Player of the Match for each match pom = matches[[\u0026#34;id\u0026#34;, \u0026#34;player_of_match\u0026#34;]].dropna() # Step 2: Merge deliveries with PoM info # We want to know: in the matches where player X won PoM, how many runs did they score? deliveries_pom = deliveries.merge( pom, left_on = \u0026#34;match_id\u0026#34;, right_on = \u0026#34;id\u0026#34;, how = \u0026#34;inner\u0026#34; ) # Step 3: Filter to rows where the batsman IS the PoM winner pom_matches_batting = deliveries_pom[ deliveries_pom[\u0026#34;batsman\u0026#34;] == deliveries_pom[\u0026#34;player_of_match\u0026#34;] ] # Runs scored by each player in their PoM matches pom_runs = ( pom_matches_batting.groupby(\u0026#34;player_of_match\u0026#34;) .agg( pom_count = (\u0026#34;match_id\u0026#34;, \u0026#34;nunique\u0026#34;), # unique matches = PoM count pom_total_runs = (\u0026#34;batsman_runs\u0026#34;, \u0026#34;sum\u0026#34;), ) .reset_index() .rename(columns={\u0026#34;player_of_match\u0026#34;: \u0026#34;player\u0026#34;}) ) pom_runs[\u0026#34;avg_runs_pom_match\u0026#34;] = ( pom_runs[\u0026#34;pom_total_runs\u0026#34;] / pom_runs[\u0026#34;pom_count\u0026#34;] ).round(1) # Step 4: Overall batting average per player overall_avg = ( deliveries[deliveries[\u0026#34;wide_runs\u0026#34;] == 0] .groupby(\u0026#34;batsman\u0026#34;)[\u0026#34;batsman_runs\u0026#34;] .mean() .round(1) .reset_index() .rename(columns={\u0026#34;batsman\u0026#34;: \u0026#34;player\u0026#34;, \u0026#34;batsman_runs\u0026#34;: \u0026#34;overall_avg_runs_per_ball\u0026#34;}) ) # Step 5: Merge and compare result = pom_runs.merge(overall_avg, on=\u0026#34;player\u0026#34;, how=\u0026#34;left\u0026#34;) result = result[result[\u0026#34;pom_count\u0026#34;] \u0026gt;= 3].sort_values(\u0026#34;pom_count\u0026#34;, ascending=False) print(\u0026#34;Players with 3+ PoM awards - PoM match avg vs overall avg:\u0026#34;) print(result[[\u0026#34;player\u0026#34;, \u0026#34;pom_count\u0026#34;, \u0026#34;avg_runs_pom_match\u0026#34;, \u0026#34;overall_avg_runs_per_ball\u0026#34;]].to_string(index=False)) nunique() in groupby is the correct way to count distinct match IDs - if a player faces multiple balls in a match (which they always do), counting rows would massively overcount. nunique() on match_id gives the number of distinct matches.\nSECTION 4 - Advanced Aggregation \u0026amp; Window Analysis (Questions 14–18) Question 14 Calculate each team\u0026rsquo;s net run rate (NRR) for each season. NRR = (runs scored / overs faced) − (runs conceded / overs bowled). Concepts: Multi-groupby on two different perspectives (batting/bowling), merging two aggregations, formula application\nAnswer # Merge season into deliveries first del_s = deliveries.merge( matches[[\u0026#34;id\u0026#34;, \u0026#34;season\u0026#34;]], left_on=\u0026#34;match_id\u0026#34;, right_on=\u0026#34;id\u0026#34;, how=\u0026#34;left\u0026#34; ) # --- Runs scored and balls faced (batting perspective) --- runs_scored = ( del_s.groupby([\u0026#34;season\u0026#34;, \u0026#34;batting_team\u0026#34;]) .agg(runs_for=(\u0026#34;total_runs\u0026#34;, \u0026#34;sum\u0026#34;), balls_faced=(\u0026#34;total_runs\u0026#34;, \u0026#34;count\u0026#34;)) .reset_index() .rename(columns={\u0026#34;batting_team\u0026#34;: \u0026#34;team\u0026#34;}) ) # --- Runs conceded and balls bowled (bowling perspective) --- runs_conceded = ( del_s.groupby([\u0026#34;season\u0026#34;, \u0026#34;bowling_team\u0026#34;]) .agg(runs_against=(\u0026#34;total_runs\u0026#34;, \u0026#34;sum\u0026#34;), balls_bowled=(\u0026#34;total_runs\u0026#34;, \u0026#34;count\u0026#34;)) .reset_index() .rename(columns={\u0026#34;bowling_team\u0026#34;: \u0026#34;team\u0026#34;}) ) # --- Merge both perspectives --- nrr_df = runs_scored.merge(runs_conceded, on=[\u0026#34;season\u0026#34;, \u0026#34;team\u0026#34;]) # --- Calculate NRR --- nrr_df[\u0026#34;overs_batted\u0026#34;] = nrr_df[\u0026#34;balls_faced\u0026#34;] / 6 nrr_df[\u0026#34;overs_bowled_\u0026#34;] = nrr_df[\u0026#34;balls_bowled\u0026#34;] / 6 nrr_df[\u0026#34;run_rate_for\u0026#34;] = (nrr_df[\u0026#34;runs_for\u0026#34;] / nrr_df[\u0026#34;overs_batted\u0026#34;]).round(3) nrr_df[\u0026#34;run_rate_against\u0026#34;] = (nrr_df[\u0026#34;runs_against\u0026#34;] / nrr_df[\u0026#34;overs_bowled_\u0026#34;]).round(3) nrr_df[\u0026#34;nrr\u0026#34;] = (nrr_df[\u0026#34;run_rate_for\u0026#34;] - nrr_df[\u0026#34;run_rate_against\u0026#34;]).round(3) # Top NRR by team-season print(\u0026#34;Best NRR performances (team-season):\u0026#34;) print(nrr_df.nlargest(10, \u0026#34;nrr\u0026#34;)[[\u0026#34;season\u0026#34;, \u0026#34;team\u0026#34;, \u0026#34;run_rate_for\u0026#34;, \u0026#34;run_rate_against\u0026#34;, \u0026#34;nrr\u0026#34;]].to_string(index=False)) NRR requires two separate groupbys - one where your team is batting, one where your team is bowling - then merging on team+season. This is a classic \u0026ldquo;self-join by role\u0026rdquo; pattern in sports analytics. The same player or team appears in two different roles (batter/bowler, home/away), and you must aggregate each role separately before combining.\nQuestion 15 Use groupby().transform() to add context columns to the deliveries table: (a) total match runs for that innings, (b) each delivery\u0026rsquo;s contribution as a percentage of innings total, (c) cumulative runs in the innings up to that ball. Concepts: groupby().transform(), groupby().cumsum(), percentage contribution within group\nAnswer # (a) Total innings runs - broadcast back to every ball in that innings deliveries[\u0026#34;innings_total\u0026#34;] = ( deliveries.groupby([\u0026#34;match_id\u0026#34;, \u0026#34;inning\u0026#34;])[\u0026#34;total_runs\u0026#34;] .transform(\u0026#34;sum\u0026#34;) ) # (b) Each ball\u0026#39;s contribution as % of innings total deliveries[\u0026#34;pct_of_innings\u0026#34;] = ( deliveries[\u0026#34;total_runs\u0026#34;] / deliveries[\u0026#34;innings_total\u0026#34;] * 100 ).round(3) # (c) Cumulative runs within the innings up to and including this ball # cumsum() within a group = running total deliveries[\u0026#34;cumulative_runs\u0026#34;] = ( deliveries.groupby([\u0026#34;match_id\u0026#34;, \u0026#34;inning\u0026#34;])[\u0026#34;total_runs\u0026#34;] .cumsum() ) # Verify: last ball of each innings should equal innings_total last_ball = deliveries.groupby([\u0026#34;match_id\u0026#34;, \u0026#34;inning\u0026#34;]).last().reset_index() matches_check = (last_ball[\u0026#34;cumulative_runs\u0026#34;] == last_ball[\u0026#34;innings_total\u0026#34;]).all() print(f\u0026#34;Cumulative runs verified: {matches_check}\u0026#34;) # Show a sample innings build-up sample_innings = deliveries[ (deliveries[\u0026#34;match_id\u0026#34;] == deliveries[\u0026#34;match_id\u0026#34;].iloc[0]) \u0026amp; (deliveries[\u0026#34;inning\u0026#34;] == 1) ][[\u0026#34;over\u0026#34;, \u0026#34;ball\u0026#34;, \u0026#34;batsman\u0026#34;, \u0026#34;bowler\u0026#34;, \u0026#34;batsman_runs\u0026#34;, \u0026#34;total_runs\u0026#34;, \u0026#34;cumulative_runs\u0026#34;, \u0026#34;innings_total\u0026#34;]].head(20) print(\u0026#34;\\nSample innings ball-by-ball with cumulative runs:\u0026#34;) print(sample_innings.to_string(index=False)) groupby().cumsum() vs groupby().transform(\u0026quot;sum\u0026quot;):\ntransform(\u0026quot;sum\u0026quot;) → broadcasts the FINAL total back to every row (same value for all rows in the group) cumsum() → gives a RUNNING total that grows with each row Both use the same groupby keys. transform answers \u0026ldquo;what was the total?\u0026rdquo; and cumsum answers \u0026ldquo;how much had been scored at this point in time?\u0026rdquo; They serve completely different analytical purposes.\nQuestion 16 Identify \u0026ldquo;momentum shifts\u0026rdquo;: balls in the 2nd innings where the required run rate (RRR) dropped by more than 2 runs per over in a single over. Use pairwise differences to detect these. Concepts: groupby().transform(), diff(), required run rate calculation, derived analytics\nAnswer # Step 1: Merge target score for each match # Target = total runs scored in innings 1 + 1 innings1_totals = ( deliveries.groupby([\u0026#34;match_id\u0026#34;]) .apply(lambda g: g[g[\u0026#34;inning\u0026#34;] == 1][\u0026#34;total_runs\u0026#34;].sum()) .reset_index() .rename(columns={0: \u0026#34;innings1_total\u0026#34;}) ) innings1_totals[\u0026#34;target\u0026#34;] = innings1_totals[\u0026#34;innings1_total\u0026#34;] + 1 # Step 2: Work on innings 2 only innings2 = deliveries[deliveries[\u0026#34;inning\u0026#34;] == 2].copy() innings2 = innings2.merge(innings1_totals[[\u0026#34;match_id\u0026#34;, \u0026#34;target\u0026#34;]], on=\u0026#34;match_id\u0026#34;) # Step 3: At the END of each over, compute the required run rate innings2[\u0026#34;cumulative_runs_i2\u0026#34;] = ( innings2.groupby(\u0026#34;match_id\u0026#34;)[\u0026#34;total_runs\u0026#34;].cumsum() ) innings2[\u0026#34;runs_still_needed\u0026#34;] = innings2[\u0026#34;target\u0026#34;] - innings2[\u0026#34;cumulative_runs_i2\u0026#34;] # Count balls remaining (approximate) innings2[\u0026#34;ball_number\u0026#34;] = innings2.groupby(\u0026#34;match_id\u0026#34;).cumcount() + 1 innings2[\u0026#34;balls_remaining\u0026#34;] = 120 - innings2[\u0026#34;ball_number\u0026#34;] innings2[\u0026#34;overs_remaining\u0026#34;] = (innings2[\u0026#34;balls_remaining\u0026#34;] / 6).clip(lower=0.01) innings2[\u0026#34;rrr\u0026#34;] = ( innings2[\u0026#34;runs_still_needed\u0026#34;] / innings2[\u0026#34;overs_remaining\u0026#34;] ).round(2) # Step 4: Get end-of-over RRR and compute change between overs end_of_over_rrr = ( innings2.groupby([\u0026#34;match_id\u0026#34;, \u0026#34;over\u0026#34;])[\u0026#34;rrr\u0026#34;] .last() # last ball of each over .reset_index() ) # Diff within each match\u0026#39;s innings 2 end_of_over_rrr[\u0026#34;rrr_change\u0026#34;] = ( end_of_over_rrr.groupby(\u0026#34;match_id\u0026#34;)[\u0026#34;rrr\u0026#34;] .diff() # change from previous over ) # Big positive drops in RRR = batsmen took pressure off momentum_shifts = end_of_over_rrr[end_of_over_rrr[\u0026#34;rrr_change\u0026#34;] \u0026lt; -2] print(f\u0026#34;Overs with RRR drop \u0026gt; 2: {len(momentum_shifts)}\u0026#34;) print(momentum_shifts.nsmallest(10, \u0026#34;rrr_change\u0026#34;)[[\u0026#34;match_id\u0026#34;, \u0026#34;over\u0026#34;, \u0026#34;rrr\u0026#34;, \u0026#34;rrr_change\u0026#34;]].to_string()) groupby().diff() computes the difference between consecutive rows within a group. Combined with groupby().last() to get end-of-over snapshots, this creates a powerful pattern for detecting changes over time - wicket clusters, run-rate swings, or any \u0026ldquo;before vs after\u0026rdquo; within a grouped sequence.\nQuestion 17 For each bowler, calculate their performance in the powerplay vs death overs separately. Then rank them within each phase using groupby().rank(). Concepts: Multi-phase analysis, wide pivot from long format, groupby().rank(), phase-comparison metrics\nAnswer # Step 1: Assign phase to each delivery is_one_indexed = deliveries[\u0026#34;over\u0026#34;].min() == 1 def assign_phase(over): if is_one_indexed: if over \u0026lt;= 6: return \u0026#34;Powerplay\u0026#34; if over \u0026lt;= 15: return \u0026#34;Middle\u0026#34; return \u0026#34;Death\u0026#34; else: if over \u0026lt;= 5: return \u0026#34;Powerplay\u0026#34; if over \u0026lt;= 14: return \u0026#34;Middle\u0026#34; return \u0026#34;Death\u0026#34; deliveries[\u0026#34;phase\u0026#34;] = deliveries[\u0026#34;over\u0026#34;].apply(assign_phase) # Step 2: Bowling stats by (bowler, phase) # Legal balls only for economy calculation legal = deliveries[deliveries[\u0026#34;wide_runs\u0026#34;] == 0].copy() legal[\u0026#34;phase\u0026#34;] = legal[\u0026#34;over\u0026#34;].apply(assign_phase) # Wickets (non-run-out only) wicket_balls = legal[ legal[\u0026#34;player_dismissed\u0026#34;].notna() \u0026amp; ~legal[\u0026#34;dismissal_kind\u0026#34;].isin([\u0026#34;run out\u0026#34;, \u0026#34;retired hurt\u0026#34;]) ].copy() wicket_balls[\u0026#34;phase\u0026#34;] = wicket_balls[\u0026#34;over\u0026#34;].apply(assign_phase) wickets_by_phase = ( wicket_balls.groupby([\u0026#34;bowler\u0026#34;, \u0026#34;phase\u0026#34;])[\u0026#34;player_dismissed\u0026#34;] .count() .reset_index() .rename(columns={\u0026#34;player_dismissed\u0026#34;: \u0026#34;wickets\u0026#34;}) ) bowling_phase = ( legal.groupby([\u0026#34;bowler\u0026#34;, \u0026#34;phase\u0026#34;]) .agg( balls = (\u0026#34;total_runs\u0026#34;, \u0026#34;count\u0026#34;), runs = (\u0026#34;total_runs\u0026#34;, \u0026#34;sum\u0026#34;), ) .reset_index() ) bowling_phase = bowling_phase.merge(wickets_by_phase, on=[\u0026#34;bowler\u0026#34;, \u0026#34;phase\u0026#34;], how=\u0026#34;left\u0026#34;) bowling_phase[\u0026#34;wickets\u0026#34;] = bowling_phase[\u0026#34;wickets\u0026#34;].fillna(0).astype(int) bowling_phase[\u0026#34;overs\u0026#34;] = (bowling_phase[\u0026#34;balls\u0026#34;] / 6).round(2) bowling_phase[\u0026#34;economy\u0026#34;] = (bowling_phase[\u0026#34;runs\u0026#34;] / bowling_phase[\u0026#34;overs\u0026#34;]).round(2) # Step 3: Filter meaningful sample (20+ overs per phase) bowling_phase = bowling_phase[bowling_phase[\u0026#34;overs\u0026#34;] \u0026gt;= 20] # Step 4: Rank within each phase by economy (lower = better rank) bowling_phase[\u0026#34;economy_rank\u0026#34;] = ( bowling_phase.groupby(\u0026#34;phase\u0026#34;)[\u0026#34;economy\u0026#34;] .rank(method=\u0026#34;dense\u0026#34;, ascending=True) # lower economy = rank 1 ) # Show top 5 in each phase for phase in [\u0026#34;Powerplay\u0026#34;, \u0026#34;Middle\u0026#34;, \u0026#34;Death\u0026#34;]: top = ( bowling_phase[bowling_phase[\u0026#34;phase\u0026#34;] == phase] .nsmallest(5, \u0026#34;economy\u0026#34;) [[\u0026#34;bowler\u0026#34;, \u0026#34;overs\u0026#34;, \u0026#34;wickets\u0026#34;, \u0026#34;economy\u0026#34;, \u0026#34;economy_rank\u0026#34;]] ) print(f\u0026#34;\\nBest {phase} bowlers (by economy, min 20 overs):\u0026#34;) print(top.to_string(index=False)) rank(method='dense') ensures no gaps in the ranking sequence when there are ties - if two bowlers have the same economy, both get rank 1 and the next gets rank 2 (not rank 3). For leaderboards and \u0026ldquo;top N\u0026rdquo; selections, dense is almost always what you want.\nQuestion 18 Build a complete batsman scorecard: for the top 20 run-scorers, show all key batting metrics including average, strike rate, highest score in a single innings, number of 50s, and number of 100s. Concepts: Multiple complex aggregations, nunique() for innings count, max() for highest score, custom lambda for milestone counts, merging dismissal data\nAnswer # Step 1: Innings-level runs per batsman (runs per match-innings combination) legal_balls = deliveries[deliveries[\u0026#34;wide_runs\u0026#34;] == 0].copy() innings_runs = ( legal_balls.groupby([\u0026#34;match_id\u0026#34;, \u0026#34;inning\u0026#34;, \u0026#34;batsman\u0026#34;])[\u0026#34;batsman_runs\u0026#34;] .sum() .reset_index() .rename(columns={\u0026#34;batsman_runs\u0026#34;: \u0026#34;innings_runs\u0026#34;}) ) # Step 2: Dismissal count (how many times was this batsman out?) dismissals = ( deliveries[deliveries[\u0026#34;player_dismissed\u0026#34;].notna()] .groupby(\u0026#34;player_dismissed\u0026#34;)[\u0026#34;match_id\u0026#34;] .count() .reset_index() .rename(columns={\u0026#34;player_dismissed\u0026#34;: \u0026#34;batsman\u0026#34;, \u0026#34;match_id\u0026#34;: \u0026#34;times_out\u0026#34;}) ) # Step 3: Aggregate innings data batting_scorecard = ( innings_runs.groupby(\u0026#34;batsman\u0026#34;) .agg( innings = (\u0026#34;innings_runs\u0026#34;, \u0026#34;count\u0026#34;), total_runs = (\u0026#34;innings_runs\u0026#34;, \u0026#34;sum\u0026#34;), highest_score = (\u0026#34;innings_runs\u0026#34;, \u0026#34;max\u0026#34;), fifties = (\u0026#34;innings_runs\u0026#34;, lambda x: ((x \u0026gt;= 50) \u0026amp; (x \u0026lt; 100)).sum()), hundreds = (\u0026#34;innings_runs\u0026#34;, lambda x: (x \u0026gt;= 100).sum()), ducks = (\u0026#34;innings_runs\u0026#34;, lambda x: (x == 0).sum()), ) .reset_index() ) # Step 4: Merge dismissals for average calculation batting_scorecard = batting_scorecard.merge(dismissals, on=\u0026#34;batsman\u0026#34;, how=\u0026#34;left\u0026#34;) batting_scorecard[\u0026#34;times_out\u0026#34;] = batting_scorecard[\u0026#34;times_out\u0026#34;].fillna(0).astype(int) # Batting average = total runs / times out (not innings) batting_scorecard[\u0026#34;batting_avg\u0026#34;] = ( batting_scorecard[\u0026#34;total_runs\u0026#34;] / batting_scorecard[\u0026#34;times_out\u0026#34;].replace(0, np.nan) ).round(2) # Step 5: Ball-by-ball strike rate balls_faced_total = ( legal_balls.groupby(\u0026#34;batsman\u0026#34;)[\u0026#34;batsman_runs\u0026#34;] .count() .reset_index() .rename(columns={\u0026#34;batsman_runs\u0026#34;: \u0026#34;balls_faced\u0026#34;}) ) batting_scorecard = batting_scorecard.merge(balls_faced_total, on=\u0026#34;batsman\u0026#34;) batting_scorecard[\u0026#34;strike_rate\u0026#34;] = ( batting_scorecard[\u0026#34;total_runs\u0026#34;] / batting_scorecard[\u0026#34;balls_faced\u0026#34;] * 100 ).round(2) # Top 20 by total runs top20 = ( batting_scorecard .nlargest(20, \u0026#34;total_runs\u0026#34;) .reset_index(drop=True) ) top20.index += 1 print(\u0026#34;Complete IPL Batting Scorecard - Top 20 Run Scorers:\u0026#34;) cols = [\u0026#34;batsman\u0026#34;, \u0026#34;innings\u0026#34;, \u0026#34;total_runs\u0026#34;, \u0026#34;batting_avg\u0026#34;, \u0026#34;strike_rate\u0026#34;, \u0026#34;highest_score\u0026#34;, \u0026#34;fifties\u0026#34;, \u0026#34;hundreds\u0026#34;, \u0026#34;ducks\u0026#34;, \u0026#34;balls_faced\u0026#34;] print(top20[cols].to_string()) Batting average ≠ runs per innings. In cricket, batting average is total runs ÷ number of times dismissed. A batsman who scored 1000 runs in 20 innings with 15 dismissals has an average of 66.67, not 50. The distinction matters enormously - it penalises batsmen who get out often, and rewards those who are not out frequently. Always compute average from dismissal count, not innings count.\nSECTION 5 - Season Trends \u0026amp; Time Analysis (Questions 19–22) Question 19 How has the average first-innings score changed across IPL seasons? Is T20 becoming a more high-scoring game over time? Concepts: Filtering by innings, groupby on season, merge() to attach season to deliveries, trend analysis\nAnswer # First innings total per match first_innings = ( deliveries[deliveries[\u0026#34;inning\u0026#34;] == 1] .groupby(\u0026#34;match_id\u0026#34;)[\u0026#34;total_runs\u0026#34;] .sum() .reset_index() .rename(columns={\u0026#34;total_runs\u0026#34;: \u0026#34;first_innings_total\u0026#34;}) ) # Attach season first_innings = first_innings.merge( matches[[\u0026#34;id\u0026#34;, \u0026#34;season\u0026#34;]], left_on=\u0026#34;match_id\u0026#34;, right_on=\u0026#34;id\u0026#34; ) # Season-level stats season_scores = ( first_innings.groupby(\u0026#34;season\u0026#34;)[\u0026#34;first_innings_total\u0026#34;] .agg( matches = \u0026#34;count\u0026#34;, avg_score = \u0026#34;mean\u0026#34;, median_score = \u0026#34;median\u0026#34;, max_score = \u0026#34;max\u0026#34;, min_score = \u0026#34;min\u0026#34;, std_score = \u0026#34;std\u0026#34;, ) .round(1) .reset_index() ) print(\u0026#34;First Innings Scores by Season:\u0026#34;) print(season_scores.to_string(index=False)) # Year-over-year change season_scores[\u0026#34;avg_score_yoy_change\u0026#34;] = season_scores[\u0026#34;avg_score\u0026#34;].diff().round(1) print(\u0026#34;\\nYear-over-year change in average first innings score:\u0026#34;) print(season_scores[[\u0026#34;season\u0026#34;, \u0026#34;avg_score\u0026#34;, \u0026#34;avg_score_yoy_change\u0026#34;]].to_string(index=False)) # Rolling 3-season average (smooths out single-season anomalies) season_scores[\u0026#34;rolling_3yr_avg\u0026#34;] = ( season_scores[\u0026#34;avg_score\u0026#34;].rolling(window=3, min_periods=1).mean().round(1) ) print(\u0026#34;\\nRolling 3-season average score:\u0026#34;) print(season_scores[[\u0026#34;season\u0026#34;, \u0026#34;avg_score\u0026#34;, \u0026#34;rolling_3yr_avg\u0026#34;]].to_string(index=False)) rolling(window=3).mean() computes a moving average - the average of the current season and the two preceding ones. This smooths out noise from single anomalous seasons (like 2009\u0026rsquo;s different pitches in South Africa). Rolling averages are the standard technique for revealing trends in time series data.\nQuestion 20 Which venues are \u0026ldquo;batsman-friendly\u0026rdquo; vs \u0026ldquo;bowler-friendly\u0026rdquo;? Define this as: venues where the average first-innings score is above/below the overall median. Minimum 20 matches at the venue. Concepts: Global median as a reference point, transform() to broadcast global stats, categorisation with np.where()\nAnswer # First innings totals per match with venue first_innings_venue = ( deliveries[deliveries[\u0026#34;inning\u0026#34;] == 1] .groupby(\u0026#34;match_id\u0026#34;)[\u0026#34;total_runs\u0026#34;] .sum() .reset_index() .rename(columns={\u0026#34;total_runs\u0026#34;: \u0026#34;first_innings_total\u0026#34;}) .merge(matches[[\u0026#34;id\u0026#34;, \u0026#34;venue\u0026#34;, \u0026#34;season\u0026#34;]], left_on=\u0026#34;match_id\u0026#34;, right_on=\u0026#34;id\u0026#34;) ) # Venue-level stats venue_scores = ( first_innings_venue.groupby(\u0026#34;venue\u0026#34;) .agg( matches = (\u0026#34;first_innings_total\u0026#34;, \u0026#34;count\u0026#34;), avg_score = (\u0026#34;first_innings_total\u0026#34;, \u0026#34;mean\u0026#34;), median_score = (\u0026#34;first_innings_total\u0026#34;, \u0026#34;median\u0026#34;), highest_score = (\u0026#34;first_innings_total\u0026#34;, \u0026#34;max\u0026#34;), ) .round(1) .reset_index() ) # Filter minimum 20 matches venue_scores = venue_scores[venue_scores[\u0026#34;matches\u0026#34;] \u0026gt;= 20] # Global median across all venues global_median = first_innings_venue[\u0026#34;first_innings_total\u0026#34;].median() print(f\u0026#34;Global median first-innings score: {global_median:.1f}\u0026#34;) # Classify venues venue_scores[\u0026#34;venue_type\u0026#34;] = np.where( venue_scores[\u0026#34;avg_score\u0026#34;] \u0026gt; global_median, \u0026#34;Batsman-Friendly\u0026#34;, \u0026#34;Bowler-Friendly\u0026#34; ) # Sort by avg score venue_scores = venue_scores.sort_values(\u0026#34;avg_score\u0026#34;, ascending=False) print(\u0026#34;\\nTop batting venues:\u0026#34;) print(venue_scores[venue_scores[\u0026#34;venue_type\u0026#34;] == \u0026#34;Batsman-Friendly\u0026#34;] .head(8)[[\u0026#34;venue\u0026#34;, \u0026#34;matches\u0026#34;, \u0026#34;avg_score\u0026#34;, \u0026#34;highest_score\u0026#34;]].to_string(index=False)) print(\u0026#34;\\nTop bowling venues:\u0026#34;) print(venue_scores[venue_scores[\u0026#34;venue_type\u0026#34;] == \u0026#34;Bowler-Friendly\u0026#34;] .tail(8)[[\u0026#34;venue\u0026#34;, \u0026#34;matches\u0026#34;, \u0026#34;avg_score\u0026#34;, \u0026#34;highest_score\u0026#34;]].to_string(index=False)) np.where(condition, value_if_true, value_if_false) is the vectorized ternary operator - far faster than apply(lambda row: ...) for creating conditional columns. Use it whenever your new column depends on a single boolean condition applied element-wise.\nQuestion 21 Track each team\u0026rsquo;s cumulative wins across the seasons. Use groupby().cumsum() on the wins data to build a \u0026ldquo;running title tally\u0026rdquo; chart. Concepts: groupby().cumsum(), multi-level sort, time-ordered cumulative aggregation\nAnswer # Step 1: One row per match result per team (was this team in this match and did they win?) match_outcomes = pd.concat([ matches[[\u0026#34;season\u0026#34;, \u0026#34;team1\u0026#34;, \u0026#34;winner\u0026#34;]].rename(columns={\u0026#34;team1\u0026#34;: \u0026#34;team\u0026#34;}), matches[[\u0026#34;season\u0026#34;, \u0026#34;team2\u0026#34;, \u0026#34;winner\u0026#34;]].rename(columns={\u0026#34;team2\u0026#34;: \u0026#34;team\u0026#34;}), ], ignore_index=True) match_outcomes[\u0026#34;won\u0026#34;] = (match_outcomes[\u0026#34;team\u0026#34;] == match_outcomes[\u0026#34;winner\u0026#34;]).astype(int) # Step 2: Sort by season (essential before cumsum) match_outcomes = match_outcomes.sort_values(\u0026#34;season\u0026#34;) # Step 3: Cumulative wins per team across seasons season_wins = ( match_outcomes.groupby([\u0026#34;team\u0026#34;, \u0026#34;season\u0026#34;])[\u0026#34;won\u0026#34;] .sum() .reset_index() .rename(columns={\u0026#34;won\u0026#34;: \u0026#34;wins_this_season\u0026#34;}) .sort_values([\u0026#34;team\u0026#34;, \u0026#34;season\u0026#34;]) ) # Cumulative wins: running total across seasons for each team season_wins[\u0026#34;cumulative_wins\u0026#34;] = ( season_wins.groupby(\u0026#34;team\u0026#34;)[\u0026#34;wins_this_season\u0026#34;] .cumsum() ) # Show for major teams major_teams = [\u0026#34;Mumbai Indians\u0026#34;, \u0026#34;Chennai Super Kings\u0026#34;, \u0026#34;Kolkata Knight Riders\u0026#34;, \u0026#34;Royal Challengers Bangalore\u0026#34;] print(\u0026#34;Cumulative wins across seasons:\u0026#34;) print( season_wins[season_wins[\u0026#34;team\u0026#34;].isin(major_teams)] .pivot(index=\u0026#34;season\u0026#34;, columns=\u0026#34;team\u0026#34;, values=\u0026#34;cumulative_wins\u0026#34;) .fillna(method=\u0026#34;ffill\u0026#34;) # fill missing seasons with last value .to_string() ) Order matters for cumsum()! You must sort by the time dimension (season) BEFORE applying cumsum() within a group. If the data is in random order, the running total will be computed in the wrong sequence. Always sort_values() before any cumulative operation.\nQuestion 22 Build the ultimate all-in-one team performance summary: for every team across all seasons, show win rate, average runs scored, average runs conceded, NRR, most common captain (player who won most tosses for that team), and their best season. Concepts: Full pipeline combining matches and deliveries, multiple groupby perspectives, lambda mode, idxmax() for best season\nAnswer # --- Win rate --- all_appearances = pd.concat([ matches[[\u0026#34;season\u0026#34;, \u0026#34;team1\u0026#34;]].rename(columns={\u0026#34;team1\u0026#34;: \u0026#34;team\u0026#34;}), matches[[\u0026#34;season\u0026#34;, \u0026#34;team2\u0026#34;]].rename(columns={\u0026#34;team2\u0026#34;: \u0026#34;team\u0026#34;}), ]) total_played_team = all_appearances.groupby(\u0026#34;team\u0026#34;).size().rename(\u0026#34;total_played\u0026#34;) total_wins_team = matches[\u0026#34;winner\u0026#34;].value_counts().rename(\u0026#34;total_wins\u0026#34;) # --- Average runs scored and conceded (from deliveries merged with season) --- del_s = deliveries.merge(matches[[\u0026#34;id\u0026#34;, \u0026#34;season\u0026#34;]], left_on=\u0026#34;match_id\u0026#34;, right_on=\u0026#34;id\u0026#34;) avg_runs_scored = ( del_s.groupby(\u0026#34;batting_team\u0026#34;)[\u0026#34;total_runs\u0026#34;] .mean() .mul(6) # per-ball average × 6 ≈ per-over average .round(2) .rename(\u0026#34;avg_run_rate_scored\u0026#34;) ) avg_runs_conceded = ( del_s.groupby(\u0026#34;bowling_team\u0026#34;)[\u0026#34;total_runs\u0026#34;] .mean() .mul(6) .round(2) .rename(\u0026#34;avg_run_rate_conceded\u0026#34;) ) # --- Most common toss-winner (proxy for captain) --- toss_freq = ( matches.groupby(\u0026#34;toss_winner\u0026#34;)[\u0026#34;player_of_match\u0026#34;] .count() # just to count toss wins per team ) # Actually: captain = player who won most tosses FOR their team # Approximation using toss_winner = team and player_of_match of those matches # (Best proxy we have without captain data) # --- Best season by win rate --- season_wins_df = ( match_outcomes.groupby([\u0026#34;team\u0026#34;, \u0026#34;season\u0026#34;]) .agg( played = (\u0026#34;won\u0026#34;, \u0026#34;count\u0026#34;), wins = (\u0026#34;won\u0026#34;, \u0026#34;sum\u0026#34;), ) .reset_index() ) season_wins_df[\u0026#34;season_win_rate\u0026#34;] = season_wins_df[\u0026#34;wins\u0026#34;] / season_wins_df[\u0026#34;played\u0026#34;] best_season_idx = season_wins_df.groupby(\u0026#34;team\u0026#34;)[\u0026#34;season_win_rate\u0026#34;].idxmax() best_seasons = season_wins_df.loc[best_season_idx][[\u0026#34;team\u0026#34;, \u0026#34;season\u0026#34;, \u0026#34;season_win_rate\u0026#34;]] best_seasons = best_seasons.rename(columns={\u0026#34;season\u0026#34;: \u0026#34;best_season\u0026#34;, \u0026#34;season_win_rate\u0026#34;: \u0026#34;best_season_win_rate\u0026#34;}) # --- Assemble final table --- team_summary = pd.DataFrame({ \u0026#34;total_played\u0026#34; : total_played_team, \u0026#34;total_wins\u0026#34; : total_wins_team, \u0026#34;avg_run_rate_scored\u0026#34; : avg_runs_scored, \u0026#34;avg_run_rate_conceded\u0026#34; : avg_runs_conceded, }) team_summary[\u0026#34;win_rate\u0026#34;] = (team_summary[\u0026#34;total_wins\u0026#34;] / team_summary[\u0026#34;total_played\u0026#34;] * 100).round(1) team_summary[\u0026#34;nrr\u0026#34;] = (team_summary[\u0026#34;avg_run_rate_scored\u0026#34;] - team_summary[\u0026#34;avg_run_rate_conceded\u0026#34;]).round(3) team_summary = team_summary.reset_index().rename(columns={\u0026#34;index\u0026#34;: \u0026#34;team\u0026#34;}) team_summary = team_summary.merge(best_seasons, on=\u0026#34;team\u0026#34;, how=\u0026#34;left\u0026#34;) team_summary = team_summary.sort_values(\u0026#34;win_rate\u0026#34;, ascending=False) print(\u0026#34;Ultimate Team Performance Summary:\u0026#34;) print(team_summary[[\u0026#34;team\u0026#34;, \u0026#34;total_played\u0026#34;, \u0026#34;total_wins\u0026#34;, \u0026#34;win_rate\u0026#34;, \u0026#34;nrr\u0026#34;, \u0026#34;best_season\u0026#34;, \u0026#34;best_season_win_rate\u0026#34;]].to_string(index=False)) SECTION 6 - Advanced Patterns \u0026amp; Practice Problems (Questions 23–27) Question 23 Identify \u0026ldquo;impact players\u0026rdquo; - batsmen whose runs in the first 6 overs (powerplay) have the highest correlation with their team\u0026rsquo;s match outcome. Use groupby() and merge() to build the dataset, then compute correlations. Concepts: Multi-level groupby, merging aggregated data with match results, corr(), defining an analytical metric from scratch\nAnswer # Step 1: Powerplay runs per batsman per match is_one_indexed = deliveries[\u0026#34;over\u0026#34;].min() == 1 pp_end = 6 if is_one_indexed else 5 powerplay_batting = ( deliveries[ (deliveries[\u0026#34;over\u0026#34;] \u0026lt;= pp_end) \u0026amp; (deliveries[\u0026#34;inning\u0026#34;] == 1) \u0026amp; (deliveries[\u0026#34;wide_runs\u0026#34;] == 0) ] .groupby([\u0026#34;match_id\u0026#34;, \u0026#34;batsman\u0026#34;])[\u0026#34;batsman_runs\u0026#34;] .sum() .reset_index() .rename(columns={\u0026#34;batsman_runs\u0026#34;: \u0026#34;pp_runs\u0026#34;}) ) # Step 2: Attach match outcome - did batting team win? powerplay_batting = powerplay_batting.merge( deliveries[[\u0026#34;match_id\u0026#34;, \u0026#34;batting_team\u0026#34;]].drop_duplicates(subset=[\u0026#34;match_id\u0026#34;, \u0026#34;batting_team\u0026#34;]), on=\u0026#34;match_id\u0026#34; ) powerplay_batting = powerplay_batting.merge( matches[[\u0026#34;id\u0026#34;, \u0026#34;winner\u0026#34;]], left_on=\u0026#34;match_id\u0026#34;, right_on=\u0026#34;id\u0026#34; ) powerplay_batting[\u0026#34;team_won\u0026#34;] = ( powerplay_batting[\u0026#34;batting_team\u0026#34;] == powerplay_batting[\u0026#34;winner\u0026#34;] ).astype(int) # Step 3: For each batsman, compute correlation between pp_runs and win min_pp_appearances = 20 batsman_pp_corr = ( powerplay_batting.groupby(\u0026#34;batsman\u0026#34;) .filter(lambda g: len(g) \u0026gt;= min_pp_appearances) .groupby(\u0026#34;batsman\u0026#34;) .apply(lambda g: g[\u0026#34;pp_runs\u0026#34;].corr(g[\u0026#34;team_won\u0026#34;])) .reset_index() .rename(columns={0: \u0026#34;pp_win_correlation\u0026#34;}) .sort_values(\u0026#34;pp_win_correlation\u0026#34;, ascending=False) ) print(\u0026#34;Batsmen with highest powerplay-run / team-win correlation:\u0026#34;) print(batsman_pp_corr.head(10).to_string(index=False)) corr() within apply() computes the Pearson correlation between two columns within a group. This is a powerful pattern for \u0026ldquo;does metric X predict outcome Y, and does this relationship differ by player/team/venue?\u0026rdquo; The filter() before apply() ensures only players with enough data are included, preventing spurious correlations from small samples.\nQuestion 24 Find partnerships: pairs of batsmen (batsman + non_striker) who have scored the most partnership runs together across all IPL matches. Concepts: Groupby on two columns simultaneously, creating a canonical pair key with sorted() + apply(), sum() on partnership runs\nAnswer # Step 1: Create a canonical partnership key (alphabetical order) # So (Rohit, Tendulkar) and (Tendulkar, Rohit) are treated as the same partnership deliveries[\u0026#34;partnership_key\u0026#34;] = deliveries.apply( lambda r: \u0026#34; \u0026amp; \u0026#34;.join(sorted([str(r[\u0026#34;batsman\u0026#34;]), str(r[\u0026#34;non_striker\u0026#34;])])), axis=1 ) # Step 2: Partnership runs per match per key # Use batsman_runs (not total_runs) as extras aren\u0026#39;t partnership runs partnership_runs = ( deliveries[deliveries[\u0026#34;wide_runs\u0026#34;] == 0] .groupby([\u0026#34;match_id\u0026#34;, \u0026#34;inning\u0026#34;, \u0026#34;partnership_key\u0026#34;])[\u0026#34;batsman_runs\u0026#34;] .sum() .reset_index() .rename(columns={\u0026#34;batsman_runs\u0026#34;: \u0026#34;partnership_contribution\u0026#34;}) ) # Step 3: Total partnership runs across all instances all_time_partnerships = ( partnership_runs.groupby(\u0026#34;partnership_key\u0026#34;) .agg( total_runs = (\u0026#34;partnership_contribution\u0026#34;, \u0026#34;sum\u0026#34;), partnership_innings = (\u0026#34;partnership_contribution\u0026#34;, \u0026#34;count\u0026#34;), avg_runs = (\u0026#34;partnership_contribution\u0026#34;, \u0026#34;mean\u0026#34;), best_stand = (\u0026#34;partnership_contribution\u0026#34;, \u0026#34;max\u0026#34;), ) .round(1) .reset_index() .sort_values(\u0026#34;total_runs\u0026#34;, ascending=False) ) print(\u0026#34;Top 20 All-Time IPL Partnerships by Total Runs:\u0026#34;) print(all_time_partnerships.head(20).to_string(index=False)) Canonical key creation (sorted() + join()) solves the ordering problem: the batsman on strike and the non-striker switch roles between balls, but they\u0026rsquo;re in the same partnership. By sorting the two names alphabetically and joining them, we create a consistent key regardless of who was on strike. This pattern - creating a canonical representation to group bidirectional relationships - is essential for pair/edge analysis in any domain (user pairs, transaction pairs, etc.).\nQuestion 25 Use pd.crosstab to build a dismissal-type matrix: for each bowler (top 15 by wickets), show how many wickets they took via each dismissal type (caught, bowled, lbw, etc.). Concepts: pd.crosstab(), multi-level column analysis, filtering to top N, fillna()\nAnswer # Get wickets only (no run-outs) wicket_deliveries = deliveries[ deliveries[\u0026#34;player_dismissed\u0026#34;].notna() \u0026amp; ~deliveries[\u0026#34;dismissal_kind\u0026#34;].isin([\u0026#34;run out\u0026#34;, \u0026#34;retired hurt\u0026#34;, \u0026#34;obstructing the field\u0026#34;]) ].copy() # Top 15 bowlers by wicket count top15_bowlers = ( wicket_deliveries[\u0026#34;bowler\u0026#34;].value_counts().head(15).index.tolist() ) # Filter to top 15 bowlers top15_wickets = wicket_deliveries[wicket_deliveries[\u0026#34;bowler\u0026#34;].isin(top15_bowlers)] # Crosstab: rows = bowler, columns = dismissal_kind dismissal_matrix = pd.crosstab( index = top15_wickets[\u0026#34;bowler\u0026#34;], columns = top15_wickets[\u0026#34;dismissal_kind\u0026#34;], margins = True, margins_name = \u0026#34;Total\u0026#34; ).fillna(0).astype(int) # Sort rows by total wickets descending dismissal_matrix = dismissal_matrix.sort_values(\u0026#34;Total\u0026#34;, ascending=False) print(\u0026#34;Dismissal Type Matrix - Top 15 IPL Wicket-Takers:\u0026#34;) print(dismissal_matrix) # Normalize: what % of each bowler\u0026#39;s wickets came via each type? dismissal_pct = pd.crosstab( index = top15_wickets[\u0026#34;bowler\u0026#34;], columns = top15_wickets[\u0026#34;dismissal_kind\u0026#34;], normalize = \u0026#34;index\u0026#34; ).round(3).mul(100).round(1) print(\u0026#34;\\nDismissal type % per bowler:\u0026#34;) print(dismissal_pct.sort_values(\u0026#34;caught\u0026#34;, ascending=False)) crosstab with normalize='index' gives the row-wise percentage - for each bowler, what percentage of their wickets were caught vs bowled vs lbw. This reveals each bowler\u0026rsquo;s \u0026ldquo;style\u0026rdquo; - a spinner who takes 70% of wickets caught vs a fast bowler who takes 40% bowled.\nQuestion 26 Calculate the \u0026ldquo;Pressure Index\u0026rdquo; for each bowler in death overs: economy rate × (1 - wicket rate). Lower is better - it means they\u0026rsquo;re economical AND taking wickets. Concepts: Derived multi-metric scoring formula, groupby() with multiple agg functions, applying a custom scoring formula\nAnswer is_one_indexed = deliveries[\u0026#34;over\u0026#34;].min() == 1 death_start = 16 if is_one_indexed else 15 death_bowling = deliveries[deliveries[\u0026#34;over\u0026#34;] \u0026gt;= death_start].copy() # Wickets in death (not run-outs) death_bowling[\u0026#34;is_wicket\u0026#34;] = ( death_bowling[\u0026#34;player_dismissed\u0026#34;].notna() \u0026amp; ~death_bowling[\u0026#34;dismissal_kind\u0026#34;].isin([\u0026#34;run out\u0026#34;, \u0026#34;retired hurt\u0026#34;]) ).astype(int) # Legal balls for economy death_legal = death_bowling[death_bowling[\u0026#34;wide_runs\u0026#34;] == 0] death_bowler_stats = ( death_legal.groupby(\u0026#34;bowler\u0026#34;) .agg( balls = (\u0026#34;total_runs\u0026#34;, \u0026#34;count\u0026#34;), runs = (\u0026#34;total_runs\u0026#34;, \u0026#34;sum\u0026#34;), wickets = (\u0026#34;is_wicket\u0026#34;, \u0026#34;sum\u0026#34;), ) .reset_index() ) death_bowler_stats[\u0026#34;overs\u0026#34;] = (death_bowler_stats[\u0026#34;balls\u0026#34;] / 6).round(2) death_bowler_stats[\u0026#34;economy\u0026#34;] = (death_bowler_stats[\u0026#34;runs\u0026#34;] / death_bowler_stats[\u0026#34;overs\u0026#34;]).round(2) death_bowler_stats[\u0026#34;wicket_rate\u0026#34;] = (death_bowler_stats[\u0026#34;wickets\u0026#34;] / death_bowler_stats[\u0026#34;balls\u0026#34;]).round(4) # Pressure Index = economy × (1 - wicket_rate) # Lower = better: economical AND takes wickets death_bowler_stats[\u0026#34;pressure_index\u0026#34;] = ( death_bowler_stats[\u0026#34;economy\u0026#34;] * (1 - death_bowler_stats[\u0026#34;wicket_rate\u0026#34;]) ).round(3) # Filter: minimum 10 overs in death qualified = death_bowler_stats[death_bowler_stats[\u0026#34;overs\u0026#34;] \u0026gt;= 10] print(\u0026#34;Death Over Bowling - Pressure Index (lower is better, min 10 overs):\u0026#34;) print( qualified.nsmallest(15, \u0026#34;pressure_index\u0026#34;) [[\u0026#34;bowler\u0026#34;, \u0026#34;overs\u0026#34;, \u0026#34;wickets\u0026#34;, \u0026#34;economy\u0026#34;, \u0026#34;wicket_rate\u0026#34;, \u0026#34;pressure_index\u0026#34;]] .reset_index(drop=True) .to_string(index=False) ) Composite scoring in sports analytics always involves a tradeoff - a single metric is never sufficient. Economy rate doesn\u0026rsquo;t account for wickets; wicket rate doesn\u0026rsquo;t account for runs. The Pressure Index combines both: a bowler who goes at 12 per over but takes wickets is worse than one who goes at 9 but takes none. The formula economy × (1 - wicket_rate) rewards both dimensions simultaneously.\nQuestion 27 Build the final investor/selector report: if you were picking a team for a high-stakes match, rank the top all-rounders using a composite score based on batting (runs + SR) and bowling (wickets + economy). Concepts: Multi-table pipeline, normalising different metrics to a common scale, weighted composite scoring, merge() across three aggregated datasets\nAnswer # --- Batting score (legal balls) --- legal = deliveries[deliveries[\u0026#34;wide_runs\u0026#34;] == 0] batting = ( legal.groupby(\u0026#34;batsman\u0026#34;) .agg( bat_runs = (\u0026#34;batsman_runs\u0026#34;, \u0026#34;sum\u0026#34;), balls_bat = (\u0026#34;batsman_runs\u0026#34;, \u0026#34;count\u0026#34;), ) .reset_index() ) batting[\u0026#34;bat_sr\u0026#34;] = (batting[\u0026#34;bat_runs\u0026#34;] / batting[\u0026#34;balls_bat\u0026#34;] * 100).round(2) batting = batting[batting[\u0026#34;balls_bat\u0026#34;] \u0026gt;= 200] # minimum 200 balls faced # --- Bowling score --- wicket_balls = deliveries[ deliveries[\u0026#34;player_dismissed\u0026#34;].notna() \u0026amp; ~deliveries[\u0026#34;dismissal_kind\u0026#34;].isin([\u0026#34;run out\u0026#34;, \u0026#34;retired hurt\u0026#34;]) ] wickets_count = ( wicket_balls.groupby(\u0026#34;bowler\u0026#34;)[\u0026#34;player_dismissed\u0026#34;].count() .reset_index() .rename(columns={\u0026#34;bowler\u0026#34;: \u0026#34;player\u0026#34;, \u0026#34;player_dismissed\u0026#34;: \u0026#34;wickets\u0026#34;}) ) bowling = ( legal.groupby(\u0026#34;bowler\u0026#34;) .agg( balls_bow = (\u0026#34;total_runs\u0026#34;, \u0026#34;count\u0026#34;), runs_bow = (\u0026#34;total_runs\u0026#34;, \u0026#34;sum\u0026#34;), ) .reset_index() .rename(columns={\u0026#34;bowler\u0026#34;: \u0026#34;player\u0026#34;}) ) bowling[\u0026#34;overs\u0026#34;] = (bowling[\u0026#34;balls_bow\u0026#34;] / 6).round(2) bowling[\u0026#34;economy\u0026#34;] = (bowling[\u0026#34;runs_bow\u0026#34;] / bowling[\u0026#34;overs\u0026#34;]).round(2) bowling = bowling[bowling[\u0026#34;overs\u0026#34;] \u0026gt;= 30] # minimum 30 overs bowled bowling = bowling.merge(wickets_count, on=\u0026#34;player\u0026#34;, how=\u0026#34;left\u0026#34;) bowling[\u0026#34;wickets\u0026#34;] = bowling[\u0026#34;wickets\u0026#34;].fillna(0).astype(int) # --- Merge batting and bowling --- batting = batting.rename(columns={\u0026#34;batsman\u0026#34;: \u0026#34;player\u0026#34;}) allrounders = batting.merge(bowling, on=\u0026#34;player\u0026#34;, how=\u0026#34;inner\u0026#34;) # inner join: only players who have BOTH batting AND bowling records print(f\u0026#34;Players with both batting and bowling records: {len(allrounders)}\u0026#34;) # --- Normalise each metric to 0–1 --- def minmax(s): rng = s.max() - s.min() return (s - s.min()) / rng if rng \u0026gt; 0 else pd.Series(0, index=s.index) allrounders[\u0026#34;norm_runs\u0026#34;] = minmax(allrounders[\u0026#34;bat_runs\u0026#34;]) allrounders[\u0026#34;norm_sr\u0026#34;] = minmax(allrounders[\u0026#34;bat_sr\u0026#34;]) allrounders[\u0026#34;norm_wickets\u0026#34;] = minmax(allrounders[\u0026#34;wickets\u0026#34;]) # For economy: LOWER is BETTER - so invert allrounders[\u0026#34;norm_economy\u0026#34;] = 1 - minmax(allrounders[\u0026#34;economy\u0026#34;]) # --- Composite score (weights reflect importance) --- allrounders[\u0026#34;allrounder_score\u0026#34;] = ( allrounders[\u0026#34;norm_runs\u0026#34;] * 0.30 + allrounders[\u0026#34;norm_sr\u0026#34;] * 0.20 + allrounders[\u0026#34;norm_wickets\u0026#34;] * 0.30 + allrounders[\u0026#34;norm_economy\u0026#34;] * 0.20 ).round(4) # Top 20 all-rounders top_allrounders = ( allrounders.nlargest(20, \u0026#34;allrounder_score\u0026#34;) [[\u0026#34;player\u0026#34;, \u0026#34;bat_runs\u0026#34;, \u0026#34;bat_sr\u0026#34;, \u0026#34;wickets\u0026#34;, \u0026#34;economy\u0026#34;, \u0026#34;allrounder_score\u0026#34;]] .reset_index(drop=True) ) top_allrounders.index += 1 print(\u0026#34;\\nTop 20 IPL All-Rounders - Composite Score:\u0026#34;) print(top_allrounders.to_string()) Concepts Coverage Checklist BASICS ✅ shape, dtypes, info(), describe() on two tables ✅ value_counts(), sort_index() for time-ordered counts ✅ isnull().sum() - missing value audit ✅ nunique() - unique value counts SINGLE-TABLE AGGREGATION ✅ groupby().size() ✅ groupby().count() vs groupby().size() ✅ groupby().mean(), .sum(), .median(), .max(), .min() ✅ Named multi-metric aggregations: agg(col=(\u0026#34;src\u0026#34;, \u0026#34;fn\u0026#34;)) ✅ Lambda inside agg() - e.g., (x == 4).sum(), mode() ✅ nlargest(), nsmallest() ✅ value_counts(normalize=True) - proportions FILTERING ✅ Boolean filtering with \u0026amp; and | ✅ ~isin() - exclusion filter ✅ notna() / isna() - null filters ✅ Multi-condition filtering with complex logic ✅ .between() - range filter MERGING (THE CORE SKILL) ✅ pd.merge() - left join vs inner join ✅ merge() on different key names (left_on / right_on) ✅ Bringing only needed columns from right table ✅ Verifying row counts before and after merge ✅ Multi-step multi-table pipeline RESHAPING ✅ pd.concat([col1, col2]) - stacking columns into a series ✅ pd.crosstab() - frequency tables with margins ✅ pd.crosstab(normalize=\u0026#39;index\u0026#39;/\u0026#39;columns\u0026#39;/True) ✅ pivot() - long to wide format ✅ unstack() - multi-level index to columns ✅ pd.pivot_table() - flexible pivot with aggfunc TRANSFORMS ✅ groupby().transform(\u0026#34;sum\u0026#34;) - broadcast group total to rows ✅ groupby().transform(\u0026#34;mean\u0026#34;) - broadcast group mean to rows ✅ groupby().cumsum() - running total within groups ✅ groupby().diff() - change from previous row within group ✅ groupby().rank() - rank within group (dense, pct) ✅ groupby().transform(lambda) - custom within-group computation ADVANCED ✅ pd.cut() - custom phase bins (powerplay/middle/death) ✅ rolling(window=n).mean() - rolling average over time ✅ apply(axis=1) - row-level logic using multiple columns ✅ groupby().idxmax() - index of max value per group ✅ groupby().apply() with nlargest() - top N per group ✅ corr() within groupby - correlation by group ✅ np.where() - vectorized conditional column creation ✅ sorted() + join() - canonical pair keys ✅ Composite scoring with min-max normalisation ✅ Multi-perspective merging (batting + bowling perspectives) ✅ Wicket filtering with ~isin() for proper credit attribution ✅ Wide-ball filtering for accurate balls faced / overs ✅ Domain-specific formulas (NRR, economy, batting avg, SR) End of Practice Sheet - IPL Dataset Codeverra - codeverra.com\n","permalink":"https://blogs.codeverra.com/blog/ipl_pandas/","summary":"\u003ch1 id=\"pandas-practice-sheet-----ipl-dataset\"\u003ePandas Practice Sheet  -  IPL Dataset\u003c/h1\u003e\n\u003ch3 id=\"a-complete-two-table-exploration-from-match-summaries-to-ball-by-ball-intelligence\"\u003eA complete two-table exploration: from match summaries to ball-by-ball intelligence\u003c/h3\u003e\n\u003chr\u003e\n\u003ch2 id=\"why-ipl-is-the-best-dataset-for-pandas-after-zomato\"\u003eWhy IPL Is the Best Dataset for Pandas After Zomato\u003c/h2\u003e\n\u003cp\u003eThe Zomato dataset taught you how to clean a single messy table. The IPL dataset teaches you something more important: \u003cstrong\u003ehow to work with two related tables and merge them to answer questions neither table can answer alone.\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThis is how data exists in every real company  -  not as one giant flat file, but as a set of related tables that you must join, aggregate across levels, and reason about carefully.\u003c/p\u003e","tags":["python","data-analysis","pandas"],"title":"Practice Pandas on IPL Dataset"},{"content":"Pandas Practice Sheet - Titanic Dataset 20 Questions from Zero to Advanced, with Full Solutions Getting the Dataset The Titanic dataset is hosted publicly and can be loaded directly into pandas - no download needed.\nDirect CSV URL (no login required):\nhttps://raw.githubusercontent.com/datasciencedojo/datasets/master/titanic.csv Load it in one line:\nimport pandas as pd df = pd.read_csv(\u0026#34;https://raw.githubusercontent.com/datasciencedojo/datasets/master/titanic.csv\u0026#34;) Or load via seaborn (also free, slightly different column names):\nimport seaborn as sns df = sns.load_dataset(\u0026#34;titanic\u0026#34;) Always run df.head() and df.info() before answering any question - knowing your data is half the battle.\nColumn Dictionary Before writing a single line of pandas, understand what every column means.\n┌─────────────────┬──────────┬──────────────────────────────────────────────────────┐ │ Column │ Type │ Meaning │ ├─────────────────┼──────────┼──────────────────────────────────────────────────────┤ │ PassengerId │ int │ Unique ID for each passenger (1–891) │ │ Survived │ int │ 0 = Did not survive, 1 = Survived │ │ Pclass │ int │ Ticket class: 1 = First, 2 = Second, 3 = Third │ │ Name │ string │ Full name including title (Mr., Mrs., Miss., etc.) │ │ Sex │ string │ \u0026#34;male\u0026#34; or \u0026#34;female\u0026#34; │ │ Age │ float │ Age in years. ~177 values are missing (NaN) │ │ SibSp │ int │ # of siblings + spouses aboard │ │ Parch │ int │ # of parents + children aboard │ │ Ticket │ string │ Ticket number (mostly irrelevant for analysis) │ │ Fare │ float │ Ticket price in British pounds │ │ Cabin │ string │ Cabin number. ~687 values are missing │ │ Embarked │ string │ Port of embarkation: S=Southampton, C=Cherbourg, │ │ │ │ Q=Queenstown. 2 values missing. │ └─────────────────┴──────────┴──────────────────────────────────────────────────────┘ Key facts to keep in your head:\n891 rows total (training set - not all passengers) Survived is the target column (1 = survived) Age and Cabin have significant missing values Pclass is stored as an integer but is a categorical variable Setup Block Run this at the top of every notebook. We will reference df throughout.\nimport pandas as pd import numpy as np df = pd.read_csv( \u0026#34;https://raw.githubusercontent.com/datasciencedojo/datasets/master/titanic.csv\u0026#34; ) # Quick sanity check print(df.shape) # (891, 12) print(df.dtypes) print(df.isnull().sum()) LEVEL 1 - Basics (Questions 1–5) Core selection, filtering, and summary statistics Question 1 How many passengers are in the dataset, and what does the data look like? Concepts: shape, head(), tail(), info(), describe()\nThis is always Question 0 in any real data project. Never skip it.\nAnswer # --- Shape: rows and columns --- print(df.shape) # (891, 12) → 891 passengers, 12 columns # --- First 5 rows --- print(df.head()) # --- Last 5 rows --- print(df.tail()) # --- Data types and non-null counts --- df.info() # Tells you: dtype per column, how many non-null values exist # Key insight: Age has 714/891 non-null → 177 missing # Cabin has only 204/891 non-null → 687 missing! # --- Statistical summary for numeric columns --- print(df.describe()) # count, mean, std, min, 25%, 50%, 75%, max for each numeric column # --- Include all columns (including object/string types) --- print(df.describe(include=\u0026#34;all\u0026#34;)) # Adds: unique, top (most frequent), freq (frequency of top) for string columns What to read from describe():\nAge mean ≈ 29.7 but median (50%) ≈ 28 → slight right skew Fare max is 512 but 75th percentile is only 31 → massive outliers Survived mean ≈ 0.38 → only 38% of passengers survived Question 2 How many passengers survived, and what is the overall survival rate? Concepts: value_counts(), value_counts(normalize=True), boolean filtering, mean()\nAnswer Method 1 - value_counts() for raw counts\nprint(df[\u0026#34;Survived\u0026#34;].value_counts()) # 0 549 (did not survive) # 1 342 (survived) Method 2 - value_counts(normalize=True) for proportions\nprint(df[\u0026#34;Survived\u0026#34;].value_counts(normalize=True).round(3)) # 0 0.616 # 1 0.384 # Interpretation: 38.4% survived, 61.6% did not Method 3 - mean() on a binary column (cleanest for survival rate)\nsurvival_rate = df[\u0026#34;Survived\u0026#34;].mean() print(f\u0026#34;Survival rate: {survival_rate:.1%}\u0026#34;) # Survival rate: 38.4% Why does mean() work here? Because Survived is 0 or 1. The average of a binary column is identical to the proportion of 1s. This trick is idiomatic pandas and much cleaner than dividing counts manually.\nMethod 4 - Boolean filter to count survivors directly\nsurvivors = df[df[\u0026#34;Survived\u0026#34;] == 1] non_survivors = df[df[\u0026#34;Survived\u0026#34;] == 0] print(f\u0026#34;Survived: {len(survivors)}\u0026#34;) # 342 print(f\u0026#34;Did not: {len(non_survivors)}\u0026#34;) # 549 print(f\u0026#34;Total: {len(df)}\u0026#34;) # 891 Tradeoff: value_counts() is great for a quick count of each category. mean() is the one-liner for a rate. For larger workflows, groupby (coming later) scales better.\nQuestion 3 What are the minimum, maximum, and average age of passengers? How many passengers have a missing age? Concepts: min(), max(), mean(), median(), isnull(), sum(), dropna()\nAnswer # Basic statistics on Age print(f\u0026#34;Min age: {df[\u0026#39;Age\u0026#39;].min()}\u0026#34;) # 0.42 (infant) print(f\u0026#34;Max age: {df[\u0026#39;Age\u0026#39;].max()}\u0026#34;) # 80.0 print(f\u0026#34;Mean age: {df[\u0026#39;Age\u0026#39;].mean():.2f}\u0026#34;) # 29.70 print(f\u0026#34;Median age: {df[\u0026#39;Age\u0026#39;].median():.2f}\u0026#34;) # 28.00 # Missing age count missing_age = df[\u0026#34;Age\u0026#34;].isnull().sum() total = len(df) print(f\u0026#34;Missing age: {missing_age} out of {total} ({missing_age/total:.1%})\u0026#34;) # Missing age: 177 out of 891 (19.9%) Method 2 - describe() gives everything at once\nprint(df[\u0026#34;Age\u0026#34;].describe()) # count 714.000000 ← non-null count # mean 29.699118 # std 14.526497 # min 0.420000 # 25% 20.125000 # 50% 28.000000 # 75% 38.000000 # max 80.000000 Method 3 - agg() for custom summary in one call\nprint(df[\u0026#34;Age\u0026#34;].agg([\u0026#34;min\u0026#34;, \u0026#34;max\u0026#34;, \u0026#34;mean\u0026#34;, \u0026#34;median\u0026#34;, \u0026#34;count\u0026#34;])) Mean vs Median for Age: Mean is 29.7, median is 28.0. They\u0026rsquo;re close, meaning the age distribution is fairly symmetric. But when a column has outliers (like Fare), mean can be misleading - always check both.\nTradeoff on missing values: min(), max(), mean() all skip NaN by default (skipna=True). This is convenient but means your statistics are computed on only 714 of 891 passengers. Always note this when reporting results.\nQuestion 4 Select only the Name, Sex, Age, and Survived columns for passengers who are female. Concepts: Column selection, boolean filtering, loc[], iloc[]\nAnswer Method 1 - Boolean filter then column selection (most readable)\nfemale_passengers = df[df[\u0026#34;Sex\u0026#34;] == \u0026#34;female\u0026#34;][[\u0026#34;Name\u0026#34;, \u0026#34;Sex\u0026#34;, \u0026#34;Age\u0026#34;, \u0026#34;Survived\u0026#34;]] print(female_passengers.head(10)) Method 2 - loc[] in one step (preferred - reads naturally as \u0026ldquo;rows, columns\u0026rdquo;)\nfemale_passengers = df.loc[ df[\u0026#34;Sex\u0026#34;] == \u0026#34;female\u0026#34;, # row condition [\u0026#34;Name\u0026#34;, \u0026#34;Sex\u0026#34;, \u0026#34;Age\u0026#34;, \u0026#34;Survived\u0026#34;] # columns to select ] print(female_passengers.shape) # (314, 4) Method 3 - query() for a SQL-like feel\nfemale_passengers = df.query(\u0026#39;Sex == \u0026#34;female\u0026#34;\u0026#39;)[[\u0026#34;Name\u0026#34;, \u0026#34;Sex\u0026#34;, \u0026#34;Age\u0026#34;, \u0026#34;Survived\u0026#34;]] loc vs iloc:\nloc[row_label, col_label] - selects by label (column name, index value) iloc[row_integer, col_integer] - selects by position (0-based integers) Always prefer loc when filtering by condition. Use iloc when you need a specific row by position (e.g., df.iloc[0] for the first row).\nTradeoff - chained indexing pitfall: df[df[\u0026quot;Sex\u0026quot;] == \u0026quot;female\u0026quot;][\u0026quot;Survived\u0026quot;] = 1 → This can silently fail with a SettingWithCopyWarning! Always use loc when modifying values: df.loc[df[\u0026quot;Sex\u0026quot;] == \u0026quot;female\u0026quot;, \u0026quot;Survived\u0026quot;] = 1 → This is safe.\nQuestion 5 How many passengers are in each passenger class (1st, 2nd, 3rd)? Concepts: value_counts(), groupby().size(), groupby().count()\nAnswer Method 1 - value_counts() (fastest for a single column)\nprint(df[\u0026#34;Pclass\u0026#34;].value_counts().sort_index()) # 1 216 # 2 184 # 3 491 Method 2 - groupby().size()\nprint(df.groupby(\u0026#34;Pclass\u0026#34;).size()) # Pclass # 1 216 # 2 184 # 3 491 Method 3 - groupby().count()\n# .count() counts non-null values per column, not rows # Different from .size() when columns have missing values! print(df.groupby(\u0026#34;Pclass\u0026#34;)[\u0026#34;PassengerId\u0026#34;].count()) size() vs count() - the critical difference:\nsize() → counts ALL rows in the group, including those with NaN count() → counts only NON-NULL values per column For counting rows in a group, always use size(). If you use count() on Age, you\u0026rsquo;ll get 714 not 891 because of missing ages. Use count() only when you want \u0026ldquo;how many valid values exist in this column for this group.\u0026rdquo;\n# Proof of the difference print(df.groupby(\u0026#34;Pclass\u0026#34;).size()) # always 216, 184, 491 print(df.groupby(\u0026#34;Pclass\u0026#34;)[\u0026#34;Age\u0026#34;].count()) # only non-null ages per class # Pclass # 1 186 # 2 173 # 3 355 LEVEL 2 - Filtering \u0026amp; Grouping (Questions 6–10) Multi-condition filters, groupby aggregations, multiple aggregations Question 6 What is the survival rate for male vs female passengers? Concepts: groupby(), agg(), mean(), multiple columns in groupby\nAnswer Method 1 - groupby().mean() on the Survived column\nsurvival_by_sex = df.groupby(\u0026#34;Sex\u0026#34;)[\u0026#34;Survived\u0026#34;].mean() print(survival_by_sex) # Sex # female 0.742038 # male 0.188908 Method 2 - groupby().agg() to get both count and rate together\nsurvival_by_sex = df.groupby(\u0026#34;Sex\u0026#34;)[\u0026#34;Survived\u0026#34;].agg( total_passengers=\u0026#34;count\u0026#34;, survivors=\u0026#34;sum\u0026#34;, survival_rate=\u0026#34;mean\u0026#34; ) print(survival_by_sex.round(3)) # total_passengers survivors survival_rate # female 314 233 0.742 # male 577 109 0.189 Method 3 - value_counts() crosstab style\nprint(df.groupby([\u0026#34;Sex\u0026#34;, \u0026#34;Survived\u0026#34;]).size().unstack()) # Survived 0 1 # Sex # female 81 233 # male 468 109 Insight: Female passengers had a 74.2% survival rate vs only 18.9% for males - a massive difference reflecting the \u0026ldquo;women and children first\u0026rdquo; evacuation policy.\nNote on agg() syntax: The agg(new_col_name=\u0026quot;original_agg_func\u0026quot;) named aggregation syntax was introduced in pandas 0.25. It\u0026rsquo;s the cleanest way to rename columns during aggregation. Before that, people used .rename() after the fact.\nQuestion 7 What is the average fare paid by passengers in each class? Concepts: groupby(), agg() with multiple functions, round(), sorting\nAnswer Method 1 - Simple groupby().mean()\navg_fare = df.groupby(\u0026#34;Pclass\u0026#34;)[\u0026#34;Fare\u0026#34;].mean().round(2) print(avg_fare) # Pclass # 1 84.15 # 2 20.66 # 3 13.68 Method 2 - Multiple aggregations using agg()\nfare_stats = df.groupby(\u0026#34;Pclass\u0026#34;)[\u0026#34;Fare\u0026#34;].agg( avg_fare=\u0026#34;mean\u0026#34;, median_fare=\u0026#34;median\u0026#34;, min_fare=\u0026#34;min\u0026#34;, max_fare=\u0026#34;max\u0026#34;, passengers=\u0026#34;count\u0026#34; ).round(2) print(fare_stats) # avg_fare median_fare min_fare max_fare passengers # Pclass # 1 84.15 60.29 0.00 512.33 216 # 2 20.66 14.25 0.00 73.50 184 # 3 13.68 8.05 0.00 69.55 491 Mean vs Median for Fare: In Class 1, the mean is 84.15 but the median is only 60.29. That gap tells you there are some very expensive tickets pulling the mean up (max = 512!). The median is more representative here. Always compare both when dealing with potentially skewed distributions like prices or incomes.\nMethod 3 - agg() with a list of functions (older but still common)\nfare_stats = df.groupby(\u0026#34;Pclass\u0026#34;)[\u0026#34;Fare\u0026#34;].agg([\u0026#34;mean\u0026#34;, \u0026#34;median\u0026#34;, \u0026#34;min\u0026#34;, \u0026#34;max\u0026#34;]) # Column names will be \u0026#39;mean\u0026#39;, \u0026#39;median\u0026#39;, \u0026#39;min\u0026#39;, \u0026#39;max\u0026#39; - use rename() to relabel Tradeoff - named agg vs list agg:\nNamed: agg(avg=\u0026quot;mean\u0026quot;) → clean column names immediately, self-documenting List: agg([\u0026quot;mean\u0026quot;, \u0026quot;median\u0026quot;]) → simpler syntax but ugly default column names Prefer named aggregations in any code you\u0026rsquo;ll share or read later. Question 8 How many passengers survived AND did not survive, broken down by both Sex and Pclass? Concepts: Multi-column groupby(), unstack(), crosstab()\nAnswer Method 1 - groupby() on two columns + size() + unstack()\ncounts = df.groupby([\u0026#34;Pclass\u0026#34;, \u0026#34;Sex\u0026#34;, \u0026#34;Survived\u0026#34;]).size().unstack(fill_value=0) print(counts) # Survived 0 1 # Pclass Sex # 1 female 3 91 # male 77 45 # 2 female 6 70 # male 91 17 # 3 female 72 72 # male 300 47 Method 2 - pd.crosstab() - the cleanest for cross-tabulations\nct = pd.crosstab( index=[df[\u0026#34;Pclass\u0026#34;], df[\u0026#34;Sex\u0026#34;]], columns=df[\u0026#34;Survived\u0026#34;], margins=True, # adds row/column totals margins_name=\u0026#34;Total\u0026#34; ) ct.columns = [\u0026#34;Did Not Survive\u0026#34;, \u0026#34;Survived\u0026#34;, \u0026#34;Total\u0026#34;] print(ct) Method 3 - pd.crosstab() with normalize for proportions\n# Row-wise proportions (survival rate per Pclass-Sex group) ct_pct = pd.crosstab( index=[df[\u0026#34;Pclass\u0026#34;], df[\u0026#34;Sex\u0026#34;]], columns=df[\u0026#34;Survived\u0026#34;], normalize=\u0026#34;index\u0026#34; # normalize across rows (sum of each row = 1) ).round(3) ct_pct.columns = [\u0026#34;survival_rate_0\u0026#34;, \u0026#34;survival_rate_1\u0026#34;] print(ct_pct) # survival_rate_0 survival_rate_1 # Pclass Sex # 1 female 0.032 0.968 ← 96.8% of 1st class women survived! # male 0.631 0.369 # 2 female 0.079 0.921 # male 0.843 0.157 # 3 female 0.500 0.500 # male 0.864 0.136 groupby vs crosstab:\ngroupby is more flexible and works naturally in data pipelines crosstab is more ergonomic for 2D frequency tables with labels crosstab supports normalize directly - cleaner than doing groupby().size() / groupby().size().sum() manually For exploratory analysis, reach for crosstab. For production pipelines and chaining, use groupby.\nQuestion 9 Find all passengers who were over 60 years old OR paid a fare over £100. How many are there, and how many survived? Concepts: Multiple boolean conditions with | and \u0026amp;, query(), sum(), mean()\nAnswer Method 1 - Boolean operators | (OR) and \u0026amp; (AND)\n# IMPORTANT: Each condition must be wrapped in parentheses! mask = (df[\u0026#34;Age\u0026#34;] \u0026gt; 60) | (df[\u0026#34;Fare\u0026#34;] \u0026gt; 100) subset = df[mask] print(f\u0026#34;Passengers matching criteria: {len(subset)}\u0026#34;) print(f\u0026#34;Survivors among them: {subset[\u0026#39;Survived\u0026#39;].sum()}\u0026#34;) print(f\u0026#34;Survival rate: {subset[\u0026#39;Survived\u0026#39;].mean():.1%}\u0026#34;) Method 2 - query() - more readable for complex conditions\nsubset = df.query(\u0026#34;Age \u0026gt; 60 or Fare \u0026gt; 100\u0026#34;) print(len(subset)) Query handles NaN differently: query(\u0026quot;Age \u0026gt; 60\u0026quot;) silently drops rows where Age is NaN from the result - the same as boolean indexing. But the string syntax is cleaner for complex multi-condition filters.\nAdding an AND condition for comparison:\n# Passengers who are BOTH old AND paid high fare high_value = df[(df[\u0026#34;Age\u0026#34;] \u0026gt; 60) \u0026amp; (df[\u0026#34;Fare\u0026#34;] \u0026gt; 100)] print(f\u0026#34;Old AND expensive passengers: {len(high_value)}\u0026#34;) # Very common mistake - using Python\u0026#39;s `and`/`or` instead of `\u0026amp;`/`|` # df[(df[\u0026#34;Age\u0026#34;] \u0026gt; 60) and (df[\u0026#34;Fare\u0026#34;] \u0026gt; 100)] ← ValueError: ambiguous truth value! # Always use \u0026amp; and | for element-wise comparison on Series Tradeoff - |/\u0026amp; vs query():\n|/\u0026amp; is faster on very large DataFrames and supports variables directly query() is cleaner to read and write, especially for 3+ conditions query() has limitations with column names containing spaces (use backticks: df.query(\u0026quot;column name \u0026gt; 5\u0026quot;) → df.query(\u0026quot;`column name` \u0026gt; 5\u0026quot;)) Question 10 What is the survival rate for each combination of Pclass and Sex? Present it as a clean pivot table. Concepts: pivot_table(), aggfunc, margins\nAnswer Method 1 - pd.pivot_table() - the purpose-built tool\npivot = pd.pivot_table( df, values=\u0026#34;Survived\u0026#34;, # the column to aggregate index=\u0026#34;Pclass\u0026#34;, # rows columns=\u0026#34;Sex\u0026#34;, # columns aggfunc=\u0026#34;mean\u0026#34;, # how to aggregate margins=True, # add row/column totals margins_name=\u0026#34;Overall\u0026#34; ).round(3) print(pivot) # Sex female male Overall # Pclass # 1 0.968 0.369 0.630 # 2 0.921 0.157 0.473 # 3 0.500 0.136 0.242 # Overall 0.742 0.189 0.384 Method 2 - groupby().unstack() gives the same result\npivot2 = df.groupby([\u0026#34;Pclass\u0026#34;, \u0026#34;Sex\u0026#34;])[\u0026#34;Survived\u0026#34;].mean().unstack().round(3) print(pivot2) # Sex female male # Pclass # 1 0.968 0.369 # 2 0.921 0.157 # 3 0.500 0.136 Method 3 - Multiple aggregation functions in pivot_table()\npivot_multi = pd.pivot_table( df, values=\u0026#34;Survived\u0026#34;, index=\u0026#34;Pclass\u0026#34;, columns=\u0026#34;Sex\u0026#34;, aggfunc=[\u0026#34;mean\u0026#34;, \u0026#34;count\u0026#34;] # survival rate AND passenger count ).round(3) print(pivot_multi) pivot_table vs groupby().unstack():\npivot_table is more explicit, supports margins, fill_value, and multiple aggfunc directly groupby().unstack() is more composable - easier to chain with other operations If you\u0026rsquo;re building a display table, use pivot_table. If you\u0026rsquo;re continuing a transformation pipeline, use groupby().unstack() What the pivot table reveals: Class 1 females had a 96.8% survival rate. Class 3 males had only a 13.6% survival rate. The \u0026ldquo;Pclass\u0026rdquo; effect is dramatic and exists independently within both sexes.\nLEVEL 3 - Aggregation Depth (Questions 11–15) Multiple aggregations, transforms, and derived columns Question 11 For each passenger class, compute: total passengers, number of survivors, survival rate, average age, and average fare - all in one query. Concepts: Named multi-column agg(), groupby() on multiple metrics\nAnswer class_summary = df.groupby(\u0026#34;Pclass\u0026#34;).agg( total_passengers = (\u0026#34;PassengerId\u0026#34;, \u0026#34;count\u0026#34;), survivors = (\u0026#34;Survived\u0026#34;, \u0026#34;sum\u0026#34;), survival_rate = (\u0026#34;Survived\u0026#34;, \u0026#34;mean\u0026#34;), avg_age = (\u0026#34;Age\u0026#34;, \u0026#34;mean\u0026#34;), avg_fare = (\u0026#34;Fare\u0026#34;, \u0026#34;mean\u0026#34;), median_fare = (\u0026#34;Fare\u0026#34;, \u0026#34;median\u0026#34;), ).round(2) print(class_summary) # total_passengers survivors survival_rate avg_age avg_fare median_fare # Pclass # 1 216 136 0.63 38.23 84.15 60.29 # 2 184 87 0.47 29.88 20.66 14.25 # 3 491 119 0.24 25.14 13.68 8.05 Named aggregation syntax - the right way: new_column_name = (\u0026quot;source_column\u0026quot;, \u0026quot;aggregation_function\u0026quot;)\nThis was introduced in pandas 0.25.0 and is now the recommended approach. It produces clean column names without needing .rename() afterwards.\nBonus - adding a percentage column after the fact:\nclass_summary[\u0026#34;pct_of_total\u0026#34;] = ( class_summary[\u0026#34;total_passengers\u0026#34;] / class_summary[\u0026#34;total_passengers\u0026#34;].sum() * 100 ).round(1) print(class_summary[\u0026#34;pct_of_total\u0026#34;]) # Pclass # 1 24.2% # 2 20.6% # 3 55.1% Insight: Class 3 passengers made up 55% of all passengers but had only a 24% survival rate. Class 1 had 24% of passengers and a 63% survival rate. This directly reflects which decks lifeboats were accessible from.\nQuestion 12 Create a new column called FamilySize (total family members aboard including the passenger). Then show the survival rate for each family size. Concepts: Column creation, arithmetic on columns, groupby() on derived column\nAnswer # FamilySize = SibSp (siblings/spouses) + Parch (parents/children) + 1 (self) df[\u0026#34;FamilySize\u0026#34;] = df[\u0026#34;SibSp\u0026#34;] + df[\u0026#34;Parch\u0026#34;] + 1 # Survival rate by family size family_survival = df.groupby(\u0026#34;FamilySize\u0026#34;)[\u0026#34;Survived\u0026#34;].agg( passengers = \u0026#34;count\u0026#34;, survival_rate = \u0026#34;mean\u0026#34; ).round(3) print(family_survival) # passengers survival_rate # FamilySize # 1 537 0.304 ← traveling alone: 30.4% survival # 2 161 0.553 # 3 102 0.578 # 4 29 0.724 ← family of 4: best survival rate! # 5 15 0.200 # 6 22 0.136 # 7 12 0.333 # 8 6 0.000 ← all large families died # 11 7 0.000 Bonus - create a categorical IsAlone column:\ndf[\u0026#34;IsAlone\u0026#34;] = (df[\u0026#34;FamilySize\u0026#34;] == 1).astype(int) print(df.groupby(\u0026#34;IsAlone\u0026#34;)[\u0026#34;Survived\u0026#34;].mean().round(3)) # IsAlone # 0 0.506 (traveling with family: 50.6% survival) # 1 0.304 (traveling alone: 30.4% survival) Column creation is fundamental pandas. New columns from arithmetic operations are vectorized - they operate on the entire column at once, not row by row. Never use a for loop to create a new column. Use df[\u0026quot;new\u0026quot;] = df[\u0026quot;a\u0026quot;] + df[\u0026quot;b\u0026quot;] instead.\nInsight: Traveling alone was actually the worst scenario. Small families (2–4) had the best survival rates. Very large families (7+) had 0% survival - possibly because they couldn\u0026rsquo;t all get on the same lifeboat.\nQuestion 13 Add a column AgeBand that bins passengers into age groups: Child (0–12), Teen (13–17), Adult (18–60), Senior (60+). What is the survival rate per age band? Concepts: pd.cut(), custom bins and labels, handling NaN, groupby() on categorical\nAnswer # Method 1 - pd.cut() with explicit bins and labels df[\u0026#34;AgeBand\u0026#34;] = pd.cut( df[\u0026#34;Age\u0026#34;], bins = [0, 12, 17, 60, 100], labels = [\u0026#34;Child\u0026#34;, \u0026#34;Teen\u0026#34;, \u0026#34;Adult\u0026#34;, \u0026#34;Senior\u0026#34;], right = True # intervals are (left, right] - i.e., right-inclusive ) # Survival rate per age band age_survival = df.groupby(\u0026#34;AgeBand\u0026#34;, observed=True)[\u0026#34;Survived\u0026#34;].agg( passengers = \u0026#34;count\u0026#34;, survivors = \u0026#34;sum\u0026#34;, survival_rate = \u0026#34;mean\u0026#34; ).round(3) print(age_survival) # passengers survivors survival_rate # AgeBand # Child 69 40 0.580 ← children evacuated first # Teen 55 27 0.491 # Adult 566 215 0.380 # Senior 24 11 0.458 pd.cut() vs pd.qcut():\npd.cut() → you define the bin boundaries (e.g., 0–12, 13–17\u0026hellip;) pd.qcut() → you define the number of quantiles (e.g., 4 equal-sized groups) Use cut when the boundaries are meaningful (age groups, score bands). Use qcut when you want equal-sized groups (quartiles, deciles).\n# Method 2 - pd.qcut() for equal-frequency bins df[\u0026#34;AgeBand_q\u0026#34;] = pd.qcut(df[\u0026#34;Age\u0026#34;], q=4, labels=[\u0026#34;Q1\u0026#34;, \u0026#34;Q2\u0026#34;, \u0026#34;Q3\u0026#34;, \u0026#34;Q4\u0026#34;]) print(df.groupby(\u0026#34;AgeBand_q\u0026#34;, observed=True)[\u0026#34;Survived\u0026#34;].mean().round(3)) Watch out for observed=True: In newer pandas (2.0+), groupby on Categorical columns produces a FutureWarning if observed is not specified. Pass observed=True to only show categories that actually appear in the data.\nHandling passengers with missing Age:\n# NaN Age → NaN AgeBand (they won\u0026#39;t appear in groupby - they\u0026#39;re silently excluded) print(df[\u0026#34;AgeBand\u0026#34;].isnull().sum()) # 177 - same count as missing Age Question 14 Add a column showing each passenger\u0026rsquo;s fare relative to the average fare in their class - i.e., how much more or less they paid vs their class average. Concepts: groupby().transform() - the key concept that separates intermediate from advanced pandas users\nAnswer # transform() broadcasts the group aggregate back to the original DataFrame index # The result has the SAME shape as the original DataFrame - that\u0026#39;s what makes it special # Step 1: compute class-level average fare using transform df[\u0026#34;ClassAvgFare\u0026#34;] = df.groupby(\u0026#34;Pclass\u0026#34;)[\u0026#34;Fare\u0026#34;].transform(\u0026#34;mean\u0026#34;) # Step 2: compute how far each passenger\u0026#39;s fare is from their class average df[\u0026#34;FareDiff\u0026#34;] = df[\u0026#34;Fare\u0026#34;] - df[\u0026#34;ClassAvgFare\u0026#34;] # Step 3: verify it worked print(df[[\u0026#34;Pclass\u0026#34;, \u0026#34;Fare\u0026#34;, \u0026#34;ClassAvgFare\u0026#34;, \u0026#34;FareDiff\u0026#34;]].head(10).round(2)) # Pclass Fare ClassAvgFare FareDiff # 0 3 7.25 13.68 -6.43 ← paid 6.43 less than Class 3 avg # 1 1 71.28 84.15 -12.87 # 2 3 7.93 13.68 -5.75 # 3 1 53.10 84.15 -31.05 # ... The transform() pattern explained:\n# Without transform - result has one row per group (can\u0026#39;t assign back to df) df.groupby(\u0026#34;Pclass\u0026#34;)[\u0026#34;Fare\u0026#34;].mean() # Pclass # 1 84.15 ← only 3 rows # 2 20.66 # 3 13.68 # With transform - result has the same shape as the original (891 rows) df.groupby(\u0026#34;Pclass\u0026#34;)[\u0026#34;Fare\u0026#34;].transform(\u0026#34;mean\u0026#34;) # 0 13.68 ← row 0 is Pclass 3 → gets 13.68 # 1 84.15 ← row 1 is Pclass 1 → gets 84.15 # 2 13.68 # 3 84.15 # ... (891 rows total) Real-world use cases for transform():\n# Normalize scores within groups (z-score) df[\u0026#34;FareZScore\u0026#34;] = df.groupby(\u0026#34;Pclass\u0026#34;)[\u0026#34;Fare\u0026#34;].transform( lambda x: (x - x.mean()) / x.std() ) # Fill NaN age with the median age of that passenger\u0026#39;s class df[\u0026#34;AgeFilled\u0026#34;] = df[\u0026#34;Age\u0026#34;].fillna( df.groupby(\u0026#34;Pclass\u0026#34;)[\u0026#34;Age\u0026#34;].transform(\u0026#34;median\u0026#34;) ) # Cumulative count within a group df[\u0026#34;RankInClass\u0026#34;] = df.groupby(\u0026#34;Pclass\u0026#34;)[\u0026#34;Fare\u0026#34;].transform(\u0026#34;rank\u0026#34;, ascending=False) transform() vs apply() vs agg():\nOperation Output shape Use when\u0026hellip; agg() One row per group You want a summary table transform() Same shape as input You want to add a column back to the original df apply() Flexible Complex per-group logic that doesn\u0026rsquo;t fit agg/transform transform() is the right tool for creating new columns that depend on group-level statistics.\nQuestion 15 Extract the passenger\u0026rsquo;s title (Mr., Mrs., Miss., etc.) from the Name column and find the survival rate per title. Concepts: str.extract(), regex, string operations on Series, groupby() on extracted column\nAnswer # Every name follows the pattern: \u0026#34;Last, Title. First Middle\u0026#34; # We want to extract the title using regex # Step 1: Extract title df[\u0026#34;Title\u0026#34;] = df[\u0026#34;Name\u0026#34;].str.extract(r\u0026#34;,\\s*([A-Za-z]+)\\.\u0026#34;) # Regex: comma, optional spaces, then capture word characters, then a period print(df[\u0026#34;Title\u0026#34;].value_counts()) # Mr 517 # Miss 182 # Mrs 125 # Master 40 # Dr 7 # Rev 6 # Mlle 2 # ... (rare titles) # Step 2: Consolidate rare titles title_mapping = { \u0026#34;Mlle\u0026#34;: \u0026#34;Miss\u0026#34;, \u0026#34;Ms\u0026#34;: \u0026#34;Miss\u0026#34;, \u0026#34;Mme\u0026#34;: \u0026#34;Mrs\u0026#34;, \u0026#34;Capt\u0026#34;: \u0026#34;Rare\u0026#34;, \u0026#34;Col\u0026#34;: \u0026#34;Rare\u0026#34;, \u0026#34;Major\u0026#34;: \u0026#34;Rare\u0026#34;, \u0026#34;Jonkheer\u0026#34;: \u0026#34;Rare\u0026#34;, \u0026#34;Don\u0026#34;: \u0026#34;Rare\u0026#34;, \u0026#34;Sir\u0026#34;: \u0026#34;Rare\u0026#34;, \u0026#34;Countess\u0026#34;: \u0026#34;Rare\u0026#34;, \u0026#34;Lady\u0026#34;: \u0026#34;Rare\u0026#34;, \u0026#34;Dr\u0026#34;: \u0026#34;Rare\u0026#34;, \u0026#34;Rev\u0026#34;: \u0026#34;Rare\u0026#34;, } df[\u0026#34;Title\u0026#34;] = df[\u0026#34;Title\u0026#34;].replace(title_mapping) # Step 3: Survival rate per title title_survival = df.groupby(\u0026#34;Title\u0026#34;)[\u0026#34;Survived\u0026#34;].agg( count = \u0026#34;count\u0026#34;, survival_rate = \u0026#34;mean\u0026#34; ).round(3).sort_values(\u0026#34;survival_rate\u0026#34;, ascending=False) print(title_survival) # count survival_rate # Mrs 125 0.792 # Miss 184 0.702 # Master 40 0.575 # Rare 23 0.347 # Mr 517 0.157 Why extract titles? This is a classic feature engineering technique. The raw Name column looks useless, but the title inside it encodes gender, social class, and marital status simultaneously - all correlated with survival. str.extract() with a regex pattern is how you unlock that information.\nstr.extract() vs str.split() vs str.findall():\nstr.extract(pattern) → returns the first captured group as a new Series/DataFrame str.findall(pattern) → returns a list of all matches str.split(pat) → splits on a delimiter; good for simple cases Use str.extract() when the pattern is structured (like a title surrounded by specific characters).\nLEVEL 4 - Advanced (Questions 16–20) Window functions, complex transforms, multi-level operations, real-world workflows Question 16 Rank passengers within each class by fare paid (highest fare = rank 1). Show the top 3 fare-payers in each class. Concepts: groupby().rank(), groupby().apply(), nlargest()\nAnswer Method 1 - groupby().rank() + filter\n# rank() within each Pclass group df[\u0026#34;FareRankInClass\u0026#34;] = df.groupby(\u0026#34;Pclass\u0026#34;)[\u0026#34;Fare\u0026#34;].rank( method = \u0026#34;dense\u0026#34;, # ties get the same rank (no gaps) ascending = False # highest fare = rank 1 ) # Top 3 fare-payers per class top3 = df[df[\u0026#34;FareRankInClass\u0026#34;] \u0026lt;= 3].sort_values([\u0026#34;Pclass\u0026#34;, \u0026#34;FareRankInClass\u0026#34;]) print(top3[[\u0026#34;Pclass\u0026#34;, \u0026#34;Name\u0026#34;, \u0026#34;Fare\u0026#34;, \u0026#34;FareRankInClass\u0026#34;, \u0026#34;Survived\u0026#34;]].to_string()) Method 2 - groupby().apply() with nlargest()\ntop3_v2 = ( df.groupby(\u0026#34;Pclass\u0026#34;)[[\u0026#34;Name\u0026#34;, \u0026#34;Fare\u0026#34;, \u0026#34;Survived\u0026#34;]] .apply(lambda g: g.nlargest(3, \u0026#34;Fare\u0026#34;)) .reset_index(drop=True) ) print(top3_v2) Method 3 - sort_values + groupby().head()\ntop3_v3 = ( df.sort_values(\u0026#34;Fare\u0026#34;, ascending=False) .groupby(\u0026#34;Pclass\u0026#34;) .head(3) [[\u0026#34;Pclass\u0026#34;, \u0026#34;Name\u0026#34;, \u0026#34;Fare\u0026#34;, \u0026#34;Survived\u0026#34;]] .sort_values([\u0026#34;Pclass\u0026#34;, \u0026#34;Fare\u0026#34;], ascending=[True, False]) ) print(top3_v3) rank() method options:\n'average' (default) → tied items share the average of their ranks 'min' → tied items all get the lowest rank 'max' → tied items all get the highest rank 'dense' → like 'min' but no gaps in rank numbers (1, 2, 2, 3 not 1, 2, 2, 4) 'first' → ranked in order of appearance (no ties) For leaderboards and \u0026ldquo;top N\u0026rdquo; queries, 'dense' is usually what you want.\nQuestion 17 What percentage of passengers in each class survived? Additionally, what percentage of ALL survivors came from each class? Concepts: groupby(), percentage within groups vs percentage of total, crosstab(normalize=...)\nThese are two different questions that look similar but produce completely different numbers.\nAnswer # --- Question A: Survival RATE by class (what % of each class survived?) --- survival_rate_by_class = df.groupby(\u0026#34;Pclass\u0026#34;)[\u0026#34;Survived\u0026#34;].mean().round(3) print(\u0026#34;Survival rate by class:\u0026#34;) print(survival_rate_by_class) # Pclass # 1 0.630 ← 63% of Class 1 passengers survived # 2 0.473 # 3 0.242 # --- Question B: Composition of survivors (what % of survivors were from each class?) --- survivor_composition = ( df[df[\u0026#34;Survived\u0026#34;] == 1] .groupby(\u0026#34;Pclass\u0026#34;) .size() / df[\u0026#34;Survived\u0026#34;].sum() * 100 ).round(1) print(\u0026#34;\\nOf all survivors, % from each class:\u0026#34;) print(survivor_composition) # Pclass # 1 39.8 ← 39.8% of survivors were Class 1 (even though Class 1 = 24% of passengers) # 2 25.4 # 3 34.8 # --- Both together using crosstab with normalize --- print(\u0026#34;\\n--- Rate within class (normalize=\u0026#39;index\u0026#39;) ---\u0026#34;) print(pd.crosstab(df[\u0026#34;Pclass\u0026#34;], df[\u0026#34;Survived\u0026#34;], normalize=\u0026#34;index\u0026#34;).round(3)) # 0 1 # Pclass # 1 0.370 0.630 # 2 0.527 0.473 # 3 0.758 0.242 print(\u0026#34;\\n--- Share of total (normalize=\u0026#39;columns\u0026#39;) ---\u0026#34;) print(pd.crosstab(df[\u0026#34;Pclass\u0026#34;], df[\u0026#34;Survived\u0026#34;], normalize=\u0026#34;columns\u0026#34;).round(3)) # 0 1 # Pclass # 1 0.140 0.398 # 2 0.166 0.254 # 3 0.694 0.348 This is a critical conceptual distinction in data analysis:\n\u0026ldquo;Survival rate by class\u0026rdquo; → normalize='index' → conditioned on class (row-wise) \u0026ldquo;Share of survivors from each class\u0026rdquo; → normalize='columns' → conditioned on survival outcome (column-wise) normalize=True → as % of the entire dataset Confusing these two is one of the most common analytical errors in practice.\nQuestion 18 Find passengers who paid MORE than the median fare for their class. Add a column PaidAboveMedian (True/False). Concepts: groupby().transform() with a custom lambda, boolean column creation\nAnswer # Step 1: Use transform to broadcast the class-level median fare to each row df[\u0026#34;ClassMedianFare\u0026#34;] = df.groupby(\u0026#34;Pclass\u0026#34;)[\u0026#34;Fare\u0026#34;].transform(\u0026#34;median\u0026#34;) # Step 2: Compare each passenger\u0026#39;s fare to their class median df[\u0026#34;PaidAboveMedian\u0026#34;] = df[\u0026#34;Fare\u0026#34;] \u0026gt; df[\u0026#34;ClassMedianFare\u0026#34;] # Verify print(df[[\u0026#34;Pclass\u0026#34;, \u0026#34;Fare\u0026#34;, \u0026#34;ClassMedianFare\u0026#34;, \u0026#34;PaidAboveMedian\u0026#34;]].head(10).round(2)) # Breakdown: how many passengers paid above median in each class? print(df.groupby([\u0026#34;Pclass\u0026#34;, \u0026#34;PaidAboveMedian\u0026#34;]).size()) # Does paying above the class median correlate with better survival? print( df.groupby(\u0026#34;PaidAboveMedian\u0026#34;)[\u0026#34;Survived\u0026#34;].mean().round(3) ) # PaidAboveMedian # False 0.340 # True 0.437 ← paid more → slightly better survival Method 2 - one-liner using transform with lambda\ndf[\u0026#34;PaidAboveMedian\u0026#34;] = df[\u0026#34;Fare\u0026#34;] \u0026gt; df.groupby(\u0026#34;Pclass\u0026#34;)[\u0026#34;Fare\u0026#34;].transform( lambda x: x.median() ) Why not just use the global median? The global median fare is ~£14.45. But a £20 fare is below average for Class 1 (median ~£60) while being way above average for Class 3 (median ~£8). Context-relative comparison is only possible with transform().\nQuestion 19 Build a complete passenger profile summary: for each combination of Sex, Pclass, and AgeBand, show passenger count, survival rate, average fare, and average age. Concepts: Multi-level groupby(), agg(), handling NaN categories, reset_index(), sort_values()\nAnswer # Make sure AgeBand exists (created in Q13) if \u0026#34;AgeBand\u0026#34; not in df.columns: df[\u0026#34;AgeBand\u0026#34;] = pd.cut( df[\u0026#34;Age\u0026#34;], bins=[0, 12, 17, 60, 100], labels=[\u0026#34;Child\u0026#34;, \u0026#34;Teen\u0026#34;, \u0026#34;Adult\u0026#34;, \u0026#34;Senior\u0026#34;] ) # Multi-level groupby with multiple aggregations profile = ( df.groupby([\u0026#34;Sex\u0026#34;, \u0026#34;Pclass\u0026#34;, \u0026#34;AgeBand\u0026#34;], observed=True) .agg( passengers = (\u0026#34;PassengerId\u0026#34;, \u0026#34;count\u0026#34;), survival_rate = (\u0026#34;Survived\u0026#34;, \u0026#34;mean\u0026#34;), avg_fare = (\u0026#34;Fare\u0026#34;, \u0026#34;mean\u0026#34;), avg_age = (\u0026#34;Age\u0026#34;, \u0026#34;mean\u0026#34;), ) .round(2) .reset_index() # bring group keys back as columns .sort_values( [\u0026#34;Sex\u0026#34;, \u0026#34;Pclass\u0026#34;, \u0026#34;AgeBand\u0026#34;], ascending=[True, True, True] ) ) print(profile.to_string()) Accessing specific slices of a multi-index result:\n# If you keep the multi-index (don\u0026#39;t reset_index), you can slice with .loc profile_mi = ( df.groupby([\u0026#34;Sex\u0026#34;, \u0026#34;Pclass\u0026#34;, \u0026#34;AgeBand\u0026#34;], observed=True) [\u0026#34;Survived\u0026#34;].mean().round(3) ) # Get all Class 1 female results print(profile_mi.loc[\u0026#34;female\u0026#34;, 1]) # Get Class 1 female children specifically print(profile_mi.loc[(\u0026#34;female\u0026#34;, 1, \u0026#34;Child\u0026#34;)]) reset_index() after groupby(): By default, groupby() puts the group keys as the DataFrame index. This is useful for label-based slicing but awkward for most downstream operations. Call .reset_index() to bring them back as regular columns - almost always the right move before saving results or further filtering.\nobserved=True in multi-level groupby with Categoricals: Without it, pandas generates a row for every possible combination of categories (including combinations that don\u0026rsquo;t exist in the data, like Senior Class 3 females). With observed=True, only combinations that actually appear are shown.\nQuestion 20 Identify the 5 most expensive cabins (by average fare of passengers in that cabin prefix). Then for those cabin prefixes, show the full survival breakdown. Concepts: str[0] string indexing, groupby() on derived column, chaining, isin(), complex multi-step analysis\nThis is the kind of multi-step question you\u0026rsquo;d encounter in a real data analysis interview.\nAnswer # Step 1: Extract cabin prefix (first letter of cabin, e.g., \u0026#39;C85\u0026#39; → \u0026#39;C\u0026#39;) # About 77% of Cabin is missing - those become NaN df[\u0026#34;CabinPrefix\u0026#34;] = df[\u0026#34;Cabin\u0026#34;].str[0] # str[0] gives first character print(df[\u0026#34;CabinPrefix\u0026#34;].value_counts()) # C 59 # B 47 # D 33 # E 32 # A 15 # F 13 # G 4 # T 1 # NaN (missing) = 687 # Step 2: Average fare per cabin prefix cabin_fare = ( df.dropna(subset=[\u0026#34;CabinPrefix\u0026#34;]) # work only with known cabins .groupby(\u0026#34;CabinPrefix\u0026#34;)[\u0026#34;Fare\u0026#34;] .mean() .round(2) .sort_values(ascending=False) ) print(\u0026#34;\\nAverage fare by cabin prefix:\u0026#34;) print(cabin_fare) # CabinPrefix # T 35.50 (only 1 passenger - unreliable) # B 93.68 # C 107.25 ← most expensive on average # D 57.24 # E 46.03 # A 39.62 # F 11.13 # G 9.45 # Step 3: Top 5 prefixes by average fare top5_prefixes = cabin_fare.nlargest(5).index.tolist() print(f\u0026#34;\\nTop 5 cabin prefixes by average fare: {top5_prefixes}\u0026#34;) # Step 4: Survival breakdown for those prefixes top5_analysis = ( df[df[\u0026#34;CabinPrefix\u0026#34;].isin(top5_prefixes)] .groupby(\u0026#34;CabinPrefix\u0026#34;) .agg( passengers = (\u0026#34;PassengerId\u0026#34;, \u0026#34;count\u0026#34;), survivors = (\u0026#34;Survived\u0026#34;, \u0026#34;sum\u0026#34;), survival_rate = (\u0026#34;Survived\u0026#34;, \u0026#34;mean\u0026#34;), avg_fare = (\u0026#34;Fare\u0026#34;, \u0026#34;mean\u0026#34;), ) .round(3) .sort_values(\u0026#34;avg_fare\u0026#34;, ascending=False) ) print(\u0026#34;\\nSurvival breakdown - top 5 cabin prefixes by fare:\u0026#34;) print(top5_analysis) # Step 5: Compare with passengers who had NO cabin recorded no_cabin = df[df[\u0026#34;CabinPrefix\u0026#34;].isna()][\u0026#34;Survived\u0026#34;].mean() print(f\u0026#34;\\nSurvival rate - no cabin recorded: {no_cabin:.3f}\u0026#34;) # Result: ~0.30 - much lower than most cabin groups # Inference: having a recorded cabin (associated with Pclass 1/2) → better survival nlargest(n) vs sort_values().head(n):\nnlargest(n) is more efficient - it doesn\u0026rsquo;t fully sort the Series sort_values().head(n) is more readable and also returns the full sorted order if needed For production code with large data, use nlargest/nsmallest. For exploration, either is fine. The missing cabin problem: 77% of passengers have no cabin recorded. This is almost certainly not random - third-class passengers generally had no assigned cabin. The missing value itself is informative. Techniques for handling this: (1) treat NaN as its own category \u0026quot;Unknown\u0026quot;, (2) use it as a binary feature has_cabin = Cabin.notna(), (3) impute based on class. Each choice has analysis tradeoffs.\n# Bonus: create has_cabin feature and check its correlation with survival df[\u0026#34;has_cabin\u0026#34;] = df[\u0026#34;Cabin\u0026#34;].notna().astype(int) print(df.groupby(\u0026#34;has_cabin\u0026#34;)[\u0026#34;Survived\u0026#34;].mean().round(3)) # has_cabin # 0 0.300 (no cabin recorded) # 1 0.667 (cabin recorded) Concepts Coverage Checklist ✅ df.shape, head(), tail(), info(), describe() ✅ value_counts(), value_counts(normalize=True) ✅ Boolean filtering with single condition ✅ Boolean filtering with multiple conditions (\u0026amp; and |) ✅ loc[] and iloc[] ✅ query() ✅ Column selection (list of columns) ✅ New column creation (arithmetic) ✅ pd.cut() and pd.qcut() ✅ groupby().size() ✅ groupby().count() - and why it differs from size() ✅ groupby().mean(), .sum(), .min(), .max() ✅ groupby() single column aggregation ✅ groupby() multi-column aggregation with named agg() ✅ groupby() on multiple keys ✅ agg() with a list of functions ✅ Named aggregation syntax: agg(new_name=(\u0026#34;col\u0026#34;, \u0026#34;func\u0026#34;)) ✅ unstack() to reshape groupby results ✅ pd.crosstab() - counts and proportions ✅ pd.crosstab(normalize=\u0026#39;index\u0026#39;/\u0026#39;columns\u0026#39;/True) ✅ pd.pivot_table() with aggfunc, margins ✅ groupby().transform() - core concept, broadcasting back ✅ transform() with lambda functions ✅ str.extract() with regex ✅ str[0] string indexing ✅ replace() for value mapping ✅ rank() within groups ✅ groupby().apply() with nlargest() ✅ nlargest() / nsmallest() ✅ isin() for membership filtering ✅ dropna(subset=[...]) ✅ fillna() with transform()-derived values ✅ reset_index() after groupby ✅ sort_values() with multiple columns ✅ Multi-level groupby with observed=True ✅ Multi-index slicing with .loc[] End of Practice Sheet - Titanic Dataset with Pandas Codeverra - codeverra.com\n","permalink":"https://blogs.codeverra.com/blog/titanic_pandas/","summary":"\u003ch1 id=\"pandas-practice-sheet-----titanic-dataset\"\u003ePandas Practice Sheet  -  Titanic Dataset\u003c/h1\u003e\n\u003ch3 id=\"20-questions-from-zero-to-advanced-with-full-solutions\"\u003e20 Questions from Zero to Advanced, with Full Solutions\u003c/h3\u003e\n\u003chr\u003e\n\u003ch2 id=\"getting-the-dataset\"\u003eGetting the Dataset\u003c/h2\u003e\n\u003cp\u003eThe Titanic dataset is hosted publicly and can be loaded directly into pandas  -  no download needed.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eDirect CSV URL (no login required):\u003c/strong\u003e\u003c/p\u003e\n\u003cpre tabindex=\"0\"\u003e\u003ccode\u003ehttps://raw.githubusercontent.com/datasciencedojo/datasets/master/titanic.csv\n\u003c/code\u003e\u003c/pre\u003e\u003cp\u003e\u003cstrong\u003eLoad it in one line:\u003c/strong\u003e\u003c/p\u003e\n\u003cdiv class=\"highlight\"\u003e\u003cpre tabindex=\"0\" class=\"chroma\"\u003e\u003ccode class=\"language-python\" data-lang=\"python\"\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"kn\"\u003eimport\u003c/span\u003e \u003cspan class=\"nn\"\u003epandas\u003c/span\u003e \u003cspan class=\"k\"\u003eas\u003c/span\u003e \u003cspan class=\"nn\"\u003epd\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"n\"\u003edf\u003c/span\u003e \u003cspan class=\"o\"\u003e=\u003c/span\u003e \u003cspan class=\"n\"\u003epd\u003c/span\u003e\u003cspan class=\"o\"\u003e.\u003c/span\u003e\u003cspan class=\"n\"\u003eread_csv\u003c/span\u003e\u003cspan class=\"p\"\u003e(\u003c/span\u003e\u003cspan class=\"s2\"\u003e\u0026#34;https://raw.githubusercontent.com/datasciencedojo/datasets/master/titanic.csv\u0026#34;\u003c/span\u003e\u003cspan class=\"p\"\u003e)\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/code\u003e\u003c/pre\u003e\u003c/div\u003e\u003cp\u003e\u003cstrong\u003eOr load via seaborn (also free, slightly different column names):\u003c/strong\u003e\u003c/p\u003e\n\u003cdiv class=\"highlight\"\u003e\u003cpre tabindex=\"0\" class=\"chroma\"\u003e\u003ccode class=\"language-python\" data-lang=\"python\"\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"kn\"\u003eimport\u003c/span\u003e \u003cspan class=\"nn\"\u003eseaborn\u003c/span\u003e \u003cspan class=\"k\"\u003eas\u003c/span\u003e \u003cspan class=\"nn\"\u003esns\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"n\"\u003edf\u003c/span\u003e \u003cspan class=\"o\"\u003e=\u003c/span\u003e \u003cspan class=\"n\"\u003esns\u003c/span\u003e\u003cspan class=\"o\"\u003e.\u003c/span\u003e\u003cspan class=\"n\"\u003eload_dataset\u003c/span\u003e\u003cspan class=\"p\"\u003e(\u003c/span\u003e\u003cspan class=\"s2\"\u003e\u0026#34;titanic\u0026#34;\u003c/span\u003e\u003cspan class=\"p\"\u003e)\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/code\u003e\u003c/pre\u003e\u003c/div\u003e\u003cblockquote\u003e\n\u003cp\u003eAlways run \u003ccode\u003edf.head()\u003c/code\u003e and \u003ccode\u003edf.info()\u003c/code\u003e before answering any question  -  knowing your data is half the battle.\u003c/p\u003e","tags":["python","data-analysis","pandas"],"title":"Practice Pandas on Titanic Dataset"},{"content":"Pandas Practice Sheet - Zomato Bangalore Dataset A complete guided exploration: from raw messy data to real business insights Why This Dataset? The Titanic dataset is clean, small, and predictable. The Zomato Bangalore dataset is the opposite - it is genuinely messy in the way real-world data is messy. Ratings stored as strings like \u0026quot;3.8/5\u0026quot;. Costs formatted as \u0026quot;450\u0026quot; with comma-separators (\u0026quot;1,200\u0026quot;). The cuisines column has multiple cuisines packed into one cell separated by commas. Several columns have inconsistent values, duplicates, and missing data scattered across them.\nThis is not a bug in the practice sheet. It is the point. Working through this dataset will teach you the pandas skills that actually matter when you sit down with a real business dataset:\nHow to clean before you analyse How to extract structured data from messy string columns How to explode multi-valued columns for correct counts How to group, aggregate, pivot, and transform on a dataset with real business meaning By the end of this sheet, you will be able to answer real questions a restaurant business owner or a Zomato product manager would actually ask.\nGetting the Dataset Kaggle page (free, no login required for direct CSV):\nhttps://www.kaggle.com/datasets/himanshupoddar/zomato-bangalore-restaurants Direct download (after Kaggle login): The dataset is zomato.csv - approximately 51,700 rows and 17 columns.\nAlternative: load a hosted mirror directly in Python (no login):\nimport pandas as pd # Hosted on GitHub (public mirror of the Kaggle dataset) url = \u0026#34;https://raw.githubusercontent.com/dsrscientist/dataset1/master/zomato.csv\u0026#34; df = pd.read_csv(url) If the mirror URL is unavailable, download zomato.csv from Kaggle and load locally:\ndf = pd.read_csv(\u0026#34;zomato.csv\u0026#34;) Column Dictionary Before writing a single line of code, understand every column in the raw dataset.\n┌────────────────────────────┬──────────┬────────────────────────────────────────────────────────────┐ │ Column (raw name) │ Type │ Meaning \u0026amp; Notes │ ├────────────────────────────┼──────────┼────────────────────────────────────────────────────────────┤ │ url │ string │ Zomato URL of the restaurant. Not useful for analysis. │ │ address │ string │ Full address. Too detailed; location is more useful. │ │ name │ string │ Restaurant name. │ │ online_order │ string │ \u0026#34;Yes\u0026#34; or \u0026#34;No\u0026#34; - does the restaurant accept online orders? │ │ book_table │ string │ \u0026#34;Yes\u0026#34; or \u0026#34;No\u0026#34; - does the restaurant allow table booking? │ │ rate │ string │ Rating as \u0026#34;4.1/5\u0026#34; or \u0026#34;NEW\u0026#34; or \u0026#34;-\u0026#34;. MESSY - needs cleaning. │ │ votes │ int │ Number of votes the restaurant has received. │ │ phone │ string │ Phone number. Not useful for analysis. │ │ location │ string │ Neighbourhood/area in Bangalore (e.g., \u0026#34;Koramangala 5th\u0026#34;). │ │ rest_type │ string │ Type of restaurant (e.g., \u0026#34;Casual Dining\u0026#34;, \u0026#34;Quick Bites\u0026#34;). │ │ dish_liked │ string │ Comma-separated list of popular dishes. Very messy. │ │ cuisines │ string │ Comma-separated list of cuisines (e.g., \u0026#34;North Indian, │ │ │ │ Chinese, Mughlai\u0026#34;). MULTI-VALUED - needs explode(). │ │ approx_cost(for two people)│ string │ Approximate cost for two, stored as string (\u0026#34;300\u0026#34;, \u0026#34;1,200\u0026#34;)│ │ │ │ Has commas in numbers. Needs cleaning → numeric. │ │ reviews_list │ string │ Raw list of reviews. Not structured for analysis. │ │ menu_item │ string │ Menu items. Mostly empty. │ │ listed_in(type) │ string │ Listing category: \u0026#34;Buffet\u0026#34;, \u0026#34;Cafes\u0026#34;, \u0026#34;Delivery\u0026#34;, etc. │ │ listed_in(city) │ string │ Area-level city tag. Broader than location. │ └────────────────────────────┴──────────┴────────────────────────────────────────────────────────────┘ Dataset at a glance:\n~51,700 rows (restaurants in Bangalore listed on Zomato) Several columns are object (string) type even though they should be numeric - this is the core challenge rate has three types of non-numeric values: \u0026quot;NEW\u0026quot;, \u0026quot;-\u0026quot;, and NaN approx_cost(for two people) has commas in numbers (Indian formatting) cuisines is multi-valued - each restaurant may serve 1 to 8+ cuisines in one cell Setup Block - Run This First import pandas as pd import numpy as np # Load the dataset df = pd.read_csv(\u0026#34;zomato.csv\u0026#34;) # Quick orientation print(\u0026#34;Shape:\u0026#34;, df.shape) print(\u0026#34;\\nColumn names:\u0026#34;) print(df.columns.tolist()) print(\u0026#34;\\nData types:\u0026#34;) print(df.dtypes) print(\u0026#34;\\nMissing values:\u0026#34;) print(df.isnull().sum()) print(\u0026#34;\\nFirst 3 rows:\u0026#34;) print(df.head(3)) SECTION 1 - First Look (Questions 1–3) Understand the shape, structure, and quality of the data before touching anything Question 1 What is the shape of the dataset? How many restaurants are there? How many columns? How many missing values does each column have, and what percentage of the column is missing? Concepts: shape, isnull(), sum(), percentage calculation, info()\nAnswer # --- Basic shape --- print(f\u0026#34;Rows : {df.shape[0]:,}\u0026#34;) # number of restaurants print(f\u0026#34;Columns : {df.shape[1]}\u0026#34;) # --- Missing value audit: count AND percentage --- missing = df.isnull().sum() missing_pct = (df.isnull().sum() / len(df) * 100).round(2) missing_report = pd.DataFrame({ \u0026#34;missing_count\u0026#34; : missing, \u0026#34;missing_pct\u0026#34; : missing_pct }).sort_values(\u0026#34;missing_pct\u0026#34;, ascending=False) print(missing_report[missing_report[\u0026#34;missing_count\u0026#34;] \u0026gt; 0]) # dish_liked ~26% ← very high - mostly useless # rate ~4% ← small but critical - needs cleaning # approx_cost... ~1% ← small # cuisines ~1% # rest_type ~1% # phone ~0.3% # --- Data types check --- df.info() # Key findings: # rate → object (should become float after cleaning) # approx_cost(for two people) → object (should become int after cleaning) # votes → already int - the one clean numeric column Why do this first? Missing value percentages tell you what to clean, what to drop, and what to be careful about in analysis. A column that is 26% missing (dish_liked) is nearly useless. A column that is 4% missing (rate) is critical and must be cleaned before any rating analysis. Never skip this step.\nQuestion 2 What are the unique values in online_order, book_table, and listed_in(type)? What is the distribution of each? Concepts: unique(), nunique(), value_counts(), value_counts(normalize=True)\nAnswer # --- Unique values --- for col in [\u0026#34;online_order\u0026#34;, \u0026#34;book_table\u0026#34;, \u0026#34;listed_in(type)\u0026#34;]: print(f\u0026#34;\\n{col}\u0026#34;) print(f\u0026#34; Unique values : {df[col].unique()}\u0026#34;) print(f\u0026#34; Unique count : {df[col].nunique()}\u0026#34;) # online_order : [\u0026#39;Yes\u0026#39; \u0026#39;No\u0026#39;] # book_table : [\u0026#39;Yes\u0026#39; \u0026#39;No\u0026#39;] # listed_in(type) : [\u0026#39;Delivery\u0026#39; \u0026#39;Dine-out\u0026#39; \u0026#39;Desserts\u0026#39; \u0026#39;Cafes\u0026#39; # \u0026#39;Drinks \u0026amp; nightlife\u0026#39; \u0026#39;Buffet\u0026#39; \u0026#39;Pubs and bars\u0026#39;] # --- Distribution with counts and percentages --- print(\u0026#34;\\n--- online_order distribution ---\u0026#34;) print(df[\u0026#34;online_order\u0026#34;].value_counts()) print(df[\u0026#34;online_order\u0026#34;].value_counts(normalize=True).mul(100).round(1)) print(\u0026#34;\\n--- book_table distribution ---\u0026#34;) print(df[\u0026#34;book_table\u0026#34;].value_counts()) print(df[\u0026#34;book_table\u0026#34;].value_counts(normalize=True).mul(100).round(1)) print(\u0026#34;\\n--- listed_in(type) distribution ---\u0026#34;) print(df[\u0026#34;listed_in(type)\u0026#34;].value_counts()) Insight to draw: If ~60–65% of restaurants offer online orders but only ~15% allow table booking, that tells you Zomato Bangalore is primarily a delivery platform, not a reservation platform. These proportions are useful business context for every downstream analysis.\nunique() vs nunique():\nunique() → returns the actual array of unique values (use to inspect categories) nunique() → returns just the count of unique values (use in aggregations) Use unique() first when you\u0026rsquo;re unfamiliar with a column to check for dirty values like mixed casing (\u0026quot;yes\u0026quot; vs \u0026quot;Yes\u0026quot;) or unexpected categories. Question 3 Examine the rate column closely. What are all the unique non-numeric values? How many restaurants have each type of value? Concepts: value_counts(), str.contains(), boolean filtering to understand dirty data before cleaning\nAnswer # --- What does rate look like? --- print(df[\u0026#34;rate\u0026#34;].dtype) # object print(df[\u0026#34;rate\u0026#34;].value_counts().head(20)) # Most values look like: \u0026#34;4.1/5\u0026#34;, \u0026#34;3.8/5\u0026#34;, \u0026#34;4.5/5\u0026#34; etc. # But there are also: \u0026#34;NEW\u0026#34;, \u0026#34;-\u0026#34;, and NaN # --- Count of non-standard values --- print(\u0026#34;\\nNon-numeric rate values:\u0026#34;) print(df[\u0026#34;rate\u0026#34;].isin([\u0026#34;NEW\u0026#34;, \u0026#34;-\u0026#34;]).sum()) # total non-numeric entries print(df[\u0026#34;rate\u0026#34;].value_counts().get(\u0026#34;NEW\u0026#34;, 0)) # NEW count print(df[\u0026#34;rate\u0026#34;].value_counts().get(\u0026#34;-\u0026#34;, 0)) # \u0026#39;-\u0026#39; count print(df[\u0026#34;rate\u0026#34;].isnull().sum()) # NaN count # --- Sample of each type --- print(df[df[\u0026#34;rate\u0026#34;] == \u0026#34;NEW\u0026#34;][[\u0026#34;name\u0026#34;, \u0026#34;rate\u0026#34;, \u0026#34;votes\u0026#34;]].head(3)) print(df[df[\u0026#34;rate\u0026#34;] == \u0026#34;-\u0026#34;][[\u0026#34;name\u0026#34;, \u0026#34;rate\u0026#34;, \u0026#34;votes\u0026#34;]].head(3)) # --- What is the range of valid ratings? --- valid_rates = df[~df[\u0026#34;rate\u0026#34;].isin([\u0026#34;NEW\u0026#34;, \u0026#34;-\u0026#34;]) \u0026amp; df[\u0026#34;rate\u0026#34;].notna()] print(valid_rates[\u0026#34;rate\u0026#34;].unique()[:10]) # e.g., [\u0026#34;4.1/5\u0026#34;, \u0026#34;3.8/5\u0026#34;, ...] This is the recon step. Before you can use rate for any analysis, you need to understand exactly what \u0026ldquo;dirty\u0026rdquo; means in this column. \u0026quot;NEW\u0026quot; means the restaurant is newly listed with no ratings yet. \u0026quot;-\u0026quot; means ratings were suppressed (too few votes or a policy reason). Both should become NaN after cleaning, not 0 - treating them as 0 would devastate your average rating calculations.\nSECTION 2 - Data Cleaning (Questions 4–6) This section is not optional. The rest of the practice sheet depends on it. Question 4 Clean the rate column: strip the \u0026quot;/5\u0026quot; suffix, convert \u0026quot;NEW\u0026quot; and \u0026quot;-\u0026quot; to NaN, and store the result as a new numeric column called rating. Concepts: str.replace(), replace(), pd.to_numeric(), astype(), handling multiple dirty values at once\nAnswer # Method 1 - Step by step (most readable for learners) # Step 1: Replace \u0026#34;NEW\u0026#34; and \u0026#34;-\u0026#34; with NaN df[\u0026#34;rate\u0026#34;] = df[\u0026#34;rate\u0026#34;].replace({\u0026#34;NEW\u0026#34;: np.nan, \u0026#34;-\u0026#34;: np.nan}) # Step 2: Strip the \u0026#34;/5\u0026#34; suffix from remaining string values # e.g., \u0026#34;4.1/5\u0026#34; → \u0026#34;4.1\u0026#34; df[\u0026#34;rate\u0026#34;] = df[\u0026#34;rate\u0026#34;].str.replace(\u0026#34;/5\u0026#34;, \u0026#34;\u0026#34;, regex=False) # Step 3: Convert to numeric - any remaining non-numeric becomes NaN automatically df[\u0026#34;rating\u0026#34;] = pd.to_numeric(df[\u0026#34;rate\u0026#34;], errors=\u0026#34;coerce\u0026#34;) # Verify print(df[\u0026#34;rating\u0026#34;].dtype) # float64 print(df[\u0026#34;rating\u0026#34;].describe()) # count ~49400 (those with valid ratings) # mean ~3.72 # min 1.8 # max 4.9 print(df[\u0026#34;rating\u0026#34;].isnull().sum()) # includes original NaN + NEW + \u0026#34;-\u0026#34; # ------------------------------------------------------- # Method 2 - One-liner using a regex extract # ------------------------------------------------------- df[\u0026#34;rating\u0026#34;] = pd.to_numeric( df[\u0026#34;rate\u0026#34;].str.extract(r\u0026#34;(\\d+\\.\\d+)\u0026#34;)[0], errors=\u0026#34;coerce\u0026#34; ) # str.extract() pulls out exactly the decimal number pattern # Any row that doesn\u0026#39;t match (NEW, -, NaN) becomes NaN automatically # Verify both methods give the same result print(df[\u0026#34;rating\u0026#34;].isna().sum()) print(df[\u0026#34;rating\u0026#34;].min(), df[\u0026#34;rating\u0026#34;].max()) pd.to_numeric(errors='coerce') is your best friend for dirty numeric columns.\nerrors='raise' → throws an error on the first bad value (default; useful during development) errors='coerce' → silently converts unparseable values to NaN (useful during cleaning) errors='ignore' → returns the original value unchanged if it can\u0026rsquo;t be parsed (rarely what you want) Why not astype(float) directly? Because astype(float) on a column containing \u0026quot;NEW\u0026quot; raises a ValueError. You\u0026rsquo;d have to manually replace non-numeric values first anyway. pd.to_numeric(errors='coerce') does both steps atomically.\nQuestion 5 Clean the approx_cost(for two people) column: remove comma separators, convert to integer, and store as a new column cost_for_two. Concepts: str.replace(), pd.to_numeric(), working with Indian number formatting\nAnswer # --- First, look at the problem --- print(df[\u0026#34;approx_cost(for two people)\u0026#34;].dtype) # object print(df[\u0026#34;approx_cost(for two people)\u0026#34;].unique()[:15]) # [\u0026#39;300\u0026#39;, \u0026#39;600\u0026#39;, \u0026#39;800\u0026#39;, \u0026#39;1,200\u0026#39;, \u0026#39;1,500\u0026#39;, \u0026#39;450\u0026#39;, ...] # Numbers over 999 use a comma: \u0026#34;1,200\u0026#34; instead of \u0026#34;1200\u0026#34; # Method 1 - str.replace then pd.to_numeric df[\u0026#34;cost_for_two\u0026#34;] = pd.to_numeric( df[\u0026#34;approx_cost(for two people)\u0026#34;].str.replace(\u0026#34;,\u0026#34;, \u0026#34;\u0026#34;, regex=False), errors=\u0026#34;coerce\u0026#34; ) # Method 2 - same result, slightly different style df[\u0026#34;cost_for_two\u0026#34;] = ( df[\u0026#34;approx_cost(for two people)\u0026#34;] .str.replace(\u0026#34;,\u0026#34;, \u0026#34;\u0026#34;, regex=False) .pipe(pd.to_numeric, errors=\u0026#34;coerce\u0026#34;) ) # Verify print(df[\u0026#34;cost_for_two\u0026#34;].dtype) # float64 (NaN requires float, not int) print(df[\u0026#34;cost_for_two\u0026#34;].describe()) # min 40 # max 6000 # mean ~547 # median ~400 # Safe cast to Int64 (nullable integer - pandas 1.0+) df[\u0026#34;cost_for_two\u0026#34;] = df[\u0026#34;cost_for_two\u0026#34;].astype(\u0026#34;Int64\u0026#34;) Why Int64 (capital I) instead of int64? Standard int64 cannot hold NaN - numpy integers don\u0026rsquo;t support null values. If you call .astype(int) on a column with NaN, it raises a ValueError. Pandas nullable integer type \u0026quot;Int64\u0026quot; handles NaN correctly. Use it whenever your integer column has missing values.\nQuestion 6 Rename the messy column names to clean, usable names. Also drop columns that are not useful for analysis (url, address, phone, menu_item, reviews_list). Concepts: rename(), drop(), column renaming best practices\nAnswer # --- Rename columns --- df = df.rename(columns={ \u0026#34;approx_cost(for two people)\u0026#34; : \u0026#34;cost_for_two_raw\u0026#34;, # already cleaned → new col \u0026#34;listed_in(type)\u0026#34; : \u0026#34;listing_type\u0026#34;, \u0026#34;listed_in(city)\u0026#34; : \u0026#34;listing_city\u0026#34;, }) # --- Drop irrelevant columns --- cols_to_drop = [\u0026#34;url\u0026#34;, \u0026#34;address\u0026#34;, \u0026#34;phone\u0026#34;, \u0026#34;menu_item\u0026#34;, \u0026#34;reviews_list\u0026#34;] df = df.drop(columns=cols_to_drop, errors=\u0026#34;ignore\u0026#34;) # errors=\u0026#34;ignore\u0026#34; → no error if a column doesn\u0026#39;t exist (safe for re-runs) # --- Check final column list --- print(df.columns.tolist()) # [\u0026#39;name\u0026#39;, \u0026#39;online_order\u0026#39;, \u0026#39;book_table\u0026#39;, \u0026#39;rate\u0026#39;, \u0026#39;votes\u0026#39;, # \u0026#39;location\u0026#39;, \u0026#39;rest_type\u0026#39;, \u0026#39;dish_liked\u0026#39;, \u0026#39;cuisines\u0026#39;, # \u0026#39;cost_for_two_raw\u0026#39;, \u0026#39;listing_type\u0026#39;, \u0026#39;listing_city\u0026#39;, # \u0026#39;rating\u0026#39;, \u0026#39;cost_for_two\u0026#39;] print(f\u0026#34;Clean shape: {df.shape}\u0026#34;) Rename before analyse. Column names like \u0026quot;approx_cost(for two people)\u0026quot; and \u0026quot;listed_in(type)\u0026quot; are painful to type repeatedly and prone to typos. Renaming upfront pays dividends across the entire notebook. The rename(columns={...}) dict syntax is the cleanest approach - far safer than reassigning df.columns = [...] (which requires getting the exact position right).\nSECTION 3 - Exploration \u0026amp; Aggregation (Questions 7–13) Now that data is clean, start answering business questions Question 7 What are the top 10 locations (neighbourhoods) in Bangalore by number of restaurants? What percentage of all restaurants does each represent? Concepts: value_counts(), percentage calculation, head(), reset_index()\nAnswer # Method 1 - value_counts with percentage column location_counts = df[\u0026#34;location\u0026#34;].value_counts().reset_index() location_counts.columns = [\u0026#34;location\u0026#34;, \u0026#34;restaurant_count\u0026#34;] location_counts[\u0026#34;pct_of_total\u0026#34;] = ( location_counts[\u0026#34;restaurant_count\u0026#34;] / len(df) * 100 ).round(2) print(location_counts.head(10)) # location restaurant_count pct_of_total # BTM ~1100 ~2.1 # Koramangala 5th Block ~900 ~1.7 # HSR ~880 ~1.7 # Indiranagar ~850 ~1.6 # ... # Method 2 - groupby approach (more flexible for adding extra metrics later) location_summary = ( df.groupby(\u0026#34;location\u0026#34;) .agg(restaurant_count=(\u0026#34;name\u0026#34;, \u0026#34;count\u0026#34;)) .sort_values(\u0026#34;restaurant_count\u0026#34;, ascending=False) .head(10) .reset_index() ) location_summary[\u0026#34;pct_of_total\u0026#34;] = ( location_summary[\u0026#34;restaurant_count\u0026#34;] / len(df) * 100 ).round(2) print(location_summary) Insight: BTM, Koramangala, HSR, and Indiranagar dominate. These are all IT-heavy neighbourhoods in South Bangalore - where tech workers who rely on food delivery are concentrated. This is not a coincidence.\nWhen to use value_counts() vs groupby().count():\nvalue_counts() → fastest one-liner for \u0026ldquo;how many of each category\u0026rdquo; groupby().agg() → better when you need additional columns (average rating, average cost, etc.) alongside the count in the same table Question 8 What is the average rating, average cost for two, and total votes for each restaurant type (rest_type)? Show only types with more than 100 restaurants, sorted by average rating descending. Concepts: Multi-metric agg(), named aggregations, filtering groups with boolean indexing after groupby, sort_values()\nAnswer rest_type_summary = ( df.groupby(\u0026#34;rest_type\u0026#34;) .agg( restaurant_count = (\u0026#34;name\u0026#34;, \u0026#34;count\u0026#34;), avg_rating = (\u0026#34;rating\u0026#34;, \u0026#34;mean\u0026#34;), avg_cost = (\u0026#34;cost_for_two\u0026#34;, \u0026#34;mean\u0026#34;), total_votes = (\u0026#34;votes\u0026#34;, \u0026#34;sum\u0026#34;), median_votes = (\u0026#34;votes\u0026#34;, \u0026#34;median\u0026#34;), ) .round(2) .reset_index() ) # Filter: only types with more than 100 restaurants filtered = rest_type_summary[rest_type_summary[\u0026#34;restaurant_count\u0026#34;] \u0026gt; 100] # Sort by average rating descending result = filtered.sort_values(\u0026#34;avg_rating\u0026#34;, ascending=False) print(result.to_string(index=False)) Why filter AFTER the groupby? Because you want to calculate aggregates across ALL restaurants of each type, then filter down to types with meaningful sample sizes. If you filtered the raw df first (removing small types), you\u0026rsquo;d still get the same aggregate results - but filtering post-groupby is the correct mental model.\nNamed aggregation reminder: new_col_name = (\u0026quot;source_col\u0026quot;, \u0026quot;agg_function\u0026quot;) This is the pandas 0.25+ way. Column names are clean immediately, no need for .rename() after.\nQuestion 9 What proportion of restaurants in each neighbourhood offer online ordering? Find the top 10 and bottom 10 locations by online order adoption rate. Concepts: groupby(), converting Yes/No to binary, mean() on derived boolean, nlargest(), nsmallest()\nAnswer # Step 1: Convert Yes/No to 1/0 for easy mean calculation df[\u0026#34;online_order_flag\u0026#34;] = (df[\u0026#34;online_order\u0026#34;] == \u0026#34;Yes\u0026#34;).astype(int) # Step 2: Groupby location - mean gives adoption rate location_online = ( df.groupby(\u0026#34;location\u0026#34;) .agg( total_restaurants = (\u0026#34;name\u0026#34;, \u0026#34;count\u0026#34;), online_order_rate = (\u0026#34;online_order_flag\u0026#34;, \u0026#34;mean\u0026#34;), ) .round(3) .reset_index() ) # Filter to locations with at least 50 restaurants (avoid noise from tiny areas) location_online = location_online[location_online[\u0026#34;total_restaurants\u0026#34;] \u0026gt;= 50] # Top 10 - highest online order adoption print(\u0026#34;Top 10 locations by online order rate:\u0026#34;) print(location_online.nlargest(10, \u0026#34;online_order_rate\u0026#34;) [[\u0026#34;location\u0026#34;, \u0026#34;total_restaurants\u0026#34;, \u0026#34;online_order_rate\u0026#34;]]) # Bottom 10 - lowest online order adoption print(\u0026#34;\\nBottom 10 locations by online order rate:\u0026#34;) print(location_online.nsmallest(10, \u0026#34;online_order_rate\u0026#34;) [[\u0026#34;location\u0026#34;, \u0026#34;total_restaurants\u0026#34;, \u0026#34;online_order_rate\u0026#34;]]) The Yes/No → binary conversion pattern (== \u0026quot;Yes\u0026quot;).astype(int)) is one of the most reusable pandas patterns. Once you have a 0/1 column, mean() gives you the rate, sum() gives you the count, and it plays nicely with all groupby operations. Apply this pattern to any binary string column.\nnlargest(n, col) vs sort_values().head(n):\nnlargest() is O(n log k) - significantly faster when n is large sort_values().head() is O(n log n) - full sort For datasets under ~1M rows the difference is negligible, but nlargest also reads more clearly. Question 10 What is the average rating and average cost for two across all combinations of online_order and book_table? Present as a pivot table. Concepts: pd.pivot_table(), multiple aggfunc, margins, interpreting 2×2 business tables\nAnswer # Pivot table: rows = online_order, columns = book_table # Values = rating and cost_for_two, aggfunc = mean pivot_rating = pd.pivot_table( df, values = \u0026#34;rating\u0026#34;, index = \u0026#34;online_order\u0026#34;, columns = \u0026#34;book_table\u0026#34;, aggfunc = \u0026#34;mean\u0026#34;, margins = True, margins_name = \u0026#34;Overall\u0026#34; ).round(3) pivot_cost = pd.pivot_table( df, values = \u0026#34;cost_for_two\u0026#34;, index = \u0026#34;online_order\u0026#34;, columns = \u0026#34;book_table\u0026#34;, aggfunc = \u0026#34;mean\u0026#34;, margins = True, margins_name = \u0026#34;Overall\u0026#34; ).round(0) print(\u0026#34;Average Rating:\u0026#34;) print(pivot_rating) # book_table No Yes Overall # online_order # No 3.XX 3.XX 3.XX # Yes 3.XX 3.XX 3.XX # Overall 3.XX 3.XX 3.XX print(\u0026#34;\\nAverage Cost for Two (Rs.):\u0026#34;) print(pivot_cost) # Combined: restaurant count in each cell pivot_count = pd.pivot_table( df, values = \u0026#34;name\u0026#34;, index = \u0026#34;online_order\u0026#34;, columns = \u0026#34;book_table\u0026#34;, aggfunc = \u0026#34;count\u0026#34;, margins = True, margins_name = \u0026#34;Total\u0026#34; ) print(\u0026#34;\\nRestaurant Count:\u0026#34;) print(pivot_count) What to read from this pivot: Restaurants that offer table booking tend to have higher average costs (they are nicer, sit-down establishments). Restaurants that do NOT offer table booking but DO offer online orders are typically quick-service or delivery-first. The 2×2 table immediately segments the market into four distinct restaurant archetypes.\nMultiple values in pivot_table: You can pass values=[\u0026quot;rating\u0026quot;, \u0026quot;cost_for_two\u0026quot;] with a list. The result has a MultiIndex column. It is slightly harder to read but gives both metrics in one call. For teaching purposes, separate calls are cleaner.\nQuestion 11 Find the top 10 most expensive locations (by median cost for two). For each, also show the average rating and restaurant count. Why median instead of mean? Concepts: groupby().agg(), mean vs median for skewed data, sort_values()\nAnswer location_cost = ( df.groupby(\u0026#34;location\u0026#34;) .agg( restaurant_count = (\u0026#34;name\u0026#34;, \u0026#34;count\u0026#34;), median_cost = (\u0026#34;cost_for_two\u0026#34;, \u0026#34;median\u0026#34;), mean_cost = (\u0026#34;cost_for_two\u0026#34;, \u0026#34;mean\u0026#34;), avg_rating = (\u0026#34;rating\u0026#34;, \u0026#34;mean\u0026#34;), ) .round(2) .reset_index() ) # Filter for locations with at least 30 restaurants location_cost = location_cost[location_cost[\u0026#34;restaurant_count\u0026#34;] \u0026gt;= 30] # Sort by median cost top10_expensive = location_cost.nlargest(10, \u0026#34;median_cost\u0026#34;) print(top10_expensive[[\u0026#34;location\u0026#34;, \u0026#34;restaurant_count\u0026#34;, \u0026#34;median_cost\u0026#34;, \u0026#34;mean_cost\u0026#34;, \u0026#34;avg_rating\u0026#34;]].to_string(index=False)) # Show the difference between mean and median (skew diagnostic) location_cost[\u0026#34;cost_skew\u0026#34;] = ( location_cost[\u0026#34;mean_cost\u0026#34;] - location_cost[\u0026#34;median_cost\u0026#34;] ).round(0) print(\u0026#34;\\nLocations with most cost skew (mean \u0026gt;\u0026gt; median → expensive outliers):\u0026#34;) print(location_cost.nlargest(5, \u0026#34;cost_skew\u0026#34;) [[\u0026#34;location\u0026#34;, \u0026#34;median_cost\u0026#34;, \u0026#34;mean_cost\u0026#34;, \u0026#34;cost_skew\u0026#34;]]) Mean vs Median - always think about this for price data. A location might have 200 budget restaurants and 5 luxury fine-dining restaurants. The 5 luxury ones can pull the mean cost to Rs.1200 even if the typical restaurant costs Rs.400. The median gives you the \u0026ldquo;typical\u0026rdquo; restaurant cost for that area - which is what you actually want when answering \u0026ldquo;how expensive is Koramangala?\u0026rdquo;\nThe gap between mean and median is itself informative: a large gap means a location has high-end outliers. A small gap means costs are more homogeneous.\nQuestion 12 What is the distribution of ratings? Create rating bands (Poor: \u0026lt;3.5, Average: 3.5–4.0, Good: 4.0–4.5, Excellent: \u0026gt;4.5) and show how many restaurants fall into each band along with their average cost and average votes. Concepts: pd.cut(), custom bins, groupby() on categorical, observed=True\nAnswer # Step 1: Create rating bands using pd.cut() df[\u0026#34;rating_band\u0026#34;] = pd.cut( df[\u0026#34;rating\u0026#34;], bins = [0, 3.5, 4.0, 4.5, 5.0], labels = [\u0026#34;Poor (\u0026lt;3.5)\u0026#34;, \u0026#34;Average (3.5-4.0)\u0026#34;, \u0026#34;Good (4.0-4.5)\u0026#34;, \u0026#34;Excellent (\u0026gt;4.5)\u0026#34;], right = True, # intervals are (left, right] ) # Step 2: Analyse by rating band rating_band_analysis = ( df.groupby(\u0026#34;rating_band\u0026#34;, observed=True) .agg( restaurant_count = (\u0026#34;name\u0026#34;, \u0026#34;count\u0026#34;), avg_cost = (\u0026#34;cost_for_two\u0026#34;, \u0026#34;mean\u0026#34;), avg_votes = (\u0026#34;votes\u0026#34;, \u0026#34;mean\u0026#34;), total_votes = (\u0026#34;votes\u0026#34;, \u0026#34;sum\u0026#34;), ) .round(1) .reset_index() ) print(rating_band_analysis.to_string(index=False)) # Percentage of restaurants in each band rating_band_analysis[\u0026#34;pct\u0026#34;] = ( rating_band_analysis[\u0026#34;restaurant_count\u0026#34;] / rating_band_analysis[\u0026#34;restaurant_count\u0026#34;].sum() * 100 ).round(1) print(\u0026#34;\\nPercentage distribution:\u0026#34;) print(rating_band_analysis[[\u0026#34;rating_band\u0026#34;, \u0026#34;restaurant_count\u0026#34;, \u0026#34;pct\u0026#34;]]) pd.cut() recap:\nBins are [0, 3.5, 4.0, 4.5, 5.0] → creates 4 intervals: (0,3.5], (3.5,4.0], (4.0,4.5], (4.5,5.0] right=True means the right boundary IS included in the bin (the default) Values outside the bin range become NaN - which is why we start from 0, not 1 Insight to draw: The bulk of restaurants cluster in the 3.5–4.0 range. Very few are below 3.5 (Zomato may suppress bad listings) or above 4.5 (genuine excellence is rare). This \u0026ldquo;compressed toward the top\u0026rdquo; distribution is common in online review systems.\nQuestion 13 Which are the top 10 restaurants by number of votes? For each, show their rating, cost, location, and restaurant type. Concepts: sort_values(), nlargest(), multi-column display, reset_index()\nAnswer # Method 1 - sort_values (returns all rows, then head) top10_votes = ( df.sort_values(\u0026#34;votes\u0026#34;, ascending=False) .head(10) [[\u0026#34;name\u0026#34;, \u0026#34;location\u0026#34;, \u0026#34;rest_type\u0026#34;, \u0026#34;rating\u0026#34;, \u0026#34;cost_for_two\u0026#34;, \u0026#34;votes\u0026#34;, \u0026#34;online_order\u0026#34;, \u0026#34;listing_type\u0026#34;]] .reset_index(drop=True) ) print(top10_votes.to_string(index=False)) # Method 2 - nlargest (more efficient, direct) top10_votes_v2 = df.nlargest(10, \u0026#34;votes\u0026#34;)[ [\u0026#34;name\u0026#34;, \u0026#34;location\u0026#34;, \u0026#34;rest_type\u0026#34;, \u0026#34;rating\u0026#34;, \u0026#34;cost_for_two\u0026#34;, \u0026#34;votes\u0026#34;] ] print(top10_votes_v2) # Bonus: are highly voted restaurants also highly rated? # Check correlation correlation = df[[\u0026#34;votes\u0026#34;, \u0026#34;rating\u0026#34;]].corr() print(f\u0026#34;\\nCorrelation between votes and rating: {correlation.loc[\u0026#39;votes\u0026#39;, \u0026#39;rating\u0026#39;]:.3f}\u0026#34;) # Expect a mild positive correlation (~0.2-0.4) # High votes ≠ high rating - some famous restaurants are famous for being debated votes vs rating - an important distinction. A restaurant with 5000 votes and rating 3.8 is very well-known but not universally loved. A restaurant with 50 votes and rating 4.8 might be a hidden gem or a new place. Both have value - they answer different questions (\u0026ldquo;most popular\u0026rdquo; vs \u0026ldquo;highest quality\u0026rdquo;).\nSECTION 4 - String Operations \u0026amp; Multi-valued Columns (Questions 14–17) The most distinctly Zomato-specific skills in this sheet Question 14 The cuisines column contains multiple cuisines per restaurant separated by commas (e.g., \u0026quot;North Indian, Chinese, Mughlai\u0026quot;). What are the top 20 most popular individual cuisines across all restaurants? Concepts: str.split(), explode(), value_counts(), working with multi-valued string columns - the single most important pattern for this dataset\nAnswer # --- The problem --- print(df[\u0026#34;cuisines\u0026#34;].head(5)) # 0 North Indian, Mughlai, Chinese # 1 Chinese, North Indian, Thai # 2 Cafe, Mexican, Italian, Continental, ... # If we do value_counts() directly, each COMBINATION is counted as one category # \u0026#34;North Indian, Chinese\u0026#34; ≠ \u0026#34;North Indian\u0026#34; ≠ \u0026#34;Chinese, North Indian\u0026#34; # --- The solution: split → explode → value_counts --- # Step 1: Split each cell by \u0026#34;,\u0026#34; into a list # Step 2: explode() turns each item in the list into its own row # Step 3: value_counts() on individual cuisines cuisine_counts = ( df[\u0026#34;cuisines\u0026#34;] .dropna() # remove NaN rows first .str.split(\u0026#34;,\u0026#34;) # split \u0026#34;North Indian, Chinese\u0026#34; → [\u0026#34;North Indian\u0026#34;, \u0026#34; Chinese\u0026#34;] .explode() # one row per cuisine .str.strip() # remove leading/trailing spaces from \u0026#34; Chinese\u0026#34; → \u0026#34;Chinese\u0026#34; .value_counts() .head(20) ) print(cuisine_counts) # North Indian ~14000 # Chinese ~10000 # Continental ~ 7000 # South Indian ~ 6000 # Fast Food ~ 5000 # ... explode() is the most important method for this dataset. Without it, you can never correctly count cuisines, dishes, or any other multi-valued column. It was introduced in pandas 0.25 and transforms a column of lists into individual rows, replicating the other column values.\nBefore explode: row 0: name=\u0026#34;Rest A\u0026#34;, cuisines=[\u0026#34;North Indian\u0026#34;, \u0026#34;Chinese\u0026#34;] After explode: row 0: name=\u0026#34;Rest A\u0026#34;, cuisines=\u0026#34;North Indian\u0026#34; row 0: name=\u0026#34;Rest A\u0026#34;, cuisines=\u0026#34;Chinese\u0026#34; The str.strip() after explode is not optional. Commas in CSVs often have spaces after them. Without strip, \u0026quot;Chinese\u0026quot; and \u0026quot; Chinese\u0026quot; (with a leading space) are counted as different cuisines, giving you dozens of phantom duplicates.\nQuestion 15 How many cuisines does each restaurant serve? Create a column cuisine_count. Then show the average rating, average cost, and restaurant count for each cuisine count (1 cuisine, 2 cuisines, 3 cuisines, etc.). Concepts: str.split() + str.len(), new column from string length, groupby() on derived numeric column\nAnswer # Step 1: Count cuisines per restaurant df[\u0026#34;cuisine_count\u0026#34;] = df[\u0026#34;cuisines\u0026#34;].str.split(\u0026#34;,\u0026#34;).str.len() # str.split(\u0026#34;,\u0026#34;) → list of cuisines # str.len() → length of that list = number of cuisines # NaN cuisines → NaN cuisine_count (handled automatically) print(df[\u0026#34;cuisine_count\u0026#34;].value_counts().sort_index()) # 1 ~8000 (single cuisine restaurants) # 2 ~9000 # 3 ~9000 # 4 ~8000 # 5 ~6000 # ... # Step 2: Analyse by cuisine count cuisine_count_analysis = ( df.groupby(\u0026#34;cuisine_count\u0026#34;) .agg( restaurant_count = (\u0026#34;name\u0026#34;, \u0026#34;count\u0026#34;), avg_rating = (\u0026#34;rating\u0026#34;, \u0026#34;mean\u0026#34;), avg_cost = (\u0026#34;cost_for_two\u0026#34;, \u0026#34;mean\u0026#34;), ) .round(2) .reset_index() ) print(cuisine_count_analysis) # Insight check: do restaurants with more cuisines tend to cost more? # (multi-cuisine = bigger menu = usually higher cost?) str.split().str.len() is a compound string operation. Chaining .str accessors works because str.split() returns a Series of lists, and str.len() on a Series of lists returns the length of each list. This pattern - str.split() → str.len() - is how you extract length from any delimited string column.\nQuestion 16 For each of the top 10 cuisines, what is the average rating and average cost for two? Use explode() properly so each restaurant is counted once per cuisine it serves. Concepts: str.split() + explode() on a copy of the DataFrame, groupby on exploded data, avoiding double-counting\nAnswer # The key challenge: after explode, one restaurant appears multiple times # (once per cuisine). This is correct for cuisine-level analysis. # Don\u0026#39;t add up \u0026#34;sum of votes\u0026#34; on exploded data - it will be wrong. # Do use \u0026#34;mean of rating\u0026#34; - that\u0026#39;s fine (same value repeated is still the same mean). # Step 1: Create an exploded copy (never explode the main df in-place!) df_exploded = df.copy() df_exploded[\u0026#34;cuisines\u0026#34;] = df_exploded[\u0026#34;cuisines\u0026#34;].str.split(\u0026#34;,\u0026#34;) df_exploded = df_exploded.explode(\u0026#34;cuisines\u0026#34;) df_exploded[\u0026#34;cuisines\u0026#34;] = df_exploded[\u0026#34;cuisines\u0026#34;].str.strip() # Step 2: Find top 10 cuisines by restaurant count top10_cuisines = ( df_exploded[\u0026#34;cuisines\u0026#34;] .value_counts() .head(10) .index .tolist() ) print(\u0026#34;Top 10 cuisines:\u0026#34;, top10_cuisines) # Step 3: Filter to only top 10 cuisines and aggregate cuisine_stats = ( df_exploded[df_exploded[\u0026#34;cuisines\u0026#34;].isin(top10_cuisines)] .groupby(\u0026#34;cuisines\u0026#34;) .agg( restaurant_count = (\u0026#34;name\u0026#34;, \u0026#34;count\u0026#34;), avg_rating = (\u0026#34;rating\u0026#34;, \u0026#34;mean\u0026#34;), avg_cost = (\u0026#34;cost_for_two\u0026#34;, \u0026#34;mean\u0026#34;), avg_votes = (\u0026#34;votes\u0026#34;, \u0026#34;mean\u0026#34;), ) .round(2) .sort_values(\u0026#34;avg_rating\u0026#34;, ascending=False) .reset_index() ) print(cuisine_stats.to_string(index=False)) Critical: always explode a copy, never the main df. After exploding, a restaurant serving 3 cuisines appears 3 times. If you then try to count total restaurants (df.shape[0] gives a wrong number) or compute a global average cost (each restaurant contributes its cost 3 times instead of 1), your results will be inflated. Keep the main df intact for analyses that should be at the restaurant level.\nWhen is it correct to use exploded data? For any question about individual cuisines: \u0026ldquo;what is the average rating of restaurants serving North Indian food?\u0026rdquo; That question is naturally answered on the exploded frame. For \u0026ldquo;what is the average cost of restaurants in Koramangala?\u0026rdquo;, use the original frame.\nQuestion 17 Extract the primary cuisine (the first cuisine listed for each restaurant). Create a new column primary_cuisine. Which primary cuisine has the highest average rating? Concepts: str.split() + str[0] (indexing into a list Series), deriving structured data from unstructured strings\nAnswer # str.split(\u0026#34;,\u0026#34;) gives a list; str[0] extracts the first element of each list df[\u0026#34;primary_cuisine\u0026#34;] = ( df[\u0026#34;cuisines\u0026#34;] .str.split(\u0026#34;,\u0026#34;) .str[0] # first cuisine in the comma-separated list .str.strip() # clean whitespace ) print(df[\u0026#34;primary_cuisine\u0026#34;].value_counts().head(10)) # North Indian ~5000 # South Indian ~3000 # Chinese ~2000 # ... # Average rating by primary cuisine (only cuisines with 100+ restaurants) primary_cuisine_rating = ( df.groupby(\u0026#34;primary_cuisine\u0026#34;) .agg( count = (\u0026#34;name\u0026#34;, \u0026#34;count\u0026#34;), avg_rating = (\u0026#34;rating\u0026#34;, \u0026#34;mean\u0026#34;), avg_cost = (\u0026#34;cost_for_two\u0026#34;, \u0026#34;mean\u0026#34;), ) .round(3) .reset_index() ) # Filter for statistical reliability primary_cuisine_rating = primary_cuisine_rating[ primary_cuisine_rating[\u0026#34;count\u0026#34;] \u0026gt;= 100 ] print( primary_cuisine_rating .sort_values(\u0026#34;avg_rating\u0026#34;, ascending=False) .head(10) .to_string(index=False) ) str[0] on a Series of lists works because the .str accessor supports integer indexing - str[0] means \u0026ldquo;first element of each list\u0026rdquo;. This is the same as str.get(0) with NaN safety. Use str[0], str[1], str[-1] to extract specific positions from a split.\nSECTION 5 - Groupby Transforms \u0026amp; Derived Insights (Questions 18–22) Adding context back to individual rows using transform() and other advanced patterns Question 18 Add a column rating_vs_location_avg that shows how much each restaurant\u0026rsquo;s rating differs from the average rating of all restaurants in the same location. Concepts: groupby().transform() - the core advanced concept\nAnswer # Step 1: Compute location-level average rating and broadcast back to each row df[\u0026#34;location_avg_rating\u0026#34;] = df.groupby(\u0026#34;location\u0026#34;)[\u0026#34;rating\u0026#34;].transform(\u0026#34;mean\u0026#34;) # Step 2: Compute the difference df[\u0026#34;rating_vs_location_avg\u0026#34;] = (df[\u0026#34;rating\u0026#34;] - df[\u0026#34;location_avg_rating\u0026#34;]).round(3) # Step 3: Verify it works sample = df[[\u0026#34;name\u0026#34;, \u0026#34;location\u0026#34;, \u0026#34;rating\u0026#34;, \u0026#34;location_avg_rating\u0026#34;, \u0026#34;rating_vs_location_avg\u0026#34;]].dropna().head(10) print(sample.to_string(index=False)) # Step 4: Find restaurants that most outperform their neighbourhood outperformers = ( df[[\u0026#34;name\u0026#34;, \u0026#34;location\u0026#34;, \u0026#34;rating\u0026#34;, \u0026#34;location_avg_rating\u0026#34;, \u0026#34;rating_vs_location_avg\u0026#34;]] .dropna() .sort_values(\u0026#34;rating_vs_location_avg\u0026#34;, ascending=False) .head(10) ) print(\u0026#34;\\nTop outperformers vs their neighbourhood:\u0026#34;) print(outperformers.to_string(index=False)) # Underperformers underperformers = ( df[[\u0026#34;name\u0026#34;, \u0026#34;location\u0026#34;, \u0026#34;rating\u0026#34;, \u0026#34;location_avg_rating\u0026#34;, \u0026#34;rating_vs_location_avg\u0026#34;]] .dropna() .sort_values(\u0026#34;rating_vs_location_avg\u0026#34;, ascending=True) .head(10) ) print(\u0026#34;\\nMost underperforming vs their neighbourhood:\u0026#34;) print(underperformers.to_string(index=False)) Why transform() matters here: You want to add \u0026ldquo;context\u0026rdquo; to each restaurant row. The context is the average rating of all restaurants around it. groupby().agg() gives you one number per location. groupby().transform() broadcasts that number back to every row in that location, giving you the same shape as the original DataFrame so you can subtract directly.\nThis is the classic \u0026ldquo;deviation from group mean\u0026rdquo; pattern - used in almost every real EDA.\nQuestion 19 Add a column cost_percentile_in_type showing where each restaurant\u0026rsquo;s cost falls within its restaurant type (as a percentile). Use this to find \u0026ldquo;value for money\u0026rdquo; restaurants - those in the top 25% of their type by rating but bottom 25% by cost. Concepts: groupby().transform() with a rank-based lambda, percentile calculation, multi-condition filtering\nAnswer # Step 1: Compute cost percentile within each restaurant type # rank(pct=True) gives percentile ranks (0.0 to 1.0) within the group df[\u0026#34;cost_pctile_in_type\u0026#34;] = ( df.groupby(\u0026#34;rest_type\u0026#34;)[\u0026#34;cost_for_two\u0026#34;] .transform(lambda x: x.rank(pct=True, na_option=\u0026#34;keep\u0026#34;)) .round(3) ) # Step 2: Compute rating percentile within each restaurant type df[\u0026#34;rating_pctile_in_type\u0026#34;] = ( df.groupby(\u0026#34;rest_type\u0026#34;)[\u0026#34;rating\u0026#34;] .transform(lambda x: x.rank(pct=True, na_option=\u0026#34;keep\u0026#34;)) .round(3) ) # Step 3: Define \u0026#34;value for money\u0026#34; = high rating, low cost within its type value_for_money = df[ (df[\u0026#34;rating_pctile_in_type\u0026#34;] \u0026gt;= 0.75) \u0026amp; # top 25% by rating in its type (df[\u0026#34;cost_pctile_in_type\u0026#34;] \u0026lt;= 0.25) # bottom 25% by cost in its type ].copy() result = ( value_for_money[ [\u0026#34;name\u0026#34;, \u0026#34;location\u0026#34;, \u0026#34;rest_type\u0026#34;, \u0026#34;rating\u0026#34;, \u0026#34;cost_for_two\u0026#34;, \u0026#34;rating_pctile_in_type\u0026#34;, \u0026#34;cost_pctile_in_type\u0026#34;, \u0026#34;cuisines\u0026#34;] ] .dropna(subset=[\u0026#34;rating\u0026#34;, \u0026#34;cost_for_two\u0026#34;]) .sort_values(\u0026#34;rating\u0026#34;, ascending=False) .head(20) ) print(f\u0026#34;Value-for-money restaurants found: {len(value_for_money)}\u0026#34;) print(result.to_string(index=False)) rank(pct=True) gives a value between 0 and 1 representing the percentile of that row within the group. A restaurant at the 95th percentile of cost in its type (cost_pctile_in_type = 0.95) is among the 5% most expensive for its category. This is far more meaningful than comparing raw costs across restaurant types (a fine dining restaurant at Rs.1200 might be cheap for its type; a quick bites restaurant at Rs.600 might be expensive for its type).\nQuestion 20 For each location, find the single highest-rated restaurant. Use groupby().idxmax() to do this efficiently. Concepts: groupby().idxmax(), using index values to slice the original DataFrame, loc[]\nAnswer # Method 1 - idxmax() returns the INDEX of the max-rated row per group idx_of_best = df.groupby(\u0026#34;location\u0026#34;)[\u0026#34;rating\u0026#34;].idxmax() # idx_of_best is a Series: index=location, values=row index in df best_per_location = df.loc[idx_of_best.values][ [\u0026#34;name\u0026#34;, \u0026#34;location\u0026#34;, \u0026#34;rating\u0026#34;, \u0026#34;cost_for_two\u0026#34;, \u0026#34;rest_type\u0026#34;, \u0026#34;cuisines\u0026#34;, \u0026#34;votes\u0026#34;] ].sort_values(\u0026#34;rating\u0026#34;, ascending=False) print(best_per_location.head(20).to_string(index=False)) # Method 2 - sort + groupby().first() (alternative approach) best_per_location_v2 = ( df.sort_values(\u0026#34;rating\u0026#34;, ascending=False) .groupby(\u0026#34;location\u0026#34;) .first() .reset_index() [[\u0026#34;location\u0026#34;, \u0026#34;name\u0026#34;, \u0026#34;rating\u0026#34;, \u0026#34;cost_for_two\u0026#34;, \u0026#34;rest_type\u0026#34;]] .sort_values(\u0026#34;rating\u0026#34;, ascending=False) ) print(best_per_location_v2.head(10)) # Method 3 - groupby().apply() with nlargest (top 3 instead of top 1) top3_per_location = ( df.dropna(subset=[\u0026#34;rating\u0026#34;]) .groupby(\u0026#34;location\u0026#34;)[[\u0026#34;name\u0026#34;, \u0026#34;rating\u0026#34;, \u0026#34;cost_for_two\u0026#34;]] .apply(lambda g: g.nlargest(3, \u0026#34;rating\u0026#34;)) .reset_index(drop=True) ) print(top3_per_location.head(15)) idxmax() vs sort + first():\nidxmax() is O(n) per group - faster sort + first() is O(n log n) but more readable and generalises to \u0026ldquo;top k\u0026rdquo; easily For \u0026ldquo;top 1\u0026rdquo;, use idxmax(). For \u0026ldquo;top k\u0026rdquo;, use nlargest() inside apply(). Important gotcha with idxmax() and NaN: If a group has all NaN ratings, idxmax() raises an error. Use df.dropna(subset=[\u0026quot;rating\u0026quot;]) before calling if this is a risk.\nQuestion 21 Create a crosstab showing the count of restaurants for every combination of listing_type and online_order. Then normalise it two ways: by row and by column. What does each normalisation tell you? Concepts: pd.crosstab(), normalize='index' vs normalize='columns' vs normalize=True\nAnswer # --- Raw counts --- ct_counts = pd.crosstab( index = df[\u0026#34;listing_type\u0026#34;], columns = df[\u0026#34;online_order\u0026#34;], margins = True, margins_name = \u0026#34;Total\u0026#34; ) print(\u0026#34;Raw Counts:\u0026#34;) print(ct_counts) # --- Normalize by ROW (index) --- # \u0026#34;Of all Delivery restaurants, what % accept online orders?\u0026#34; ct_row = pd.crosstab( index = df[\u0026#34;listing_type\u0026#34;], columns = df[\u0026#34;online_order\u0026#34;], normalize = \u0026#34;index\u0026#34; ).round(3) print(\u0026#34;\\nNormalized by Row (% within each listing type):\u0026#34;) print(ct_row) # Shows: Delivery → almost all Yes. Buffet → mostly No. Etc. # --- Normalize by COLUMN --- # \u0026#34;Of all restaurants that DO accept online orders, what % are Delivery type?\u0026#34; ct_col = pd.crosstab( index = df[\u0026#34;listing_type\u0026#34;], columns = df[\u0026#34;online_order\u0026#34;], normalize = \u0026#34;columns\u0026#34; ).round(3) print(\u0026#34;\\nNormalized by Column (% within each order type):\u0026#34;) print(ct_col) # --- Normalize across entire table --- # \u0026#34;What % of ALL restaurants are Delivery + Yes online order?\u0026#34; ct_all = pd.crosstab( index = df[\u0026#34;listing_type\u0026#34;], columns = df[\u0026#34;online_order\u0026#34;], normalize = True ).round(4) print(\u0026#34;\\nNormalized across entire table:\u0026#34;) print(ct_all) The three normalizations answer three completely different questions:\nnormalize value Question answered \u0026quot;index\u0026quot; (row-wise) \u0026ldquo;Within each listing type, what share accepts online orders?\u0026rdquo; \u0026quot;columns\u0026quot; (column-wise) \u0026ldquo;Among online-order restaurants, what share is each listing type?\u0026rdquo; True (whole table) \u0026ldquo;What fraction of ALL restaurants are in each combination?\u0026rdquo; Confusing these is a very common analytical error. Always state which normalisation you applied and why before interpreting the numbers.\nQuestion 22 Identify duplicate restaurant entries. Some restaurants appear in the dataset multiple times (same name, different listing type). How many truly unique restaurant names are there? For restaurants that appear multiple times, is there any systematic difference in how they are listed? Concepts: duplicated(), nunique(), groupby().filter(), understanding what \u0026ldquo;duplicate\u0026rdquo; means in context\nAnswer # --- How many unique restaurant names? --- total_rows = len(df) unique_names = df[\u0026#34;name\u0026#34;].nunique() print(f\u0026#34;Total rows : {total_rows:,}\u0026#34;) print(f\u0026#34;Unique names : {unique_names:,}\u0026#34;) print(f\u0026#34;Duplicate rows : {total_rows - unique_names:,}\u0026#34;) # --- True duplicates: same name, same location, same listing_type --- true_dupes = df.duplicated(subset=[\u0026#34;name\u0026#34;, \u0026#34;location\u0026#34;, \u0026#34;listing_type\u0026#34;], keep=False) print(f\u0026#34;\\nTrue duplicates (same name+location+type): {true_dupes.sum():,}\u0026#34;) # --- Restaurants that appear under multiple listing_types --- listing_counts = ( df.groupby(\u0026#34;name\u0026#34;)[\u0026#34;listing_type\u0026#34;] .nunique() .reset_index() .rename(columns={\u0026#34;listing_type\u0026#34;: \u0026#34;listing_type_count\u0026#34;}) ) multi_listed = listing_counts[listing_counts[\u0026#34;listing_type_count\u0026#34;] \u0026gt; 1] print(f\u0026#34;\\nRestaurants appearing under multiple listing types: {len(multi_listed):,}\u0026#34;) print(multi_listed.sort_values(\u0026#34;listing_type_count\u0026#34;, ascending=False).head(10)) # --- See an example --- example_name = multi_listed.iloc[0][\u0026#34;name\u0026#34;] example_rows = df[df[\u0026#34;name\u0026#34;] == example_name][ [\u0026#34;name\u0026#34;, \u0026#34;location\u0026#34;, \u0026#34;listing_type\u0026#34;, \u0026#34;rating\u0026#34;, \u0026#34;votes\u0026#34;, \u0026#34;online_order\u0026#34;] ] print(f\u0026#34;\\nAll entries for \u0026#39;{example_name}\u0026#39;:\u0026#34;) print(example_rows) \u0026ldquo;Duplicate\u0026rdquo; in this dataset has a specific meaning. A restaurant like \u0026ldquo;Truffles\u0026rdquo; appears as \u0026ldquo;Delivery\u0026rdquo; AND \u0026ldquo;Dine-out\u0026rdquo; as two separate rows - because it is listed under two categories on Zomato. These are NOT errors to drop. They represent different Zomato pages for the same physical restaurant. If you want one row per restaurant, you need to decide: take the row with the most votes? Average the ratings? The right answer depends on the question you\u0026rsquo;re asking.\nduplicated(subset=[...], keep=False): The keep=False flag marks ALL occurrences of duplicates (not just the second+). This lets you see and inspect every duplicated row. keep='first' keeps the first and marks the rest; keep='last' keeps the last.\nSECTION 6 - Advanced Multi-Step Analysis (Questions 23–27) Questions that combine multiple pandas techniques in a single analytical pipeline Question 23 Build a full neighbourhood comparison report: for each location with at least 50 restaurants, show the number of restaurants, average rating, median cost, % with online orders, % with table booking, and the most common restaurant type. Concepts: Multiple aggregations in one agg(), custom lambda aggregations, mode() in groupby, reset_index(), filtering on aggregated results\nAnswer # Adding binary flags for easy averaging df[\u0026#34;online_order_flag\u0026#34;] = (df[\u0026#34;online_order\u0026#34;] == \u0026#34;Yes\u0026#34;).astype(int) df[\u0026#34;book_table_flag\u0026#34;] = (df[\u0026#34;book_table\u0026#34;] == \u0026#34;Yes\u0026#34;).astype(int) # agg() with a lambda for mode (most common value) location_report = ( df.groupby(\u0026#34;location\u0026#34;) .agg( restaurant_count = (\u0026#34;name\u0026#34;, \u0026#34;count\u0026#34;), avg_rating = (\u0026#34;rating\u0026#34;, \u0026#34;mean\u0026#34;), median_cost = (\u0026#34;cost_for_two\u0026#34;, \u0026#34;median\u0026#34;), pct_online_order = (\u0026#34;online_order_flag\u0026#34;, \u0026#34;mean\u0026#34;), pct_book_table = (\u0026#34;book_table_flag\u0026#34;, \u0026#34;mean\u0026#34;), total_votes = (\u0026#34;votes\u0026#34;, \u0026#34;sum\u0026#34;), top_rest_type = (\u0026#34;rest_type\u0026#34;, lambda x: x.mode().iloc[0] if not x.mode().empty else \u0026#34;Unknown\u0026#34;), ) .round(3) .reset_index() ) # Percentage formatting location_report[\u0026#34;pct_online_order\u0026#34;] = (location_report[\u0026#34;pct_online_order\u0026#34;] * 100).round(1) location_report[\u0026#34;pct_book_table\u0026#34;] = (location_report[\u0026#34;pct_book_table\u0026#34;] * 100).round(1) # Filter for locations with at least 50 restaurants location_report = location_report[location_report[\u0026#34;restaurant_count\u0026#34;] \u0026gt;= 50] # Sort by average rating location_report = location_report.sort_values(\u0026#34;avg_rating\u0026#34;, ascending=False) print(f\u0026#34;Locations with 50+ restaurants: {len(location_report)}\u0026#34;) print(location_report.head(15).to_string(index=False)) mode() inside agg() with a lambda. The mode (most common value) is not a built-in aggregation function in pandas groupby. You have to pass a lambda. The .mode() method returns a Series (there can be multiple modes), so we take .iloc[0] to get the first one. The if not x.mode().empty else \u0026quot;Unknown\u0026quot; guard handles groups where all values are NaN.\nlambda inside agg() - tradeoffs:\nMore flexible than built-in aggregation strings Slower on large datasets (not vectorised) For datasets under ~500k rows (like this one), performance is fine For large data, consider computing the lambda column separately and merging it in Question 24 For the top 5 locations by restaurant count, build a crosstab showing the distribution of listing types (Delivery, Dine-out, Cafes, etc.). Normalise by row to see the \u0026ldquo;character\u0026rdquo; of each neighbourhood. Concepts: Filtering to a subset, pd.crosstab() with normalize, multi-step pipeline\nAnswer # Step 1: Identify top 5 locations top5_locations = df[\u0026#34;location\u0026#34;].value_counts().head(5).index.tolist() print(\u0026#34;Top 5 locations:\u0026#34;, top5_locations) # Step 2: Filter df to only those locations df_top5 = df[df[\u0026#34;location\u0026#34;].isin(top5_locations)] # Step 3: Crosstab - location vs listing_type, normalised by row ct = pd.crosstab( index = df_top5[\u0026#34;location\u0026#34;], columns = df_top5[\u0026#34;listing_type\u0026#34;], normalize = \u0026#34;index\u0026#34; # each row sums to 1.0 ).round(3).mul(100).round(1) # convert to percentages print(\u0026#34;Listing type distribution (%) per top location:\u0026#34;) print(ct) # Step 4: Also show raw counts for context ct_raw = pd.crosstab( index = df_top5[\u0026#34;location\u0026#34;], columns = df_top5[\u0026#34;listing_type\u0026#34;], margins = True ) print(\u0026#34;\\nRaw counts:\u0026#34;) print(ct_raw) Why normalise by row here? Locations have very different numbers of restaurants (BTM might have 1100, another location 200). Raw counts would make BTM look dominant in every category simply because it is larger. Row normalisation lets you compare the profile of each area - \u0026ldquo;what proportion of BTM restaurants are Delivery vs Dine-out?\u0026rdquo; - on equal footing.\nQuestion 25 Find cuisine combinations that appear most frequently together (the top 20 most common cuisine pairings). For each pair, show average rating and average cost. Concepts: str.split(), combinations() from itertools, Counter, advanced multi-step analysis\nAnswer from itertools import combinations from collections import Counter # Step 1: Extract cuisine pairs from each restaurant pair_counter = Counter() pair_details = [] # store rating and cost alongside for _, row in df.dropna(subset=[\u0026#34;cuisines\u0026#34;]).iterrows(): cuisines = [c.strip() for c in row[\u0026#34;cuisines\u0026#34;].split(\u0026#34;,\u0026#34;)] if len(cuisines) \u0026gt;= 2: for pair in combinations(sorted(cuisines), 2): # sorted → canonical order pair_counter[pair] += 1 pair_details.append({ \u0026#34;pair\u0026#34; : pair, \u0026#34;rating\u0026#34; : row[\u0026#34;rating\u0026#34;], \u0026#34;cost_for_two\u0026#34;: row[\u0026#34;cost_for_two\u0026#34;], }) # Step 2: Top 20 pairs by frequency top20_pairs = pair_counter.most_common(20) pair_df = pd.DataFrame(top20_pairs, columns=[\u0026#34;cuisine_pair\u0026#34;, \u0026#34;frequency\u0026#34;]) # Step 3: Compute average rating and cost per pair details_df = pd.DataFrame(pair_details) pair_stats = ( details_df.groupby(\u0026#34;pair\u0026#34;) .agg( avg_rating = (\u0026#34;rating\u0026#34;, \u0026#34;mean\u0026#34;), avg_cost = (\u0026#34;cost_for_two\u0026#34;, \u0026#34;mean\u0026#34;), ) .round(2) .reset_index() ) pair_stats[\u0026#34;pair\u0026#34;] = pair_stats[\u0026#34;pair\u0026#34;].astype(str) pair_df[\u0026#34;cuisine_pair_str\u0026#34;] = pair_df[\u0026#34;cuisine_pair\u0026#34;].astype(str) # Note: for large datasets, replace iterrows() with a vectorised approach print(pair_df.head(20)) print(\u0026#34;\\nTop pairs with stats:\u0026#34;) print(pair_stats.sort_values(\u0026#34;avg_rating\u0026#34;, ascending=False).head(10)) iterrows() warning: This approach uses iterrows(), which is slow for very large DataFrames. For this ~50k row dataset it runs in a few seconds. For millions of rows, you would use a vectorised approach with explode() and a self-join or apply(). The logic here is more important than the performance for this dataset.\nsorted() for canonical pairing: combinations() preserves order. To avoid counting (\u0026quot;North Indian\u0026quot;, \u0026quot;Chinese\u0026quot;) and (\u0026quot;Chinese\u0026quot;, \u0026quot;North Indian\u0026quot;) as different pairs, we sorted() the cuisine list first. This ensures every pair is always in alphabetical order.\nQuestion 26 Using groupby().transform(), add a column votes_zscore_in_location that shows the z-score of each restaurant\u0026rsquo;s votes relative to other restaurants in the same location. Then identify the restaurants that are statistically unusual (|z-score| \u0026gt; 2). Concepts: Z-score via transform() with a lambda, statistical outlier detection, multi-step analysis\nAnswer # Z-score = (value - group_mean) / group_std # transform() broadcasts group mean and std back to each row df[\u0026#34;votes_zscore_loc\u0026#34;] = df.groupby(\u0026#34;location\u0026#34;)[\u0026#34;votes\u0026#34;].transform( lambda x: (x - x.mean()) / x.std() ).round(3) # Restaurants that are statistical outliers in their location (|z| \u0026gt; 2) outliers = df[df[\u0026#34;votes_zscore_loc\u0026#34;].abs() \u0026gt; 2].copy() print(f\u0026#34;Outlier restaurants (|z-score| \u0026gt; 2): {len(outliers)}\u0026#34;) # Positive outliers - far more popular than their neighbourhood average popular_outliers = ( outliers[outliers[\u0026#34;votes_zscore_loc\u0026#34;] \u0026gt; 2] .sort_values(\u0026#34;votes_zscore_loc\u0026#34;, ascending=False) [[\u0026#34;name\u0026#34;, \u0026#34;location\u0026#34;, \u0026#34;votes\u0026#34;, \u0026#34;votes_zscore_loc\u0026#34;, \u0026#34;rating\u0026#34;, \u0026#34;listing_type\u0026#34;]] .head(15) ) print(\u0026#34;\\nMost unusually popular restaurants:\u0026#34;) print(popular_outliers.to_string(index=False)) # Why z-score over raw vote count? print(\u0026#34;\\n--- Why z-score matters ---\u0026#34;) print(\u0026#34;A restaurant with 500 votes in BTM (avg: 200 votes) is more exceptional\u0026#34;) print(\u0026#34;than the same 500 votes in Koramangala (avg: 450 votes)\u0026#34;) Z-score normalisation is the \u0026ldquo;correct\u0026rdquo; way to compare within-group standing when groups have different scales. Using raw votes to compare restaurants across locations is misleading - an area with more restaurants naturally accumulates more votes on average. The z-score removes this location-level bias and tells you how exceptional a restaurant is relative to its own neighbourhood.\nQuestion 27 Build the final \u0026ldquo;investor report\u0026rdquo;: for a hypothetical new restaurant investor, identify the top 5 locations that have the best combination of: (a) high average rating, (b) moderate average cost (Rs.300–Rs.700), (c) high online order adoption, and (d) moderate competition (between 50 and 200 restaurants). Score each location and rank them. Concepts: Multi-condition filtering, column normalisation with transform()/manual min-max scaling, creating a composite score, sort_values()\nAnswer # Step 1: Build location-level summary location_inv = ( df.groupby(\u0026#34;location\u0026#34;) .agg( restaurant_count = (\u0026#34;name\u0026#34;, \u0026#34;count\u0026#34;), avg_rating = (\u0026#34;rating\u0026#34;, \u0026#34;mean\u0026#34;), avg_cost = (\u0026#34;cost_for_two\u0026#34;, \u0026#34;mean\u0026#34;), online_rate = (\u0026#34;online_order_flag\u0026#34;, \u0026#34;mean\u0026#34;), ) .round(3) .reset_index() ) # Step 2: Apply investor filters # - Moderate competition: 50–200 restaurants # - Average cost in the \u0026#34;accessible\u0026#34; range: Rs.300–700 filtered = location_inv[ (location_inv[\u0026#34;restaurant_count\u0026#34;].between(50, 200)) \u0026amp; (location_inv[\u0026#34;avg_cost\u0026#34;].between(300, 700)) ].copy() print(f\u0026#34;Locations passing filters: {len(filtered)}\u0026#34;) # Step 3: Normalise each metric to 0–1 scale (min-max normalisation) def minmax(s): return (s - s.min()) / (s.max() - s.min()) filtered[\u0026#34;score_rating\u0026#34;] = minmax(filtered[\u0026#34;avg_rating\u0026#34;]) filtered[\u0026#34;score_online\u0026#34;] = minmax(filtered[\u0026#34;online_rate\u0026#34;]) # For competition: FEWER restaurants in range = better opportunity # So invert the scale: lower count → higher score filtered[\u0026#34;score_competition\u0026#34;] = 1 - minmax(filtered[\u0026#34;restaurant_count\u0026#34;]) # Step 4: Composite score (equal weights - adjustable) filtered[\u0026#34;composite_score\u0026#34;] = ( filtered[\u0026#34;score_rating\u0026#34;] * 0.40 + # rating quality: 40% weight filtered[\u0026#34;score_online\u0026#34;] * 0.35 + # digital readiness: 35% weight filtered[\u0026#34;score_competition\u0026#34;] * 0.25 # lower competition: 25% weight ).round(4) # Step 5: Top 5 top5_investment = filtered.nlargest(5, \u0026#34;composite_score\u0026#34;)[ [\u0026#34;location\u0026#34;, \u0026#34;restaurant_count\u0026#34;, \u0026#34;avg_rating\u0026#34;, \u0026#34;avg_cost\u0026#34;, \u0026#34;online_rate\u0026#34;, \u0026#34;composite_score\u0026#34;] ].reset_index(drop=True) top5_investment[\u0026#34;online_rate\u0026#34;] = (top5_investment[\u0026#34;online_rate\u0026#34;] * 100).round(1) top5_investment.index = top5_investment.index + 1 # rank starts at 1 print(\u0026#34;\\nTop 5 Locations for Restaurant Investment:\u0026#34;) print(top5_investment.to_string()) Composite scoring with min-max normalisation is a standard decision analysis technique. The key steps are:\nNormalise each metric to [0,1] so they are comparable in scale Invert metrics where \u0026ldquo;lower is better\u0026rdquo; (like competition count) Apply weights based on business priorities Sum to a single score The weights (40%, 35%, 25%) are a business decision, not a statistical one. A good analyst would document these weights, explain why they were chosen, and run a sensitivity analysis - what if we gave rating 50% weight? Does the ranking change dramatically?\nConcepts Coverage Checklist BASICS ✅ shape, dtypes, info(), describe() ✅ isnull().sum() - missing value audit with percentages ✅ unique(), nunique(), value_counts(), value_counts(normalize=True) CLEANING ✅ str.replace() - removing unwanted substrings ✅ replace({...}) - mapping specific values to NaN ✅ pd.to_numeric(errors=\u0026#39;coerce\u0026#39;) - safe string-to-number conversion ✅ astype(\u0026#34;Int64\u0026#34;) - nullable integer type for columns with NaN ✅ rename(columns={...}) - clean column renaming ✅ drop(columns=[...], errors=\u0026#39;ignore\u0026#39;) - safe column dropping SELECTION \u0026amp; FILTERING ✅ Boolean filtering with single conditions ✅ Boolean filtering with \u0026amp; (AND) and | (OR) ✅ .between() - range filtering ✅ isin([...]) - membership filtering ✅ .loc[] - label-based selection ✅ nlargest() / nsmallest() STRING OPERATIONS ✅ str.split() - splitting delimited strings ✅ str.strip() - removing whitespace ✅ str.len() - length of each string ✅ str[0] - indexing into a list Series ✅ str.extract(regex) - extracting patterns MULTI-VALUED COLUMNS (CORE ZOMATO SKILL) ✅ str.split() + explode() - expanding multi-valued cells ✅ str.strip() after explode() - removing phantom duplicates ✅ Working on exploded copy vs original DataFrame AGGREGATION ✅ groupby().size() vs groupby().count() ✅ groupby().mean(), .sum(), .median(), .min(), .max() ✅ Named aggregations: agg(col_name=(\u0026#34;source\u0026#34;, \u0026#34;func\u0026#34;)) ✅ Multiple aggregations in one agg() call ✅ Lambda inside agg() - e.g., mode() ✅ idxmax() / idxmin() to find the row with max/min value GROUPBY TRANSFORMS ✅ groupby().transform(\u0026#34;mean\u0026#34;) - broadcast group mean to rows ✅ groupby().transform(lambda x: ...) - custom group-level computation ✅ rank(pct=True) inside transform - within-group percentiles ✅ Z-score normalisation via transform() RESHAPING ✅ unstack() - multi-level index to columns ✅ pd.crosstab() - frequency tables ✅ pd.crosstab(normalize=\u0026#39;index\u0026#39;/\u0026#39;columns\u0026#39;/True) - row/col/total normalisation ✅ pd.pivot_table() - flexible pivot with aggfunc and margins ✅ reset_index() - after groupby, bringing group keys back to columns ADVANCED ✅ pd.cut() - custom bin ranges ✅ pd.qcut() - quantile-based bins ✅ duplicated(subset=[...], keep=False) ✅ combinations() from itertools - pair analysis ✅ Composite scoring with min-max normalisation ✅ Multi-step investor analysis pipeline End of Practice Sheet - Zomato Bangalore Dataset Codeverra - codeverra.com\n","permalink":"https://blogs.codeverra.com/blog/pandas_zomato/","summary":"\u003ch1 id=\"pandas-practice-sheet-----zomato-bangalore-dataset\"\u003ePandas Practice Sheet  -  Zomato Bangalore Dataset\u003c/h1\u003e\n\u003ch3 id=\"a-complete-guided-exploration-from-raw-messy-data-to-real-business-insights\"\u003eA complete guided exploration: from raw messy data to real business insights\u003c/h3\u003e\n\u003chr\u003e\n\u003ch2 id=\"why-this-dataset\"\u003eWhy This Dataset?\u003c/h2\u003e\n\u003cp\u003eThe Titanic dataset is clean, small, and predictable. The Zomato Bangalore dataset is the opposite  -  it is \u003cstrong\u003egenuinely messy in the way real-world data is messy\u003c/strong\u003e. Ratings stored as strings like \u003ccode\u003e\u0026quot;3.8/5\u0026quot;\u003c/code\u003e. Costs formatted as \u003ccode\u003e\u0026quot;450\u0026quot;\u003c/code\u003e with comma-separators (\u003ccode\u003e\u0026quot;1,200\u0026quot;\u003c/code\u003e). The \u003ccode\u003ecuisines\u003c/code\u003e column has multiple cuisines packed into one cell separated by commas. Several columns have inconsistent values, duplicates, and missing data scattered across them.\u003c/p\u003e","tags":["python","pandas","data-analysis"],"title":"Practice Pandas on Zomato Dataset"},{"content":" ## Pratice easy, medium and hard questions on cricket dataset. \u003c!DOCTYPE html\u003e SQL Practice: CricketDB - Codeverra Codeverra · SQL Practice SQL Practice Set\nCricketDB 12 questions on a cricket dataset - players, matches, and ball-by-ball performances. Easy to hard. Every question has a one-line approach, a clean solution, and alternatives where they exist.\nDataset Schema players -- 15 rows id SERIAL PRIMARY KEY name VARCHAR role VARCHAR -- 'Batsman' | 'Bowler' | 'All-rounder' | 'Wicket-keeper' batting_style VARCHAR -- 'Right-hand' | 'Left-hand' bowling_style VARCHAR -- NULL for batsmen and wicket-keepers is_active BOOLEAN matches -- 8 rows id SERIAL PRIMARY KEY opponent VARCHAR venue VARCHAR format VARCHAR -- 'Test' | 'ODI' | 'T20' match_date DATE result VARCHAR -- 'Won' | 'Lost' | 'Draw' performances -- 30 rows - one per player per match id SERIAL PRIMARY KEY match_id INT → matches.id player_id INT → players.id runs INT wickets INT catches INT 📄 Download the setup file to create and populate CricketDB in PostgreSQL (pgAdmin): → cricketdb_setup.sql ↓ \u0026nbsp;·\u0026nbsp; Run it once and all 15 players, 8 matches, and 30 performances are ready to query. Easy - single table, basic clauses Medium - JOINs, GROUP BY, HAVING Hard - window functions, CTEs, subqueries Easy Single table · WHERE · ORDER BY · basic aggregates 01 Easy List all active players and their roles, sorted alphabetically SELECT · WHERE · ORDER BY You're building a squad page. You need every currently active player's name and role - nothing from retired players, sorted so it's easy to scan. Approach Filter is_active = TRUE on the players table and sort by name. Single table, no JOIN needed. SELECT name, role, batting_style FROM players WHERE is_active = TRUE ORDER BY name; What to observe 11 rows - the 4 retired players (Dhoni, Sachin, Kumble, Dhawan) are excluded. is_active is a BOOLEAN column - comparing it to TRUE is the same as just writing WHERE is_active, which also works in PostgreSQL. 02 Easy How many players are in each role? GROUP BY · COUNT · ORDER BY The selection committee wants a quick squad breakdown - how many batsmen, bowlers, all-rounders, and wicket-keepers are in the 15-man squad. Approach Group the players table by role and count rows per group. No JOIN, no WHERE - aggregate the whole table. SELECT role, COUNT(*) AS player_count FROM players GROUP BY role ORDER BY player_count DESC; What to observe Batsmen: 7 · Bowlers: 4 · All-rounders: 3 · Wicket-keepers: 1. This includes both active and retired players - add WHERE is_active = TRUE before GROUP BY if you want only current players. 03 Easy Show all players who have a bowling style recorded - use COALESCE for the rest IS NULL · COALESCE · WHERE You need a bowling roster. Pure batsmen and the wicket-keeper have NULL in bowling_style. Display \"N/A\" for them instead of a blank, and only show players who actually bowl. Approach Use WHERE bowling_style IS NOT NULL for the bowling roster. For the full squad with \"N/A\" fallback, remove the WHERE and wrap the column in COALESCE(bowling_style, 'N/A'). -- Bowlers only SELECT name, role, bowling_style FROM players WHERE bowling_style IS NOT NULL ORDER BY bowling_style, name; -- Full squad with 'N/A' for non-bowlers SELECT name, role, COALESCE(bowling_style, 'N/A') AS bowling_style FROM players ORDER BY role, name; What to observe Never write WHERE bowling_style = NULL - that always returns zero rows. NULL comparisons require IS NULL / IS NOT NULL. COALESCE returns the first non-NULL argument, so COALESCE(bowling_style, 'N/A') gives 'N/A' only when bowling_style is NULL. 04 Easy India's win/loss record - how many matches won, lost, and drawn per format? GROUP BY · COUNT · FILTER A journalist wants India's record broken down by format - Tests, ODIs, and T20s separately, with wins, losses, and draws in each. Approach Group matches by format, then use COUNT(*) FILTER (WHERE result = '...') to count each outcome within the same row. Avoids writing three separate queries. SELECT format, COUNT(*) AS played, COUNT(*) FILTER (WHERE result = 'Won') AS wins, COUNT(*) FILTER (WHERE result = 'Lost') AS losses, COUNT(*) FILTER (WHERE result = 'Draw') AS draws FROM matches GROUP BY format ORDER BY format; Alternative - CASE WHEN (works in all databases) Replace each FILTER with SUM(CASE WHEN result = 'Won' THEN 1 ELSE 0 END). FILTER is PostgreSQL-specific and cleaner; CASE WHEN is standard SQL that works everywhere including MySQL and SQLite. What to observe ODI: 2 played, 2 wins · T20: 3 played, 2 wins, 1 loss · Test: 3 played, 2 wins, 1 loss. This dataset has no draws - all draw counts will be 0, which is correct. Medium JOINs · GROUP BY + HAVING · multi-table aggregation 05 Medium Show each player's career totals - runs, wickets, and matches played JOIN · GROUP BY · SUM · COUNT You need a career statistics page - one row per player showing their aggregate numbers across all matches they appeared in. Approach JOIN performances → players, then GROUP BY player to aggregate. Always include the player's id in GROUP BY alongside name - two players could theoretically share a name, and the id prevents incorrect grouping. SELECT pl.name, pl.role, COUNT(*) AS matches_played, SUM(p.runs) AS total_runs, SUM(p.wickets) AS total_wickets, SUM(p.catches) AS total_catches, MAX(p.runs) AS highest_score, ROUND(AVG(p.runs), 1) AS batting_avg FROM performances AS p JOIN players AS pl ON pl.id = p.player_id GROUP BY pl.id, pl.name, pl.role ORDER BY total_runs DESC; What to observe Kohli tops the run chart at 388 across 6 appearances. Only 13 players appear - 4 in the squad (Dhoni, Sachin, Kumble, Dhawan) were never selected for these 8 matches and have no rows in performances. An INNER JOIN only returns players who played. 06 Medium Find players who never took a single wicket in any match LEFT JOIN · anti-join · IS NULL · NOT EXISTS The bowling coach wants a list of players in the squad who contributed zero wickets across all recorded matches - pure batsmen who were selected but never bowled. Approach Two valid patterns: (1) LEFT JOIN from players to performances and filter on IS NULL (players with no performances at all), or (2) JOIN then HAVING to find players who played but had total wickets = 0. -- Approach A: Players in squad but never selected (no performances) SELECT pl.name, pl.role FROM players AS pl LEFT JOIN performances AS p ON p.player_id = pl.id WHERE p.id IS NULL ORDER BY pl.name; -- Approach B: Players who played but took 0 wickets SELECT pl.name, pl.role, SUM(p.wickets) AS total_wickets FROM performances AS p JOIN players AS pl ON pl.id = p.player_id GROUP BY pl.id, pl.name, pl.role HAVING SUM(p.wickets) = 0 ORDER BY pl.name; What to observe Approach A returns 4 players never selected at all (Dhoni, Sachin, Kumble, Dhawan). Approach B returns players who did play but bowled zero wickets - Kohli, Rohit, KL Rahul, Shreyas, Suryakumar. These are different questions despite sounding similar. Read carefully which one you actually need. 07 Medium Which all-rounders contributed both runs AND wickets in the same match? JOIN · WHERE with AND · multiple conditions The team analyst wants to identify matches where all-rounders delivered on both fronts - not just with the bat or ball, but both in the same game. Approach Three-table JOIN (performances → players → matches), then filter with WHERE pl.role = 'All-rounder' AND p.runs \u003e 0 AND p.wickets \u003e 0. All conditions go in WHERE - no aggregation needed because we want individual match rows. SELECT pl.name, m.opponent, m.format, m.result, p.runs, p.wickets FROM performances AS p JOIN players AS pl ON pl.id = p.player_id JOIN matches AS m ON m.id = p.match_id WHERE pl.role = 'All-rounder' AND p.runs \u003e 0 AND p.wickets \u003e 0 ORDER BY p.runs DESC; What to observe Jadeja appears 3 times - he consistently contributed with both bat and ball. Ashwin shows up twice. Pandya twice. This is the kind of query coaches use for form analysis - identifying players whose match impact is genuinely two-dimensional. 08 Medium Players with over 50 runs AND over 3 wickets - the true all-rounders JOIN · GROUP BY · HAVING with multiple conditions You want to rank players by genuine all-round contribution - not just role label, but actual numbers. Show only players who cleared both the batting and bowling thresholds. Approach Aggregate total runs and wickets per player with GROUP BY, then use HAVING to filter on both aggregates simultaneously. WHERE runs before grouping - HAVING filters after. You can't use WHERE here because SUM(runs) doesn't exist until after aggregation. SELECT pl.name, pl.role, SUM(p.runs) AS total_runs, SUM(p.wickets) AS total_wickets FROM performances AS p JOIN players AS pl ON pl.id = p.player_id GROUP BY pl.id, pl.name, pl.role HAVING SUM(p.runs) \u003e 50 AND SUM(p.wickets) \u003e 3 ORDER BY total_runs DESC; What to observe Jadeja (165 runs, 7 wickets) and Ashwin (48 runs, 10 wickets) clear both bars - though Ashwin is close on runs. Pandya (62 runs, 3 wickets) falls just short on wickets if you use \u003e 3 strictly. Try changing the thresholds to see who enters and exits the list. Hard Window functions · CTEs · subqueries · complex aggregation 09 Hard Top scorer in each match - without removing other players from the result ROW_NUMBER · OVER · PARTITION BY · subquery For every match, identify who scored the most runs. The catch: you want to see all players in each match, but flag who the top scorer was. A GROUP BY would collapse the rows - you need a window function. Approach Use ROW_NUMBER() OVER (PARTITION BY match_id ORDER BY runs DESC) to rank players within each match. Then wrap it in a subquery (or CTE) and filter on rn = 1 to get only the top scorer per match. -- Full ranking (all players visible, with their rank per match) SELECT pl.name, m.opponent, m.format, p.runs, ROW_NUMBER() OVER ( PARTITION BY p.match_id ORDER BY p.runs DESC ) AS rank_in_match FROM performances AS p JOIN players AS pl ON pl.id = p.player_id JOIN matches AS m ON m.id = p.match_id ORDER BY m.match_date, rank_in_match; -- Top scorer only per match SELECT name, opponent, format, runs FROM ( SELECT pl.name, m.opponent, m.format, p.runs, ROW_NUMBER() OVER ( PARTITION BY p.match_id ORDER BY p.runs DESC ) AS rn FROM performances AS p JOIN players AS pl ON pl.id = p.player_id JOIN matches AS m ON m.id = p.match_id ) AS ranked WHERE rn = 1 ORDER BY runs DESC; Alternative - RANK() instead of ROW_NUMBER() If two players score the same runs in a match, ROW_NUMBER picks one arbitrarily. Use RANK() instead - tied players both get rn = 1 and both appear in the filtered result. Use ROW_NUMBER when you want exactly one row per group regardless of ties; RANK when ties should both show. What to observe Kohli's 121 against England (Test, Ahmedabad) is the highest individual score. You cannot do this with a plain GROUP BY - MAX(runs) gives you the number but not the player's name without a second join. Window functions solve this cleanly by operating across rows without collapsing them. 10 Hard Kohli's running career total - cumulative runs match by match SUM OVER · PARTITION BY · ORDER BY · running total Show Kohli's runs in each match chronologically, and alongside each match show his cumulative career total at that point - the running total that grows with every game. Approach Filter performances to Kohli only, then use SUM(runs) OVER (ORDER BY match_date). The PARTITION BY is omitted here - since we've already filtered to one player, the window covers all his rows, accumulating in date order. SELECT m.match_date, m.opponent, m.format, m.result, p.runs AS runs_this_match, SUM(p.runs) OVER ( ORDER BY m.match_date ) AS career_total FROM performances AS p JOIN players AS pl ON pl.id = p.player_id JOIN matches AS m ON m.id = p.match_id WHERE pl.name = 'Virat Kohli' ORDER BY m.match_date; Extension - all players at once using PARTITION BY Remove the WHERE clause and add PARTITION BY pl.id inside the OVER - the running total then resets per player. Without PARTITION BY on multi-player data, the total would accumulate across all players combined, which is wrong. What to observe The final row shows career_total = 388 (Kohli's total across 6 matches in this dataset). Key insight: SUM() OVER keeps every row intact while adding the accumulated value. A plain GROUP BY + SUM would give you one total row, not a row-by-row progression. 11 Hard Players who scored above the average score across all performances CTE · subquery · AVG · correlated comparison Find players whose career total runs exceed the average career total across all players who appeared in matches - a measure of who's punching above the squad average. Approach Two-step problem: first calculate total runs per player, then find the average of those totals. Use a CTE to name the first step cleanly, then compare each player's total against the average in the outer query. WITH player_totals AS ( SELECT pl.id, pl.name, pl.role, SUM(p.runs) AS total_runs FROM performances AS p JOIN players AS pl ON pl.id = p.player_id GROUP BY pl.id, pl.name, pl.role ) SELECT name, role, total_runs, ROUND(AVG(total_runs) OVER (), 1) AS squad_avg FROM player_totals WHERE total_runs \u003e (SELECT AVG(total_runs) FROM player_totals) ORDER BY total_runs DESC; Alternative - scalar subquery inline (no CTE) You can compute the average inline: HAVING SUM(p.runs) \u003e (SELECT AVG(sub.total) FROM (SELECT SUM(runs) AS total FROM performances GROUP BY player_id) sub). This works but is harder to read - the CTE version names each step clearly and is easier to debug. What to observe AVG() OVER () with an empty OVER clause computes the average across the entire result - a window function trick that lets you show the squad average alongside each player's number without a second query or a self-join. 12 Hard Full player report card with CASE WHEN performance tier CTE · multiple aggregates · CASE WHEN classification · ROUND Build a one-page squad report: every player's matches, runs, wickets, batting average, highest score, and a performance tier label (Star / Solid / Support) based on their numbers. Approach Use a CTE to compute all raw aggregates in one place, then apply CASE WHEN in the outer SELECT to assign the tier. Separating aggregation from classification makes both steps readable and individually testable. WITH stats AS ( SELECT pl.id, pl.name, pl.role, pl.is_active, COUNT(*) AS matches_played, SUM(p.runs) AS total_runs, SUM(p.wickets) AS total_wickets, MAX(p.runs) AS highest_score, ROUND(AVG(p.runs), 1) AS batting_avg FROM performances AS p JOIN players AS pl ON pl.id = p.player_id GROUP BY pl.id, pl.name, pl.role, pl.is_active ) SELECT name, role, CASE WHEN is_active THEN 'Active' ELSE 'Retired' END AS status, matches_played, total_runs, total_wickets, highest_score, batting_avg, CASE WHEN total_runs \u003e= 200 OR total_wickets \u003e= 8 THEN '⭐ Star' WHEN total_runs \u003e= 80 OR total_wickets \u003e= 4 THEN '✅ Solid' ELSE '🔧 Support' END AS tier FROM stats ORDER BY total_runs DESC; What to observe Kohli (388 runs) and Rohit (243) are clear Stars by runs. Bumrah (11 wickets) and Ashwin (10 wickets) earn Star by the bowling route. The CASE WHEN checks conditions top to bottom - first match wins. Adjust the thresholds and rerun to see the tier boundaries shift. Concepts Covered Run these 12 questions in order - each builds on what came before.\nEasySELECT, WHERE, IS NULL, COALESCE, GROUP BY, COUNT, FILTER MediumINNER JOIN, LEFT JOIN, anti-join, GROUP BY + HAVING, multi-condition filters HardROW_NUMBER, RANK, SUM OVER, PARTITION BY, CTEs, CASE WHEN classification Downloadcricketdb_setup.sql ↓ ","permalink":"https://blogs.codeverra.com/blog/sql-practice-cricketdb/","summary":"## Pratice easy, medium and hard questions on cricket dataset.\n\n\u003c!DOCTYPE html\u003e\n\u003chtml lang=\"en\"\u003e\n\u003chead\u003e\n\u003cmeta charset=\"UTF-8\"\u003e\n\u003cmeta name=\"viewport\" content=\"width=device-width, initial-scale=1.0\"\u003e\n\u003ctitle\u003eSQL Practice: CricketDB  -  Codeverra\u003c/title\u003e\n\u003clink href=\"https://fonts.googleapis.com/css2?family=Fraunces:ital,wght@0,400;0,700;0,900;1,400\u0026family=DM+Mono:wght@400;500\u0026family=DM+Sans:wght@300;400;500\u0026display=swap\" rel=\"stylesheet\"\u003e\n\u003cstyle\u003e\n  :root {\n    --bg: #faf8f3;\n    --surface: #ffffff;\n    --border: #e8e2d9;\n    --accent: #1a6b3c;\n    --accent-light: #e8f5ee;\n    --accent2: #c8440a;\n    --text: #1c1a18;\n    --muted: #6b6560;\n    --code-bg: #1e2227;\n    --easy: #1a6b3c;\n    --easy-bg: #e8f5ee;\n    --medium: #b45309;\n    --medium-bg: #fef3c7;\n    --hard: #be123c;\n    --hard-bg: #ffe4e6;\n    --alt: #1d4ed8;\n    --alt-bg: #eff6ff;\n  }\n\n  * { margin: 0; padding: 0; box-sizing: border-box; }\n\n  body {\n    background: var(--bg);\n    color: var(--text);\n    font-family: 'DM Sans', sans-serif;\n    font-weight: 300;\n    line-height: 1.75;\n    padding-bottom: 80px;\n  }\n\n  /* HERO */\n  .hero {\n    background: var(--accent);\n    color: white;\n    padding: 64px 40px 56px;\n    position: relative;\n    overflow: hidden;\n  }\n  .hero::before {\n    content: '🏏';\n    position: absolute;\n    right: 40px;\n    top: 50%;\n    transform: translateY(-50%);\n    font-size: 140px;\n    opacity: 0.12;\n    line-height: 1;\n  }\n  .hero-tag {\n    display: inline-block;\n    background: rgba(255,255,255,0.2);\n    font-family: 'DM Mono', monospace;\n    font-size: 11px;\n    letter-spacing: 2px;\n    text-transform: uppercase;\n    padding: 4px 12px;\n    margin-bottom: 20px;\n  }\n  .hero h1 {\n    font-family: 'Fraunces', serif;\n    font-size: clamp(32px, 5vw, 58px);\n    font-weight: 900;\n    line-height: 1.1;\n    max-width: 700px;\n    margin-bottom: 16px;\n  }\n  .hero p {\n    font-size: 15px;\n    opacity: 0.85;\n    max-width: 560px;\n    line-height: 1.7;\n  }\n\n  /* SCHEMA BOX */\n  .schema-section {\n    max-width: 860px;\n    margin: 48px auto;\n    padding: 0 40px;\n  }\n  .schema-card {\n    background: var(--code-bg);\n    border-left: 4px solid var(--accent);\n    padding: 28px 32px;\n  }\n  .schema-label {\n    font-family: 'DM Mono', monospace;\n    font-size: 11px;\n    letter-spacing: 2px;\n    text-transform: uppercase;\n    color: #6b7280;\n    margin-bottom: 14px;\n  }\n  .schema-card pre {\n    font-family: 'DM Mono', monospace;\n    font-size: 13px;\n    color: #abb2bf;\n    line-height: 1.8;\n    overflow-x: auto;\n  }\n  .schema-card .tbl { color: #e06c75; }\n  .schema-card .col { color: #98c379; }\n  .schema-card .typ { color: #d19a66; }\n  .schema-card .cmt { color: #5c6370; font-style: italic; }\n\n  /* SETUP LINK */\n  .setup-banner {\n    max-width: 860px;\n    margin: 0 auto 48px;\n    padding: 0 40px;\n  }\n  .setup-link-box {\n    background: var(--accent-light);\n    border: 1px solid #b7dfc8;\n    padding: 16px 24px;\n    display: flex;\n    align-items: center;\n    gap: 14px;\n  }\n  .setup-link-box .icon { font-size: 22px; }\n  .setup-link-box p { font-size: 14px; color: var(--accent); }\n  .setup-link-box a {\n    color: var(--accent);\n    font-weight: 500;\n    font-family: 'DM Mono', monospace;\n    font-size: 13px;\n    text-decoration: underline;\n  }\n\n  /* DIFFICULTY LEGEND */\n  .legend {\n    max-width: 860px;\n    margin: 0 auto 40px;\n    padding: 0 40px;\n    display: flex;\n    gap: 24px;\n    flex-wrap: wrap;\n  }\n  .legend-item {\n    display: flex;\n    align-items: center;\n    gap: 8px;\n    font-size: 13px;\n    font-family: 'DM Mono', monospace;\n    color: var(--muted);\n  }\n  .legend-dot { width: 10px; height: 10px; border-radius: 50%; }\n\n  /* SECTION HEADER */\n  .section-header {\n    max-width: 860px;\n    margin: 0 auto;\n    padding: 0 40px;\n  }\n  .difficulty-banner {\n    padding: 12px 20px;\n    margin-bottom: 32px;\n    display: flex;\n    align-items: center;\n    gap: 12px;\n  }\n  .difficulty-banner .d-label {\n    font-family: 'Fraunces', serif;\n    font-size: 22px;\n    font-weight: 700;\n  }\n  .difficulty-banner .d-desc {\n    font-size: 13px;\n    opacity: 0.8;\n  }\n  .easy-banner { background: var(--easy-bg); color: var(--easy); }\n  .medium-banner { background: var(--medium-bg); color: var(--medium); }\n  .hard-banner { background: var(--hard-bg); color: var(--hard); }\n\n  /* QUESTION CARD */\n  .questions-wrap {\n    max-width: 860px;\n    margin: 0 auto;\n    padding: 0 40px;\n  }\n  .q-card {\n    background: var(--surface);\n    border: 1px solid var(--border);\n    margin-bottom: 32px;\n    box-shadow: 0 1px 3px rgba(0,0,0,0.04);\n  }\n  .q-header {\n    padding: 24px 28px 0;\n    display: flex;\n    gap: 16px;\n    align-items: flex-start;\n  }\n  .q-num {\n    font-family: 'Fraunces', serif;\n    font-size: 36px;\n    font-weight: 900;\n    line-height: 1;\n    min-width: 44px;\n    color: var(--border);\n  }\n  .q-card.easy .q-num { color: #b7dfc8; }\n  .q-card.medium .q-num { color: #fcd34d; }\n  .q-card.hard .q-num { color: #fca5a5; }\n\n  .q-meta { flex: 1; }\n  .q-badge {\n    display: inline-block;\n    font-family: 'DM Mono', monospace;\n    font-size: 10px;\n    font-weight: 500;\n    letter-spacing: 1.5px;\n    text-transform: uppercase;\n    padding: 3px 10px;\n    margin-bottom: 10px;\n  }\n  .easy .q-badge { background: var(--easy-bg); color: var(--easy); }\n  .medium .q-badge { background: var(--medium-bg); color: var(--medium); }\n  .hard .q-badge { background: var(--hard-bg); color: var(--hard); }\n\n  .q-title {\n    font-family: 'Fraunces', serif;\n    font-size: 18px;\n    font-weight: 700;\n    color: var(--text);\n    line-height: 1.4;\n    margin-bottom: 8px;\n  }\n  .q-concepts {\n    font-family: 'DM Mono', monospace;\n    font-size: 11px;\n    color: var(--muted);\n    letter-spacing: 0.5px;\n  }\n\n  .q-body { padding: 20px 28px 28px; }\n\n  .q-scenario {\n    background: #f9f7f3;\n    border-left: 3px solid var(--border);\n    padding: 14px 18px;\n    font-size: 14px;\n    color: var(--muted);\n    margin-bottom: 18px;\n    line-height: 1.7;\n  }\n  .q-scenario strong { color: var(--text); font-weight: 500; }\n\n  .approach-label {\n    font-family: 'DM Mono', monospace;\n    font-size: 10px;\n    letter-spacing: 2px;\n    text-transform: uppercase;\n    color: var(--accent);\n    margin-bottom: 6px;\n    margin-top: 18px;\n  }\n  .approach-text {\n    font-size: 14px;\n    color: var(--text);\n    background: var(--accent-light);\n    border-left: 3px solid var(--accent);\n    padding: 10px 14px;\n    margin-bottom: 14px;\n    line-height: 1.6;\n  }\n  .approach-text code {\n    font-family: 'DM Mono', monospace;\n    font-size: 12px;\n    background: rgba(26,107,60,0.1);\n    padding: 1px 5px;\n  }\n\n  /* ALT APPROACH */\n  .alt-label {\n    font-family: 'DM Mono', monospace;\n    font-size: 10px;\n    letter-spacing: 2px;\n    text-transform: uppercase;\n    color: var(--alt);\n    margin-bottom: 6px;\n    margin-top: 20px;\n  }\n  .alt-note {\n    font-size: 13.5px;\n    color: var(--text);\n    background: var(--alt-bg);\n    border-left: 3px solid var(--alt);\n    padding: 10px 14px;\n    margin-bottom: 10px;\n    line-height: 1.6;\n  }\n  .alt-note code {\n    font-family: 'DM Mono', monospace;\n    font-size: 12px;\n    background: rgba(29,78,216,0.1);\n    padding: 1px 5px;\n  }\n\n  /* CODE */\n  pre.sql {\n    background: var(--code-bg);\n    padding: 20px 24px;\n    overflow-x: auto;\n    font-family: 'DM Mono', monospace;\n    font-size: 13px;\n    line-height: 1.8;\n    color: #abb2bf;\n    margin-bottom: 6px;\n  }\n  .kw { color: #c678dd; }\n  .fn { color: #61afef; }\n  .str { color: #98c379; }\n  .num { color: #d19a66; }\n  .cm { color: #5c6370; font-style: italic; }\n  .tbl-name { color: #e06c75; }\n  .op { color: #56b6c2; }\n\n  .explain-label {\n    font-family: 'DM Mono', monospace;\n    font-size: 10px;\n    letter-spacing: 2px;\n    text-transform: uppercase;\n    color: var(--muted);\n    margin-top: 16px;\n    margin-bottom: 6px;\n  }\n  .explain-text {\n    font-size: 13.5px;\n    color: var(--muted);\n    line-height: 1.75;\n    border-top: 1px solid var(--border);\n    padding-top: 14px;\n  }\n  .explain-text code {\n    font-family: 'DM Mono', monospace;\n    font-size: 12px;\n    background: #f0ece6;\n    padding: 1px 5px;\n    color: var(--accent2);\n  }\n  .explain-text strong { color: var(--text); font-weight: 500; }\n\n  /* DIVIDER */\n  .section-divider {\n    max-width: 860px;\n    margin: 16px auto 40px;\n    padding: 0 40px;\n    border: none;\n    border-top: 2px dashed var(--border);\n  }\n\n  /* SUMMARY */\n  .summary {\n    max-width: 860px;\n    margin: 48px auto 0;\n    padding: 0 40px;\n  }\n  .summary-card {\n    background: var(--accent);\n    color: white;\n    padding: 36px 40px;\n  }\n  .summary-card h2 {\n    font-family: 'Fraunces', serif;\n    font-size: 26px;\n    font-weight: 700;\n    margin-bottom: 16px;\n  }\n  .summary-grid {\n    display: grid;\n    grid-template-columns: repeat(auto-fit, minmax(200px, 1fr));\n    gap: 12px;\n    margin-top: 20px;\n  }\n  .summary-item {\n    background: rgba(255,255,255,0.1);\n    padding: 14px 16px;\n    font-size: 13px;\n    line-height: 1.6;\n  }\n  .summary-item strong {\n    display: block;\n    font-family: 'DM Mono', monospace;\n    font-size: 10px;\n    letter-spacing: 1.5px;\n    text-transform: uppercase;\n    opacity: 0.7;\n    margin-bottom: 4px;\n  }\n\n  @media (max-width: 640px) {\n    .hero, .schema-section, .setup-banner, .legend, .section-header, .questions-wrap, .summary { padding-left: 20px; padding-right: 20px; }\n    .hero::before { display: none; }\n    .q-num { font-size: 28px; min-width: 36px; }\n  }\n\u003c/style\u003e\n\u003c/head\u003e\n\u003cbody\u003e\n\n\u003cdiv class=\"hero\"\u003e\n  \u003cdiv class=\"hero-tag\"\u003eCodeverra · SQL Practice\u003c/div\u003e\n  \u003ch1\u003eSQL Practice Set\u003cbr\u003eCricketDB\u003c/h1\u003e\n  \u003cp\u003e12 questions on a cricket dataset  -  players, matches, and ball-by-ball performances. Easy to hard. Every question has a one-line approach, a clean solution, and alternatives where they exist.\u003c/p\u003e","tags":["sql","data-analysis"],"title":"SQL Practice on Cricket Dataset: 12 Questions with Solutions"},{"content":" ## Practice SQL questions on the shopping dataset. \u003c!DOCTYPE html\u003e SQL Practice: ShopDB - Codeverra Codeverra · SQL Practice SQL Practice Set\nShopDB 12 questions on an Indian e-commerce dataset - customers, products, orders, and line items. Easy to hard. Every question includes the business context, a one-line approach, a clean query, and alternatives where they exist.\nDataset Schema · 5 tables customers - 20 rows id, name, email city, state joined_on DATE categories - 6 rows id, name description nullable products - 25 rows id, name category_id → categories price_inr, stock_quantity is_available BOOLEAN orders - 15 rows id, customer_id → customers order_date DATE status Delivered/Shipped/Pending/Cancelled payment_method UPI/Card/COD/NetBanking order_items - 30 rows id, order_id → orders product_id → products quantity, unit_price 📄 Create ShopDB in PostgreSQL with one command - 20 customers, 25 products, 15 orders, 30 line items, all Indian-themed: → shopdb_setup.sql ↓ \u0026nbsp;·\u0026nbsp; Run it in pgAdmin or psql before starting. Easy - single/two table Medium - multi-table JOINs, HAVING Hard - CTEs, window functions, subqueries Easy Single or two tables · WHERE · basic aggregates · CASE WHEN 01 Easy How many products are in each category? GROUP BY · COUNT · JOIN The product team wants a quick count of how many SKUs live in each category - Electronics, Books, Sports, etc. Show the category name, not the id. Approach JOIN products to categories to get the name, then GROUP BY category and COUNT(*). The JOIN is necessary because the name lives in a separate table - products only stores category_id. SELECT c.name AS category, COUNT(*) AS product_count, ROUND(AVG(p.price_inr), 0) AS avg_price_inr FROM products AS p JOIN categories AS c ON c.id = p.category_id GROUP BY c.id, c.name ORDER BY product_count DESC; Always include the JOIN key (c.id) in GROUP BY alongside any display column - if two categories accidentally shared a name, GROUP BY c.name alone would merge them incorrectly. 02 Easy Label every product as Budget, Mid-range, or Premium CASE WHEN · SELECT Marketing needs to segment the product catalogue by price band for a promotional email - budget items for one campaign, premium for another. Approach Use CASE WHEN to create a computed column. Conditions are checked top to bottom - first match wins, so order matters. No JOIN needed - everything lives on products. SELECT name, price_inr, stock_quantity, CASE WHEN price_inr \u003c 1000 THEN 'Budget' WHEN price_inr BETWEEN 1000 AND 9999 THEN 'Mid-range' ELSE 'Premium' END AS price_segment FROM products ORDER BY price_inr; Alternative - add a count summary Wrap the above in a subquery (or CTE) and GROUP BY price_segment to get how many products fall into each band - useful as a dashboard metric. BETWEEN 1000 AND 9999 is inclusive on both ends. A product priced exactly ₹1000 matches the Mid-range condition. CASE WHEN conditions short-circuit - once a row matches, remaining conditions are skipped. 03 Easy Monthly order count for 2024 - how busy was each month? EXTRACT · GROUP BY · ORDER BY Operations wants to know which months in 2024 saw the highest order volume - to plan staffing and logistics for the next year. Approach Use EXTRACT(MONTH FROM order_date) to pull the month number out of the date, then GROUP BY that. Filter the year with WHERE EXTRACT(YEAR FROM order_date) = 2024. SELECT EXTRACT(MONTH FROM order_date) AS month, TO_CHAR(order_date, 'Month') AS month_name, COUNT(*) AS orders_placed FROM orders WHERE EXTRACT(YEAR FROM order_date) = 2024 GROUP BY month, month_name ORDER BY month; TO_CHAR(order_date, 'Month') gives a readable month name alongside the number. PostgreSQL's EXTRACT returns a number (1–12) - you need it in GROUP BY to aggregate correctly. TO_CHAR is just for display. 04 Easy Order status breakdown - delivered, pending, shipped, cancelled GROUP BY · COUNT · FILTER pivot The ops dashboard needs a single-row summary of all orders by status - how many delivered, how many still pending, how many cancelled. Approach Two ways: GROUP BY status gives one row per status (vertical), or use COUNT(*) FILTER (WHERE status = '...') to get all statuses in one horizontal row - the \"pivot\" pattern. -- Vertical: one row per status SELECT status, COUNT(*) AS count FROM orders GROUP BY status ORDER BY count DESC; -- Horizontal pivot: everything in one row SELECT COUNT(*) AS total, COUNT(*) FILTER (WHERE status = 'Delivered') AS delivered, COUNT(*) FILTER (WHERE status = 'Pending') AS pending, COUNT(*) FILTER (WHERE status = 'Shipped') AS shipped, COUNT(*) FILTER (WHERE status = 'Cancelled') AS cancelled FROM orders; The pivot form is useful when the result needs to feed a dashboard widget that expects one row. The GROUP BY form is more flexible - add a new status value and it appears automatically. FILTER is PostgreSQL-specific; use SUM(CASE WHEN status = 'X' THEN 1 ELSE 0 END) for portability. Medium Multi-table JOINs · revenue calculations · HAVING · anti-join 05 Medium Total revenue by category - excluding cancelled orders 4-table JOIN · SUM · GROUP BY · WHERE Finance needs a revenue breakdown by product category. Revenue = quantity × unit_price at the line item level. Cancelled orders must be excluded. Approach Chain four tables: categories → products → order_items → orders. Filter out cancelled orders in WHERE. Group and sum. Use COUNT(DISTINCT o.id) for order count - without DISTINCT, you'd count line items instead of orders. SELECT cat.name AS category, SUM(oi.quantity * oi.unit_price) AS revenue_inr, COUNT(DISTINCT o.id) AS orders_count, SUM(oi.quantity) AS units_sold FROM categories AS cat JOIN products AS p ON p.category_id = cat.id JOIN order_items AS oi ON oi.product_id = p.id JOIN orders AS o ON oi.order_id = o.id WHERE o.status != 'Cancelled' GROUP BY cat.id, cat.name ORDER BY revenue_inr DESC; Revenue is calculated at line item level (quantity × price per item), not order level. Electronics dominates because phones and laptops are high-value. COUNT(DISTINCT o.id) counts unique orders - without DISTINCT an order with 3 items would be counted 3 times. 06 Medium Products that have never been ordered - dead stock alert LEFT JOIN anti-join · IS NULL · NOT EXISTS · NOT IN The inventory team wants to flag products that have sat in the catalogue with zero orders - candidates for discounting or delisting. Approach Three valid patterns all produce the same result: (1) LEFT JOIN + WHERE IS NULL (anti-join), (2) NOT EXISTS subquery, (3) NOT IN subquery. The LEFT JOIN pattern is the most commonly seen in production. -- Method 1: LEFT JOIN anti-join (most common) SELECT p.name, cat.name AS category, p.price_inr, p.stock_quantity FROM products AS p JOIN categories AS cat ON cat.id = p.category_id LEFT JOIN order_items AS oi ON oi.product_id = p.id WHERE oi.id IS NULL ORDER BY cat.name, p.name; -- Method 2: NOT EXISTS (safest with NULLs) SELECT p.name, p.price_inr FROM products AS p WHERE NOT EXISTS ( SELECT 1 FROM order_items oi WHERE oi.product_id = p.id ); -- Method 3: NOT IN (add NULL guard!) SELECT name, price_inr FROM products WHERE id NOT IN ( SELECT product_id FROM order_items WHERE product_id IS NOT NULL -- critical ); Why NOT IN needs the NULL guard If SELECT product_id FROM order_items returns even one NULL, the entire NOT IN returns zero rows - a silent bug. NOT EXISTS handles NULLs correctly by design and is generally the safer choice. The LEFT JOIN approach works because a product with no order_items gets NULL on the right side of the join. WHERE oi.id IS NULL keeps only those unmatched rows. It is called an anti-join - finding rows in A that don't exist in B. 07 Medium Top 5 customers by total spend JOIN · GROUP BY · SUM · LIMIT Marketing wants to identify the highest-value customers for a loyalty programme - top 5 by total spend on non-cancelled orders, with their city and state. Approach Join the full chain from customers through to order_items, filter out cancelled orders, group by customer, sum the revenue, and take the top 5 with ORDER BY + LIMIT. SELECT c.name AS customer, c.city, c.state, COUNT(DISTINCT o.id) AS orders_placed, SUM(oi.quantity * oi.unit_price) AS total_spent_inr FROM customers AS c JOIN orders AS o ON o.customer_id = c.id JOIN order_items AS oi ON oi.order_id = o.id WHERE o.status != 'Cancelled' GROUP BY c.id, c.name, c.city, c.state ORDER BY total_spent_inr DESC LIMIT 5; Customers with no orders don't appear - INNER JOIN excludes them. If you want all customers including those with zero spend, switch to LEFT JOINs and use COALESCE(SUM(...), 0). The top customer's spend should be well above ₹40,000 - likely a phone or laptop order. 08 Medium Customers who placed more than one order - repeat buyers GROUP BY · HAVING · COUNT DISTINCT The retention team wants to identify repeat buyers - customers who have placed at least 2 orders - as the foundation for a loyalty programme analysis. Approach Join customers → orders → order_items, group by customer, then use HAVING COUNT(DISTINCT o.id) \u003e 1. The HAVING clause filters after grouping - you cannot use WHERE here because COUNT doesn't exist until after aggregation. SELECT c.name AS customer, c.city, COUNT(DISTINCT o.id) AS total_orders, SUM(oi.quantity * oi.unit_price) AS total_spent_inr FROM customers AS c JOIN orders AS o ON o.customer_id = c.id JOIN order_items AS oi ON oi.order_id = o.id WHERE o.status != 'Cancelled' GROUP BY c.id, c.name, c.city HAVING COUNT(DISTINCT o.id) \u003e 1 ORDER BY total_orders DESC, total_spent_inr DESC; Common mistake Using COUNT(o.id) instead of COUNT(DISTINCT o.id) counts order items, not orders. A single order with 3 items would count as 3 - incorrectly showing the customer as a repeat buyer. WHERE runs before grouping, HAVING runs after. This is part of SQL's execution order: FROM → JOIN → WHERE → GROUP BY → HAVING → SELECT → ORDER BY → LIMIT. Any condition on an aggregate must go in HAVING. Hard CTEs · window functions · nested aggregation · full report queries 09 Hard Best-selling product in each category CTE · RANK() OVER PARTITION BY · window function Merchandising wants the single best-selling product (by units sold) in each category - to feature it in a homepage banner. If two products tie, both should appear. Approach Two-step: first aggregate sales per product per category in a CTE, then apply RANK() OVER (PARTITION BY category ORDER BY units_sold DESC). Filter WHERE rnk = 1 in the outer query. RANK handles ties - ROW_NUMBER would arbitrarily pick one. WITH product_sales AS ( SELECT cat.id AS cat_id, cat.name AS category, p.name AS product, SUM(oi.quantity) AS units_sold, SUM(oi.quantity * oi.unit_price) AS revenue_inr, RANK() OVER ( PARTITION BY cat.id ORDER BY SUM(oi.quantity) DESC ) AS rnk FROM categories AS cat JOIN products AS p ON p.category_id = cat.id JOIN order_items AS oi ON oi.product_id = p.id JOIN orders AS o ON oi.order_id = o.id WHERE o.status != 'Cancelled' GROUP BY cat.id, cat.name, p.id, p.name ) SELECT category, product, units_sold, revenue_inr FROM product_sales WHERE rnk = 1 ORDER BY category; You cannot filter on a window function in the same SELECT - WHERE RANK() OVER (...) = 1 is invalid. The CTE (or subquery) materialises the ranked result first, then the outer query can filter on rnk. This two-step pattern appears constantly in analytics work. 10 Hard Monthly revenue with running cumulative total DATE_TRUNC · GROUP BY · SUM OVER · running total Finance wants a month-by-month revenue table with a running cumulative total - so they can see at a glance when the business crossed key revenue milestones across the year. Approach Group by month to get monthly revenue, then apply SUM(monthly_revenue) OVER (ORDER BY month) for the running total. The tricky part: you cannot directly nest SUM(SUM(...)) - use a CTE or SUM(SUM()) OVER which is valid in one pass. SELECT EXTRACT(MONTH FROM o.order_date) AS month, TO_CHAR(o.order_date, 'Mon YYYY') AS month_label, SUM(oi.quantity * oi.unit_price) AS monthly_revenue, SUM(SUM(oi.quantity * oi.unit_price)) OVER (ORDER BY EXTRACT(MONTH FROM o.order_date)) AS running_total FROM orders AS o JOIN order_items AS oi ON oi.order_id = o.id WHERE o.status != 'Cancelled' GROUP BY month, month_label ORDER BY month; Why SUM(SUM(...)) OVER works The inner SUM(oi.quantity * oi.unit_price) is evaluated per GROUP BY group first, producing one revenue number per month. The outer SUM(...) OVER (ORDER BY month) then accumulates those monthly totals in date order - a two-level aggregation in one query. The final row's running_total equals the gross revenue across all non-cancelled orders. Verify it against SELECT SUM(quantity * unit_price) FROM order_items JOIN orders... WHERE status != 'Cancelled' - they should match exactly. 11 Hard Average order value by payment method - using a CTE CTE · nested aggregation · AVG · ROUND The payments team wants to know if customers who pay by card tend to spend more per order than UPI or COD customers - average order value broken down by payment method. Approach You cannot write AVG(SUM(...)) directly - aggregate functions can't be nested. Use a CTE to compute each order's total first, then AVG those totals in the outer query. This is a very common two-step pattern. WITH order_totals AS ( SELECT o.id AS order_id, o.payment_method, SUM(oi.quantity * oi.unit_price) AS order_total FROM orders AS o JOIN order_items AS oi ON oi.order_id = o.id WHERE o.status != 'Cancelled' GROUP BY o.id, o.payment_method ) SELECT payment_method, COUNT(*) AS order_count, ROUND(AVG(order_total), 2) AS avg_order_value_inr, ROUND(MIN(order_total), 2) AS min_order, ROUND(MAX(order_total), 2) AS max_order FROM order_totals GROUP BY payment_method ORDER BY avg_order_value_inr DESC; The CTE order_totals runs as a named subquery - you can test it independently before adding the outer aggregation. This is the main reason CTEs exist: making complex two-step queries readable and debuggable one step at a time. 12 Hard Full customer report - spend, tier, and order breakdown in one query CTE · LEFT JOIN · FILTER · CASE WHEN · COALESCE · ROUND The CEO wants one table: every customer, how many orders they placed, how many were delivered vs cancelled, their total spend, average order value, last order date, and a tier label (Platinum / Gold / Silver / No Orders). Approach Use a CTE with LEFT JOINs (so all 20 customers appear, even those with no orders). Aggregate everything in the CTE, then apply CASE WHEN for tier in the outer SELECT. Separate the aggregation step from the classification step - it makes both readable. WITH customer_stats AS ( SELECT c.id, c.name, c.city, c.state, COUNT(DISTINCT o.id) AS total_orders, COUNT(DISTINCT o.id) FILTER (WHERE o.status = 'Delivered') AS delivered, COUNT(DISTINCT o.id) FILTER (WHERE o.status = 'Cancelled') AS cancelled, COALESCE( SUM(oi.quantity * oi.unit_price) FILTER (WHERE o.status != 'Cancelled'), 0 ) AS total_spent, MAX(o.order_date) AS last_order_date FROM customers AS c LEFT JOIN orders AS o ON o.customer_id = c.id LEFT JOIN order_items AS oi ON oi.order_id = o.id GROUP BY c.id, c.name, c.city, c.state ) SELECT name, city, state, total_orders, delivered, cancelled, total_spent AS total_spent_inr, CASE WHEN total_orders \u003e 0 THEN ROUND(total_spent / total_orders, 0) ELSE 0 END AS avg_order_value, last_order_date, CASE WHEN total_spent \u003e 50000 THEN '💎 Platinum' WHEN total_spent \u003e= 10000 THEN '🥇 Gold' WHEN total_spent \u003e 0 THEN '🥈 Silver' ELSE ' - No Orders' END AS tier FROM customer_stats ORDER BY total_spent DESC; All 20 customers appear because of LEFT JOINs. COALESCE(..., 0) converts NULL revenue (customers with no orders) to 0. The division-by-zero guard in CASE WHEN is essential - without it, customers with zero orders would cause a divide-by-zero error when calculating average order value. What this practice set covers EasyGROUP BY, COUNT, EXTRACT, CASE WHEN, FILTER pivot, IS NULL, COALESCE Medium4-table INNER JOIN, LEFT JOIN anti-join, HAVING, COUNT DISTINCT, revenue pattern HardCTEs, RANK() OVER PARTITION BY, SUM OVER running total, nested aggregation, full report query Next stepTry the same questions on a real Kaggle dataset - Zomato Bangalore or Indian Superstore Sales. Downloadshopdb_setup.sql ↓ ","permalink":"https://blogs.codeverra.com/blog/sql-practice-shopdb/","summary":"## Practice SQL questions on the shopping dataset. \n\n\u003c!DOCTYPE html\u003e\n\u003chtml lang=\"en\"\u003e\n\u003chead\u003e\n\u003cmeta charset=\"UTF-8\"\u003e\n\u003cmeta name=\"viewport\" content=\"width=device-width, initial-scale=1.0\"\u003e\n\u003ctitle\u003eSQL Practice: ShopDB  -  Codeverra\u003c/title\u003e\n\u003clink href=\"https://fonts.googleapis.com/css2?family=Syne:wght@700;800\u0026family=JetBrains+Mono:wght@400;500\u0026family=DM+Sans:wght@300;400;500\u0026display=swap\" rel=\"stylesheet\"\u003e\n\u003cstyle\u003e\n  :root {\n    --bg: #0c0f1a;\n    --surface: #121628;\n    --surface2: #181d2e;\n    --border: #1e2438;\n    --accent: #f97316;\n    --accent2: #3b82f6;\n    --accent3: #10b981;\n    --text: #e2e8f0;\n    --muted: #64748b;\n    --code-bg: #080b14;\n    --easy: #10b981;\n    --easy-bg: rgba(16,185,129,0.1);\n    --medium: #f59e0b;\n    --medium-bg: rgba(245,158,11,0.1);\n    --hard: #ef4444;\n    --hard-bg: rgba(239,68,68,0.1);\n  }\n\n  * { margin: 0; padding: 0; box-sizing: border-box; }\n\n  body {\n    background: var(--bg);\n    color: var(--text);\n    font-family: 'DM Sans', sans-serif;\n    font-weight: 300;\n    line-height: 1.75;\n    padding-bottom: 80px;\n  }\n\n  /* HERO */\n  .hero {\n    padding: 72px 40px 60px;\n    border-bottom: 1px solid var(--border);\n    position: relative;\n    background: linear-gradient(135deg, #0c0f1a 0%, #111827 100%);\n    overflow: hidden;\n  }\n  .hero::after {\n    content: '🛒';\n    position: absolute;\n    right: 60px;\n    top: 50%;\n    transform: translateY(-50%);\n    font-size: 120px;\n    opacity: 0.07;\n  }\n  .hero-tag {\n    display: inline-flex;\n    align-items: center;\n    gap: 8px;\n    font-family: 'JetBrains Mono', monospace;\n    font-size: 11px;\n    letter-spacing: 2px;\n    text-transform: uppercase;\n    color: var(--accent);\n    margin-bottom: 20px;\n    border: 1px solid rgba(249,115,22,0.3);\n    padding: 4px 12px;\n  }\n  .hero h1 {\n    font-family: 'Syne', sans-serif;\n    font-size: clamp(36px, 6vw, 68px);\n    font-weight: 800;\n    line-height: 1.05;\n    color: #fff;\n    max-width: 680px;\n    margin-bottom: 20px;\n  }\n  .hero h1 span { color: var(--accent); }\n  .hero p {\n    font-size: 15px;\n    color: var(--muted);\n    max-width: 560px;\n    line-height: 1.75;\n  }\n\n  /* SCHEMA */\n  .schema-section {\n    max-width: 900px;\n    margin: 48px auto;\n    padding: 0 40px;\n  }\n  .schema-grid {\n    display: grid;\n    grid-template-columns: repeat(auto-fit, minmax(260px, 1fr));\n    gap: 12px;\n  }\n  .schema-table {\n    background: var(--code-bg);\n    border: 1px solid var(--border);\n    padding: 20px;\n  }\n  .schema-table-name {\n    font-family: 'JetBrains Mono', monospace;\n    font-size: 13px;\n    font-weight: 500;\n    color: #e06c75;\n    margin-bottom: 10px;\n    padding-bottom: 8px;\n    border-bottom: 1px solid var(--border);\n  }\n  .schema-row {\n    display: flex;\n    gap: 10px;\n    font-family: 'JetBrains Mono', monospace;\n    font-size: 11.5px;\n    line-height: 2;\n  }\n  .schema-col { color: #98c379; }\n  .schema-type { color: #d19a66; }\n  .schema-note { color: #5c6370; font-style: italic; font-size: 10px; }\n\n  .section-label {\n    font-family: 'JetBrains Mono', monospace;\n    font-size: 11px;\n    letter-spacing: 2px;\n    text-transform: uppercase;\n    color: var(--muted);\n    margin-bottom: 16px;\n  }\n\n  /* SETUP */\n  .setup-banner {\n    max-width: 900px;\n    margin: 0 auto 48px;\n    padding: 0 40px;\n  }\n  .setup-box {\n    background: var(--surface2);\n    border: 1px solid var(--border);\n    border-left: 3px solid var(--accent);\n    padding: 16px 22px;\n    display: flex;\n    gap: 14px;\n    align-items: center;\n  }\n  .setup-box .icon { font-size: 20px; }\n  .setup-box p { font-size: 13.5px; color: var(--muted); }\n  .setup-box a { color: var(--accent); font-family: 'JetBrains Mono', monospace; font-size: 12px; text-decoration: underline; }\n\n  /* LEGEND */\n  .legend {\n    max-width: 900px;\n    margin: 0 auto 40px;\n    padding: 0 40px;\n    display: flex;\n    gap: 28px;\n    flex-wrap: wrap;\n  }\n  .legend-item {\n    display: flex;\n    align-items: center;\n    gap: 8px;\n    font-size: 12px;\n    font-family: 'JetBrains Mono', monospace;\n    color: var(--muted);\n  }\n  .legend-dot { width: 8px; height: 8px; border-radius: 50%; }\n\n  /* DIFFICULTY HEADER */\n  .diff-header {\n    max-width: 900px;\n    margin: 0 auto 28px;\n    padding: 0 40px;\n  }\n  .diff-strip {\n    display: flex;\n    align-items: center;\n    gap: 16px;\n    padding: 14px 22px;\n    border: 1px solid;\n  }\n  .diff-strip.easy { border-color: var(--easy); background: var(--easy-bg); }\n  .diff-strip.medium { border-color: var(--medium); background: var(--medium-bg); }\n  .diff-strip.hard { border-color: var(--hard); background: var(--hard-bg); }\n  .diff-title {\n    font-family: 'Syne', sans-serif;\n    font-size: 20px;\n    font-weight: 800;\n  }\n  .easy .diff-title { color: var(--easy); }\n  .medium .diff-title { color: var(--medium); }\n  .hard .diff-title { color: var(--hard); }\n  .diff-desc { font-size: 13px; color: var(--muted); }\n\n  /* QUESTION CARD */\n  .q-wrap {\n    max-width: 900px;\n    margin: 0 auto;\n    padding: 0 40px;\n  }\n  .q-card {\n    background: var(--surface);\n    border: 1px solid var(--border);\n    margin-bottom: 28px;\n    transition: border-color 0.2s;\n  }\n  .q-card:hover { border-color: #2d3548; }\n  .q-card.easy:hover { border-color: rgba(16,185,129,0.3); }\n  .q-card.medium:hover { border-color: rgba(245,158,11,0.3); }\n  .q-card.hard:hover { border-color: rgba(239,68,68,0.3); }\n\n  .q-top {\n    padding: 24px 28px 0;\n    display: flex;\n    gap: 18px;\n  }\n  .q-number {\n    font-family: 'Syne', sans-serif;\n    font-size: 40px;\n    font-weight: 800;\n    line-height: 1;\n    color: var(--border);\n    min-width: 50px;\n    transition: color 0.2s;\n  }\n  .q-card.easy:hover .q-number { color: rgba(16,185,129,0.3); }\n  .q-card.medium:hover .q-number { color: rgba(245,158,11,0.3); }\n  .q-card.hard:hover .q-number { color: rgba(239,68,68,0.3); }\n\n  .q-info { flex: 1; }\n  .q-pill {\n    display: inline-block;\n    font-family: 'JetBrains Mono', monospace;\n    font-size: 9px;\n    font-weight: 500;\n    letter-spacing: 2px;\n    text-transform: uppercase;\n    padding: 3px 10px;\n    margin-bottom: 10px;\n    border: 1px solid;\n  }\n  .easy .q-pill { color: var(--easy); border-color: var(--easy); }\n  .medium .q-pill { color: var(--medium); border-color: var(--medium); }\n  .hard .q-pill { color: var(--hard); border-color: var(--hard); }\n\n  .q-title {\n    font-family: 'Syne', sans-serif;\n    font-size: 17px;\n    font-weight: 700;\n    color: #fff;\n    line-height: 1.4;\n    margin-bottom: 6px;\n  }\n  .q-tags {\n    font-family: 'JetBrains Mono', monospace;\n    font-size: 10px;\n    color: var(--muted);\n    letter-spacing: 0.5px;\n  }\n\n  .q-body { padding: 18px 28px 28px; }\n\n  .scenario {\n    background: var(--surface2);\n    border-left: 2px solid var(--border);\n    padding: 12px 16px;\n    font-size: 13.5px;\n    color: var(--muted);\n    margin-bottom: 18px;\n    line-height: 1.7;\n  }\n  .scenario strong { color: var(--text); font-weight: 500; }\n\n  .label {\n    font-family: 'JetBrains Mono', monospace;\n    font-size: 9px;\n    letter-spacing: 2px;\n    text-transform: uppercase;\n    margin-bottom: 6px;\n    margin-top: 16px;\n  }\n  .label.approach { color: var(--accent3); }\n  .label.alt { color: var(--accent2); }\n  .label.explain { color: var(--muted); }\n\n  .approach-box {\n    background: rgba(16,185,129,0.08);\n    border-left: 2px solid var(--accent3);\n    padding: 10px 14px;\n    font-size: 13.5px;\n    color: var(--text);\n    line-height: 1.65;\n    margin-bottom: 14px;\n  }\n  .approach-box code {\n    font-family: 'JetBrains Mono', monospace;\n    font-size: 11.5px;\n    background: rgba(16,185,129,0.15);\n    color: var(--accent3);\n    padding: 1px 5px;\n  }\n\n  .alt-box {\n    background: rgba(59,130,246,0.08);\n    border-left: 2px solid var(--accent2);\n    padding: 10px 14px;\n    font-size: 13px;\n    color: #94a3b8;\n    line-height: 1.65;\n    margin-bottom: 10px;\n  }\n  .alt-box code {\n    font-family: 'JetBrains Mono', monospace;\n    font-size: 11px;\n    background: rgba(59,130,246,0.15);\n    color: var(--accent2);\n    padding: 1px 5px;\n  }\n\n  pre.sql {\n    background: var(--code-bg);\n    border: 1px solid var(--border);\n    padding: 18px 22px;\n    overflow-x: auto;\n    font-family: 'JetBrains Mono', monospace;\n    font-size: 12.5px;\n    line-height: 1.85;\n    color: #abb2bf;\n    margin-bottom: 8px;\n  }\n  .kw { color: #c678dd; }\n  .fn { color: #61afef; }\n  .str { color: #98c379; }\n  .num { color: #d19a66; }\n  .cm { color: #5c6370; font-style: italic; }\n  .tbl { color: #e06c75; }\n\n  .explain-box {\n    border-top: 1px solid var(--border);\n    padding-top: 14px;\n    font-size: 13px;\n    color: var(--muted);\n    line-height: 1.8;\n    margin-top: 14px;\n  }\n  .explain-box code {\n    font-family: 'JetBrains Mono', monospace;\n    font-size: 11px;\n    background: rgba(255,255,255,0.05);\n    padding: 1px 5px;\n    color: #e2e8f0;\n  }\n  .explain-box strong { color: var(--text); font-weight: 500; }\n\n  .divider {\n    max-width: 900px;\n    margin: 16px auto 36px;\n    padding: 0 40px;\n    border: none;\n    border-top: 1px solid var(--border);\n  }\n\n  /* FOOTER */\n  .footer-card {\n    max-width: 900px;\n    margin: 48px auto 0;\n    padding: 0 40px;\n  }\n  .footer-inner {\n    background: var(--surface2);\n    border: 1px solid var(--border);\n    border-top: 3px solid var(--accent);\n    padding: 36px;\n  }\n  .footer-inner h2 {\n    font-family: 'Syne', sans-serif;\n    font-size: 22px;\n    font-weight: 800;\n    color: #fff;\n    margin-bottom: 20px;\n  }\n  .concept-grid {\n    display: grid;\n    grid-template-columns: repeat(auto-fit, minmax(180px, 1fr));\n    gap: 10px;\n  }\n  .concept-item {\n    background: var(--surface);\n    border: 1px solid var(--border);\n    padding: 12px 14px;\n    font-size: 12px;\n    color: var(--muted);\n    line-height: 1.6;\n  }\n  .concept-item strong {\n    display: block;\n    font-family: 'JetBrains Mono', monospace;\n    font-size: 9px;\n    letter-spacing: 1.5px;\n    text-transform: uppercase;\n    color: var(--accent);\n    margin-bottom: 5px;\n  }\n\n  @media (max-width: 640px) {\n    .hero, .schema-section, .setup-banner, .legend, .diff-header, .q-wrap, .footer-card { padding-left: 20px; padding-right: 20px; }\n    .hero::after { display: none; }\n    .q-number { font-size: 30px; min-width: 38px; }\n  }\n\u003c/style\u003e\n\u003c/head\u003e\n\u003cbody\u003e\n\n\u003cdiv class=\"hero\"\u003e\n  \u003cdiv class=\"hero-tag\"\u003eCodeverra · SQL Practice\u003c/div\u003e\n  \u003ch1\u003eSQL Practice Set\u003cbr\u003e\u003cspan\u003eShopDB\u003c/span\u003e\u003c/h1\u003e\n  \u003cp\u003e12 questions on an Indian e-commerce dataset  -  customers, products, orders, and line items. Easy to hard. Every question includes the business context, a one-line approach, a clean query, and alternatives where they exist.\u003c/p\u003e","tags":["sql","data-analysis"],"title":"SQL Practice on Shopping Dataset: SQL Questions with Solutions"},{"content":"Data Structures and Algorithms (DSA) are often treated as a necessary evil, something you grind through only to clear technical interviews. This narrow framing has done more harm than good. It leads to shallow learning, memorized solutions, and frustration when problems slightly change.\nThe truth is simple: DSA is not just about interviews. It is about learning how to think clearly, reason about efficiency, and design solutions that scale.\nThis blog lays out:\nWhy DSA matters beyond interviews\nHow to approach DSA learning the right way A structured roadmap of topics and patterns Practical guidance on how to practice effectively This is not a problem list. It is a thinking roadmap. Why Study DSA Beyond Interviews 1. DSA Builds Strong Problem-Solving Skills At its core, DSA teaches you how to:\nBreak a complex problem into smaller parts Identify constraints and edge cases Choose the right representation for data Reduce unnecessary complexity Good data structures simplify logic. Poor choices complicate even simple tasks. This habit of structured thinking carries into every area of software engineering.\n2. DSA Helps You Write Efficient and Scalable Code Efficiency is not about premature optimization. It is about avoiding obvious inefficiencies.\nUnderstanding algorithms helps you:\nRecognize when a solution will not scale Make informed trade-offs between time and space Avoid brute-force approaches that fail on large inputs Even with powerful hardware and high-level languages, algorithmic thinking still matters.\n3. DSA Improves Debugging and Code Reviews When you understand how an algorithm works internally:\nYou can mentally trace execution You spot logical flaws faster You understand why certain edge cases fail This makes debugging less about trial and error and more about reasoning.\n4. DSA Is the Foundation for Advanced Topics Many advanced concepts are built directly on DSA fundamentals:\nDSA Concept\tReal-World Applications Trees\tDatabases, file systems, indexing Graphs\tNetworks, dependencies, schedulers Heaps\tPriority queues, streaming systems Dynamic Programming\tOptimization problems Hashing\tCaching, fast lookups\nDSA is not isolated knowledge. It is the foundation layer.\nHow Most People Learn DSA (And Why It Fails) Common mistakes: Solving random problems without a plan Jumping to hard problems too early Memorizing solutions instead of understanding patterns Ignoring time and space complexity Skipping recursion and fundamentals This approach creates the illusion of progress but fails under pressure. A structured roadmap fixes this.\nHow to Start Learning DSA the Right Way Before starting DSA, you should be comfortable with:\nOne programming language (Python is a great choice) Variables, loops, conditionals Functions and basic recursion Language confidence matters more than rushing into algorithms. Complete DSA Learning Roadmap (Structured) 1. Core Foundations (Mandatory Before Everything Else) Area Topics to Cover Why It Matters Programming Basics Variables, loops, conditionals, functions DSA logic fails without language fluency Problem Constraints Input size, limits, edge cases Drives algorithm choice Time Complexity Big-O, best/average/worst case Prevents non-scalable solutions Space Complexity Auxiliary vs input space Important for memory-heavy problems Recursion Basics Call stack, base case, recursive flow Foundation for trees, backtracking, DP Iteration vs Recursion When to use which Improves code clarity and performance Dry Runs Manual tracing Builds debugging and reasoning skills 2. Linear Data Structures Data Structure Concepts Key Skills Developed Arrays Indexing, traversal, resizing, in-place ops Memory awareness, iteration logic Strings Character access, substrings, immutability Text processing, pattern matching Linked Lists Pointers, insertion, deletion, reversal Pointer manipulation Stacks LIFO, recursion simulation Expression parsing, undo logic Queues FIFO, circular queues Scheduling, buffering 3. Searching, Sorting, and Basic Techniques Area Topics Why Important Searching Linear search, binary search Fundamental optimization Sorting Selection, insertion, merge, quick Comparison-based thinking Binary Search Variants First/last occurrence, search on answer Logarithmic problem solving 3: Hashing and Prefix Techniques Technique Topics Problem-Solving Value Hash Maps Key-value storage, collisions Constant-time lookups Hash Sets Uniqueness, membership checks De-duplication Frequency Counting Counting patterns Many array/string problems Prefix Sums Range queries Converts O(n²) → O(n) 4: Two Pointer \u0026amp; Sliding Window Patterns Pattern Applied On Why It Matters Two Pointers Arrays, strings Reduces nested loops Sliding Window Subarrays, substrings Efficient range processing Fast–Slow Pointers Linked lists Cycle detection 5. Recursion and Backtracking Concept Topics Core Learning Recursion Design Base case, recursive case Problem decomposition Backtracking Subsets, permutations Decision tree traversal Constraint-Based Recursion Pruning paths Optimization 6. Trees Tree Type Concepts Skills Developed Binary Trees DFS, BFS, height, depth Recursive reasoning Binary Search Trees Insert, delete, search Ordered data handling Tree Properties Diameter, balance Structural analysis 7. Heaps and Greedy Algorithms Area Topics Application Heaps Min heap, max heap Priority-based logic Priority Queues Scheduling Real-time systems Greedy Strategy Local vs global optimum Optimization intuition 8. Graphs Graph Topic Concepts Real-World Modeling Graph Representation Adjacency list/matrix Data modeling Traversals BFS, DFS Reachability Topological Sort Kahn’s, DFS-based Dependency resolution Shortest Path Dijkstra, BFS Routing, latency MST Prim’s, Kruskal’s Network optimization 9. Dynamic Programming Area Topics Why It’s Critical Problem Identification Overlapping subproblems When DP applies State Definition What uniquely defines a subproblem Core DP skill Transitions State relationships Logic formulation Approaches Top-down, bottom-up Implementation clarity Optimization Space optimization Performance tuning 10: Advanced Patterns Pattern Use Case Bit Manipulation Low-level optimization Union-Find Connectivity problems Monotonic Stack Range queries Trie Prefix-based searching Segment Tree / Fenwick Tree Range updates and queries Learning playlists Here are some curated playlists you can follow:\nPython \u0026amp; DSA Leetcode with Python in Hindi #100DaysOfLeetcode More structured posts and learning paths coming soon.\n","permalink":"https://blogs.codeverra.com/blog/dsa_roadmap/","summary":"\u003cp\u003eData Structures and Algorithms (DSA) are often treated as a necessary evil, something you grind through only to clear technical interviews. This narrow framing has done more harm than good. It leads to shallow learning, memorized solutions, and frustration when problems slightly change.\u003c/p\u003e\n\u003cp\u003eThe truth is simple: DSA is not just about interviews. It is about learning how to think clearly, reason about efficiency, and design solutions that scale.\u003c/p\u003e\n\u003cp\u003eThis blog lays out:\u003c/p\u003e","tags":["dsa","learning-roadmap"],"title":"A Practical DSA Learning Roadmap"},{"content":"Time complexity is one of the most important concepts in Data Structures and Algorithms. It helps you understand how an algorithm behaves as the input size grows and why some solutions fail to scale.\nThis guide explains time complexity step-by-step using simple explanations and Python examples written purely as text, focusing on intuition rather than heavy mathematics.\n1. What Is Time Complexity? Time complexity measures how the running time of an algorithm grows as the input size increases.\nIt does not measure actual seconds taken by the program.\nInstead, it focuses on how fast execution grows relative to input size n.\nThe focus is on growth rate, not machine speed.\n2. Why Time Complexity Matters Time complexity helps you:\nCompare different algorithms Predict performance for large inputs Perform better in coding interviews and competitive programming Avoid inefficient solutions early Efficient code is essential for scalable real-world systems.\n3. Asymptotic Notations Big O (O) Represents the worst-case time complexity.\nIt is the most commonly used notation in interviews.\nBig Ω (Omega) Represents the best-case time complexity.\nBig Θ (Theta) Represents the average-case time complexity.\n4. Common Time Complexities (Overview) O(1) – Constant\nO(log n) – Logarithmic\nO(n) – Linear\nO(n log n) – Linearithmic\nO(n²) – Quadratic\nO(2ⁿ) – Exponential\n5. O(1) - Constant Time Explanation The execution time does not depend on the input size.\nPython Example (Text Only) A function that returns the first element of an array always runs in constant time, regardless of how large the array is.\nFor example:\ndef get_first_element(arr): return arr[0] This is the fastest possible time complexity.\n6. O(log n) - Logarithmic Time Explanation The input size is reduced by half in every step.\nPython Example (Binary Search – Text Only) Binary search repeatedly divides the search space into two halves.\nExample logic:\nInitialize low and high pointers.\nFind the middle element.\nCompare the middle element with the target.\nDiscard half of the array in each step.\nBecause the input is halved every time, the complexity is O(log n).\n7. O(n) - Linear Time Explanation Execution time grows directly proportional to input size.\nPython Example (Text Only) Finding the maximum element in an array requires checking every element once.\nExample logic:\nInitialize maximum with the first element.\nTraverse the array.\nUpdate maximum whenever a larger value is found.\nIf the input size doubles, execution time also doubles.\n8. O(n log n) - Linearithmic Time Explanation This complexity appears in efficient sorting algorithms.\nPython Example (Merge Sort Idea – Text Only) Merge sort divides the array into halves recursively and then merges them.\nDivide step takes log n time.\nMerge step takes n time.\nOverall complexity becomes O(n log n), which is optimal for sorting.\n9. O(n²) - Quadratic Time Explanation Occurs when nested loops are used over the input.\nPython Example (Text Only) Printing all pairs of elements in an array requires two loops.\nExample logic:\nFor each element i in the array,\nloop again through the array for j,\nprint the pair (i, j).\nIf input size doubles, execution time becomes four times slower.\n10. O(2ⁿ) - Exponential Time Explanation Each input creates multiple recursive calls.\nPython Example (Naive Fibonacci – Text Only) The naive recursive Fibonacci function calls itself twice for each value of n.\nThis leads to exponential growth in function calls.\nSuch solutions are extremely slow for large inputs and should be avoided unless optimized using Dynamic Programming.\n11. Loop-Based Complexity Analysis Single Loop A loop that runs from 1 to n executes n times.\nTime complexity: O(n)\nNested Loop A loop inside another loop, each running n times.\nTime complexity: O(n²)\nLoop with Halving A loop where the value is divided by 2 in every iteration.\nTime complexity: O(log n)\n12. Combining Complexities Example If one loop runs n times and another independent loop also runs n times:\nTotal time = O(n + n)\nFinal complexity = O(n)\nDominant Term Rule O(n² + n + 1) simplifies to O(n²)\nAlways keep the term with the highest growth rate.\n13. Best, Average, and Worst Case Linear Search Example (Text Only) Best case: Target found at the first position → O(1)\nAverage case: Target found somewhere in the array → O(n)\nWorst case: Target not present or at last position → O(n)\n14. Space vs Time Tradeoff Sometimes extra memory is used to reduce time complexity.\nExamples include:\nUsing hash sets to reduce lookup time Using dynamic programming arrays to avoid recomputation This tradeoff is very common in real-world applications.\nConclusion Understanding time complexity helps you:\nWrite efficient and scalable code Choose the right algorithm Crack coding interviews Build real-world systems Mastering time complexity is non-negotiable for anyone serious about Data Structures and Algorithms.\n","permalink":"https://blogs.codeverra.com/blog/time_complexity/","summary":"\u003cp\u003eTime complexity is one of the most important concepts in Data Structures and Algorithms. It helps you understand how an algorithm behaves as the input size grows and why some solutions fail to scale.\u003c/p\u003e\n\u003cp\u003eThis guide explains time complexity step-by-step using simple explanations and Python examples written purely as text, focusing on intuition rather than heavy mathematics.\u003c/p\u003e\n\u003chr\u003e\n\u003ch2 id=\"1-what-is-time-complexity\"\u003e1. What Is Time Complexity?\u003c/h2\u003e\n\u003cp\u003eTime complexity measures how the running time of an algorithm grows as the input size increases.\u003c/p\u003e","tags":["dsa","time-complexity"],"title":"Analysis of Time Complexity Explained with Python Examples"},{"content":"Array patterns are recurring problem-solving techniques used to solve a wide variety of array problems efficiently.\nInstead of memorizing individual solutions, understanding these patterns helps you recognize how to approach a problem and design an optimal solution.\n1. Traversal Pattern Concept The traversal pattern involves visiting each element of the array once or in a fixed manner.\nWhen to Use Finding maximum or minimum elements Printing all elements Counting frequency Simple condition checks Explanation Example Traverse the array from the first element to the last, updating the required result as you move forward.\nTime Complexity O(n)\n2. Sliding Window Pattern Concept The sliding window pattern processes a fixed-size or variable-size window that slides over the array, avoiding repeated computation.\nWhen to Use Subarray sum problems Maximum or minimum in a window Longest or shortest subarray problems Explanation Example Instead of recalculating the sum for every subarray, update the window by adding the next element and removing the previous one.\nTime Complexity O(n)\n3. Two Pointer Pattern Concept The two pointer pattern uses two pointers that move through the array, either from both ends or in the same direction.\nWhen to Use Sorted arrays Pair sum problems Removing duplicates Reversing arrays Explanation Example One pointer starts at the beginning and another at the end. Based on conditions, one of the pointers is moved inward.\nTime Complexity O(n)\n4. Prefix Sum Pattern Concept The prefix sum pattern precomputes cumulative sums so that range queries can be answered efficiently.\nWhen to Use Range sum queries Subarray sum problems Performance optimization scenarios Explanation Example Store cumulative sums in an auxiliary array where each index represents the sum of elements up to that index.\nTime Complexity Preprocessing: O(n)\nQuery: O(1)\n5. Kadane’s Algorithm (Maximum Subarray) Concept Kadane’s algorithm is a Dynamic Programming technique used to find the maximum sum of a contiguous subarray.\nWhen to Use Maximum sum subarray problems Problems involving contiguous segments Explanation Example Maintain a running sum and reset it whenever it becomes worse than starting fresh from the current element.\nTime Complexity O(n)\n6. Frequency Count Pattern Concept This pattern uses extra space, such as an array or hashmap, to count the frequency of elements.\nWhen to Use Detecting duplicates Anagram checking Majority element problems Explanation Example Traverse the array and update a frequency record for each element encountered.\nTime Complexity O(n)\n7. Sorting-Based Pattern Concept Sorting the array first can simplify the logic of many problems.\nWhen to Use Pair or triplet problems Merging intervals Duplicate handling Explanation Example After sorting, elements that satisfy conditions often appear next to each other, reducing problem complexity.\nTime Complexity O(n log n)\n8. Binary Search on Arrays Concept Binary search repeatedly divides the search space into half to efficiently locate elements or answers.\nWhen to Use Sorted arrays Search problems Optimization problems such as binary search on answer Explanation Example Compare the middle element with the target and eliminate half of the array in each step.\nTime Complexity O(log n)\n9. Subarray Pattern Concept This pattern deals with problems involving continuous segments of an array.\nWhen to Use Subarray sum problems Longest or shortest subarray problems Zero-sum subarray problems Key Insight Sliding window or prefix sum techniques are often applied to solve these problems efficiently.\n10. In-Place Modification Pattern Concept The in-place modification pattern updates the array without using extra space.\nWhen to Use Removing elements Rearranging arrays Memory-constrained problems Explanation Example Use an index to overwrite unwanted elements while traversing the array.\nTime and Space Complexity Time complexity: O(n)\nSpace complexity: O(1)\nSummary of Array Patterns Traversal Pattern\nUse case: Basic operations\nSliding Window Pattern\nUse case: Subarrays\nTwo Pointer Pattern\nUse case: Sorted arrays\nPrefix Sum Pattern\nUse case: Range queries\nKadane’s Algorithm\nUse case: Maximum subarray\nFrequency Count Pattern\nUse case: Duplicates and counting\nSorting-Based Pattern\nUse case: Simplifying logic\nBinary Search Pattern\nUse case: Fast searching\nSubarray Pattern\nUse case: Contiguous segments\nIn-Place Modification Pattern\nUse case: Space optimization\nKey Learning Advice Identify array constraints Check whether the array is sorted Look for subarray or pair conditions Try to reduce nested loops Think in terms of patterns, not individual problems Conclusion Mastering array patterns significantly improves your problem-solving ability.\nOnce you recognize the underlying pattern, designing an efficient solution becomes much easier and faster.\n","permalink":"https://blogs.codeverra.com/blog/array_patterns/","summary":"\u003cp\u003eArray patterns are recurring problem-solving techniques used to solve a wide variety of array problems efficiently.\u003c/p\u003e\n\u003cp\u003eInstead of memorizing individual solutions, understanding these patterns helps you recognize how to approach a problem and design an optimal solution.\u003c/p\u003e\n\u003chr\u003e\n\u003ch2 id=\"1-traversal-pattern\"\u003e1. Traversal Pattern\u003c/h2\u003e\n\u003ch3 id=\"concept\"\u003eConcept\u003c/h3\u003e\n\u003cp\u003eThe traversal pattern involves visiting each element of the array once or in a fixed manner.\u003c/p\u003e\n\u003chr\u003e\n\u003ch3 id=\"when-to-use\"\u003eWhen to Use\u003c/h3\u003e\n\u003cul\u003e\n\u003cli\u003eFinding maximum or minimum elements\u003c/li\u003e\n\u003cli\u003ePrinting all elements\u003c/li\u003e\n\u003cli\u003eCounting frequency\u003c/li\u003e\n\u003cli\u003eSimple condition checks\u003c/li\u003e\n\u003c/ul\u003e\n\u003chr\u003e\n\u003ch3 id=\"explanation-example\"\u003eExplanation Example\u003c/h3\u003e\n\u003cp\u003eTraverse the array from the first element to the last, updating the required result as you move forward.\u003c/p\u003e","tags":["dsa","arrays"],"title":"Array Patterns for Problem Solving"},{"content":"Graphs and trees are non-linear data structures. To process or explore them, traversal algorithms are used.\nThe two most important traversal techniques are:\nBreadth-First Search (BFS) Depth-First Search (DFS) Both algorithms visit all nodes in a graph or tree, but their strategies and use-cases differ significantly.\n1. Breadth-First Search (BFS) Concept Breadth-First Search explores a graph level by level.\nIt visits all neighbors of a node before moving deeper into the graph.\nIn simple terms, BFS explores nodes in increasing order of distance from the starting node.\nHow BFS Works Start from a source node Visit the node and mark it as visited Visit all its immediate neighbors Then visit neighbors of neighbors Continue until all reachable nodes are visited Data Structure Used Queue\nWhy Queue Is Used BFS follows the First In, First Out (FIFO) principle.\nNodes discovered earlier are processed first, ensuring level-wise traversal.\nBFS Example (Step-by-Step) Graph structure:\nA connected to B and C\nB connected to D\nStarting from node A:\nVisit A, queue contains A Visit B and C, queue contains B and C Visit D, queue contains C and D Traversal order becomes:\nA → B → C → D\nTime Complexity Time complexity of BFS is O(V + E)\nWhere:\nV is the number of vertices E is the number of edges Each vertex and edge is processed once.\nSpace Complexity Space complexity is O(V)\nThis is because the queue and visited structure may store all vertices.\nApplications of BFS Finding shortest path in unweighted graphs Level-order traversal of trees Finding minimum number of steps Social networks (degrees of connection) Web crawling 2. Depth-First Search (DFS) Concept Depth-First Search explores a graph by going as deep as possible before backtracking.\nIn simple terms, DFS explores one complete path before trying another path.\nHow DFS Works Start from a source node Visit a neighbor Continue visiting deeper neighbors When no unvisited neighbor exists, backtrack Repeat until all nodes are visited Data Structure Used Stack\nThis can be:\nAn explicit stack Or an implicit recursion stack Why Stack Is Used DFS follows the Last In, First Out (LIFO) principle.\nThe most recently visited node is explored first.\nDFS Example (Step-by-Step) Graph structure:\nA connected to B and C\nB connected to D\nStarting from node A:\nVisit A Go to B Go to D Backtrack Visit C Traversal order becomes:\nA → B → D → C\nTime Complexity Time complexity of DFS is O(V + E)\nSame as BFS.\nSpace Complexity Space complexity is O(V)\nThis is due to the recursion stack or explicit stack used during traversal.\nApplications of DFS Cycle detection Topological sorting Finding connected components Path existence problems Solving mazes and puzzles Backtracking problems 3. Key Conceptual Differences Traversal Strategy BFS explores breadth first.\nDFS explores depth first.\nData Structure BFS uses a queue.\nDFS uses a stack or recursion.\nShortest Path BFS guarantees the shortest path in unweighted graphs.\nDFS does not guarantee the shortest path.\n4. BFS vs DFS Comparison Traversal Style\nBFS: Level-wise\nDFS: Depth-wise\nData Structure\nBFS: Queue\nDFS: Stack or recursion\nShortest Path\nBFS: Yes (for unweighted graphs)\nDFS: No\nMemory Usage\nBFS: Higher\nDFS: Lower\nImplementation\nBFS: Iterative\nDFS: Recursive or iterative\nUse Case\nBFS: Minimum distance problems\nDFS: Exhaustive search problems\n5. When to Use BFS vs DFS Use BFS When You need the shortest path Distance or level matters The graph is unweighted You are searching for the nearest solution Use DFS When You need to explore all possibilities Memory is limited Solving puzzles or backtracking problems Detecting cycles or connectivity Conclusion Breadth-First Search and Depth-First Search are foundational graph traversal algorithms.\nUnderstanding their traversal strategies, data structures, complexities, and use-cases is essential for solving graph problems efficiently in interviews and real-world applications.\n","permalink":"https://blogs.codeverra.com/blog/bfs_dfs/","summary":"\u003cp\u003eGraphs and trees are non-linear data structures. To process or explore them, traversal algorithms are used.\u003c/p\u003e\n\u003cp\u003eThe two most important traversal techniques are:\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003eBreadth-First Search (BFS)\u003c/li\u003e\n\u003cli\u003eDepth-First Search (DFS)\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003eBoth algorithms visit all nodes in a graph or tree, but their strategies and use-cases differ significantly.\u003c/p\u003e\n\u003chr\u003e\n\u003ch2 id=\"1-breadth-first-search-bfs\"\u003e1. Breadth-First Search (BFS)\u003c/h2\u003e\n\u003ch3 id=\"concept\"\u003eConcept\u003c/h3\u003e\n\u003cp\u003eBreadth-First Search explores a graph level by level.\u003c/p\u003e\n\u003cp\u003eIt visits all neighbors of a node before moving deeper into the graph.\u003c/p\u003e","tags":["dsa","bfs","dfs","graphs","traversal","algorithms"],"title":"BFS vs DFS Explained Clearly"},{"content":"Binary Search is an efficient searching algorithm used to find an element in a sorted array by repeatedly dividing the search space into half.\nUnlike linear search, which checks elements one by one, binary search eliminates half of the remaining elements at each step. This makes it significantly faster for large datasets.\nKey Requirement Binary search can only be applied when the array is already sorted.\nThe sorting order can be:\nAscending order Descending order If the array is not sorted, binary search will not work correctly.\nHow Binary Search Works The algorithm follows these steps:\nFind the middle element of the array Compare the middle element with the target value If both are equal, the search is complete If the target is smaller, search the left half If the target is larger, search the right half Repeat the process until the element is found or the search space becomes empty At each step, half of the remaining elements are discarded.\nExample Walkthrough Sorted array:\n[10, 20, 30, 40, 50, 60]\nTarget element:\n40\nStep-by-step process:\nMiddle element is 30, target is larger, so search the right half New middle element is 50, target is smaller, so search the left half Middle element becomes 40, element found This demonstrates how binary search reduces the search space efficiently.\nPython Example (Text Only) The iterative binary search approach works as follows:\nInitialize two pointers named low and high Calculate the middle index Compare the middle element with the target Update the low or high pointer accordingly Continue until the element is found or the range becomes invalid If the element is found, its index is returned.\nIf not found, the function returns -1.\nTime Complexity Best case: O(1) when the middle element is the target Average case: O(log n) Worst case: O(log n) Binary search remains efficient even for very large datasets.\nSpace Complexity Iterative binary search: O(1) because no extra space is used Recursive binary search: O(log n) due to the recursion call stack When to Use Binary Search Binary search is ideal when:\nThe data is already sorted The dataset is large Fast searching is required Multiple search operations are expected When Not to Use Binary Search Binary search should be avoided when:\nThe array is unsorted The dataset is very small Only one search is required and sorting cost is high Interview Insight Sorting an unsorted array takes O(n log n) time.\nFor a single search operation, sorting the array first and then applying binary search is inefficient.\nHowever, when multiple searches are required, sorting once and using binary search multiple times becomes a better approach.\nConclusion Binary search is one of the most important searching algorithms in computer science.\nIt provides excellent performance on sorted data and is widely used in real-world systems such as databases, indexing engines, and search services.\nUnderstanding binary search is fundamental for mastering Data Structures and Algorithms.\n","permalink":"https://blogs.codeverra.com/blog/binary_search/","summary":"\u003cp\u003eBinary Search is an efficient searching algorithm used to find an element in a sorted array by repeatedly dividing the search space into half.\u003c/p\u003e\n\u003cp\u003eUnlike linear search, which checks elements one by one, binary search eliminates half of the remaining elements at each step. This makes it significantly faster for large datasets.\u003c/p\u003e\n\u003chr\u003e\n\u003ch2 id=\"key-requirement\"\u003eKey Requirement\u003c/h2\u003e\n\u003cp\u003eBinary search can only be applied when the array is already sorted.\u003c/p\u003e\n\u003cp\u003eThe sorting order can be:\u003c/p\u003e","tags":["dsa","binary-search","searching"],"title":"Binary Search Explained Clearly"},{"content":"Python is one of the most versatile and beginner-friendly programming languages in the world. From web development and automation to data science and AI, Python forms the backbone of modern software development.\nThis syllabus is designed as a structured learning roadmap, not just a list of topics. It focuses on building strong fundamentals, clear thinking, and professional-level coding skills.\nThis guide covers:\nPython fundamentals from scratch Core programming concepts Advanced Python features Career-oriented applications Interview and professional preparation Beginner Level - Python Foundations 1. Introduction to Python What is Python History and features of Python Applications of Python Python vs other languages Installing Python Python versions Running Python programs Python interactive shell Writing your first Python program 2. Python Basics \u0026amp; Syntax Python keywords Identifiers Indentation and code structure Comments (single-line and multi-line) Variables and naming conventions Data types overview Type checking using type() Type conversion 3. Input, Output \u0026amp; Operators Input using input() Output using print() Formatted output Arithmetic operators Relational operators Logical operators Assignment operators Operator precedence 4. Control Flow Conditional Statements if if-else elif Nested conditions Looping Statements for loop while loop Loop Control break continue pass Intermediate Level - Core Python Concepts 5. Strings String creation Indexing and slicing String methods String formatting Escape characters Immutability String operations 6. Lists Creating lists Indexing and slicing List methods Nested lists Iterating through lists List comprehensions 7. Tuples Creating tuples Tuple operations Tuple unpacking Difference between list and tuple Use cases 8. Sets Set creation Set methods Set operations Difference between set and list Use cases 9. Dictionaries Dictionary creation Accessing elements Dictionary methods Iterating dictionaries Nested dictionaries Real-world use cases 10. Functions Defining functions Function parameters Return values Default arguments Keyword arguments Variable-length arguments (*args, **kwargs) Scope (local and global) Lambda functions 11. Modules \u0026amp; Packages Importing modules Built-in modules Creating custom modules Python packages __name__ == \u0026quot;__main__\u0026quot; Virtual environments Advanced Level - Professional Python 12. File Handling Reading files Writing files File modes Working with CSV files JSON handling Context managers using with 13. Exception Handling Types of errors try and except else and finally Multiple exceptions Custom exceptions Debugging techniques 14. Object-Oriented Programming (OOP) Classes and objects Constructors Instance and class variables Methods Encapsulation Inheritance Polymorphism Abstraction Magic methods (__str__, __len__, etc.) 15. Advanced Python Concepts List, set, and dictionary comprehensions Iterators and generators Decorators Closures Shallow vs deep copy Memory management Garbage collection 16. Python Standard Library math random datetime collections itertools functools os sys 17. Testing \u0026amp; Debugging Assertions Unit testing Writing test cases Testing frameworks overview Debugging tools Logging Specialized \u0026amp; Career-Oriented Topics 18. Data Structures \u0026amp; Algorithms with Python Time and space complexity Arrays and strings Stacks and queues Linked lists Trees Graphs Dynamic programming Greedy algorithms Recursion and backtracking 19. Automation \u0026amp; Scripting File and folder automation Web scraping basics Task scheduling Command-line tools Automation projects 20. Python for Web Development Backend fundamentals REST APIs JSON and HTTP Web frameworks overview Request–response cycle 21. Python for Data \u0026amp; AI NumPy basics Pandas basics Data cleaning Basic data visualization Working with datasets 22. Performance \u0026amp; Optimization Code optimization Profiling Efficient data handling Best practices PEP 8 guidelines Type hints 23. Real-World Projects Beginner mini projects Automation projects API-based projects Data-driven projects End-to-end Python applications 24. Interview \u0026amp; Professional Topics Common Python interview questions DSA problem solving Design patterns Code optimization Packaging and distribution Resume and project preparation 25. Python Programming Playlist --- More structured Python learning guides and hands-on roadmaps coming soon.\n","permalink":"https://blogs.codeverra.com/blog/python-syllabus/","summary":"\u003cp\u003ePython is one of the most versatile and beginner-friendly programming languages in the world. From web development and automation to data science and AI, Python forms the backbone of modern software development.\u003c/p\u003e\n\u003cp\u003eThis syllabus is designed as a \u003cstrong\u003estructured learning roadmap\u003c/strong\u003e, not just a list of topics. It focuses on building strong fundamentals, clear thinking, and professional-level coding skills.\u003c/p\u003e\n\u003cp\u003eThis guide covers:\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003ePython fundamentals from scratch\u003c/li\u003e\n\u003cli\u003eCore programming concepts\u003c/li\u003e\n\u003cli\u003eAdvanced Python features\u003c/li\u003e\n\u003cli\u003eCareer-oriented applications\u003c/li\u003e\n\u003cli\u003eInterview and professional preparation\u003c/li\u003e\n\u003c/ul\u003e\n\u003chr\u003e\n\u003ch2 id=\"beginner-level-----python-foundations\"\u003eBeginner Level  -  Python Foundations\u003c/h2\u003e\n\u003ch3 id=\"1-introduction-to-python\"\u003e1. Introduction to Python\u003c/h3\u003e\n\u003cul\u003e\n\u003cli\u003eWhat is Python\u003c/li\u003e\n\u003cli\u003eHistory and features of Python\u003c/li\u003e\n\u003cli\u003eApplications of Python\u003c/li\u003e\n\u003cli\u003ePython vs other languages\u003c/li\u003e\n\u003cli\u003eInstalling Python\u003c/li\u003e\n\u003cli\u003ePython versions\u003c/li\u003e\n\u003cli\u003eRunning Python programs\u003c/li\u003e\n\u003cli\u003ePython interactive shell\u003c/li\u003e\n\u003cli\u003eWriting your first Python program\u003c/li\u003e\n\u003c/ul\u003e\n\u003chr\u003e\n\u003ch3 id=\"2-python-basics--syntax\"\u003e2. Python Basics \u0026amp; Syntax\u003c/h3\u003e\n\u003cul\u003e\n\u003cli\u003ePython keywords\u003c/li\u003e\n\u003cli\u003eIdentifiers\u003c/li\u003e\n\u003cli\u003eIndentation and code structure\u003c/li\u003e\n\u003cli\u003eComments (single-line and multi-line)\u003c/li\u003e\n\u003cli\u003eVariables and naming conventions\u003c/li\u003e\n\u003cli\u003eData types overview\u003c/li\u003e\n\u003cli\u003eType checking using \u003ccode\u003etype()\u003c/code\u003e\u003c/li\u003e\n\u003cli\u003eType conversion\u003c/li\u003e\n\u003c/ul\u003e\n\u003chr\u003e\n\u003ch3 id=\"3-input-output--operators\"\u003e3. Input, Output \u0026amp; Operators\u003c/h3\u003e\n\u003cul\u003e\n\u003cli\u003eInput using \u003ccode\u003einput()\u003c/code\u003e\u003c/li\u003e\n\u003cli\u003eOutput using \u003ccode\u003eprint()\u003c/code\u003e\u003c/li\u003e\n\u003cli\u003eFormatted output\u003c/li\u003e\n\u003cli\u003eArithmetic operators\u003c/li\u003e\n\u003cli\u003eRelational operators\u003c/li\u003e\n\u003cli\u003eLogical operators\u003c/li\u003e\n\u003cli\u003eAssignment operators\u003c/li\u003e\n\u003cli\u003eOperator precedence\u003c/li\u003e\n\u003c/ul\u003e\n\u003chr\u003e\n\u003ch3 id=\"4-control-flow\"\u003e4. Control Flow\u003c/h3\u003e\n\u003ch4 id=\"conditional-statements\"\u003eConditional Statements\u003c/h4\u003e\n\u003cul\u003e\n\u003cli\u003e\u003ccode\u003eif\u003c/code\u003e\u003c/li\u003e\n\u003cli\u003e\u003ccode\u003eif-else\u003c/code\u003e\u003c/li\u003e\n\u003cli\u003e\u003ccode\u003eelif\u003c/code\u003e\u003c/li\u003e\n\u003cli\u003eNested conditions\u003c/li\u003e\n\u003c/ul\u003e\n\u003ch4 id=\"looping-statements\"\u003eLooping Statements\u003c/h4\u003e\n\u003cul\u003e\n\u003cli\u003e\u003ccode\u003efor\u003c/code\u003e loop\u003c/li\u003e\n\u003cli\u003e\u003ccode\u003ewhile\u003c/code\u003e loop\u003c/li\u003e\n\u003c/ul\u003e\n\u003ch4 id=\"loop-control\"\u003eLoop Control\u003c/h4\u003e\n\u003cul\u003e\n\u003cli\u003e\u003ccode\u003ebreak\u003c/code\u003e\u003c/li\u003e\n\u003cli\u003e\u003ccode\u003econtinue\u003c/code\u003e\u003c/li\u003e\n\u003cli\u003e\u003ccode\u003epass\u003c/code\u003e\u003c/li\u003e\n\u003c/ul\u003e\n\u003chr\u003e\n\u003ch2 id=\"intermediate-level-----core-python-concepts\"\u003eIntermediate Level  -  Core Python Concepts\u003c/h2\u003e\n\u003ch3 id=\"5-strings\"\u003e5. Strings\u003c/h3\u003e\n\u003cul\u003e\n\u003cli\u003eString creation\u003c/li\u003e\n\u003cli\u003eIndexing and slicing\u003c/li\u003e\n\u003cli\u003eString methods\u003c/li\u003e\n\u003cli\u003eString formatting\u003c/li\u003e\n\u003cli\u003eEscape characters\u003c/li\u003e\n\u003cli\u003eImmutability\u003c/li\u003e\n\u003cli\u003eString operations\u003c/li\u003e\n\u003c/ul\u003e\n\u003chr\u003e\n\u003ch3 id=\"6-lists\"\u003e6. Lists\u003c/h3\u003e\n\u003cul\u003e\n\u003cli\u003eCreating lists\u003c/li\u003e\n\u003cli\u003eIndexing and slicing\u003c/li\u003e\n\u003cli\u003eList methods\u003c/li\u003e\n\u003cli\u003eNested lists\u003c/li\u003e\n\u003cli\u003eIterating through lists\u003c/li\u003e\n\u003cli\u003eList comprehensions\u003c/li\u003e\n\u003c/ul\u003e\n\u003chr\u003e\n\u003ch3 id=\"7-tuples\"\u003e7. Tuples\u003c/h3\u003e\n\u003cul\u003e\n\u003cli\u003eCreating tuples\u003c/li\u003e\n\u003cli\u003eTuple operations\u003c/li\u003e\n\u003cli\u003eTuple unpacking\u003c/li\u003e\n\u003cli\u003eDifference between list and tuple\u003c/li\u003e\n\u003cli\u003eUse cases\u003c/li\u003e\n\u003c/ul\u003e\n\u003chr\u003e\n\u003ch3 id=\"8-sets\"\u003e8. Sets\u003c/h3\u003e\n\u003cul\u003e\n\u003cli\u003eSet creation\u003c/li\u003e\n\u003cli\u003eSet methods\u003c/li\u003e\n\u003cli\u003eSet operations\u003c/li\u003e\n\u003cli\u003eDifference between set and list\u003c/li\u003e\n\u003cli\u003eUse cases\u003c/li\u003e\n\u003c/ul\u003e\n\u003chr\u003e\n\u003ch3 id=\"9-dictionaries\"\u003e9. Dictionaries\u003c/h3\u003e\n\u003cul\u003e\n\u003cli\u003eDictionary creation\u003c/li\u003e\n\u003cli\u003eAccessing elements\u003c/li\u003e\n\u003cli\u003eDictionary methods\u003c/li\u003e\n\u003cli\u003eIterating dictionaries\u003c/li\u003e\n\u003cli\u003eNested dictionaries\u003c/li\u003e\n\u003cli\u003eReal-world use cases\u003c/li\u003e\n\u003c/ul\u003e\n\u003chr\u003e\n\u003ch3 id=\"10-functions\"\u003e10. Functions\u003c/h3\u003e\n\u003cul\u003e\n\u003cli\u003eDefining functions\u003c/li\u003e\n\u003cli\u003eFunction parameters\u003c/li\u003e\n\u003cli\u003eReturn values\u003c/li\u003e\n\u003cli\u003eDefault arguments\u003c/li\u003e\n\u003cli\u003eKeyword arguments\u003c/li\u003e\n\u003cli\u003eVariable-length arguments (\u003ccode\u003e*args\u003c/code\u003e, \u003ccode\u003e**kwargs\u003c/code\u003e)\u003c/li\u003e\n\u003cli\u003eScope (local and global)\u003c/li\u003e\n\u003cli\u003eLambda functions\u003c/li\u003e\n\u003c/ul\u003e\n\u003chr\u003e\n\u003ch3 id=\"11-modules--packages\"\u003e11. Modules \u0026amp; Packages\u003c/h3\u003e\n\u003cul\u003e\n\u003cli\u003eImporting modules\u003c/li\u003e\n\u003cli\u003eBuilt-in modules\u003c/li\u003e\n\u003cli\u003eCreating custom modules\u003c/li\u003e\n\u003cli\u003ePython packages\u003c/li\u003e\n\u003cli\u003e\u003ccode\u003e__name__ == \u0026quot;__main__\u0026quot;\u003c/code\u003e\u003c/li\u003e\n\u003cli\u003eVirtual environments\u003c/li\u003e\n\u003c/ul\u003e\n\u003chr\u003e\n\u003ch2 id=\"advanced-level-----professional-python\"\u003eAdvanced Level  -  Professional Python\u003c/h2\u003e\n\u003ch3 id=\"12-file-handling\"\u003e12. File Handling\u003c/h3\u003e\n\u003cul\u003e\n\u003cli\u003eReading files\u003c/li\u003e\n\u003cli\u003eWriting files\u003c/li\u003e\n\u003cli\u003eFile modes\u003c/li\u003e\n\u003cli\u003eWorking with CSV files\u003c/li\u003e\n\u003cli\u003eJSON handling\u003c/li\u003e\n\u003cli\u003eContext managers using \u003ccode\u003ewith\u003c/code\u003e\u003c/li\u003e\n\u003c/ul\u003e\n\u003chr\u003e\n\u003ch3 id=\"13-exception-handling\"\u003e13. Exception Handling\u003c/h3\u003e\n\u003cul\u003e\n\u003cli\u003eTypes of errors\u003c/li\u003e\n\u003cli\u003e\u003ccode\u003etry\u003c/code\u003e and \u003ccode\u003eexcept\u003c/code\u003e\u003c/li\u003e\n\u003cli\u003e\u003ccode\u003eelse\u003c/code\u003e and \u003ccode\u003efinally\u003c/code\u003e\u003c/li\u003e\n\u003cli\u003eMultiple exceptions\u003c/li\u003e\n\u003cli\u003eCustom exceptions\u003c/li\u003e\n\u003cli\u003eDebugging techniques\u003c/li\u003e\n\u003c/ul\u003e\n\u003chr\u003e\n\u003ch3 id=\"14-object-oriented-programming-oop\"\u003e14. Object-Oriented Programming (OOP)\u003c/h3\u003e\n\u003cul\u003e\n\u003cli\u003eClasses and objects\u003c/li\u003e\n\u003cli\u003eConstructors\u003c/li\u003e\n\u003cli\u003eInstance and class variables\u003c/li\u003e\n\u003cli\u003eMethods\u003c/li\u003e\n\u003cli\u003eEncapsulation\u003c/li\u003e\n\u003cli\u003eInheritance\u003c/li\u003e\n\u003cli\u003ePolymorphism\u003c/li\u003e\n\u003cli\u003eAbstraction\u003c/li\u003e\n\u003cli\u003eMagic methods (\u003ccode\u003e__str__\u003c/code\u003e, \u003ccode\u003e__len__\u003c/code\u003e, etc.)\u003c/li\u003e\n\u003c/ul\u003e\n\u003chr\u003e\n\u003ch3 id=\"15-advanced-python-concepts\"\u003e15. Advanced Python Concepts\u003c/h3\u003e\n\u003cul\u003e\n\u003cli\u003eList, set, and dictionary comprehensions\u003c/li\u003e\n\u003cli\u003eIterators and generators\u003c/li\u003e\n\u003cli\u003eDecorators\u003c/li\u003e\n\u003cli\u003eClosures\u003c/li\u003e\n\u003cli\u003eShallow vs deep copy\u003c/li\u003e\n\u003cli\u003eMemory management\u003c/li\u003e\n\u003cli\u003eGarbage collection\u003c/li\u003e\n\u003c/ul\u003e\n\u003chr\u003e\n\u003ch3 id=\"16-python-standard-library\"\u003e16. Python Standard Library\u003c/h3\u003e\n\u003cul\u003e\n\u003cli\u003e\u003ccode\u003emath\u003c/code\u003e\u003c/li\u003e\n\u003cli\u003e\u003ccode\u003erandom\u003c/code\u003e\u003c/li\u003e\n\u003cli\u003e\u003ccode\u003edatetime\u003c/code\u003e\u003c/li\u003e\n\u003cli\u003e\u003ccode\u003ecollections\u003c/code\u003e\u003c/li\u003e\n\u003cli\u003e\u003ccode\u003eitertools\u003c/code\u003e\u003c/li\u003e\n\u003cli\u003e\u003ccode\u003efunctools\u003c/code\u003e\u003c/li\u003e\n\u003cli\u003e\u003ccode\u003eos\u003c/code\u003e\u003c/li\u003e\n\u003cli\u003e\u003ccode\u003esys\u003c/code\u003e\u003c/li\u003e\n\u003c/ul\u003e\n\u003chr\u003e\n\u003ch3 id=\"17-testing--debugging\"\u003e17. Testing \u0026amp; Debugging\u003c/h3\u003e\n\u003cul\u003e\n\u003cli\u003eAssertions\u003c/li\u003e\n\u003cli\u003eUnit testing\u003c/li\u003e\n\u003cli\u003eWriting test cases\u003c/li\u003e\n\u003cli\u003eTesting frameworks overview\u003c/li\u003e\n\u003cli\u003eDebugging tools\u003c/li\u003e\n\u003cli\u003eLogging\u003c/li\u003e\n\u003c/ul\u003e\n\u003chr\u003e\n\u003ch2 id=\"specialized--career-oriented-topics\"\u003eSpecialized \u0026amp; Career-Oriented Topics\u003c/h2\u003e\n\u003ch3 id=\"18-data-structures--algorithms-with-python\"\u003e18. Data Structures \u0026amp; Algorithms with Python\u003c/h3\u003e\n\u003cul\u003e\n\u003cli\u003eTime and space complexity\u003c/li\u003e\n\u003cli\u003eArrays and strings\u003c/li\u003e\n\u003cli\u003eStacks and queues\u003c/li\u003e\n\u003cli\u003eLinked lists\u003c/li\u003e\n\u003cli\u003eTrees\u003c/li\u003e\n\u003cli\u003eGraphs\u003c/li\u003e\n\u003cli\u003eDynamic programming\u003c/li\u003e\n\u003cli\u003eGreedy algorithms\u003c/li\u003e\n\u003cli\u003eRecursion and backtracking\u003c/li\u003e\n\u003c/ul\u003e\n\u003chr\u003e\n\u003ch3 id=\"19-automation--scripting\"\u003e19. Automation \u0026amp; Scripting\u003c/h3\u003e\n\u003cul\u003e\n\u003cli\u003eFile and folder automation\u003c/li\u003e\n\u003cli\u003eWeb scraping basics\u003c/li\u003e\n\u003cli\u003eTask scheduling\u003c/li\u003e\n\u003cli\u003eCommand-line tools\u003c/li\u003e\n\u003cli\u003eAutomation projects\u003c/li\u003e\n\u003c/ul\u003e\n\u003chr\u003e\n\u003ch3 id=\"20-python-for-web-development\"\u003e20. Python for Web Development\u003c/h3\u003e\n\u003cul\u003e\n\u003cli\u003eBackend fundamentals\u003c/li\u003e\n\u003cli\u003eREST APIs\u003c/li\u003e\n\u003cli\u003eJSON and HTTP\u003c/li\u003e\n\u003cli\u003eWeb frameworks overview\u003c/li\u003e\n\u003cli\u003eRequest–response cycle\u003c/li\u003e\n\u003c/ul\u003e\n\u003chr\u003e\n\u003ch3 id=\"21-python-for-data--ai\"\u003e21. Python for Data \u0026amp; AI\u003c/h3\u003e\n\u003cul\u003e\n\u003cli\u003eNumPy basics\u003c/li\u003e\n\u003cli\u003ePandas basics\u003c/li\u003e\n\u003cli\u003eData cleaning\u003c/li\u003e\n\u003cli\u003eBasic data visualization\u003c/li\u003e\n\u003cli\u003eWorking with datasets\u003c/li\u003e\n\u003c/ul\u003e\n\u003chr\u003e\n\u003ch3 id=\"22-performance--optimization\"\u003e22. Performance \u0026amp; Optimization\u003c/h3\u003e\n\u003cul\u003e\n\u003cli\u003eCode optimization\u003c/li\u003e\n\u003cli\u003eProfiling\u003c/li\u003e\n\u003cli\u003eEfficient data handling\u003c/li\u003e\n\u003cli\u003eBest practices\u003c/li\u003e\n\u003cli\u003ePEP 8 guidelines\u003c/li\u003e\n\u003cli\u003eType hints\u003c/li\u003e\n\u003c/ul\u003e\n\u003chr\u003e\n\u003ch3 id=\"23-real-world-projects\"\u003e23. Real-World Projects\u003c/h3\u003e\n\u003cul\u003e\n\u003cli\u003eBeginner mini projects\u003c/li\u003e\n\u003cli\u003eAutomation projects\u003c/li\u003e\n\u003cli\u003eAPI-based projects\u003c/li\u003e\n\u003cli\u003eData-driven projects\u003c/li\u003e\n\u003cli\u003eEnd-to-end Python applications\u003c/li\u003e\n\u003c/ul\u003e\n\u003chr\u003e\n\u003ch3 id=\"24-interview--professional-topics\"\u003e24. Interview \u0026amp; Professional Topics\u003c/h3\u003e\n\u003cul\u003e\n\u003cli\u003eCommon Python interview questions\u003c/li\u003e\n\u003cli\u003eDSA problem solving\u003c/li\u003e\n\u003cli\u003eDesign patterns\u003c/li\u003e\n\u003cli\u003eCode optimization\u003c/li\u003e\n\u003cli\u003ePackaging and distribution\u003c/li\u003e\n\u003cli\u003eResume and project preparation\u003c/li\u003e\n\u003c/ul\u003e\n\u003ch3 id=\"25-python-programming-playlist\"\u003e25. Python Programming Playlist\u003c/h3\u003e\n\u003ciframe\n  width=\"560\"\n  height=\"315\"\n  src=\"https://youtube.com/playlist?list=PLo8TPLc6QbB8xV-Jzh0JYQ8fmckdh--pu\u0026si=8oIfIdpaqQxYKpXR\"\n  title=\"Python Programming playlist\"\n  frameborder=\"0\"\n  allowfullscreen\u003e\n\u003c/iframe\u003e\n\u003ciframe\n  width=\"560\"\n  height=\"315\"\n  src=\"https://youtube.com/playlist?list=PLo8TPLc6QbB9R6knaQqcZ58JIVVRYu0tb\u0026si=MKraBdmi0DBuuD-0\"\n  title=\"Python Programming playlist\"\n  frameborder=\"0\"\n  allowfullscreen\u003e\n\u003c/iframe\u003e\n\u003ciframe\n  width=\"560\"\n  height=\"315\"\n  src=\"https://youtube.com/playlist?list=PLo8TPLc6QbB_oVJJeNpd3wpGAjf-gYCgU\u0026si=bGjNgZw_B2-0z5F4\"\n  title=\"Python Programming playlist\"\n  frameborder=\"0\"\n  allowfullscreen\u003e\n\u003c/iframe\u003e\n---\n\u003cp\u003eMore structured Python learning guides and hands-on roadmaps coming soon.\u003c/p\u003e","tags":["python","learning-roadmap"],"title":"Complete Python Programming Syllabus (Beginner to Advanced)"},{"content":"Dynamic Programming (DP) is a powerful algorithmic technique used to solve problems that can be broken down into smaller subproblems whose solutions are reused to efficiently compute the final result.\nUnlike brute-force approaches or naive recursion, Dynamic Programming avoids unnecessary recomputation and significantly improves performance.\n1. What Is Dynamic Programming? Dynamic Programming is a method of solving problems by:\nDividing a problem into smaller subproblems Solving each subproblem only once Storing the solution of each subproblem Using stored solutions to build the final answer The word dynamic refers to solving problems step by step, while programming refers to structured problem-solving rather than coding.\n2. Overlapping Subproblems Definition A problem has overlapping subproblems if the same subproblems are solved multiple times while solving the larger problem.\nRepeated computation of the same subproblems makes naive solutions inefficient.\nWhy Overlapping Subproblems Matter When subproblems overlap:\nRecursive solutions perform redundant computations Time complexity increases rapidly The same values are recalculated again and again Dynamic Programming addresses this issue by storing the result of each subproblem so that it is computed only once.\nExample: Fibonacci Numbers The Fibonacci sequence is defined as:\nF(n) depends on F(n−1) and F(n−2)\nWhile computing F(5):\nF(5) depends on F(4) and F(3) F(4) again depends on F(3) and F(2) F(3) is calculated multiple times Here, F(3) and F(2) are overlapping subproblems.\nImpact Without Dynamic Programming Time complexity becomes exponential A large number of repeated calculations occur Impact With Dynamic Programming Computed values are stored in a table or array Each subproblem is solved once Time complexity reduces significantly, often to linear Key Takeaway The presence of overlapping subproblems indicates that memoization or tabulation can greatly improve efficiency.\n3. Optimal Substructure Definition A problem has optimal substructure if its optimal solution can be constructed from optimal solutions of its subproblems.\nIn simpler terms, solving smaller parts optimally leads to an optimal solution for the entire problem.\nWhy Optimal Substructure Matters Dynamic Programming assumes that:\nSubproblems can be solved optimally Combining optimal subproblem solutions produces a global optimal solution If this assumption does not hold, Dynamic Programming cannot guarantee correct results.\nExample: Shortest Path Problem Consider finding the shortest path from city A to city D via city B.\nThe shortest path from A to D consists of:\nThe shortest path from A to B The shortest path from B to D If either subpath is not optimal, the final path will also not be optimal.\nThis confirms that the shortest path problem has optimal substructure.\nAnother Example: Knapsack Problem In the 0/1 Knapsack problem:\nThe optimal solution for capacity W Depends on optimal solutions for smaller capacities This confirms the existence of optimal substructure.\nKey Takeaway Optimal substructure ensures that local optimal solutions contribute to a globally optimal solution.\n4. Relationship Between Overlapping Subproblems and Optimal Substructure Dynamic Programming is applicable only when both properties exist.\nOverlapping subproblems help avoid repeated work.\nOptimal substructure ensures correctness of the solution.\nIf subproblems do not overlap, Dynamic Programming provides no efficiency benefit.\nIf optimal substructure does not exist, Dynamic Programming may produce incorrect results.\n5. How Dynamic Programming Uses These Concepts Together To apply Dynamic Programming effectively:\nIdentify the subproblems Check whether subproblems repeat Verify the presence of optimal substructure Define a DP state to represent subproblems Formulate a recurrence relation Store and reuse results 6. Example Combining Both Concepts Problem: Minimum cost to reach the end.\nCost to reach a step depends on minimum cost of previous steps The same steps are evaluated multiple times Optimal cost at each step depends on optimal costs of earlier steps This problem has both overlapping subproblems and optimal substructure.\nTherefore, Dynamic Programming is applicable.\n7. When Dynamic Programming Should Not Be Used Dynamic Programming is not suitable when:\nSubproblems are independent A greedy approach produces better results Recursion depth is small and no repetition occurs Summary Overlapping subproblems reduce efficiency without Dynamic Programming Optimal substructure ensures correctness of solutions Both properties are essential for applying Dynamic Programming A solid understanding of these concepts is crucial for mastering advanced algorithmic problem solving.\nDynamic Programming playlists Here are some curated playlists you can follow:\nPython \u0026amp; DSA More structured posts and learning paths coming soon.\n","permalink":"https://blogs.codeverra.com/blog/dynamic_programming/","summary":"\u003cp\u003eDynamic Programming (DP) is a powerful algorithmic technique used to solve problems that can be broken down into smaller subproblems whose solutions are reused to efficiently compute the final result.\u003c/p\u003e\n\u003cp\u003eUnlike brute-force approaches or naive recursion, Dynamic Programming avoids unnecessary recomputation and significantly improves performance.\u003c/p\u003e\n\u003chr\u003e\n\u003ch2 id=\"1-what-is-dynamic-programming\"\u003e1. What Is Dynamic Programming?\u003c/h2\u003e\n\u003cp\u003eDynamic Programming is a method of solving problems by:\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003eDividing a problem into smaller subproblems\u003c/li\u003e\n\u003cli\u003eSolving each subproblem only once\u003c/li\u003e\n\u003cli\u003eStoring the solution of each subproblem\u003c/li\u003e\n\u003cli\u003eUsing stored solutions to build the final answer\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003eThe word dynamic refers to solving problems step by step, while programming refers to structured problem-solving rather than coding.\u003c/p\u003e","tags":["dsa","dynamic-programming"],"title":"Concept of Dynamic Programming Explained Clearly"},{"content":"Sorting is the process of arranging elements in a specific order, usually ascending or descending, to make data easier to search, analyze, and process.\nSorting plays a critical role in computer science because many algorithms perform efficiently only on sorted data.\n1. Bubble Sort Concept Bubble Sort repeatedly compares adjacent elements and swaps them if they are in the wrong order. Larger elements gradually move to the end of the array, similar to bubbles rising to the surface.\nWorking Idea Compare adjacent elements Swap them if they are in the wrong order Repeat the process for all elements Time Complexity Best case: O(n) Average case: O(n²) Worst case: O(n²) Space Complexity O(1)\nKey Points Very simple to understand Highly inefficient for large datasets 2. Selection Sort Concept Selection Sort repeatedly selects the smallest element from the unsorted portion of the array and places it at its correct position.\nWorking Idea Find the minimum element Swap it with the first unsorted element Repeat for the remaining elements Time Complexity Best case: O(n²) Average case: O(n²) Worst case: O(n²) Space Complexity O(1)\nKey Points Requires fewer swaps than Bubble Sort Still slow for large inputs 3. Insertion Sort Concept Insertion Sort builds the sorted array one element at a time by inserting each element into its correct position in the already sorted part.\nWorking Idea Take one element at a time Insert it into the correct position in the sorted portion Time Complexity Best case: O(n) Average case: O(n²) Worst case: O(n²) Space Complexity O(1)\nKey Points Efficient for small or nearly sorted datasets Used in real-world hybrid sorting algorithms 4. Merge Sort Concept Merge Sort follows the divide-and-conquer approach by dividing the array into halves, sorting each half, and then merging them back together.\nWorking Idea Divide the array into two halves Recursively sort both halves Merge the sorted halves Time Complexity Best case: O(n log n) Average case: O(n log n) Worst case: O(n log n) Space Complexity O(n)\nKey Points Stable sorting algorithm Very efficient for large datasets 5. Quick Sort Concept Quick Sort selects a pivot element and arranges elements smaller than the pivot on one side and larger elements on the other side.\nWorking Idea Choose a pivot element Partition the array around the pivot Recursively sort the subarrays Time Complexity Best case: O(n log n) Average case: O(n log n) Worst case: O(n²) Space Complexity O(log n) due to recursion stack\nKey Points Very fast in practice Not stable by default 6. Heap Sort Concept Heap Sort uses a heap data structure to repeatedly extract the maximum element and place it at the end of the array.\nWorking Idea Build a max heap Remove the largest element Place it at the end Repeat the process Time Complexity Best case: O(n log n) Average case: O(n log n) Worst case: O(n log n) Space Complexity O(1)\nKey Points In-place sorting algorithm Not stable 7. Counting Sort Concept Counting Sort counts the occurrences of each element and uses this information to place elements directly into their correct positions.\nWorking Idea Count the frequency of each element Compute cumulative counts Place elements in sorted order Time Complexity O(n + k), where k is the range of values\nSpace Complexity O(n + k)\nKey Points Extremely fast for small range data Not a comparison-based sorting algorithm 8. Radix Sort Concept Radix Sort sorts numbers digit by digit, starting from the least significant digit and moving to the most significant digit.\nWorking Idea Sort numbers by each digit Use a stable sorting method at each step Time Complexity O(n × d), where d is the number of digits\nSpace Complexity O(n)\nKey Points Works well for integers Not comparison-based 9. Bucket Sort Concept Bucket Sort distributes elements into different buckets, sorts each bucket individually, and then combines all buckets to form the sorted array.\nWorking Idea Create buckets Distribute elements into buckets Sort each bucket Combine the results Time Complexity Average case: O(n + k) Worst case: O(n²) Space Complexity O(n)\nKey Points Efficient for uniformly distributed data Comparison of Sorting Algorithms Bubble Sort\nBest: O(n)\nAverage: O(n²)\nWorst: O(n²)\nStable: Yes\nSelection Sort\nBest: O(n²)\nAverage: O(n²)\nWorst: O(n²)\nStable: No\nInsertion Sort\nBest: O(n)\nAverage: O(n²)\nWorst: O(n²)\nStable: Yes\nMerge Sort\nBest: O(n log n)\nAverage: O(n log n)\nWorst: O(n log n)\nStable: Yes\nQuick Sort\nBest: O(n log n)\nAverage: O(n log n)\nWorst: O(n²)\nStable: No\nHeap Sort\nBest: O(n log n)\nAverage: O(n log n)\nWorst: O(n log n)\nStable: No\nCounting Sort\nBest: O(n + k)\nAverage: O(n + k)\nWorst: O(n + k)\nStable: Yes\nRadix Sort\nBest: O(n × d)\nAverage: O(n × d)\nWorst: O(n × d)\nStable: Yes\nChoosing the Right Sorting Algorithm Small dataset → Insertion Sort Large dataset → Merge Sort or Quick Sort Memory constraints → Heap Sort Known range of values → Counting Sort Real-world systems → Hybrid algorithms Conclusion No single sorting algorithm is best for all situations.\nUnderstanding the strengths and limitations of each sorting algorithm allows you to choose the most efficient approach for a given problem and dataset.\n","permalink":"https://blogs.codeverra.com/blog/sorting_algo/","summary":"\u003cp\u003eSorting is the process of arranging elements in a specific order, usually ascending or descending, to make data easier to search, analyze, and process.\u003c/p\u003e\n\u003cp\u003eSorting plays a critical role in computer science because many algorithms perform efficiently only on sorted data.\u003c/p\u003e\n\u003chr\u003e\n\u003ch2 id=\"1-bubble-sort\"\u003e1. Bubble Sort\u003c/h2\u003e\n\u003ch3 id=\"concept\"\u003eConcept\u003c/h3\u003e\n\u003cp\u003eBubble Sort repeatedly compares adjacent elements and swaps them if they are in the wrong order. Larger elements gradually move to the end of the array, similar to bubbles rising to the surface.\u003c/p\u003e","tags":["dsa","sorting"],"title":"Different Sorting Algorithms Explained Clearly"},{"content":"About This Guide Welcome to the DSA with Python: Zero to Hero comprehensive learning roadmap! This guide serves as the complete reference material for our live course, organizing all topics, concepts, and learning outcomes in a structured, progressive manner.\nFor Course Students If you\u0026rsquo;re enrolled in the live course, this blog is your companion resource that:\nMirrors the course structure - Every module taught in class is documented here Serves as a reference - Quickly look up concepts, syntax, and examples Tracks your progress - Follow along and check off completed modules Provides continuity - Never miss a beat even if you miss a session For Self-Learners Even if you\u0026rsquo;re not enrolled in the course, this roadmap is designed to be:\nCompletely self-contained - Learn at your own pace with clear learning outcomes Beginner-friendly - No prior programming experience required Comprehensive - Covers everything from Python basics to advanced algorithms Practical - Focus on real-world applications and problem-solving What This Blog Includes This is a living document that will be continuously updated with:\nDetailed explanations for every topic and concept Code examples with clear comments and explanations Practice problems curated from LeetCode, HackerRank, and other platforms Solutions and explanations for all practice problems Video tutorials (where applicable) Additional resources - articles, visualizations, and external references Cheat sheets for quick revision Common pitfalls and debugging tips Interview tips and frequently asked questions How to Use This Guide Linear Learning: Start from Module 1 and progress sequentially - each module builds on previous knowledge Topic Review: Use the table of contents to jump to specific topics you want to revisit Practice First: Try practice problems before looking at solutions Active Learning: Code along with examples rather than just reading Track Progress: Check off modules as you complete them Stay Updated This blog is regularly updated with:\nNew practice problems and solutions Additional explanations based on common questions Links to helpful external resources Real-world applications and case studies Interview preparation materials\nBookmark this page and check back regularly for updates!\nHow to Get the Most Out of This Course Practice daily - Consistency beats intensity Code everything yourself - Don\u0026rsquo;t just copy-paste Understand, don\u0026rsquo;t memorize - Focus on concepts, not syntax Solve problems - Apply what you learn immediately Ask questions - No question is too basic Build projects - Apply concepts to real-world scenarios Enroll Now DSA with Python: Zero to Hero - Course Structure PHASE 1: PYTHON FOUNDATIONS Module 1: Getting Started with Python Subtopics:\nIntroduction to programming and Python Installing Python and setting up IDE Writing and running first program Understanding the Python interpreter Learning Outcomes:\nSet up Python development environment Write, save, and execute basic Python programs Understand how Python code is interpreted Navigate basic IDE features Module 2: Variables, Data Types \u0026amp; Basic Operations Subtopics:\nVariables and naming conventions Basic data types: int, float, str, bool Type conversion and type checking Arithmetic operators String operations and formatting User input and output Comments and documentation Learning Outcomes:\nDeclare and use variables following Python conventions Work with all basic data types confidently Perform type conversions when needed Create formatted string outputs Build simple interactive programs Module 3: Control Flow - Decision Making Subtopics:\nBoolean logic and comparison operators if, elif, else statements Nested conditionals Logical operators (and, or, not) Truthy and falsy values Ternary operators Learning Outcomes:\nWrite programs that make decisions based on conditions Use complex boolean expressions effectively Implement multi-way decision structures Apply logical operators to combine conditions Module 4: Control Flow - Loops Subtopics:\nwhile loops for loops and range() Nested loops break, continue, pass statements Loop control and infinite loops Iterating over sequences Learning Outcomes:\nChoose appropriate loop structures Write and control loops with precision Avoid and debug infinite loops Implement nested loop patterns Solve repetitive tasks programmatically Module 5: Functions Subtopics:\nDefining and calling functions Parameters and arguments (positional, keyword, default) Return values and multiple returns Variable scope (local, global, nonlocal) Docstrings Lambda functions *args and **kwargs Recursion basics Learning Outcomes:\nDesign and implement reusable functions Use different parameter types effectively Understand and apply variable scope rules Write well-documented functions Solve problems using recursion Module 6: Lists Subtopics:\nCreating and accessing lists List methods (append, insert, remove, pop, etc.) List slicing and indexing List comprehensions Nested lists and 2D lists Common list operations Learning Outcomes:\nCreate and manipulate lists effectively Use list comprehensions for elegant code Work with multi-dimensional lists Choose appropriate list methods Module 7: Tuples \u0026amp; Sets Subtopics:\nTuples: creation, immutability, use cases Tuple packing and unpacking Sets: creation and characteristics Set operations (union, intersection, difference) Frozen sets When to use tuples vs lists vs sets Learning Outcomes:\nUse tuples for immutable data storage Implement set operations for unique collections Choose the right data structure for each use case Apply set theory operations in programming Module 8: Dictionaries Subtopics:\nCreating and accessing dictionaries Dictionary methods Iterating over dictionaries Nested dictionaries Dictionary comprehensions defaultdict and Counter Learning Outcomes:\nModel real-world data using dictionaries Perform efficient lookups and updates Work with complex nested structures Use dictionary comprehensions effectively Module 9: Strings Deep Dive Subtopics:\nString methods and operations String slicing and indexing String formatting (f-strings, format(), %) Regular expressions basics String immutability Common string patterns Learning Outcomes:\nManipulate strings using various methods Use advanced formatting techniques Apply basic regular expressions Solve string-based problems efficiently Module 10: File Handling Subtopics:\nReading and writing text files File modes and context managers Working with CSV files JSON file handling Exception handling with files File paths and directories Learning Outcomes:\nRead from and write to files safely Handle different file formats Implement proper error handling for file operations Manage file resources using context managers Module 11: Exception Handling Subtopics:\nUnderstanding exceptions and errors try, except, else, finally blocks Raising exceptions Custom exceptions Best practices for error handling Debugging techniques Learning Outcomes:\nWrite robust code that handles errors gracefully Create and raise custom exceptions Debug programs effectively Implement defensive programming practices Module 12: Object-Oriented Programming - Part 1 Subtopics:\nClasses and objects init method Instance vs class attributes Methods Encapsulation and private attributes Property decorators str and repr Learning Outcomes:\nDesign and implement classes Create objects and use them effectively Apply encapsulation principles Write clean, object-oriented code Module 13: Object-Oriented Programming - Part 2 Subtopics:\nInheritance and super() Method overriding Polymorphism Abstract classes Multiple inheritance Composition vs inheritance Magic/dunder methods Learning Outcomes:\nImplement inheritance hierarchies Use polymorphism effectively Choose between inheritance and composition Leverage Python\u0026rsquo;s special methods Module 14: Python Standard Library \u0026amp; Tools Subtopics:\nModules and packages math, random, datetime, collections Generator expressions itertools and functools map, filter, reduce Installing packages with pip Learning Outcomes:\nNavigate and use the standard library Write efficient code using built-in tools Install and manage external packages Apply functional programming tools PHASE 2: ALGORITHM FOUNDATIONS Module 15: Introduction to Algorithms \u0026amp; Complexity Subtopics:\nWhat are algorithms? Algorithm characteristics Big O notation (O(1), O(n), O(n²), O(log n), O(n log n)) Best, average, worst case analysis Space complexity Analyzing code complexity Learning Outcomes:\nUnderstand what makes a good algorithm Analyze time and space complexity Use Big O notation correctly Compare algorithms based on efficiency Module 16: Searching Algorithms Subtopics:\nLinear search Binary search Jump search Interpolation search When to use each algorithm Learning Outcomes:\nImplement various searching algorithms Choose the appropriate search algorithm Understand trade-offs between algorithms Apply searching techniques to real problems Module 17: Sorting Algorithms - Part 1 Subtopics:\nBubble sort Selection sort Insertion sort Comparison and analysis Stability in sorting Learning Outcomes:\nImplement basic sorting algorithms Understand sorting stability Analyze sorting algorithm efficiency Recognize when to use simple sorts Module 18: Sorting Algorithms - Part 2 Subtopics:\nMerge sort Quick sort Heap sort Counting sort Python\u0026rsquo;s built-in sort (Timsort) Learning Outcomes:\nImplement advanced sorting algorithms Understand divide-and-conquer strategy Apply efficient sorting in practical scenarios Use Python\u0026rsquo;s sorting effectively Module 19: Recursion Deep Dive Subtopics:\nUnderstanding recursion thoroughly Base cases and recursive cases Stack overflow and limits Tail recursion Recursion vs iteration Backtracking introduction Learning Outcomes:\nWrite recursive solutions confidently Identify problems suitable for recursion Convert between recursive and iterative solutions Debug recursive code PHASE 3: DATA STRUCTURES Module 20: Arrays and Dynamic Arrays Subtopics:\nArray fundamentals Static vs dynamic arrays Python lists as dynamic arrays Array operations and complexity Multi-dimensional arrays Learning Outcomes:\nUnderstand array internals and behavior Work efficiently with arrays Implement array-based solutions Optimize array operations Module 21: Linked Lists - Part 1 Subtopics:\nSingly linked lists Node structure and implementation Insertion operations Deletion operations Traversal and searching Linked list vs array Learning Outcomes:\nImplement singly linked lists from scratch Perform all basic operations efficiently Understand pointer/reference manipulation Choose between arrays and linked lists Module 22: Linked Lists - Part 2 Subtopics:\nDoubly linked lists Circular linked lists Common linked list problems Floyd\u0026rsquo;s cycle detection Reversing linked lists Learning Outcomes:\nImplement different linked list variants Solve intermediate linked list problems Apply two-pointer techniques Handle edge cases properly Module 23: Stacks Subtopics:\nStack ADT and LIFO principle Stack implementation (array and linked list) Stack operations: push, pop, peek Applications: balanced parentheses, expression evaluation Infix, prefix, postfix conversions Learning Outcomes:\nImplement stacks using different methods Solve stack-based problems Apply stacks to real-world scenarios Understand recursion through stacks Module 24: Queues Subtopics:\nQueue ADT and FIFO principle Queue implementation (array and linked list) Circular queues Deque (double-ended queue) Priority queues (introduction) Learning Outcomes:\nImplement various queue types Use queues for problem-solving Understand queue applications Apply appropriate queue structures Module 25: Hash Tables Subtopics:\nHashing concept and hash functions Collision resolution (chaining, open addressing) Hash table implementation Load factor and rehashing Python dictionaries and sets internals Learning Outcomes:\nUnderstand hashing thoroughly Implement hash tables from scratch Design good hash functions Solve problems using hash tables Module 26: Trees - Part 1 (Binary Trees) Subtopics:\nTree terminology and concepts Binary tree structure and implementation Tree traversals (inorder, preorder, postorder, level-order) Recursive vs iterative traversals Tree height, depth, and size Learning Outcomes:\nImplement binary trees Perform all tree traversals Calculate tree properties Solve basic tree problems Module 27: Trees - Part 2 (Binary Search Trees) Subtopics:\nBST properties and implementation BST operations (insert, search, delete) BST traversal applications Balanced vs unbalanced BSTs Time complexity analysis Learning Outcomes:\nImplement fully functional BSTs Perform efficient BST operations Understand BST advantages and limitations Solve BST-specific problems Module 28: Trees - Part 3 (Advanced Trees) Subtopics:\nAVL trees (self-balancing) Red-Black trees (overview) B-trees (concept) Tries (prefix trees) Segment trees (introduction) Learning Outcomes:\nUnderstand self-balancing concepts Implement tries Know when to use advanced tree structures Appreciate real-world tree applications Module 29: Heaps and Priority Queues Subtopics:\nHeap property (min-heap, max-heap) Heap implementation using arrays Heapify operations Heap sort Priority queue implementation Python\u0026rsquo;s heapq module Learning Outcomes:\nImplement heaps from scratch Use heaps for priority queues Apply heaps to solve problems Use Python\u0026rsquo;s heap module effectively Module 30: Graphs - Part 1 (Fundamentals) Subtopics:\nGraph terminology Graph representations (adjacency matrix, list) Graph implementation Graph traversals (BFS, DFS) Connected components Cycle detection Learning Outcomes:\nImplement graphs using different representations Perform graph traversals Detect graph properties Choose appropriate representations Module 31: Graphs - Part 2 (Advanced Algorithms) Subtopics:\nShortest path (Dijkstra\u0026rsquo;s, Bellman-Ford) Minimum spanning trees (Prim\u0026rsquo;s, Kruskal\u0026rsquo;s) Topological sorting Strongly connected components Union-Find data structure Learning Outcomes:\nImplement classic graph algorithms Solve shortest path problems Find minimum spanning trees Apply graphs to real-world problems PHASE 4: ADVANCED ALGORITHMS Module 32: Greedy Algorithms Subtopics:\nGreedy algorithm paradigm Activity selection problem Huffman coding Fractional knapsack Job sequencing When greedy works and limitations Learning Outcomes:\nIdentify greedy algorithm opportunities Implement greedy solutions Prove greedy correctness Recognize greedy limitations Module 33: Dynamic Programming - Part 1 Subtopics:\nDP fundamentals Memoization vs tabulation Fibonacci and basic problems 0/1 Knapsack problem Longest common subsequence Coin change problem Learning Outcomes:\nUnderstand DP principles thoroughly Implement memoization and tabulation Solve classic DP problems Recognize DP problem patterns Module 34: Dynamic Programming - Part 2 Subtopics:\nMatrix chain multiplication Edit distance Longest increasing subsequence Partition problems DP on trees and graphs Space optimization techniques Learning Outcomes:\nSolve advanced DP problems Optimize DP solutions Apply DP to complex scenarios Develop DP intuition Module 35: Backtracking Subtopics:\nBacktracking concept and template N-Queens problem Sudoku solver Subset sum Permutations and combinations Constraint satisfaction problems Learning Outcomes:\nImplement backtracking solutions Optimize backtracking with pruning Solve combinatorial problems Recognize backtracking opportunities Module 36: Divide and Conquer Subtopics:\nDivide and conquer paradigm Recurrence relations Master theorem Binary search analysis Merge and quick sort analysis Closest pair of points Learning Outcomes:\nApply divide and conquer strategy Analyze divide and conquer complexity Solve problems using this paradigm Understand when to divide and conquer Module 37: String Algorithms Subtopics:\nPattern matching (KMP, Rabin-Karp) Longest common substring String hashing Anagram and palindrome problems Edit distance applications Suffix arrays (introduction) Learning Outcomes:\nImplement efficient string algorithms Solve string manipulation problems Apply pattern matching techniques Handle string problems optimally Module 38: Bit Manipulation Subtopics:\nBinary representation Bitwise operators Common bit tricks Counting set bits Power of two problems Bit masking techniques XOR properties Learning Outcomes:\nUse bitwise operations effectively Solve problems using bit manipulation Optimize solutions with bit tricks Understand binary arithmetic PHASE 5: PROBLEM-SOLVING \u0026amp; PRACTICE Module 39: Problem-Solving Strategies Subtopics:\nHow to approach DSA problems Pattern recognition Breaking down complex problems Debugging strategies Writing clean, efficient code Test case design Learning Outcomes:\nDevelop systematic problem-solving approach Recognize common problem patterns Debug code efficiently Write production-quality solutions Module 40: Competitive Programming Essentials Subtopics:\nFast I/O techniques Common templates and snippets Time and space optimization Online judge platforms Contest strategies Practice techniques Learning Outcomes:\nSolve problems under time constraints Optimize code for competitions Use online platforms effectively Build competitive programming skills Module 42: Capstone Projects Duration: 3-4 weeks\nProjects:\nCustom data structure library Text editor with undo/redo Task scheduler application Route optimization system Spell checker with tries LRU cache implementation Learning Outcomes:\nApply all learned concepts to real projects Make architectural decisions Write production-ready code Document and present technical work ","permalink":"https://blogs.codeverra.com/blog/dsa-with-python-curriculum/","summary":"\u003ch1 id=\"about-this-guide\"\u003eAbout This Guide\u003c/h1\u003e\n\u003chr\u003e\n\u003cp\u003eWelcome to the DSA with Python: Zero to Hero comprehensive learning roadmap! This guide serves as the complete reference material for our live course, organizing all topics, concepts, and learning outcomes in a structured, progressive manner.\u003c/p\u003e\n\u003ch3 id=\"for-course-students\"\u003eFor Course Students\u003c/h3\u003e\n\u003cp\u003eIf you\u0026rsquo;re enrolled in the live course, this blog is your companion resource that:\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003e\u003cstrong\u003eMirrors the course structure\u003c/strong\u003e - Every module taught in class is documented here\u003c/li\u003e\n\u003cli\u003e\u003cstrong\u003eServes as a reference\u003c/strong\u003e - Quickly look up concepts, syntax, and examples\u003c/li\u003e\n\u003cli\u003e\u003cstrong\u003eTracks your progress\u003c/strong\u003e - Follow along and check off completed modules\u003c/li\u003e\n\u003cli\u003e\u003cstrong\u003eProvides continuity\u003c/strong\u003e - Never miss a beat even if you miss a session\u003c/li\u003e\n\u003c/ul\u003e\n\u003ch3 id=\"for-self-learners\"\u003eFor Self-Learners\u003c/h3\u003e\n\u003cp\u003eEven if you\u0026rsquo;re not enrolled in the course, this roadmap is designed to be:\u003c/p\u003e","tags":["dsa","learning-roadmap"],"title":"DSA With Python - Zero to Hero (Curriculum)"},{"content":"Recursion is a programming technique in which a function calls itself to solve a problem by breaking it down into smaller and simpler subproblems.\nInstead of solving the entire problem at once, recursion focuses on solving a reduced version of the problem repeatedly until a stopping condition is reached.\nWhat Is Recursion? In recursion, a function solves a problem by calling itself with a smaller input.\nEach recursive call works on a reduced version of the original problem until it reaches a point where no further calls are needed.\nRecursion is commonly used in problems that naturally break down into similar subproblems.\nKey Components of Recursion Base Case The base case is the condition that stops the recursion.\nWithout a base case, recursion will continue indefinitely and eventually cause a stack overflow error.\nThe base case represents the simplest version of the problem.\nRecursive Case The recursive case is the part of the function where it calls itself.\nEach recursive call must move closer to the base case by reducing the input size.\nHow Recursion Works Internally When a recursive function is called:\nThe function call is placed on the call stack Each recursive call gets its own stack frame Local variables are stored separately for each call Once the base case is reached, functions start returning values in reverse order This process is known as stack unwinding.\nExample: Factorial Using Recursion The factorial of a number is calculated by multiplying the number with the factorial of the number minus one.\nExecution for factorial of 3:\nfactorial(3) becomes 3 multiplied by factorial(2) factorial(2) becomes 2 multiplied by factorial(1) factorial(1) becomes 1 multiplied by factorial(0) factorial(0) returns 1 Once the base case is reached, the values return back step by step.\nTime and Space Complexity For the factorial problem:\nTime complexity is O(n) because the function is called n times.\nSpace complexity is O(n) because each recursive call occupies space on the call stack.\nTypes of Recursion Direct Recursion The function calls itself directly within its own body.\nIndirect Recursion The function calls another function, which eventually calls the original function again.\nTail Recursion The recursive call is the last operation performed in the function.\nNo additional computation is done after the recursive call returns.\nNon-Tail Recursion The recursive call is followed by additional operations.\nMost common recursive problems fall into this category.\nAdvantages of Recursion Simplifies complex problems Produces cleaner and more readable code for divide-and-conquer problems Very useful for tree and graph traversal Matches the natural structure of many problems Disadvantages of Recursion Uses extra memory due to the call stack Can cause stack overflow if the base case is missing or incorrect Sometimes slower than iterative solutions When to Use Recursion Recursion should be used when:\nThe problem can be divided into similar subproblems Tree, graph, or divide-and-conquer problems are involved Code clarity is more important than performance Conclusion Recursion is a powerful problem-solving technique when used correctly.\nUnderstanding base cases, recursive cases, and call stack behavior is essential for mastering recursion.\nA strong grasp of recursion is fundamental for advanced topics such as trees, graphs, backtracking, and dynamic programming.\n","permalink":"https://blogs.codeverra.com/blog/recursion/","summary":"\u003cp\u003eRecursion is a programming technique in which a function calls itself to solve a problem by breaking it down into smaller and simpler subproblems.\u003c/p\u003e\n\u003cp\u003eInstead of solving the entire problem at once, recursion focuses on solving a reduced version of the problem repeatedly until a stopping condition is reached.\u003c/p\u003e\n\u003chr\u003e\n\u003ch2 id=\"what-is-recursion\"\u003eWhat Is Recursion?\u003c/h2\u003e\n\u003cp\u003eIn recursion, a function solves a problem by calling itself with a smaller input.\u003c/p\u003e\n\u003cp\u003eEach recursive call works on a reduced version of the original problem until it reaches a point where no further calls are needed.\u003c/p\u003e","tags":["dsa","recursion"],"title":"Recursion Explained from Basics"},{"content":"Searching is the process of determining whether a given element exists in an array and, if required, returning its index.\nThe choice of searching technique depends entirely on whether the array is sorted or unsorted. Selecting the correct approach directly impacts performance and scalability.\n1. Searching in an Unsorted Array Characteristics Elements are not arranged in any specific order No assumptions can be made about element positions Linear search is the primary technique used Linear Search Concept Each element of the array is checked one by one until the target element is found or the array ends.\nPython Example (Text Only) The algorithm starts from the first element and compares each element with the target value.\nIf a match is found, the index is returned.\nIf the loop finishes without a match, -1 is returned.\nExample logic:\nTraverse the array from index 0 to the last index.\nCompare each element with the target value.\nStop when the element is found.\nTime Complexity Best case: O(1) Average case: O(n) Worst case: O(n) When to Use Linear Search The array is unsorted The array size is small Only a few searches are required Simplicity is more important than performance 2. Searching in a Sorted Array Characteristics Elements are arranged in ascending or descending order Efficient searching techniques can be applied Binary search is commonly used Binary Search Concept Binary search repeatedly divides the search space into halves by comparing the target value with the middle element.\nIf the target is smaller than the middle element, the search continues in the left half.\nIf the target is larger, the search continues in the right half.\nPython Example (Text Only) The algorithm maintains two pointers: low and high.\nThe middle element is calculated.\nThe target is compared with the middle element.\nHalf of the array is discarded in each step.\nThis process continues until the element is found or the search space becomes empty.\nTime Complexity Best case: O(1) Average case: O(log n) Worst case: O(log n) When to Use Binary Search The array is already sorted The dataset is large Multiple searches are required Faster performance is needed 3. Comparison: Sorted vs Unsorted Searching Aspect: Searching Method\nUnsorted Array: Linear search\nSorted Array: Binary search\nAspect: Time Complexity\nUnsorted Array: O(n)\nSorted Array: O(log n)\nAspect: Precondition\nUnsorted Array: No requirement\nSorted Array: Array must be sorted\nAspect: Speed\nUnsorted Array: Slower\nSorted Array: Faster\nAspect: Implementation\nUnsorted Array: Simple\nSorted Array: Moderate\n4. Important Interview Insight Sorting an unsorted array takes O(n log n) time.\nFor a single search operation, sorting first and then applying binary search is inefficient.\nHowever, if multiple searches are required, sorting the array once and then using binary search becomes beneficial.\nThis tradeoff is commonly tested in interviews.\n5. Real-World Examples Unsorted Array Searching User-entered data Log files Sensor readings Sorted Array Searching Contact lists Dictionaries Database indexes Ranking systems 6. Common Mistakes to Avoid Applying binary search on an unsorted array Ignoring edge cases such as empty arrays Not analyzing time complexity Using inefficient searching methods for large datasets Conclusion Linear search is suitable for unsorted arrays and small datasets.\nBinary search is ideal for sorted arrays and large datasets.\nChoosing the correct searching technique is fundamental to writing efficient and scalable algorithms.\n","permalink":"https://blogs.codeverra.com/blog/searching_in_arrays/","summary":"\u003cp\u003eSearching is the process of determining whether a given element exists in an array and, if required, returning its index.\u003c/p\u003e\n\u003cp\u003eThe choice of searching technique depends entirely on whether the array is sorted or unsorted. Selecting the correct approach directly impacts performance and scalability.\u003c/p\u003e\n\u003chr\u003e\n\u003ch2 id=\"1-searching-in-an-unsorted-array\"\u003e1. Searching in an Unsorted Array\u003c/h2\u003e\n\u003ch3 id=\"characteristics\"\u003eCharacteristics\u003c/h3\u003e\n\u003cul\u003e\n\u003cli\u003eElements are not arranged in any specific order\u003c/li\u003e\n\u003cli\u003eNo assumptions can be made about element positions\u003c/li\u003e\n\u003cli\u003eLinear search is the primary technique used\u003c/li\u003e\n\u003c/ul\u003e\n\u003chr\u003e\n\u003ch3 id=\"linear-search\"\u003eLinear Search\u003c/h3\u003e\n\u003ch4 id=\"concept\"\u003eConcept\u003c/h4\u003e\n\u003cp\u003eEach element of the array is checked one by one until the target element is found or the array ends.\u003c/p\u003e","tags":["dsa","searching","algorithms"],"title":"Searching in Arrays (Sorted and Unsorted)"},{"content":"Low-Level Design Interview Questions — A Complete Walkthrough Who this is for: Anyone preparing for low-level design interviews — or anyone who wants to see the OOP, SOLID, and design-pattern concepts from the previous documents come together into real, working designs. We\u0026rsquo;ll walk through six classic problems end-to-end.\nTable of Contents How to Approach Any LLD Interview Problem The Universal Framework Common Pitfalls in LLD Interviews Problem 1 — Design a Parking Lot Problem 2 — Design an Elevator System Problem 3 — Design a Library Management System Problem 4 — Design a Vending Machine Problem 5 — Design a Splitwise-Like Expense Sharing System Problem 6 — Design a Ride-Hailing System (Uber-Lite) Meta-Patterns Across All Six Problems Final Interview Tips 1. How to Approach Any LLD Interview Problem The LLD interview isn\u0026rsquo;t really about the specific problem — parking lot, elevator, chess, whatever. It\u0026rsquo;s about demonstrating a structured thinking process. The interviewer watches how you decompose ambiguity, handle tradeoffs, and communicate design decisions.\nMost candidates fail not because they can\u0026rsquo;t code, but because they:\nDive into writing classes before clarifying requirements. Design for imagined scope instead of actual scope. Can\u0026rsquo;t explain why they chose a particular class structure. Miss the chance to apply SOLID principles or known patterns when they fit naturally. We\u0026rsquo;ll beat all of that by following a consistent framework for every problem.\nThe Mental Model An LLD interview is a conversation, not a monologue. Think of yourself as a design consultant: the interviewer is your client, they\u0026rsquo;ve given you vague requirements, and your job is to produce a clean, extensible design while thinking out loud.\nYou will be judged on:\nClarity of thought — Can you reduce ambiguity into specific decisions? Object identification — Can you find the right classes (not too many, not too few)? Relationships — Do your classes compose in ways that make sense (is-a vs has-a)? SOLID awareness — Are you subtly baking in extensibility? Pattern fluency — Do you reach for patterns when genuinely appropriate? Communication — Can you explain your decisions and weigh tradeoffs? Writing perfect code is not the primary criterion. Writing thoughtful, defensible design is.\n2. The Universal Framework Apply these six steps to every LLD problem. Internalize this; it\u0026rsquo;s your skeleton for the whole interview.\nStep 1 — Clarify Requirements (5-10 min) Never assume. Ask clarifying questions to define scope. Categorize them:\nFunctional requirements — what the system must do.\nWhat are the core actions the system supports? Who are the users/actors? What are the happy-path scenarios? What edge cases exist? Non-functional requirements — qualities of the system.\nScale? (How many users, items, events/sec?) Concurrency? (Single-threaded vs multi-threaded?) Persistence? (In-memory vs database?) Real-time constraints? Out of scope — explicitly state what you won\u0026rsquo;t design.\nPayment gateway internals? No. Frontend UI? No. Authentication? Probably no. State these back to the interviewer: \u0026ldquo;So I\u0026rsquo;ll focus on X, Y, Z, and assume A, B are out of scope — does that work?\u0026rdquo;\nStep 2 — Identify Core Entities (Nouns) Read your requirements. Pull out the nouns. These are your candidate classes.\nParking Lot → Lot, Floor, Spot, Vehicle, Ticket, Payment, Gate, User. Library → Book, Member, Loan, Librarian, Shelf, Reservation.\nNot every noun becomes a class. Filter:\nDistinct identity? A \u0026ldquo;parking spot\u0026rdquo; has identity; a \u0026ldquo;free space\u0026rdquo; doesn\u0026rsquo;t. Distinct behavior or state? If two \u0026ldquo;nouns\u0026rdquo; behave identically, collapse them. Meaningful in the domain? Words like \u0026ldquo;process,\u0026rdquo; \u0026ldquo;system,\u0026rdquo; \u0026ldquo;data\u0026rdquo; are usually not classes. Step 3 — Identify Actions (Verbs) and Actors What are the use cases? For each use case, identify:\nActor — who initiates it (Customer, Admin, System). Action — what happens (Park a car, Issue a book). Collaborators — which entities participate. This shapes your methods. Actions like \u0026ldquo;park a car\u0026rdquo; become methods on ParkingLot or ParkingSpot. Methods follow the entities they logically belong to.\nStep 4 — Define Relationships For each pair of related classes, ask:\nIs-a (inheritance)? Car is-a Vehicle. Has-a (composition)? ParkingLot has-a collection of Floors. Uses-a (dependency)? PaymentProcessor uses a PricingStrategy. Prefer composition over inheritance — this avoids rigid hierarchies (remember the \u0026ldquo;Favor composition over inheritance\u0026rdquo; principle from the OOP doc).\nMultiplicities matter:\nOne-to-one: a Ticket belongs to one Vehicle. One-to-many: a Floor has many Spots. Many-to-many: Members can have many Books, Books can be read by many members over time. Step 5 — Apply SOLID and Patterns Now, with the skeleton in place, look for opportunities:\nStrategy — is there an algorithm that might vary? (Pricing, discount, matching.) State — does an entity have a lifecycle with distinct behaviors? (Elevator, order, vehicle in lot.) Observer — is there a \u0026ldquo;when X changes, notify Y\u0026rdquo; relationship? (Display boards, notifications.) Factory — is object creation complex or variable? (Different vehicle types, different spot types.) Singleton — genuinely one-of-a-kind resource? (Pool manager, coordinator — but be wary of overuse.) Chain of Responsibility — escalation or ordered handling? (Approvals, matching priorities.) Command — actions that need to be queued, undone, or logged? (Bookings, transactions.) You don\u0026rsquo;t have to force patterns. If none apply, that\u0026rsquo;s fine — simple code is often best.\nCheck SOLID:\nSRP — does each class have one reason to change? OCP — can I add a new variant without modifying existing classes? LSP — do subclasses honor their parent\u0026rsquo;s contract? ISP — are interfaces small and focused? DIP — are high-level classes depending on abstractions? Step 6 — Write Code (The Design You\u0026rsquo;ve Designed) Now, only now, write code. If you\u0026rsquo;ve done steps 1-5 well, this part is fast and clean. You\u0026rsquo;re transcribing, not inventing.\nUse:\nPython (or whatever language the interviewer prefers). Clear class and method names. Type hints (shows discipline). Docstrings for classes and non-obvious methods. @dataclass where appropriate to cut boilerplate. Abstract base classes (abc) for contracts. Short working examples showing the API in action. Resist the urge to implement everything. Sketch the key classes; leave simple methods as skeletons. An interview is about design, not LeetCode.\nStep 7 — Discuss Extensions \u0026amp; Tradeoffs Great candidates anticipate follow-ups. Proactively discuss:\n\u0026ldquo;What if we needed to add X?\u0026rdquo; \u0026ldquo;This design assumes single-threaded; here\u0026rsquo;s how we\u0026rsquo;d handle concurrency.\u0026rdquo; \u0026ldquo;We could swap this strategy for that one easily because of X.\u0026rdquo; \u0026ldquo;If scale grew to 1M events/sec, this in-memory map wouldn\u0026rsquo;t work — we\u0026rsquo;d need Y.\u0026rdquo; This shows design maturity. You\u0026rsquo;re not just solving the problem; you\u0026rsquo;re owning it.\n3. Common Pitfalls Avoid these recurring mistakes:\nPitfall 1 — Jumping to Code Too Early You hear \u0026ldquo;parking lot\u0026rdquo; and start writing class ParkingLot:. Don\u0026rsquo;t. Spend 5-10 minutes clarifying first. Interviewers expect this — if you skip it, you\u0026rsquo;re signaling poor process.\nPitfall 2 — Over-Engineering A parking lot for an interview doesn\u0026rsquo;t need a Visitor pattern for pricing strategies combined with an Abstract Factory for spot types. Match complexity to the problem. Start simple; add sophistication only where justified.\nPitfall 3 — Under-Engineering The opposite: one giant ParkingLot class with 40 methods. No inheritance, no strategy for pricing, no separation of concerns. This fails every SOLID test and interviewers notice immediately.\nPitfall 4 — Ignoring the Interviewer The interviewer drops hints. \u0026ldquo;What if the parking lot had multiple floors?\u0026rdquo; isn\u0026rsquo;t a casual question — they want you to account for it. Listen actively and adjust your design.\nPitfall 5 — Silence Think out loud. \u0026ldquo;I\u0026rsquo;m considering making Vehicle abstract with Car/Bike/Truck subclasses — the alternative would be a single class with a type enum. I\u0026rsquo;ll go with inheritance because the pricing and spot-size logic differs genuinely per type.\u0026rdquo; That narration is half the evaluation.\nPitfall 6 — Not Validating the Design After coding, run a mental walkthrough: \u0026ldquo;A customer enters with a car. They grab a ticket. We find a free spot. They park. Later they pay and leave.\u0026rdquo; Catch any gaps live.\nPitfall 7 — Forgetting Edge Cases What if the lot is full? What if a user loses their ticket? What if payment fails? What if the elevator is called on two floors at once? Even if you don\u0026rsquo;t implement every edge case, mentioning them shows maturity.\n4. Problem 1 — Design a Parking Lot The Prompt: Design the object model for a multi-floor parking lot system.\nThis is perhaps the most-asked LLD question in existence. Let\u0026rsquo;s walk through it by-the-book.\nStep 1 — Clarify Requirements Questions to ask:\nMultiple floors? → Yes (assume 3-5 floors). Different vehicle types? → Yes: Bike, Car, Truck. Different spot sizes? → Yes: Small (bikes), Medium (cars), Large (trucks). Pricing model? → Hourly. First hour flat; subsequent hours variable. Entry/exit gates? → Multiple of each. Payments? → Assume cash + card, but payment gateway integration is out of scope. Display boards showing free spots? → Yes, per floor. Concurrency? → Assume single-threaded for simplicity; mention locking in extensions. Persistence? → In-memory. Scope summary: \u0026ldquo;I\u0026rsquo;ll design a multi-floor lot with typed vehicles, typed spots, entry/exit flow, ticket issuance, hourly billing with pluggable pricing, and display boards that show real-time availability. Concurrency handling and database persistence are out of scope.\u0026rdquo;\nStep 2 — Identify Entities (Nouns) From the requirements:\nParkingLot — top-level entity. Floor — a level of the lot. ParkingSpot — a specific spot (sized: small/medium/large). Vehicle — abstract; Bike, Car, Truck are concrete. Ticket — issued on entry; used to compute billing. EntryGate, ExitGate — gates (likely just one class with a role). Payment — records payment for a ticket. DisplayBoard — shows availability per floor. PricingStrategy — pluggable billing algorithm. Reject: \u0026ldquo;customer\u0026rdquo; (they\u0026rsquo;re just an external actor), \u0026ldquo;system\u0026rdquo; (too vague), \u0026ldquo;data\u0026rdquo; (not a thing).\nStep 3 — Identify Actions Park a vehicle: actor = driver + entry gate; produces a ticket. Exit: actor = driver + exit gate; consumes ticket, computes bill, accepts payment, frees the spot. View availability: actor = display board consuming floor state. Step 4 — Relationships ParkingLot has-many Floors (composition). Floor has-many ParkingSpots (composition). Vehicle is the abstract parent; Bike/Car/Truck are concrete subclasses (is-a). ParkingSpot is associated with a SpotSize enum or subtypes. Ticket references one Vehicle and one ParkingSpot. ExitGate uses a PricingStrategy (dependency injection). DisplayBoard observes its Floor (Observer pattern). Step 5 — SOLID and Patterns Strategy — PricingStrategy is pluggable (hourly, flat-rate, weekend-special). Observer — DisplayBoard watches Floor changes. When a spot fills/empties, the board updates automatically. Factory (optional) — VehicleFactory could create vehicles from input, but we can skip this for simplicity. Singleton — ParkingLot is sometimes modeled as a singleton. We\u0026rsquo;ll avoid this unless justified (prefer dependency injection). SRP — pricing lives in the strategy, not the gate; display logic in display board, not floor. OCP — new vehicle types, new spot types, new pricing models: add classes, don\u0026rsquo;t modify. Step 6 — Code from abc import ABC, abstractmethod from enum import Enum from datetime import datetime from typing import Optional import uuid # --- Enums --- class VehicleType(Enum): BIKE = \u0026#34;bike\u0026#34; CAR = \u0026#34;car\u0026#34; TRUCK = \u0026#34;truck\u0026#34; class SpotSize(Enum): SMALL = 1 # fits Bike MEDIUM = 2 # fits Bike, Car LARGE = 3 # fits Bike, Car, Truck # --- Vehicle hierarchy (inheritance for type-specific behavior) --- class Vehicle(ABC): def __init__(self, license_plate: str): self.license_plate = license_plate @property @abstractmethod def type(self) -\u0026gt; VehicleType: pass @property @abstractmethod def required_size(self) -\u0026gt; SpotSize: pass class Bike(Vehicle): @property def type(self): return VehicleType.BIKE @property def required_size(self): return SpotSize.SMALL class Car(Vehicle): @property def type(self): return VehicleType.CAR @property def required_size(self): return SpotSize.MEDIUM class Truck(Vehicle): @property def type(self): return VehicleType.TRUCK @property def required_size(self): return SpotSize.LARGE # --- Parking spot --- class ParkingSpot: def __init__(self, spot_id: str, size: SpotSize): self.spot_id = spot_id self.size = size self.vehicle: Optional[Vehicle] = None @property def is_free(self) -\u0026gt; bool: return self.vehicle is None def can_fit(self, vehicle: Vehicle) -\u0026gt; bool: # A spot can fit a vehicle if it\u0026#39;s at least as large as the vehicle needs return self.size.value \u0026gt;= vehicle.required_size.value def park(self, vehicle: Vehicle): if not self.is_free: raise ValueError(f\u0026#34;Spot {self.spot_id} is occupied\u0026#34;) if not self.can_fit(vehicle): raise ValueError(f\u0026#34;Vehicle doesn\u0026#39;t fit in spot {self.spot_id}\u0026#34;) self.vehicle = vehicle def vacate(self): self.vehicle = None # --- Observer pattern for display boards --- class FloorObserver(ABC): @abstractmethod def update(self, floor): pass class Floor: def __init__(self, floor_number: int, spots: list[ParkingSpot]): self.floor_number = floor_number self.spots = spots self._observers: list[FloorObserver] = [] def attach(self, observer: FloorObserver): self._observers.append(observer) def _notify(self): for o in self._observers: o.update(self) def find_free_spot(self, vehicle: Vehicle) -\u0026gt; Optional[ParkingSpot]: # Best-fit: smallest spot that fits candidates = [s for s in self.spots if s.is_free and s.can_fit(vehicle)] return min(candidates, key=lambda s: s.size.value, default=None) def park(self, vehicle: Vehicle) -\u0026gt; Optional[ParkingSpot]: spot = self.find_free_spot(vehicle) if spot: spot.park(vehicle) self._notify() return spot def vacate(self, spot: ParkingSpot): spot.vacate() self._notify() def free_spots_by_size(self) -\u0026gt; dict[SpotSize, int]: result = {s: 0 for s in SpotSize} for spot in self.spots: if spot.is_free: result[spot.size] += 1 return result class DisplayBoard(FloorObserver): def update(self, floor: Floor): counts = floor.free_spots_by_size() print(f\u0026#34;[Floor {floor.floor_number}] Free — \u0026#34; f\u0026#34;Small: {counts[SpotSize.SMALL]}, \u0026#34; f\u0026#34;Medium: {counts[SpotSize.MEDIUM]}, \u0026#34; f\u0026#34;Large: {counts[SpotSize.LARGE]}\u0026#34;) # --- Ticket --- class Ticket: def __init__(self, vehicle: Vehicle, spot: ParkingSpot, floor_number: int): self.id = str(uuid.uuid4())[:8] self.vehicle = vehicle self.spot = spot self.floor_number = floor_number self.entry_time = datetime.now() self.exit_time: Optional[datetime] = None self.amount: Optional[float] = None # --- Strategy pattern for pricing --- class PricingStrategy(ABC): @abstractmethod def calculate(self, ticket: Ticket) -\u0026gt; float: pass class HourlyPricing(PricingStrategy): RATES = { VehicleType.BIKE: 10, VehicleType.CAR: 30, VehicleType.TRUCK: 60, } def calculate(self, ticket: Ticket) -\u0026gt; float: duration = ticket.exit_time - ticket.entry_time hours = max(1, duration.total_seconds() / 3600) # min 1 hour return self.RATES[ticket.vehicle.type] * hours class FlatRatePricing(PricingStrategy): RATES = {VehicleType.BIKE: 50, VehicleType.CAR: 150, VehicleType.TRUCK: 300} def calculate(self, ticket: Ticket) -\u0026gt; float: return self.RATES[ticket.vehicle.type] # --- Gates --- class EntryGate: def __init__(self, lot): self.lot = lot def admit(self, vehicle: Vehicle) -\u0026gt; Optional[Ticket]: return self.lot.park(vehicle) class ExitGate: def __init__(self, lot, pricing: PricingStrategy): self.lot = lot self.pricing = pricing def process(self, ticket: Ticket) -\u0026gt; float: ticket.exit_time = datetime.now() ticket.amount = self.pricing.calculate(ticket) self.lot.vacate(ticket) return ticket.amount # --- ParkingLot --- class ParkingLot: def __init__(self, floors: list[Floor]): self.floors = floors self.active_tickets: dict[str, Ticket] = {} # ticket_id -\u0026gt; Ticket def park(self, vehicle: Vehicle) -\u0026gt; Optional[Ticket]: for floor in self.floors: spot = floor.park(vehicle) if spot: ticket = Ticket(vehicle, spot, floor.floor_number) self.active_tickets[ticket.id] = ticket return ticket return None # lot full def vacate(self, ticket: Ticket): floor = self.floors[ticket.floor_number] floor.vacate(ticket.spot) self.active_tickets.pop(ticket.id, None) # --- Usage --- def build_lot(): floors = [] for f in range(3): spots = ( [ParkingSpot(f\u0026#34;F{f}-S{i}\u0026#34;, SpotSize.SMALL) for i in range(5)] + [ParkingSpot(f\u0026#34;F{f}-M{i}\u0026#34;, SpotSize.MEDIUM) for i in range(10)] + [ParkingSpot(f\u0026#34;F{f}-L{i}\u0026#34;, SpotSize.LARGE) for i in range(3)] ) floor = Floor(f, spots) floor.attach(DisplayBoard()) floors.append(floor) return ParkingLot(floors) lot = build_lot() entry = EntryGate(lot) exit_gate = ExitGate(lot, HourlyPricing()) t1 = entry.admit(Car(\u0026#34;KA-01-1234\u0026#34;)) t2 = entry.admit(Bike(\u0026#34;KA-01-5678\u0026#34;)) t3 = entry.admit(Truck(\u0026#34;KA-01-9999\u0026#34;)) import time; time.sleep(0.1) amount = exit_gate.process(t1) print(f\u0026#34;\\nCar paid: ₹{amount:.2f}\u0026#34;) Step 7 — Extensions \u0026amp; Tradeoffs If the interviewer asks \u0026ldquo;What if we needed\u0026hellip;\u0026rdquo;:\nReserved spots (electric, handicapped, VIP): add a reserved_for field on ParkingSpot and filter accordingly during find_free_spot. Concurrency (multiple gates admitting at once): wrap spot acquisition in a lock per floor. Or use a concurrent reservation system. Different pricing per day/time: introduce a TimeBasedPricingStrategy that wraps hourly rates with multipliers. Membership-based discounts: introduce a DiscountStrategy applied on top of PricingStrategy (Decorator pattern). Lost ticket: define a special LostTicketPricing strategy (flat heavy fee). Persistence: extract a ParkingRepository interface (DIP); back it with in-memory or DB. Multiple entry/exit gates: gates are already independent; they all share the same lot reference. With concurrency, we\u0026rsquo;d need lot-level locking. Common Pitfalls in This Problem Putting pricing logic inside ExitGate or Ticket — violates SRP and OCP. Isolate it as a Strategy. One ParkingSpot class handling all sizes via conditionals — works, but rigid. Using a SpotSize enum + can_fit() is clean and extensible. Making ParkingLot a Singleton — often unnecessary. If you want \u0026ldquo;one global lot,\u0026rdquo; pass it around via DI. Ignoring display boards — interviewers often ask; always have a story for \u0026ldquo;how does the user see availability?\u0026rdquo; Forgetting the \u0026ldquo;full lot\u0026rdquo; case — your park() must return something meaningful (None, or raise). Patterns Used Strategy — PricingStrategy (pluggable billing). Observer — DisplayBoard observes Floor changes. Template Method (subtle) — Vehicle sets up the skeleton; subclasses fill in type and required_size. SOLID Checks SRP ✓ — Vehicle knows vehicle stuff; Spot knows spot stuff; Pricing knows pricing; DisplayBoard knows display; Gate knows entry/exit flow. OCP ✓ — Add a new vehicle type, new spot size, new pricing strategy, new display type — all without modifying existing code. LSP ✓ — All Vehicle subclasses honor the same contract. ISP ✓ — Interfaces (PricingStrategy, FloorObserver) are narrow. DIP ✓ — ExitGate depends on PricingStrategy (abstraction), not a concrete pricer. 5. Problem 2 — Design an Elevator System The Prompt: Design a multi-elevator system for a building with N floors.\nElevators are trickier than parking lots because of state transitions and request scheduling. This is a great problem to showcase the State pattern.\nStep 1 — Clarify Requirements Questions to ask:\nHow many elevators and floors? → M elevators, N floors (configurable). Two types of requests: external (press button on a floor, want to go up/down) and internal (inside elevator, pick destination). Support both? → Yes. Elevator directions? → Moves up, down, or idle. Door opens/closes? → Yes, as part of the state. Multiple elevators — how are requests assigned? → Need a scheduler/dispatcher. Scheduling algorithm? → Nearest-idle-elevator for this interview, but make it pluggable. Capacity limits? → Optional; mention as an extension. Emergency stop? → Out of scope for core design; mention as extension. Real-time / concurrency? → Simplify: one request at a time, single-threaded. Scope summary: \u0026ldquo;I\u0026rsquo;ll design N floors, M elevators, each elevator having states (idle, moving up/down, doors opening/closing), external and internal request types, and a pluggable dispatcher that selects which elevator handles a request. Concurrency, capacity, and emergency modes are extensions.\u0026rdquo;\nStep 2 — Identify Entities Building — holds floors and elevators. Floor — represents a level; has up/down external buttons. Elevator — the car itself. ElevatorState — Idle, MovingUp, MovingDown, DoorOpen, DoorClosed. Request — abstract; ExternalRequest (from floor) and InternalRequest (from inside). Direction — enum (UP, DOWN, IDLE). Dispatcher — assigns requests to elevators. SchedulingStrategy — pluggable algorithm for selection. Step 3 — Identify Actions User presses floor button (external request): dispatcher picks an elevator, routes the request. User inside elevator presses floor number (internal request): added to that elevator\u0026rsquo;s queue. Elevator moves between floors, opens doors, closes doors, continues or idles. Display shows current floor per elevator. Step 4 — Relationships Building has-many Floors and has-many Elevators. Elevator has-one ElevatorState (changes over time). Elevator maintains a set of target floors (pending stops). Dispatcher uses-a SchedulingStrategy (DIP). Request → ExternalRequest and InternalRequest (inheritance). Step 5 — SOLID and Patterns State — ElevatorState hierarchy models the elevator\u0026rsquo;s lifecycle cleanly. Idle, MovingUp, MovingDown, DoorOpen each encapsulate their behavior and transitions. Strategy — SchedulingStrategy lets us swap algorithms (nearest, round-robin, least-busy). Command (subtle) — Request objects carry the action to perform. Observer (optional) — floor displays could observe elevator position. DIP — Dispatcher depends on SchedulingStrategy abstraction. OCP — new scheduling strategies or states can be added cleanly. Step 6 — Code from abc import ABC, abstractmethod from enum import Enum from typing import Optional import heapq class Direction(Enum): UP = 1 DOWN = -1 IDLE = 0 # --- Request types --- class Request(ABC): @property @abstractmethod def floor(self) -\u0026gt; int: pass class ExternalRequest(Request): def __init__(self, source_floor: int, direction: Direction): self._floor = source_floor self.direction = direction @property def floor(self): return self._floor def __repr__(self): return f\u0026#34;ExternalRequest(floor={self._floor}, dir={self.direction.name})\u0026#34; class InternalRequest(Request): def __init__(self, destination: int): self._floor = destination @property def floor(self): return self._floor def __repr__(self): return f\u0026#34;InternalRequest(dest={self._floor})\u0026#34; # --- State pattern for elevator --- class ElevatorState(ABC): @abstractmethod def step(self, elevator): pass @abstractmethod def name(self) -\u0026gt; str: pass class Idle(ElevatorState): def step(self, elevator): if elevator.has_pending_requests(): target = elevator.next_target() if target \u0026gt; elevator.current_floor: elevator.set_state(MovingUp()) elif target \u0026lt; elevator.current_floor: elevator.set_state(MovingDown()) else: elevator.set_state(DoorOpen()) def name(self): return \u0026#34;IDLE\u0026#34; class MovingUp(ElevatorState): def step(self, elevator): elevator.current_floor += 1 print(f\u0026#34;[Elevator {elevator.id}] Moving UP → floor {elevator.current_floor}\u0026#34;) if elevator.current_floor in elevator.stops: elevator.set_state(DoorOpen()) def name(self): return \u0026#34;MOVING_UP\u0026#34; class MovingDown(ElevatorState): def step(self, elevator): elevator.current_floor -= 1 print(f\u0026#34;[Elevator {elevator.id}] Moving DOWN → floor {elevator.current_floor}\u0026#34;) if elevator.current_floor in elevator.stops: elevator.set_state(DoorOpen()) def name(self): return \u0026#34;MOVING_DOWN\u0026#34; class DoorOpen(ElevatorState): def step(self, elevator): print(f\u0026#34;[Elevator {elevator.id}] Doors open at floor {elevator.current_floor}\u0026#34;) elevator.stops.discard(elevator.current_floor) elevator.set_state(DoorClosed()) def name(self): return \u0026#34;DOOR_OPEN\u0026#34; class DoorClosed(ElevatorState): def step(self, elevator): print(f\u0026#34;[Elevator {elevator.id}] Doors closed\u0026#34;) if elevator.has_pending_requests(): target = elevator.next_target() if target \u0026gt; elevator.current_floor: elevator.set_state(MovingUp()) elif target \u0026lt; elevator.current_floor: elevator.set_state(MovingDown()) else: elevator.set_state(DoorOpen()) else: elevator.set_state(Idle()) def name(self): return \u0026#34;DOOR_CLOSED\u0026#34; # --- Elevator --- class Elevator: def __init__(self, id: int, num_floors: int): self.id = id self.num_floors = num_floors self.current_floor = 0 self.stops: set[int] = set() self._state: ElevatorState = Idle() def set_state(self, state: ElevatorState): self._state = state @property def state_name(self): return self._state.name() def step(self): self._state.step(self) def accept(self, request: Request): self.stops.add(request.floor) if isinstance(self._state, Idle): self._state.step(self) # Wake it up def has_pending_requests(self): return bool(self.stops) def next_target(self) -\u0026gt; int: # Simple: go to nearest stop return min(self.stops, key=lambda f: abs(f - self.current_floor)) @property def direction(self) -\u0026gt; Direction: if isinstance(self._state, MovingUp): return Direction.UP if isinstance(self._state, MovingDown): return Direction.DOWN return Direction.IDLE # --- Strategy for dispatching --- class SchedulingStrategy(ABC): @abstractmethod def select(self, elevators: list[Elevator], request: Request) -\u0026gt; Elevator: pass class NearestCarStrategy(SchedulingStrategy): def select(self, elevators, request): # Idle elevators are best; else pick the one whose current trajectory matches def score(e: Elevator): distance = abs(e.current_floor - request.floor) # Prefer idle (bonus of 0); penalize if direction disagrees if e.direction == Direction.IDLE: return distance # Same direction as request is better moving_toward = ( (e.direction == Direction.UP and request.floor \u0026gt;= e.current_floor) or (e.direction == Direction.DOWN and request.floor \u0026lt;= e.current_floor) ) return distance + (0 if moving_toward else 100) return min(elevators, key=score) class RoundRobinStrategy(SchedulingStrategy): def __init__(self): self._index = 0 def select(self, elevators, request): chosen = elevators[self._index % len(elevators)] self._index += 1 return chosen # --- Dispatcher --- class Dispatcher: def __init__(self, elevators: list[Elevator], strategy: SchedulingStrategy): self.elevators = elevators self.strategy = strategy def submit(self, request: Request, for_elevator: Optional[Elevator] = None): if isinstance(request, InternalRequest) and for_elevator: for_elevator.accept(request) else: chosen = self.strategy.select(self.elevators, request) print(f\u0026#34;\u0026gt;\u0026gt;\u0026gt; Routing {request} to Elevator {chosen.id}\u0026#34;) chosen.accept(request) # --- Simulation tick --- class Building: def __init__(self, num_floors: int, num_elevators: int, strategy: SchedulingStrategy): self.num_floors = num_floors self.elevators = [Elevator(i, num_floors) for i in range(num_elevators)] self.dispatcher = Dispatcher(self.elevators, strategy) def tick(self): \u0026#34;\u0026#34;\u0026#34;One time unit — every elevator advances its state.\u0026#34;\u0026#34;\u0026#34; for e in self.elevators: e.step() # --- Usage --- building = Building(num_floors=10, num_elevators=2, strategy=NearestCarStrategy()) # Simulate some requests building.dispatcher.submit(ExternalRequest(5, Direction.UP)) for _ in range(12): building.tick() # After elevator reaches floor 5, rider presses \u0026#34;floor 8\u0026#34; if building.elevators[0].current_floor == 5 and 8 not in building.elevators[0].stops: building.dispatcher.submit(InternalRequest(8), for_elevator=building.elevators[0]) Step 7 — Extensions \u0026amp; Tradeoffs Scheduling algorithm sophistication — real elevator algorithms (SCAN, LOOK, SSTF) can replace NearestCarStrategy as new strategies. OCP wins. Capacity limit — add max_weight or max_persons; reject internal requests that would overflow. Concurrency — in reality, requests come from multiple floors concurrently. Wrap dispatcher state with a lock, or use a queue processed by a single worker. Emergency stop / maintenance mode — add EmergencyStopped, UnderMaintenance states. Floor displays — Floor could observe each elevator\u0026rsquo;s position via the Observer pattern. Predictive dispatching — assign elevators based on historical traffic patterns (ML-heavy; out of interview scope but worth mentioning). Common Pitfalls Representing states as strings/enums — works but pushes all behavior into if/elif. The State pattern is cleaner here and is exactly the kind of thing interviewers love to see. Coupling dispatcher to nearest-car logic — hard-wiring the algorithm. Use Strategy so alternatives swap in. One massive Elevator.move() method with nested conditionals — fragile and untestable. State objects fix this. Forgetting external vs internal requests have different routing rules — internal requests go to the specific elevator; external go through the dispatcher. Patterns Used State — elevator states (Idle, MovingUp, MovingDown, DoorOpen, DoorClosed). Strategy — SchedulingStrategy for dispatching. Command (subtle) — Request objects encapsulate actions. SOLID Checks SRP ✓ — State transitions in state classes, dispatching in dispatcher, scheduling in strategy. OCP ✓ — Add a new State (emergency), new Strategy (SCAN algorithm) — no existing code changes. LSP ✓ — All states substitute for ElevatorState properly. ISP ✓ — Interfaces are narrow (SchedulingStrategy.select, ElevatorState.step). DIP ✓ — Dispatcher depends on abstract SchedulingStrategy. 6. Problem 3 — Design a Library Management System The Prompt: Design a library system that supports members borrowing books, returning, and reserving.\nLibrary systems showcase relationships (members ↔ books), lifecycle (book states), and policy (borrowing rules).\nStep 1 — Clarify Requirements Questions to ask:\nBook copies? → Yes; multiple physical copies per title. Member types? → Student, Faculty, Guest — different borrowing limits and durations. Reservations? → A member can reserve a currently-borrowed book; first in queue gets it when returned. Fines for late returns? → Yes; per day after due date. Search capabilities? → By title, author, category — basic in-memory search. Multiple branches/libraries? → Single library for core design; extension for multi-branch. Concurrency? → Single-threaded for simplicity. Payment? → Out of scope; just track fine amounts. Scope summary: \u0026ldquo;I\u0026rsquo;ll design a library with book titles and copies, member types with different policies, a borrowing flow with due dates and fines, a reservation queue, and search. Multi-branch, payment processing, and concurrency are extensions.\u0026rdquo;\nStep 2 — Identify Entities Library — the system root. BookItem (a physical copy) vs Book (the title). Member — abstract; Student, Faculty, Guest subclasses. Loan — represents a borrowing record. Reservation — a waiting request for a book. MembershipPolicy — limits + durations + fine rates per member type. Librarian — actor who performs administrative actions. Search — a component for querying books. Key distinction: Book is the abstract title (1984 by Orwell); BookItem is a specific copy (ISBN + physical copy ID). Many interviewers look for this distinction.\nStep 3 — Identify Actions Member borrows a book copy → creates a Loan. Member returns a book → closes the Loan, potentially assesses fine, triggers next reservation. Member reserves a book → adds to reservation queue. Librarian adds/removes books. System searches by title/author/category. Step 4 — Relationships Book has-many BookItem (copies). BookItem has a status: AVAILABLE, BORROWED, RESERVED. Member has-many Loans. Member is linked to a MembershipPolicy (Strategy-like). Reservation links a Member to a Book. Library has-many Books, Members, Loans, Reservations. Step 5 — SOLID and Patterns Strategy — MembershipPolicy (varies per member type: borrowing limits, loan durations, fine rates). Observer (optional) — reservations notify members when their book becomes available. State (optional) — BookItem.status could use State pattern, but since logic is simple, an enum + checks is adequate. This is a judgment call — interviewers appreciate hearing \u0026ldquo;I could use State here but it\u0026rsquo;s overkill for this flow.\u0026rdquo; Factory — MemberFactory could produce the right member type + policy. SRP — policy logic separated from member; search separated from library core. Step 6 — Code from abc import ABC, abstractmethod from enum import Enum from datetime import datetime, timedelta from typing import Optional from collections import deque import uuid # --- Enums --- class BookStatus(Enum): AVAILABLE = \u0026#34;available\u0026#34; BORROWED = \u0026#34;borrowed\u0026#34; LOST = \u0026#34;lost\u0026#34; # --- Domain entities --- class Book: \u0026#34;\u0026#34;\u0026#34;The title (abstract book).\u0026#34;\u0026#34;\u0026#34; def __init__(self, isbn: str, title: str, author: str, category: str): self.isbn = isbn self.title = title self.author = author self.category = category self.copies: list[BookItem] = [] def add_copy(self, copy: \u0026#34;BookItem\u0026#34;): self.copies.append(copy) def available_copy(self) -\u0026gt; Optional[\u0026#34;BookItem\u0026#34;]: return next((c for c in self.copies if c.status == BookStatus.AVAILABLE), None) class BookItem: \u0026#34;\u0026#34;\u0026#34;A physical copy of a book.\u0026#34;\u0026#34;\u0026#34; def __init__(self, book: Book, copy_id: str): self.book = book self.copy_id = copy_id self.status = BookStatus.AVAILABLE # --- Strategy for membership policy --- class MembershipPolicy(ABC): @property @abstractmethod def max_books(self) -\u0026gt; int: pass @property @abstractmethod def loan_days(self) -\u0026gt; int: pass @property @abstractmethod def fine_per_day(self) -\u0026gt; float: pass class StudentPolicy(MembershipPolicy): @property def max_books(self): return 3 @property def loan_days(self): return 14 @property def fine_per_day(self): return 5.0 class FacultyPolicy(MembershipPolicy): @property def max_books(self): return 10 @property def loan_days(self): return 30 @property def fine_per_day(self): return 2.0 class GuestPolicy(MembershipPolicy): @property def max_books(self): return 1 @property def loan_days(self): return 7 @property def fine_per_day(self): return 10.0 # --- Member --- class Member: def __init__(self, member_id: str, name: str, policy: MembershipPolicy): self.member_id = member_id self.name = name self.policy = policy self.active_loans: list[Loan] = [] def can_borrow_more(self) -\u0026gt; bool: return len(self.active_loans) \u0026lt; self.policy.max_books # --- Loan --- class Loan: def __init__(self, member: Member, book_item: BookItem): self.id = str(uuid.uuid4())[:8] self.member = member self.book_item = book_item self.issue_date = datetime.now() self.due_date = self.issue_date + timedelta(days=member.policy.loan_days) self.return_date: Optional[datetime] = None @property def is_overdue(self) -\u0026gt; bool: return datetime.now() \u0026gt; self.due_date and self.return_date is None def calculate_fine(self) -\u0026gt; float: end_date = self.return_date or datetime.now() if end_date \u0026lt;= self.due_date: return 0.0 overdue_days = (end_date - self.due_date).days + 1 return overdue_days * self.member.policy.fine_per_day # --- Reservation --- class Reservation: def __init__(self, member: Member, book: Book): self.member = member self.book = book self.reserved_on = datetime.now() # --- Search --- class BookSearch: def __init__(self, books: list[Book]): self.books = books def by_title(self, title: str) -\u0026gt; list[Book]: return [b for b in self.books if title.lower() in b.title.lower()] def by_author(self, author: str) -\u0026gt; list[Book]: return [b for b in self.books if author.lower() in b.author.lower()] def by_category(self, category: str) -\u0026gt; list[Book]: return [b for b in self.books if b.category.lower() == category.lower()] # --- Library (the facade/coordinator) --- class Library: def __init__(self): self.books: dict[str, Book] = {} # isbn -\u0026gt; Book self.members: dict[str, Member] = {} self.active_loans: dict[str, Loan] = {} # loan_id -\u0026gt; Loan self.reservations: dict[str, deque[Reservation]] = {} # isbn -\u0026gt; queue # Book management def add_book(self, book: Book): self.books[book.isbn] = book def add_member(self, member: Member): self.members[member.member_id] = member @property def search(self) -\u0026gt; BookSearch: return BookSearch(list(self.books.values())) # Core flow def borrow(self, member_id: str, isbn: str) -\u0026gt; Optional[Loan]: member = self.members[member_id] book = self.books[isbn] if not member.can_borrow_more(): print(f\u0026#34;{member.name} has reached borrowing limit.\u0026#34;) return None copy = book.available_copy() if not copy: print(f\u0026#34;No copies available for \u0026#39;{book.title}\u0026#39;. Consider reserving.\u0026#34;) return None copy.status = BookStatus.BORROWED loan = Loan(member, copy) member.active_loans.append(loan) self.active_loans[loan.id] = loan print(f\u0026#34;Issued \u0026#39;{book.title}\u0026#39; to {member.name}. Due: {loan.due_date.date()}\u0026#34;) return loan def return_book(self, loan_id: str) -\u0026gt; float: loan = self.active_loans.pop(loan_id) loan.return_date = datetime.now() fine = loan.calculate_fine() loan.book_item.status = BookStatus.AVAILABLE loan.member.active_loans.remove(loan) print(f\u0026#34;Returned \u0026#39;{loan.book_item.book.title}\u0026#39;. Fine: ₹{fine:.2f}\u0026#34;) # Check reservation queue isbn = loan.book_item.book.isbn queue = self.reservations.get(isbn) if queue: next_reservation = queue.popleft() print(f\u0026#34; Notifying {next_reservation.member.name}: \u0026#34; f\u0026#34;\u0026#39;{loan.book_item.book.title}\u0026#39; is now available for you.\u0026#34;) return fine def reserve(self, member_id: str, isbn: str): member = self.members[member_id] book = self.books[isbn] if book.available_copy(): print(f\u0026#34;Book \u0026#39;{book.title}\u0026#39; is available — just borrow it.\u0026#34;) return self.reservations.setdefault(isbn, deque()).append(Reservation(member, book)) print(f\u0026#34;{member.name} reserved \u0026#39;{book.title}\u0026#39;.\u0026#34;) # --- Usage --- library = Library() # Add books b1 = Book(\u0026#34;978-001\u0026#34;, \u0026#34;The Pragmatic Programmer\u0026#34;, \u0026#34;Hunt \u0026amp; Thomas\u0026#34;, \u0026#34;Tech\u0026#34;) b1.add_copy(BookItem(b1, \u0026#34;c1\u0026#34;)) b1.add_copy(BookItem(b1, \u0026#34;c2\u0026#34;)) library.add_book(b1) # Add members amit = Member(\u0026#34;M1\u0026#34;, \u0026#34;Amit\u0026#34;, StudentPolicy()) priya = Member(\u0026#34;M2\u0026#34;, \u0026#34;Priya\u0026#34;, FacultyPolicy()) library.add_member(amit) library.add_member(priya) # Flow loan1 = library.borrow(\u0026#34;M1\u0026#34;, \u0026#34;978-001\u0026#34;) loan2 = library.borrow(\u0026#34;M2\u0026#34;, \u0026#34;978-001\u0026#34;) loan3 = library.borrow(\u0026#34;M1\u0026#34;, \u0026#34;978-001\u0026#34;) # No copies — should reserve library.reserve(\u0026#34;M1\u0026#34;, \u0026#34;978-001\u0026#34;) library.return_book(loan1.id) # Should notify Amit Step 7 — Extensions \u0026amp; Tradeoffs Multiple branches — introduce a Branch class; Library becomes a network of branches. Books can be transferred between branches. Digital books / e-books — abstract BorrowableItem; BookItem and EbookLicense become subtypes. Notifications — Observer pattern: when a reservation becomes available, notify the member via email/SMS. Fine payment flow — add a Payment component; extend return_book to accept payment. Membership upgrade — Policy is already a strategy — just swap the policy attribute. Popularity tracking — add a BorrowingHistory per book; use for recommendations (out of scope but a great mention). Common Pitfalls Not distinguishing Book (title) from BookItem (copy) — very common mistake; leads to modeling headaches when the library has 10 copies of the same book. Stuffing all policy logic inside Member — violates SRP and makes it hard to add new member types. Ignoring the reservation queue — interviewers ask about this often. Fines computed in Library — should be on Loan (it has the data) or a dedicated FineCalculator if logic grows complex. Patterns Used Strategy — MembershipPolicy (student/faculty/guest). Facade — Library provides a simple interface over the internal moving parts. Repository-like pattern (implicitly) — self.books, self.members are in-memory repositories. SOLID Checks SRP ✓ — Policies, loans, searches, reservations, and library administration are separate. OCP ✓ — New member types (Premium, Alumni) = new policy classes, no modification. LSP ✓ — All MembershipPolicy variants are substitutable. ISP ✓ — MembershipPolicy has a focused interface. DIP ✓ — Member depends on MembershipPolicy abstraction, not a concrete policy. 7. Problem 4 — Design a Vending Machine The Prompt: Design a vending machine that sells items, accepts coins/notes, returns change, and handles inventory.\nVending machines are the canonical State pattern interview problem. A clean design here immediately shouts \u0026ldquo;this candidate knows the State pattern.\u0026rdquo;\nStep 1 — Clarify Requirements Questions to ask:\nItems have a fixed set of prices and quantities? → Yes; inventory is pre-loaded. Accepted denominations? → Indian coins + notes: ₹1, ₹2, ₹5, ₹10, ₹20, ₹50, ₹100. Exact change required? → No; machine gives change if possible. If not enough change, reject or notify. Cancel button (refund without purchase)? → Yes. Restocking? → Yes; administrator interface. Concurrency? → Assume one user at a time. Scope summary: \u0026ldquo;I\u0026rsquo;ll design a vending machine with product inventory, multi-denomination payment, a clear state lifecycle (idle → has money → dispensing), change-giving, cancel/refund, and restock. Concurrency and payment gateways are extensions.\u0026rdquo;\nStep 2 — Identify Entities VendingMachine — the overall system. Product — an item sold (name, price, code). Inventory — tracks product stock and coin/note availability. VendingMachineState — Idle, HasMoney, Dispensing, OutOfStock (State pattern). Coin / Note — denominations. PaymentProcessor — handles coin/note insertion and change calculation. Administrator — restocks and collects money. Step 3 — Identify Actions Insert coin/note → adds to current balance; transitions Idle → HasMoney. Select product → checks stock and balance; dispenses + returns change; transitions HasMoney → Dispensing → Idle. Cancel → refund the balance; transitions back to Idle. Restock (admin) → add items / denominations. Step 4 — Relationships VendingMachine has-a Inventory, has-a current VendingMachineState, has-a current balance. State classes know how to transition the machine. Inventory has-many product slots + denomination counts. Step 5 — SOLID and Patterns State — the heart of the design. Behavior changes drastically based on current state. Strategy (optional) — ChangeStrategy for computing change (greedy vs exact-only). Command (optional) — represent actions as commands for logging/undo. Often overkill for this problem. SRP — inventory management separate from state management, payment separate from dispensing. Step 6 — Code from abc import ABC, abstractmethod from enum import Enum from typing import Optional # --- Product --- class Product: def __init__(self, code: str, name: str, price: int): self.code = code self.name = name self.price = price # in rupees (integer for simplicity) # --- Inventory --- class Inventory: def __init__(self): self.products: dict[str, tuple[Product, int]] = {} # code -\u0026gt; (product, qty) self.coins: dict[int, int] = {1: 0, 2: 0, 5: 0, 10: 0, 20: 0, 50: 0, 100: 0} def add_product(self, product: Product, qty: int): if product.code in self.products: p, existing = self.products[product.code] self.products[product.code] = (p, existing + qty) else: self.products[product.code] = (product, qty) def stock_coins(self, denomination: int, count: int): if denomination not in self.coins: raise ValueError(f\u0026#34;Invalid denomination: {denomination}\u0026#34;) self.coins[denomination] += count def get_product(self, code: str) -\u0026gt; Optional[Product]: if code not in self.products: return None return self.products[code][0] def has_stock(self, code: str) -\u0026gt; bool: return code in self.products and self.products[code][1] \u0026gt; 0 def remove_one(self, code: str): p, qty = self.products[code] if qty == 0: raise ValueError(\u0026#34;Out of stock\u0026#34;) self.products[code] = (p, qty - 1) def can_provide_change(self, amount: int) -\u0026gt; bool: # Greedy check — can we make this amount? remaining = amount for denom in sorted(self.coins.keys(), reverse=True): use = min(self.coins[denom], remaining // denom) remaining -= use * denom return remaining == 0 def dispense_change(self, amount: int) -\u0026gt; dict[int, int]: dispensed = {} remaining = amount for denom in sorted(self.coins.keys(), reverse=True): use = min(self.coins[denom], remaining // denom) if use \u0026gt; 0: dispensed[denom] = use self.coins[denom] -= use remaining -= use * denom if remaining != 0: raise ValueError(\u0026#34;Can\u0026#39;t make exact change\u0026#34;) return dispensed # --- State pattern --- class VendingState(ABC): @abstractmethod def insert_money(self, machine, denomination: int): pass @abstractmethod def select_product(self, machine, code: str): pass @abstractmethod def cancel(self, machine): pass class Idle(VendingState): def insert_money(self, machine, denomination): machine.inventory.stock_coins(denomination, 1) # machine absorbs it machine.balance += denomination print(f\u0026#34;[Idle → HasMoney] Balance: ₹{machine.balance}\u0026#34;) machine.set_state(HasMoney()) def select_product(self, machine, code): print(\u0026#34;Insert money first.\u0026#34;) def cancel(self, machine): print(\u0026#34;Nothing to cancel.\u0026#34;) class HasMoney(VendingState): def insert_money(self, machine, denomination): machine.inventory.stock_coins(denomination, 1) machine.balance += denomination print(f\u0026#34;Balance: ₹{machine.balance}\u0026#34;) def select_product(self, machine, code): product = machine.inventory.get_product(code) if not product: print(f\u0026#34;Unknown product code: {code}\u0026#34;); return if not machine.inventory.has_stock(code): print(f\u0026#34;\u0026#39;{product.name}\u0026#39; is out of stock\u0026#34;); return if machine.balance \u0026lt; product.price: print(f\u0026#34;Insufficient balance. Need ₹{product.price - machine.balance} more.\u0026#34;); return change = machine.balance - product.price if change \u0026gt; 0 and not machine.inventory.can_provide_change(change): print(f\u0026#34;Can\u0026#39;t give ₹{change} change. Please use exact amount or cancel.\u0026#34;) return print(f\u0026#34;[HasMoney → Dispensing]\u0026#34;) machine.set_state(Dispensing()) machine.dispense(product, change) def cancel(self, machine): # Refund — but we\u0026#39;ve already pooled the coins. In reality we\u0026#39;d track separately. refund = machine.inventory.dispense_change(machine.balance) print(f\u0026#34;Refunding ₹{machine.balance} → {refund}\u0026#34;) machine.balance = 0 machine.set_state(Idle()) class Dispensing(VendingState): def insert_money(self, m, d): print(\u0026#34;Busy dispensing. Please wait.\u0026#34;) def select_product(self, m, code): print(\u0026#34;Busy dispensing.\u0026#34;) def cancel(self, m): print(\u0026#34;Can\u0026#39;t cancel during dispense.\u0026#34;) # --- Vending machine --- class VendingMachine: def __init__(self, inventory: Inventory): self.inventory = inventory self.balance = 0 self._state: VendingState = Idle() def set_state(self, state: VendingState): self._state = state def insert_money(self, denomination: int): self._state.insert_money(self, denomination) def select_product(self, code: str): self._state.select_product(self, code) def cancel(self): self._state.cancel(self) def dispense(self, product: Product, change: int): self.inventory.remove_one(product.code) print(f\u0026#34;Dispensed: {product.name}\u0026#34;) if change \u0026gt; 0: change_dispensed = self.inventory.dispense_change(change) print(f\u0026#34;Change: ₹{change} → {change_dispensed}\u0026#34;) self.balance = 0 self.set_state(Idle()) # --- Usage --- inv = Inventory() inv.add_product(Product(\u0026#34;A1\u0026#34;, \u0026#34;Coke\u0026#34;, 30), 5) inv.add_product(Product(\u0026#34;A2\u0026#34;, \u0026#34;Chips\u0026#34;, 20), 5) inv.add_product(Product(\u0026#34;A3\u0026#34;, \u0026#34;Water\u0026#34;, 15), 5) for denom in [1, 2, 5, 10, 20]: inv.stock_coins(denom, 20) # pre-stocked change vm = VendingMachine(inv) vm.select_product(\u0026#34;A1\u0026#34;) # Error: no money vm.insert_money(20) vm.insert_money(10) vm.insert_money(5) # Balance ₹35 vm.select_product(\u0026#34;A1\u0026#34;) # Dispense Coke, ₹5 change Step 7 — Extensions \u0026amp; Tradeoffs Exact-change-only mode — when low on change denominations, machine refuses overpayments. Add a ChangeStrategy. Multiple products in one transaction — change the state machine to allow cart-like behavior. Significantly more complex — mention but don\u0026rsquo;t over-engineer. Digital payment — add PaymentMethod abstraction; CashPayment, UPIPayment, CardPayment as subtypes. Hardware failure handling — states like Jammed or OutOfService. Admin portal — restocking, coin retrieval, sales reports — probably its own class. Concurrency — locking around state transitions (only one user at a time anyway; often not needed). Common Pitfalls Not using State pattern — easy to fall into one big class with if self.state == \u0026quot;idle\u0026quot; everywhere. Interviewers often explicitly hint: \u0026ldquo;think about how the machine\u0026rsquo;s behavior differs based on its current state.\u0026rdquo; Confusing balance (inserted money) with inventory coins (available for change) — in reality these are separate pools; simplifying them into one (as I did) is a modeling choice worth noting. Forgetting the \u0026ldquo;no change available\u0026rdquo; case — a real vending-machine pain; candidates miss this frequently. Treating product code as name — keep them distinct (code for selection, name for display). Patterns Used State — core of the design; Idle, HasMoney, Dispensing. Strategy (implicitly in change logic; explicit if extended). Facade — VendingMachine fronts a subsystem (inventory + state + dispenser). SOLID Checks SRP ✓ — Inventory, State classes, and VendingMachine each have focused roles. OCP ✓ — New states (jammed, maintenance), new denominations, new products — all additive. LSP ✓ — All states substitute uniformly. ISP ✓ — State interface is narrow (3 methods). DIP ✓ — VendingMachine holds a state interface, not a concrete state. 8. Problem 5 — Design a Splitwise-Like Expense Sharing System The Prompt: Design a system where users can record shared expenses (e.g., \u0026ldquo;I paid ₹1200 for dinner; split equally among 4 people\u0026rdquo;) and track who owes whom.\nThis problem stretches beyond simple entity modeling — it exercises policy (split strategies), graph-like debt tracking, and observability of complex state.\nStep 1 — Clarify Requirements Questions to ask:\nHow can expenses be split? → Equally, by exact amounts, by percentage, or by shares. Currency handling? → Single currency for simplicity. Groups? → Yes; users can be in multiple groups; expenses are typically group-scoped but can be one-off. Debt simplification? → Should the system suggest the minimum number of settlements to clear debts? → Yes — a nice add-on showcasing algorithms. Payments/settlements? → Recording a \u0026ldquo;I paid X to Y\u0026rdquo; action reduces debt. Notifications? → Out of scope; mention Observer pattern as extension. Scope summary: \u0026ldquo;Users, groups, expenses split by various strategies, a balance ledger showing who owes whom, settlement recording, and optional debt simplification. Notifications and multi-currency are extensions.\u0026rdquo;\nStep 2 — Identify Entities User — a person in the system. Group — a set of users sharing expenses. Expense — a record of who paid how much and who shares in what way. SplitStrategy — equal, exact, percentage, shares. Split — an individual user\u0026rsquo;s portion of an expense. BalanceSheet — tracks pairwise balances (who owes whom). ExpenseManager — the facade coordinating it all. Step 3 — Identify Actions Create user, group. Add expense (paid by X, split among Y with strategy Z). Record settlement (user A pays user B). View balance (for a user, or for a group). Simplify debts in a group. Step 4 — Relationships Group has-many Users. Expense has-one payer (User), has-many Splits (each tied to a user). BalanceSheet is a map {(user_a, user_b): amount} where positive means a owes b. ExpenseManager orchestrates. Step 5 — SOLID and Patterns Strategy — SplitStrategy is the cleanest fit: different algorithms for splitting. Facade — ExpenseManager provides a simple API over users/groups/expenses/balances. Command (optional) — each action (add expense, settle) could be a Command for undo/audit. Usually out of scope. Observer (optional) — balance updates could notify UI. SRP — each concern isolated. Step 6 — Code from abc import ABC, abstractmethod from dataclasses import dataclass, field from typing import Optional import uuid # --- User / Group --- @dataclass class User: user_id: str name: str email: str def __hash__(self): return hash(self.user_id) def __eq__(self, other): return isinstance(other, User) and self.user_id == other.user_id @dataclass class Group: group_id: str name: str members: list[User] = field(default_factory=list) def add_member(self, user: User): if user not in self.members: self.members.append(user) # --- Split --- @dataclass class Split: user: User amount: float # --- Strategy for splitting --- class SplitStrategy(ABC): @abstractmethod def calculate(self, total: float, participants: list[User], **kwargs) -\u0026gt; list[Split]: pass class EqualSplit(SplitStrategy): def calculate(self, total, participants, **kwargs): n = len(participants) share = round(total / n, 2) splits = [Split(u, share) for u in participants] # Adjust last split for rounding drift drift = total - share * n splits[-1] = Split(splits[-1].user, round(splits[-1].amount + drift, 2)) return splits class ExactSplit(SplitStrategy): def calculate(self, total, participants, *, exact_amounts: dict[User, float], **kwargs): if abs(sum(exact_amounts.values()) - total) \u0026gt; 0.01: raise ValueError(\u0026#34;Exact amounts don\u0026#39;t sum to total\u0026#34;) return [Split(u, exact_amounts[u]) for u in participants] class PercentSplit(SplitStrategy): def calculate(self, total, participants, *, percentages: dict[User, float], **kwargs): if abs(sum(percentages.values()) - 100) \u0026gt; 0.01: raise ValueError(\u0026#34;Percentages don\u0026#39;t sum to 100\u0026#34;) return [Split(u, round(total * percentages[u] / 100, 2)) for u in participants] # --- Expense --- @dataclass class Expense: id: str description: str amount: float payer: User splits: list[Split] group: Optional[Group] = None # --- Balance sheet --- class BalanceSheet: \u0026#34;\u0026#34;\u0026#34; Stores pairwise debts. balances[a][b] = how much a owes b. Invariant: balances[a][b] and balances[b][a] are not both nonzero; we net them out. \u0026#34;\u0026#34;\u0026#34; def __init__(self): self._balances: dict[User, dict[User, float]] = {} def _get(self, a: User, b: User) -\u0026gt; float: return self._balances.get(a, {}).get(b, 0.0) def _set(self, a: User, b: User, amount: float): self._balances.setdefault(a, {})[b] = amount def add_debt(self, debtor: User, creditor: User, amount: float): \u0026#34;\u0026#34;\u0026#34;debtor owes creditor `amount` more.\u0026#34;\u0026#34;\u0026#34; if debtor == creditor: return # Net against reverse direction reverse = self._get(creditor, debtor) if reverse \u0026gt;= amount: self._set(creditor, debtor, round(reverse - amount, 2)) else: # Clear reverse, add remainder to forward remainder = amount - reverse self._set(creditor, debtor, 0.0) self._set(debtor, creditor, round(self._get(debtor, creditor) + remainder, 2)) def get_balance(self, user: User) -\u0026gt; dict[User, float]: \u0026#34;\u0026#34;\u0026#34;Returns dict of other_user -\u0026gt; amount (positive = user owes them, negative = they owe user).\u0026#34;\u0026#34;\u0026#34; result = {} for other, amt in self._balances.get(user, {}).items(): if amt \u0026gt; 0: result[other] = amt for other, inner in self._balances.items(): if other == user: continue amt = inner.get(user, 0.0) if amt \u0026gt; 0: result[other] = result.get(other, 0) - amt return result def all_pairwise(self) -\u0026gt; list[tuple[User, User, float]]: result = [] for debtor, inner in self._balances.items(): for creditor, amt in inner.items(): if amt \u0026gt; 0: result.append((debtor, creditor, amt)) return result # --- ExpenseManager (Facade) --- class ExpenseManager: def __init__(self): self.users: dict[str, User] = {} self.groups: dict[str, Group] = {} self.expenses: list[Expense] = [] self.balance_sheet = BalanceSheet() def add_user(self, user: User): self.users[user.user_id] = user def add_group(self, group: Group): self.groups[group.group_id] = group def add_expense(self, description: str, amount: float, payer: User, participants: list[User], strategy: SplitStrategy, group: Optional[Group] = None, **strategy_kwargs) -\u0026gt; Expense: splits = strategy.calculate(amount, participants, **strategy_kwargs) expense = Expense( id=str(uuid.uuid4())[:8], description=description, amount=amount, payer=payer, splits=splits, group=group, ) self.expenses.append(expense) # Update balances: each non-payer participant owes the payer their split for s in splits: if s.user != payer: self.balance_sheet.add_debt(s.user, payer, s.amount) return expense def record_settlement(self, from_user: User, to_user: User, amount: float): \u0026#34;\u0026#34;\u0026#34;A direct payment reduces debt from_user → to_user.\u0026#34;\u0026#34;\u0026#34; self.balance_sheet.add_debt(to_user, from_user, amount) print(f\u0026#34;Settled: {from_user.name} paid {to_user.name} ₹{amount}\u0026#34;) def show_balances(self, user: User): balances = self.balance_sheet.get_balance(user) if not balances: print(f\u0026#34;{user.name}: all settled!\u0026#34;) return for other, amt in balances.items(): if amt \u0026gt; 0: print(f\u0026#34;{user.name} owes {other.name}: ₹{amt:.2f}\u0026#34;) else: print(f\u0026#34;{other.name} owes {user.name}: ₹{-amt:.2f}\u0026#34;) # --- Usage --- mgr = ExpenseManager() amit = User(\u0026#34;u1\u0026#34;, \u0026#34;Amit\u0026#34;, \u0026#34;amit@x.com\u0026#34;) priya = User(\u0026#34;u2\u0026#34;, \u0026#34;Priya\u0026#34;, \u0026#34;priya@x.com\u0026#34;) rohan = User(\u0026#34;u3\u0026#34;, \u0026#34;Rohan\u0026#34;, \u0026#34;rohan@x.com\u0026#34;) for u in [amit, priya, rohan]: mgr.add_user(u) trip = Group(\u0026#34;g1\u0026#34;, \u0026#34;Goa Trip\u0026#34;) for u in [amit, priya, rohan]: trip.add_member(u) mgr.add_group(trip) # Amit pays ₹1200 for dinner; split equally among all 3 mgr.add_expense(\u0026#34;Dinner\u0026#34;, 1200, amit, [amit, priya, rohan], EqualSplit(), group=trip) # Priya pays ₹900 for cab; Amit owes 500, Rohan owes 400 (exact) mgr.add_expense(\u0026#34;Cab\u0026#34;, 900, priya, [amit, priya, rohan], ExactSplit(), group=trip, exact_amounts={amit: 500, priya: 0, rohan: 400}) mgr.show_balances(amit) mgr.show_balances(priya) mgr.show_balances(rohan) # Settlement mgr.record_settlement(amit, priya, 100) print(\u0026#34;---\u0026#34;) mgr.show_balances(amit) mgr.show_balances(priya) Step 7 — Extensions \u0026amp; Tradeoffs Debt simplification — given current pairwise balances, find the minimum-transaction way to settle all. This is a graph problem (min-cashflow); use a greedy algorithm or netting approach. This is a common follow-up to this problem in interviews. Multiple currencies — store expenses in native currency; convert via a CurrencyConverter when computing balances in a user\u0026rsquo;s preferred currency. Multi-payer expenses — someone pays part, another pays part. Extend the payer field to payers: list[Split]. Activity feed — Observer pattern: UI observes balance updates. Immutable history — represent each expense as a Command for auditability and potential undo. Concurrency — if multiple users add expenses simultaneously to a group, use proper transactions. Interview-level: mention and move on. Common Pitfalls Conflating Expense and Split — Expense is the event; Split is one person\u0026rsquo;s share. Keep them separate. Not using Strategy for splitting — hardcoding equal/exact/percent with conditionals loses OCP. Double-counting balances — forgetting to subtract/net against reverse direction. Floating-point for money — interviewers often ask; in production use Decimal or cents-as-int. Mention this tradeoff even if you use floats for demo. Over-engineering the balance sheet — a simple dict-of-dicts works for interviews; distributed ledger stuff is out of scope. Patterns Used Strategy — SplitStrategy variants. Facade — ExpenseManager provides a simple API over everything. Composite (subtle) — Groups contain users; expenses compose splits. SOLID Checks SRP ✓ — Split strategies, balance sheet, manager all separate. OCP ✓ — New split algorithm (e.g., SharesSplit) = new class. LSP ✓ — All strategies substitutable. ISP ✓ — SplitStrategy is narrow. DIP ✓ — Manager depends on SplitStrategy abstraction. 9. Problem 6 — Design a Ride-Hailing System The Prompt: Design a ride-hailing service (like Uber/Ola) at the LLD level — riders request rides, drivers accept, trips complete, fares are computed.\nThis is the most open-ended of the six. It\u0026rsquo;s a great showcase for multiple patterns working together and for thinking about extensibility and scale.\nStep 1 — Clarify Requirements Questions to ask:\nCore flow? → Rider requests ride → system matches nearby driver → driver accepts → trip happens → fare calculated → payment recorded. Multiple ride types? → Yes: mini, sedan, SUV; each with different pricing. Surge pricing? → Yes, during high demand. Matching algorithm? → Nearest available driver to pickup; pluggable. Payment methods? → Card, UPI, wallet, cash; external gateway is out of scope. Ratings? → Both rider and driver rate each other. Concurrency? → Real system is distributed; for LLD, assume in-memory, optional mention of locking. Cancellation? → Yes; with potential fee depending on when canceled. Scope summary: \u0026ldquo;I\u0026rsquo;ll design entities (User, Rider, Driver, Vehicle, Trip), a matching flow with pluggable strategies, a lifecycle state for trips, pricing with vehicle type and surge, rating capture, and payment recording. Real-time geolocation, persistence, and concurrency are extensions.\u0026rdquo;\nStep 2 — Identify Entities User (base), Rider, Driver — inheritance. Vehicle — driver owns one; attributes (type, plate). Trip — the core unit: rider, driver, vehicle, source, destination, fare, state. Location — (lat, lng) + helpers. TripState — Requested, DriverAssigned, Started, Completed, Cancelled (State pattern). MatchingStrategy — picks a driver for a ride request. PricingStrategy — computes fare. SurgeCalculator — multiplier for peak hours. PaymentMethod — abstract; subtypes for each method. RideManager — the orchestrator/facade. Step 3 — Identify Actions Rider requests a ride. System matches a driver; driver accepts (or declines, falling through to next). Driver starts the trip upon pickup. Driver ends the trip at drop-off; fare is calculated. Payment is processed. Rider/driver rate each other. Cancellation can happen before driver accepts, before trip starts, or mid-trip — different fee rules. Step 4 — Relationships Driver has-one Vehicle. Trip has-one Rider, has-one Driver, has-one Vehicle, has-one TripState. RideManager has-many drivers, has-many trips. MatchingStrategy, PricingStrategy, SurgeCalculator are dependencies of RideManager. Step 5 — SOLID and Patterns State — TripState (Requested, DriverAssigned, Started, Completed, Cancelled). Strategy — matching algorithm, pricing algorithm, payment method — three strategies! This problem naturally attracts multiple Strategy applications. Observer (optional) — drivers observe ride-request events in their area; riders observe trip state changes for the app UI. Factory (optional) — PaymentMethodFactory for creating payment objects. Facade — RideManager as the orchestrator. SRP — each component owns one concern. DIP — manager depends on strategy abstractions. Step 6 — Code from abc import ABC, abstractmethod from enum import Enum from dataclasses import dataclass, field from typing import Optional from datetime import datetime import uuid import math # --- Location helper --- @dataclass class Location: lat: float lng: float def distance_to(self, other: \u0026#34;Location\u0026#34;) -\u0026gt; float: # Simplified: treat lat/lng as flat plane; enough for LLD return math.sqrt((self.lat - other.lat) ** 2 + (self.lng - other.lng) ** 2) * 111 # km # --- Users --- class User: def __init__(self, user_id: str, name: str, phone: str): self.user_id = user_id self.name = name self.phone = phone class Rider(User): pass class VehicleType(Enum): MINI = \u0026#34;mini\u0026#34; SEDAN = \u0026#34;sedan\u0026#34; SUV = \u0026#34;suv\u0026#34; @dataclass class Vehicle: plate: str type: VehicleType class Driver(User): def __init__(self, user_id, name, phone, vehicle: Vehicle, location: Location): super().__init__(user_id, name, phone) self.vehicle = vehicle self.location = location self.is_available = True self.rating = 5.0 # --- Trip state --- class TripState(ABC): @abstractmethod def name(self) -\u0026gt; str: pass class Requested(TripState): def name(self): return \u0026#34;REQUESTED\u0026#34; class DriverAssigned(TripState): def name(self): return \u0026#34;DRIVER_ASSIGNED\u0026#34; class Started(TripState): def name(self): return \u0026#34;STARTED\u0026#34; class Completed(TripState): def name(self): return \u0026#34;COMPLETED\u0026#34; class Cancelled(TripState): def name(self): return \u0026#34;CANCELLED\u0026#34; # --- Trip --- @dataclass class Trip: id: str rider: Rider source: Location destination: Location vehicle_type: VehicleType state: TripState = field(default_factory=Requested) driver: Optional[Driver] = None started_at: Optional[datetime] = None completed_at: Optional[datetime] = None fare: Optional[float] = None rider_rating_of_driver: Optional[int] = None driver_rating_of_rider: Optional[int] = None # --- Matching strategy --- class MatchingStrategy(ABC): @abstractmethod def select(self, trip: Trip, drivers: list[Driver]) -\u0026gt; Optional[Driver]: pass class NearestDriverStrategy(MatchingStrategy): def select(self, trip: Trip, drivers): candidates = [d for d in drivers if d.is_available and d.vehicle.type == trip.vehicle_type] if not candidates: return None return min(candidates, key=lambda d: d.location.distance_to(trip.source)) class HighestRatedStrategy(MatchingStrategy): \u0026#34;\u0026#34;\u0026#34;Among nearby drivers, pick the highest-rated.\u0026#34;\u0026#34;\u0026#34; def __init__(self, radius_km: float = 5): self.radius_km = radius_km def select(self, trip, drivers): candidates = [ d for d in drivers if d.is_available and d.vehicle.type == trip.vehicle_type and d.location.distance_to(trip.source) \u0026lt;= self.radius_km ] if not candidates: return None return max(candidates, key=lambda d: d.rating) # --- Pricing strategy --- class PricingStrategy(ABC): @abstractmethod def calculate(self, trip: Trip, surge: float) -\u0026gt; float: pass class DistanceBasedPricing(PricingStrategy): BASE_FARE = {VehicleType.MINI: 50, VehicleType.SEDAN: 80, VehicleType.SUV: 120} PER_KM = {VehicleType.MINI: 10, VehicleType.SEDAN: 15, VehicleType.SUV: 20} def calculate(self, trip, surge): distance = trip.source.distance_to(trip.destination) fare = self.BASE_FARE[trip.vehicle_type] + self.PER_KM[trip.vehicle_type] * distance return round(fare * surge, 2) class SurgeCalculator: \u0026#34;\u0026#34;\u0026#34;Computes surge based on supply/demand. Stub for interview.\u0026#34;\u0026#34;\u0026#34; def current_multiplier(self, pickup: Location, available_drivers: int) -\u0026gt; float: if available_drivers \u0026lt; 2: return 2.0 if available_drivers \u0026lt; 5: return 1.3 return 1.0 # --- Payment --- class PaymentMethod(ABC): @abstractmethod def charge(self, amount: float, user: User) -\u0026gt; bool: pass class CardPayment(PaymentMethod): def charge(self, amount, user): print(f\u0026#34;Charging ₹{amount} on {user.name}\u0026#39;s card\u0026#34;) return True class UPIPayment(PaymentMethod): def charge(self, amount, user): print(f\u0026#34;Charging ₹{amount} via UPI to {user.name}\u0026#34;) return True class CashPayment(PaymentMethod): def charge(self, amount, user): print(f\u0026#34;{user.name} to pay ₹{amount} cash to driver\u0026#34;) return True # --- Ride Manager (Facade) --- class RideManager: def __init__(self, matching: MatchingStrategy, pricing: PricingStrategy, surge: SurgeCalculator): self.drivers: dict[str, Driver] = {} self.trips: dict[str, Trip] = {} self.matching = matching self.pricing = pricing self.surge = surge def register_driver(self, d: Driver): self.drivers[d.user_id] = d def request_ride(self, rider: Rider, source: Location, destination: Location, vehicle_type: VehicleType) -\u0026gt; Optional[Trip]: trip = Trip( id=str(uuid.uuid4())[:8], rider=rider, source=source, destination=destination, vehicle_type=vehicle_type, ) self.trips[trip.id] = trip # Match driver = self.matching.select(trip, list(self.drivers.values())) if not driver: print(f\u0026#34;No drivers available for {vehicle_type.value}\u0026#34;) trip.state = Cancelled() return trip trip.driver = driver driver.is_available = False trip.state = DriverAssigned() print(f\u0026#34;Trip {trip.id}: Assigned {driver.name} to {rider.name}\u0026#34;) return trip def start_trip(self, trip_id: str): trip = self.trips[trip_id] if not isinstance(trip.state, DriverAssigned): print(f\u0026#34;Can\u0026#39;t start: trip is {trip.state.name()}\u0026#34;) return trip.state = Started() trip.started_at = datetime.now() print(f\u0026#34;Trip {trip_id}: Started\u0026#34;) def complete_trip(self, trip_id: str, payment: PaymentMethod): trip = self.trips[trip_id] if not isinstance(trip.state, Started): print(f\u0026#34;Can\u0026#39;t complete: trip is {trip.state.name()}\u0026#34;) return trip.state = Completed() trip.completed_at = datetime.now() surge = self.surge.current_multiplier( trip.source, sum(1 for d in self.drivers.values() if d.is_available), ) trip.fare = self.pricing.calculate(trip, surge) print(f\u0026#34;Trip {trip_id}: Completed. Fare: ₹{trip.fare} (surge x{surge})\u0026#34;) payment.charge(trip.fare, trip.rider) trip.driver.is_available = True trip.driver.location = trip.destination def cancel_trip(self, trip_id: str): trip = self.trips[trip_id] if isinstance(trip.state, (Completed, Cancelled)): print(f\u0026#34;Can\u0026#39;t cancel: trip already {trip.state.name()}\u0026#34;) return # Cancellation fees could depend on current state fee = 0 if isinstance(trip.state, DriverAssigned): fee = 30 elif isinstance(trip.state, Started): fee = 100 trip.state = Cancelled() if trip.driver: trip.driver.is_available = True print(f\u0026#34;Trip {trip_id}: Cancelled. Fee: ₹{fee}\u0026#34;) def rate_trip(self, trip_id: str, by_rider: Optional[int] = None, by_driver: Optional[int] = None): trip = self.trips[trip_id] if not isinstance(trip.state, Completed): print(\u0026#34;Can only rate completed trips\u0026#34;) return if by_rider is not None: trip.rider_rating_of_driver = by_rider # update driver\u0026#39;s average rating (simplified) trip.driver.rating = (trip.driver.rating + by_rider) / 2 if by_driver is not None: trip.driver_rating_of_rider = by_driver # --- Usage --- mgr = RideManager( matching=NearestDriverStrategy(), pricing=DistanceBasedPricing(), surge=SurgeCalculator(), ) # Register drivers d1 = Driver(\u0026#34;d1\u0026#34;, \u0026#34;Raj\u0026#34;, \u0026#34;9999\u0026#34;, Vehicle(\u0026#34;KA01-AA\u0026#34;, VehicleType.SEDAN), Location(12.97, 77.59)) d2 = Driver(\u0026#34;d2\u0026#34;, \u0026#34;Sita\u0026#34;, \u0026#34;8888\u0026#34;, Vehicle(\u0026#34;KA02-BB\u0026#34;, VehicleType.SEDAN), Location(12.96, 77.58)) d3 = Driver(\u0026#34;d3\u0026#34;, \u0026#34;Ahmed\u0026#34;, \u0026#34;7777\u0026#34;, Vehicle(\u0026#34;KA03-CC\u0026#34;, VehicleType.MINI), Location(12.98, 77.60)) for d in [d1, d2, d3]: mgr.register_driver(d) # Create a rider and request rider = Rider(\u0026#34;r1\u0026#34;, \u0026#34;Amit\u0026#34;, \u0026#34;6666\u0026#34;) trip = mgr.request_ride(rider, Location(12.97, 77.59), Location(12.95, 77.55), VehicleType.SEDAN) mgr.start_trip(trip.id) mgr.complete_trip(trip.id, UPIPayment()) mgr.rate_trip(trip.id, by_rider=5, by_driver=4) Step 7 — Extensions \u0026amp; Tradeoffs Real geolocation — Haversine formula or a geo index (quadtree, geohash, H3) for efficient nearest-driver queries. Driver pool scalability — in-memory list is fine for LLD; real system uses a geospatial index (Redis GEO, PostGIS). Event-driven architecture — request, driver acceptance, pickup, drop-off become events on a queue. Observer pattern on top. Cancellation policy refinement — externalize as CancellationPolicy strategy. Driver matching with preferences — e.g., prefer drivers with matching language, female driver option — another layer of strategies or filters. Payment retries and failures — introduce PaymentResult and retry logic. Scheduled / outstation trips — extend Trip with scheduling fields; different state machine. Trip history, receipts — repository pattern for persistence. Common Pitfalls Putting matching logic inside RideManager — violates SRP and OCP. Externalize as Strategy. Hardcoding fare formulas — same issue. Strategy fixes it. Forgetting trip states — State pattern is natural here; without it, you get if/elif everywhere. Having one User class with a flag is_driver — works, but loses polymorphism; interviewers prefer a class per role. Coupling Trip to payment — trip and payment should be separable concerns; manager orchestrates. Ignoring cancellation — a realistic system needs cancellation at multiple states with different rules. Patterns Used State — TripState lifecycle. Strategy — matching, pricing, payment method. Facade — RideManager. Factory (could be introduced) — for creating payment methods by type string. Observer (extension) — for event-driven UI updates. SOLID Checks SRP ✓ — Matching, pricing, surge, payment, manager — all distinct concerns. OCP ✓ — New vehicle types, new matching/pricing/payment strategies — all additive. LSP ✓ — All strategy variants substitutable. ISP ✓ — Narrow interfaces. DIP ✓ — RideManager depends on abstractions. 10. Meta-Patterns Across All Six Problems Look back at the six problems. Patterns keep appearing for the same reasons. Internalizing these meta-patterns lets you recognize design choices quickly in any LLD problem.\nMeta-Pattern 1 — \u0026ldquo;Pluggable Algorithm\u0026rdquo; → Strategy Almost every problem has at least one. Parking has PricingStrategy. Elevator has SchedulingStrategy. Library has MembershipPolicy. Vending machine has change-giving (implicit). Splitwise has SplitStrategy. Ride-hailing has MatchingStrategy, PricingStrategy, PaymentMethod.\nTrigger: Is there an operation that might reasonably be done in more than one way? → Strategy.\nMeta-Pattern 2 — \u0026ldquo;Lifecycle with Distinct Behaviors\u0026rdquo; → State Problems where an entity transitions through named stages, each with different allowed actions, scream State: Elevator (Idle/Moving/DoorOpen), Vending Machine (Idle/HasMoney/Dispensing), Trip (Requested/Assigned/Started/Completed).\nTrigger: Does an entity behave differently based on where it is in its lifecycle, with strict transition rules? → State.\nMeta-Pattern 3 — \u0026ldquo;Orchestrator of Many Parts\u0026rdquo; → Facade Every problem has a top-level coordinator. ParkingLot, Library, VendingMachine, ExpenseManager, RideManager. Each exposes a simple API and internally coordinates sub-components.\nTrigger: Is the system made of several moving parts that clients shouldn\u0026rsquo;t juggle directly? → Facade.\nMeta-Pattern 4 — \u0026ldquo;Notify Interested Parties\u0026rdquo; → Observer Parking\u0026rsquo;s DisplayBoard watching Floor. Library\u0026rsquo;s reservation notifications. Ride-hailing\u0026rsquo;s driver receiving ride requests.\nTrigger: When X happens, does Y need to know, without X knowing Y explicitly? → Observer.\nMeta-Pattern 5 — \u0026ldquo;Is-a Hierarchy with Behavioral Variance\u0026rdquo; → Inheritance + Polymorphism Vehicle → Bike/Car/Truck. User → Rider/Driver. Member → Student/Faculty/Guest (though we used Strategy for their policies, which is often cleaner).\nTrigger: Do I have related types with genuinely different behaviors or attributes? → Inheritance.\nMeta-Pattern 6 — \u0026ldquo;Depend on Abstraction, Inject Concrete\u0026rdquo; → DIP Every orchestrator in these problems receives strategies via its constructor rather than newing them up internally. This enables testing (inject a fake), swapping implementations, and configuration.\nTrigger: Always. You should almost always inject dependencies rather than hardcoding them.\nHow to Use This Meta-View in Interviews When you hear a new LLD problem, run through these triggers:\nWhat varies in algorithm? → Strategy candidates. What has a lifecycle? → State candidates. What\u0026rsquo;s the top-level orchestrator? → Facade. Who watches what? → Observer candidates. What are my main noun hierarchies? → Inheritance. Where should I inject vs construct? → DIP everywhere. Most LLD problems will hit 3-5 of these. Hitting all six is unusual but impressive.\n11. Final Interview Tips Before the Interview Study the six problems in this document thoroughly. Most actual LLD interview questions are variations or combinations of these. Practice verbalizing your thinking. Design well internally, but also articulate well externally. A great design poorly explained is a mediocre interview. Practice on paper or a whiteboard without running code. You won\u0026rsquo;t have python to validate syntax — you need to write clean code from memory. Memorize the 6-step framework. Clarify → Entities → Actions → Relationships → SOLID/Patterns → Code → Extensions. Know your language well. Type hints, ABCs, dataclasses, enums — these show polish. During the Interview Always start by clarifying. If you dive into code without clarifying, the interviewer will steer you back — that\u0026rsquo;s wasted time. Speak constantly. Silence is the enemy. Even \u0026ldquo;I\u0026rsquo;m considering X vs Y; I\u0026rsquo;ll go with X because\u0026hellip;\u0026rdquo; is valuable. Write clean code, but not complete code. Sketch the main classes fully; use method stubs for secondary ones. Say \u0026ldquo;I\u0026rsquo;ll stub this; the logic would be\u0026hellip;\u0026rdquo;. Explicitly call out patterns and SOLID. \u0026ldquo;I\u0026rsquo;m using Strategy here for pricing because\u0026hellip;\u0026rdquo; — interviewers love this narration. Handle pushback gracefully. If the interviewer pokes at your design, they\u0026rsquo;re probably hinting at a problem. Listen, reconsider, adjust. Stay flexible. If you over-designed, admit it: \u0026ldquo;Actually, for this scope, a simple dict would be clearer than a full factory.\u0026rdquo; That\u0026rsquo;s a win, not a loss. Common Follow-Up Questions Interviewers often probe with:\n\u0026ldquo;How would you test this?\u0026rdquo; → Talk about unit tests for each strategy/state; mock the DB/repository; test the orchestrator with fakes. Show you understand dependency injection makes testing easy. \u0026ldquo;How would this scale to 1 million users?\u0026rdquo; → Move in-memory maps to a database; introduce caching (Redis); shard by user_id; async processing via queues. \u0026ldquo;What if X was distributed?\u0026rdquo; → Mention event-driven architecture, idempotency, eventual consistency. \u0026ldquo;Are there concurrency issues?\u0026rdquo; → Identify critical sections; use locks or transactional boundaries; mention optimistic vs pessimistic locking. \u0026ldquo;What patterns did you use and why?\u0026rdquo; → Be ready to articulate not just what pattern, but what problem it solved in your design. \u0026ldquo;What would you change?\u0026rdquo; → Always have at least one self-identified improvement. Shows self-awareness. Pitfalls Interviewers Specifically Look For From the interviewer\u0026rsquo;s perspective, red flags include:\nWriting code before clarifying. One giant class handling everything. Hardcoded strategies (if type == \u0026quot;premium\u0026quot;:). No inheritance or polymorphism when they\u0026rsquo;d obviously fit. No state pattern where a clear lifecycle exists. Ignoring the interviewer\u0026rsquo;s hints. Being unable to explain any design choice. Perfect code but zero design thinking. The Three Things You Must Demonstrate Regardless of the problem, by the end of the interview you should have clearly demonstrated:\nStructured thinking — a clean, deliberate decomposition of the problem. Design fluency — natural use of SOLID and a few well-chosen patterns. Communication — explaining your reasoning and handling feedback well. If you nail these three, the specific problem almost doesn\u0026rsquo;t matter.\nFinal Thought Designing software is a craft. Patterns, principles, and frameworks like the ones in this series are the tools of the craft — but craftsmanship is what emerges when you apply them with judgment, restraint, and a clear eye for the problem in front of you.\nYou\u0026rsquo;ve walked through OOP fundamentals, SOLID principles, 23 design patterns, and six realistic interview designs. You now have the vocabulary and the judgment to approach any LLD problem with confidence.\nGo practice. Design things. Build things. Break them. Refactor them. That\u0026rsquo;s where real mastery comes from.\nGood luck in your interviews — you\u0026rsquo;ve got the toolkit. Now make it yours.\n","permalink":"https://blogs.codeverra.com/blog/low_level_design/06_lld_interview_questions/","summary":"\u003ch1 id=\"low-level-design-interview-questions--a-complete-walkthrough\"\u003eLow-Level Design Interview Questions — A Complete Walkthrough\u003c/h1\u003e\n\u003cblockquote\u003e\n\u003cp\u003e\u003cstrong\u003eWho this is for:\u003c/strong\u003e Anyone preparing for low-level design interviews — or anyone who wants to see the OOP, SOLID, and design-pattern concepts from the previous documents come together into real, working designs. We\u0026rsquo;ll walk through six classic problems end-to-end.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003chr\u003e\n\u003ch2 id=\"table-of-contents\"\u003eTable of Contents\u003c/h2\u003e\n\u003col\u003e\n\u003cli\u003e\u003ca href=\"#1-how-to-approach-any-lld-interview-problem\"\u003eHow to Approach Any LLD Interview Problem\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#2-the-universal-framework\"\u003eThe Universal Framework\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#3-common-pitfalls\"\u003eCommon Pitfalls in LLD Interviews\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#4-problem-1--design-a-parking-lot\"\u003eProblem 1 — Design a Parking Lot\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#5-problem-2--design-an-elevator-system\"\u003eProblem 2 — Design an Elevator System\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#6-problem-3--design-a-library-management-system\"\u003eProblem 3 — Design a Library Management System\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#7-problem-4--design-a-vending-machine\"\u003eProblem 4 — Design a Vending Machine\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#8-problem-5--design-a-splitwise-like-expense-sharing-system\"\u003eProblem 5 — Design a Splitwise-Like Expense Sharing System\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#9-problem-6--design-a-ride-hailing-system\"\u003eProblem 6 — Design a Ride-Hailing System (Uber-Lite)\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#10-meta-patterns\"\u003eMeta-Patterns Across All Six Problems\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#11-final-interview-tips\"\u003eFinal Interview Tips\u003c/a\u003e\u003c/li\u003e\n\u003c/ol\u003e\n\u003chr\u003e\n\u003ch2 id=\"1-how-to-approach-any-lld-interview-problem\"\u003e1. How to Approach Any LLD Interview Problem\u003c/h2\u003e\n\u003cp\u003eThe LLD interview isn\u0026rsquo;t really about the specific problem — parking lot, elevator, chess, whatever. \u003cstrong\u003eIt\u0026rsquo;s about demonstrating a structured thinking process.\u003c/strong\u003e The interviewer watches \u003cem\u003ehow\u003c/em\u003e you decompose ambiguity, handle tradeoffs, and communicate design decisions.\u003c/p\u003e","tags":null,"title":""},{"content":"LLD Deep Dive #1 — Designing a Parking Lot What this document is: A genuine, step-by-step walkthrough of how to think through a Parking Lot LLD problem in an interview. Not a finished design dropped on you — but the reasoning, the false starts, the alternatives weighed, and the justifications that lead to each design decision.\nTime investment: Read this slowly. If you\u0026rsquo;re doing this for interview prep, it\u0026rsquo;s worth treating each section as its own thinking exercise — pause and ask yourself \u0026ldquo;what would I do here?\u0026rdquo; before reading on.\nTable of Contents Why This Problem? Why First? Stage 1 — Receiving the Problem Stage 2 — Clarifying Requirements Stage 3 — Identifying Entities (And Rejecting Some) Stage 4 — Deciding the Vehicle Hierarchy Stage 5 — Modeling Spots and Sizes Stage 6 — Modeling Floors and the Spot-Finding Logic Stage 7 — Tickets, Gates, and the Park/Exit Flow Stage 8 — Pricing as a Strategy Stage 9 — Display Boards and the Observer Decision Stage 10 — Putting It All Together Stage 11 — Validating with a Mental Walkthrough Stage 12 — Anticipating Follow-Up Questions Final Reflection — What This Problem Teaches 1. Why This Problem? Why First? Parking Lot is the most frequently asked LLD question — for good reason. It\u0026rsquo;s deceptively simple-sounding (\u0026ldquo;just track which spots are full, right?\u0026rdquo;), but every meaningful design decision in it foreshadows a real-world architecture concern:\nHow do you model a hierarchy of related types? (Vehicles.) How do you make policy pluggable? (Pricing.) How do you separate state from notification? (Spots and display boards.) How do you keep the top-level orchestrator from becoming a god class? If you can do this problem well — with full reasoning, not pattern-matching — you\u0026rsquo;ve demonstrated most of what an LLD interview tests. So we\u0026rsquo;re going to do it slowly, deliberately, and out loud.\n2. Stage 1 — Receiving the Problem The interviewer says:\n\u0026ldquo;Design a parking lot.\u0026rdquo;\nThat\u0026rsquo;s it. That\u0026rsquo;s all they said. No floors, no fees, no vehicle types, no scale.\nThis vagueness is intentional. They\u0026rsquo;re testing whether you\u0026rsquo;ll dive in or pause. Most candidates dive in and lose immediately. Let\u0026rsquo;s pause.\nWhat do we actually know? Almost nothing. What do we think the interviewer probably has in mind? A multi-floor commercial parking lot — that\u0026rsquo;s the canonical version. But \u0026ldquo;thinking\u0026rdquo; isn\u0026rsquo;t enough. We must ask.\nWhy is asking better than assuming?\nWrong assumption = wasted design time. If you spend 20 minutes designing for a single-floor lot and they wanted multi-floor, you\u0026rsquo;ve burned half the interview on a wrong premise. Asking shows process. A senior engineer wouldn\u0026rsquo;t start coding without scope. Behaving like a senior engineer is half the evaluation. It elicits hints. When you ask \u0026ldquo;should this support multiple floors?\u0026rdquo;, the interviewer\u0026rsquo;s tone might give it away — they may say \u0026ldquo;yes, definitely\u0026rdquo; (signal: scale matters) or \u0026ldquo;let\u0026rsquo;s keep it simple\u0026rdquo; (signal: don\u0026rsquo;t over-engineer). So our first move is not to think. It\u0026rsquo;s to ask.\n3. Stage 2 — Clarifying Requirements A clarification round is structured. We don\u0026rsquo;t fire random questions; we organize them into:\nFunctional requirements — what does it do? Non-functional requirements — what qualities does it have? Out of scope — what we\u0026rsquo;re explicitly not doing. Let\u0026rsquo;s walk through what we\u0026rsquo;d actually ask, and why each question matters.\nQuestion 1: Single floor or multiple floors? Why ask: Affects the entire structure. Multi-floor means we need a Floor entity; single-floor means ParkingLot directly holds spots. Conflating them is bad design.\nSuppose the interviewer says: \u0026ldquo;Multiple — 3 to 5 floors.\u0026rdquo;\nThat tells us: ParkingLot → Floor → Spot is going to be our containment hierarchy. We\u0026rsquo;re definitely modeling Floor as a first-class concept.\nQuestion 2: Different vehicle types? Why ask: If everything is a generic \u0026ldquo;vehicle,\u0026rdquo; our design collapses to a much simpler form. If types differ in how they\u0026rsquo;re parked, priced, or sized, we need polymorphism.\nSuppose: \u0026ldquo;Yes — bikes, cars, trucks.\u0026rdquo;\nNow we have a hierarchy decision to make later. We\u0026rsquo;ve also implicitly learned that size matters — bikes, cars, and trucks need different-sized spots. Note this.\nQuestion 3: How are spots assigned? Why ask: This affects the spot-finding algorithm. Options:\nAny free spot of correct size? Closest to entrance? Best fit (smallest spot that fits)? Suppose: \u0026ldquo;Best fit — smallest spot that fits the vehicle.\u0026rdquo;\nThis is a single-line answer that drives a real algorithm choice. We\u0026rsquo;ll come back to it.\nQuestion 4: Pricing model? Why ask: Pricing is one of the most varied real-world concerns — flat, hourly, surge, discounts, memberships. If we hardcode it now, we\u0026rsquo;ll regret it.\nSuppose: \u0026ldquo;Hourly. Different rates for bikes, cars, trucks. Round up to next hour.\u0026rdquo;\nThat\u0026rsquo;s enough for v1. The \u0026ldquo;round up to next hour\u0026rdquo; is a small but specific detail — note it; we\u0026rsquo;ll implement it precisely.\nQuestion 5: Multiple gates? Why ask: One entry/exit gate vs many affects concurrency considerations and whether tickets need globally unique IDs.\nSuppose: \u0026ldquo;Multiple entries and exits.\u0026rdquo;\nWe\u0026rsquo;ll keep this in mind. For our single-threaded design we\u0026rsquo;ll mostly ignore it but mention concurrency as an extension.\nQuestion 6: Display boards? Why ask: Cars circling looking for spots is a real UX problem. Display boards solve it. They\u0026rsquo;re also a great chance to show the Observer pattern — interviewers often want to see this.\nSuppose: \u0026ldquo;Yes — per floor, showing spots free per size category.\u0026rdquo;\nExcellent. We\u0026rsquo;ve now committed ourselves to thinking about how Floor notifies a display when its state changes.\nQuestion 7: Payment methods? Why ask: Payment is its own rabbit hole. We want to scope-bound it.\nSuppose: \u0026ldquo;Cash and card. But payment gateway integration is out of scope — just track the amount.\u0026rdquo;\nGood — we don\u0026rsquo;t have to model complex payment flows. We just compute and \u0026ldquo;accept\u0026rdquo; payment.\nQuestion 8: Concurrency? Persistence? Scale? Why ask: These are the non-functional requirements that make or break a real system but often slow down an LLD interview if overemphasized.\nSuppose: \u0026ldquo;Single-threaded is fine. In-memory storage. Mention concurrency as an extension.\u0026rdquo;\nWe\u0026rsquo;re cleared to make a clean, in-memory, single-threaded design and discuss the rest at the end.\nRecap to the Interviewer After the clarification round, state your understanding back:\n\u0026ldquo;To make sure I have this right: a multi-floor lot (3-5 floors), three vehicle types (bike, car, truck) needing differently-sized spots, best-fit allocation, hourly pricing per vehicle type with round-up, multiple entry/exit gates, per-floor display boards showing free spots, and cash/card payments where the gateway is out of scope. I\u0026rsquo;ll design it single-threaded and in-memory, mentioning concurrency and persistence as extensions. Anyone bit you\u0026rsquo;d like to add or change?\u0026rdquo;\nThis recap is critical — it forces the interviewer to confirm your scope. If you missed something, they\u0026rsquo;ll add it now. If you over-included something, they\u0026rsquo;ll trim. Either way, you\u0026rsquo;re aligned before writing a single class.\n4. Stage 3 — Identifying Entities (And Rejecting Some) Now we look at our requirements and pull out nouns. Nouns are candidate classes. But not every noun deserves a class — some are attributes, some are external actors, some are too vague.\nLet me list everything noun-ish that came up:\nparking lot floor spot vehicle (bike, car, truck) entry gate exit gate ticket payment pricing display board driver / customer size system amount hours Now I filter. For each, I ask three questions:\nDoes it have its own identity? (i.e., do I need to reference this specific one later?) Does it have its own behavior or state? (vs being just a value) Is it part of the domain, or is it just generic infrastructure language? Let\u0026rsquo;s apply this:\nNoun Identity? Behavior/State? Verdict parking lot Yes Yes (orchestrates) Class floor Yes Yes (holds spots, notifies) Class spot Yes Yes (occupied/free) Class vehicle Yes Yes (type-specific) Class hierarchy entry gate Yes (multiple) Yes (admit) Class exit gate Yes (multiple) Yes (process exit) Class ticket Yes Yes (entry time, etc.) Class payment Maybe Has amount, method Probably class (small) pricing No identity Has algorithm Strategy/algorithm, not entity display board Yes (multiple) Yes (renders state) Class driver / customer External actor We don\u0026rsquo;t model them — they trigger actions Not a class in this design size No identity Just a label Enum, not class system Too vague — Reject amount Just a number — Reject (it\u0026rsquo;s a field) hours Just a number — Reject (it\u0026rsquo;s a field) Why did we reject \u0026ldquo;driver / customer\u0026rdquo;? Because in this scope we don\u0026rsquo;t need to model the customer. They don\u0026rsquo;t log in, don\u0026rsquo;t have accounts, don\u0026rsquo;t have a history we track. They\u0026rsquo;re an actor outside the system, not a thing inside the system. Adding a Customer class with no fields and no methods would be noise.\nCould that change? Sure — if the next requirement is \u0026ldquo;loyalty programs,\u0026rdquo; we\u0026rsquo;d add Customer. But for now, reject. YAGNI — You Aren\u0026rsquo;t Gonna Need It.\nWe\u0026rsquo;ve now identified our core classes:\nParkingLot, Floor, ParkingSpot, Vehicle (+ Bike, Car, Truck), EntryGate, ExitGate, Ticket, Payment, DisplayBoard Plus a Strategy: PricingStrategy (with implementations).\nPlus an Enum: SpotSize.\nThat\u0026rsquo;s our skeleton. Now we design each piece.\n5. Stage 4 — Deciding the Vehicle Hierarchy This is the first real design decision. Three approaches present themselves; let me lay them out.\nApproach A: Single class with a type field class Vehicle: def __init__(self, plate, type): self.plate = plate self.type = type # \u0026#34;bike\u0026#34;, \u0026#34;car\u0026#34;, \u0026#34;truck\u0026#34; Pros:\nSimplest possible. One class to maintain. Easy to add a new type — just a new string. Cons:\nBehavior that varies by type ends up as if vehicle.type == \u0026quot;bike\u0026quot;: everywhere. Violates OCP. No type safety — what if someone passes type=\u0026quot;crab\u0026quot;? We can\u0026rsquo;t attach type-specific data (e.g., bikes have helmets stored, trucks have permits) without a bag of optional fields. Approach B: Inheritance — abstract Vehicle, concrete Bike/Car/Truck class Vehicle(ABC): @property @abstractmethod def required_size(self) -\u0026gt; SpotSize: ... class Bike(Vehicle): @property def required_size(self) -\u0026gt; SpotSize: return SpotSize.SMALL class Car(Vehicle): ... class Truck(Vehicle): ... Pros:\nType-specific behavior lives on the type itself. Clean polymorphism. New vehicle type → new class. No if/elif ladder. Type checker can verify usage. Can add type-specific fields naturally. Cons:\nMore classes upfront. Slight ceremony for what may be just \u0026ldquo;different label.\u0026rdquo; Approach C: Composition — Vehicle has a VehicleType object class VehicleType: def __init__(self, name, required_size, base_rate): ... class Vehicle: def __init__(self, plate, type: VehicleType): self.plate = plate self.type = type Pros:\nMaximum flexibility — types can be created at runtime, configured externally. Adding a new type doesn\u0026rsquo;t require code changes — just data. Cons:\nLoses the \u0026ldquo;Bike is a kind of Vehicle\u0026rdquo; semantic from the type system. Harder to add type-specific methods (would have to dispatch via the VehicleType object). Overkill for our scope. Decision I\u0026rsquo;ll go with Approach B (inheritance).\nReasoning: The vehicle types in our problem genuinely differ in behavior and properties (size requirement is the obvious one; in real systems they\u0026rsquo;d differ in pricing too). The set of vehicle types is small and stable (bike/car/truck — we\u0026rsquo;re not constantly adding new types at runtime). Approach B gives us clean polymorphism without unnecessary abstraction.\nApproach A would be tempting if we had only the type label and no per-type behavior — but we already have one (required_size), and pricing might use it later. Approach C is interesting for systems where types are data-driven (e.g., a CMS), but here it\u0026rsquo;s overengineering.\nIn the interview, I\u0026rsquo;d say this aloud:\n\u0026ldquo;I\u0026rsquo;m going with an abstract Vehicle base class and concrete Bike/Car/Truck subclasses. The alternative would be a single class with a type enum, but since vehicles genuinely differ in their required spot size — and possibly more behaviors as the system grows — polymorphism keeps the design open to extension. A composition-based approach (Vehicle has a VehicleType) would be more flexible but feels overkill for three stable types.\u0026rdquo;\nThat sentence shows: I considered alternatives, I weighed them, I chose with reasoning. That sentence is worth more than perfect code.\nThe Code from abc import ABC, abstractmethod from enum import Enum class SpotSize(Enum): \u0026#34;\u0026#34;\u0026#34; Discrete spot sizes. The integer values define a natural ordering: a larger spot can fit any vehicle that fits in a smaller spot. Why an Enum (not strings)? - Type safety: you can\u0026#39;t pass an arbitrary string by mistake. - Comparable via `.value` for the \u0026#34;can fit\u0026#34; check. - Self-documenting in IDE autocomplete. \u0026#34;\u0026#34;\u0026#34; SMALL = 1 # Fits bikes only MEDIUM = 2 # Fits bikes and cars LARGE = 3 # Fits bikes, cars, and trucks class Vehicle(ABC): \u0026#34;\u0026#34;\u0026#34; Abstract base for all vehicle types in the lot. Design choice: - We could have used a single Vehicle class with a type field, but each vehicle type has a different `required_size`, and that logic belongs on the type itself (polymorphism \u0026gt; conditionals). - Subclasses are expected to be small — they exist to declare their type-specific properties. Adding a new type (e.g., ElectricScooter): - Create a new subclass. - Override `required_size`. - No existing code changes (Open/Closed Principle). \u0026#34;\u0026#34;\u0026#34; def __init__(self, license_plate: str): # license_plate is the vehicle\u0026#39;s identity in our system. # In a real system this would also have validation (length, format). self.license_plate = license_plate @property @abstractmethod def required_size(self) -\u0026gt; SpotSize: \u0026#34;\u0026#34;\u0026#34; The smallest spot size that can hold this vehicle. Used by the spot-finding algorithm to filter candidates. \u0026#34;\u0026#34;\u0026#34; ... def __repr__(self): # Useful for logging and debugging. return f\u0026#34;{self.__class__.__name__}({self.license_plate})\u0026#34; class Bike(Vehicle): @property def required_size(self) -\u0026gt; SpotSize: return SpotSize.SMALL class Car(Vehicle): @property def required_size(self) -\u0026gt; SpotSize: return SpotSize.MEDIUM class Truck(Vehicle): @property def required_size(self) -\u0026gt; SpotSize: return SpotSize.LARGE Notice the documentation. The class docstring explains why we chose this design, not just what the class is. That\u0026rsquo;s the level interviewers reward — and that students learn most from.\nAlso notice: we kept Bike, Car, Truck empty beyond the property. They don\u0026rsquo;t need to do anything else right now. Don\u0026rsquo;t add fields just because \u0026ldquo;real bikes have helmets.\u0026rdquo; Add when needed.\n6. Stage 5 — Modeling Spots and Sizes A spot is a fixed physical location in the lot. It has a size. It\u0026rsquo;s either occupied or free. Sometimes a vehicle is in it.\nThis is dead simple — but there are still decisions to make.\nDecision Point 1: How do we represent occupancy? Option A: Boolean is_occupied. Option B: Reference to the vehicle: vehicle: Optional[Vehicle].\nA is simpler. B is more useful — if I have a spot, I can ask \u0026ldquo;what\u0026rsquo;s parked here?\u0026rdquo;, which is needed when we vacate.\nGoing with B. The boolean is implied: spot.is_free == (spot.vehicle is None).\nDecision Point 2: Should \u0026ldquo;can this vehicle fit?\u0026rdquo; live on Spot or Vehicle? Option A: vehicle.fits_in(spot) — vehicle knows what it fits in. Option B: spot.can_fit(vehicle) — spot knows what it can hold.\nBoth are valid. I lean B because:\nThe spot is the constraint. The check is naturally phrased \u0026ldquo;does this spot accept this vehicle?\u0026rdquo; When iterating spots looking for a fit, the loop reads naturally: for spot in spots: if spot.can_fit(v): .... Decision Point 3: Where does the parking transition happen? When a vehicle parks, several things change: the spot becomes occupied, the vehicle is associated with the spot, and (eventually) the floor needs to know to update its display board. Where does this logic go?\nOption A: On ParkingSpot.park(vehicle) — the spot manages its own state. Option B: On Floor.park(vehicle, spot) — the floor manages its child spots. Option C: On ParkingLot.park(vehicle, floor, spot) — the lot manages everything top-down.\nI\u0026rsquo;ll do A and B together: the spot manages its own state (it knows how to be parked-in), and the floor coordinates the which spot selection plus notification. The lot just selects which floor to try.\nThis is a layered responsibility: each level handles what it owns.\nThe Code from typing import Optional class ParkingSpot: \u0026#34;\u0026#34;\u0026#34; A single physical parking spot. Responsibilities (Single Responsibility Principle): - Track its own occupancy. - Decide whether a given vehicle fits. - Get parked in / vacated. NOT its responsibility: - Knowing about other spots, floors, or the lot. - Pricing. - Notifying display boards (that\u0026#39;s Floor\u0026#39;s job). The spot is intentionally \u0026#34;dumb\u0026#34; about its surroundings. This keeps it reusable and testable. \u0026#34;\u0026#34;\u0026#34; def __init__(self, spot_id: str, size: SpotSize): # spot_id is a unique identifier — useful for tickets and logs. # We make it human-readable (e.g., \u0026#34;F0-M3\u0026#34; = Floor 0, Medium spot 3). self.spot_id = spot_id self.size = size # The currently parked vehicle, or None if free. # Storing the vehicle (vs just a boolean) lets us know # who occupies the spot — needed when validating that a # vacate request matches the parked vehicle. self.vehicle: Optional[Vehicle] = None @property def is_free(self) -\u0026gt; bool: \u0026#34;\u0026#34;\u0026#34;Convenience boolean for the most-common check.\u0026#34;\u0026#34;\u0026#34; return self.vehicle is None def can_fit(self, vehicle: Vehicle) -\u0026gt; bool: \u0026#34;\u0026#34;\u0026#34; A spot can fit a vehicle if (1) it\u0026#39;s currently free AND (2) it\u0026#39;s at least as large as the vehicle requires. Why compare via .value? - SpotSize is an Enum where larger spots have larger integer values. - This gives us a natural ordering without writing custom comparators. Note: we deliberately include the `is_free` check here so callers get a single yes/no answer. If a spot is occupied, it can\u0026#39;t fit anything new — that\u0026#39;s a fact, not two facts. \u0026#34;\u0026#34;\u0026#34; return self.is_free and self.size.value \u0026gt;= vehicle.required_size.value def park(self, vehicle: Vehicle) -\u0026gt; None: \u0026#34;\u0026#34;\u0026#34; Park a vehicle in this spot. Raises ValueError if the spot is occupied or the vehicle doesn\u0026#39;t fit. We raise (rather than returning False) because parking a vehicle in an invalid spot is a *bug* in the caller\u0026#39;s logic — they should have checked `can_fit` first. Loud failure is better than silent. \u0026#34;\u0026#34;\u0026#34; if not self.is_free: raise ValueError( f\u0026#34;Spot {self.spot_id} is already occupied by \u0026#34; f\u0026#34;{self.vehicle.license_plate}\u0026#34; ) if not self.can_fit(vehicle): raise ValueError( f\u0026#34;Vehicle {vehicle.license_plate} (needs {vehicle.required_size.name}) \u0026#34; f\u0026#34;doesn\u0026#39;t fit in spot {self.spot_id} (size {self.size.name})\u0026#34; ) self.vehicle = vehicle def vacate(self) -\u0026gt; Vehicle: \u0026#34;\u0026#34;\u0026#34; Free up the spot. Returns the vehicle that was parked (useful for logging or further processing). Raises ValueError if the spot was already empty — again, this is a bug in the caller\u0026#39;s logic. \u0026#34;\u0026#34;\u0026#34; if self.is_free: raise ValueError(f\u0026#34;Spot {self.spot_id} was already empty\u0026#34;) vehicle = self.vehicle self.vehicle = None return vehicle def __repr__(self): status = f\u0026#34;occupied by {self.vehicle.license_plate}\u0026#34; if not self.is_free else \u0026#34;free\u0026#34; return f\u0026#34;Spot({self.spot_id}, size={self.size.name}, {status})\u0026#34; A few details worth highlighting:\nspot.park(vehicle) raises on invalid input. Why not return False? Because at this layer, an invalid call is a programming error, not a user error. The caller (Floor) is supposed to have validated can_fit already. Raising is louder and more debuggable. vacate() returns the vehicle. Why? Because the caller often wants to know who left — perhaps to log it, perhaps to validate the ticket matches. Returning the vehicle costs nothing and adds information. No Floor reference, no DisplayBoard reference. The spot has zero knowledge of where it lives. This is intentional. If we ever want to reuse ParkingSpot in a different system, we can. 7. Stage 6 — Modeling Floors and the Spot-Finding Logic A floor holds spots. Its job is to find a spot for a parking vehicle, and to know the total state of its spots (so display boards can show counts).\nDecision Point 1: How does Floor find a spot? Several algorithms are reasonable. Let\u0026rsquo;s enumerate them.\nOption A — First-fit: loop through spots, return the first that can fit.\nfor spot in self.spots: if spot.can_fit(vehicle): return spot Simple. Fast. But may leave large spots filled with bikes — wasting capacity.\nOption B — Best-fit: find the smallest spot that can fit.\ncandidates = [s for s in self.spots if s.can_fit(vehicle)] return min(candidates, key=lambda s: s.size.value, default=None) Optimal in terms of spot utilization. Slightly more work (iterates whole list).\nOption C — Best-fit by data structure: maintain a sorted multiset of free spots per size, pop the smallest that fits.\n# Pre-sort by size, then pop best match Most efficient. But complicates the design — now we have to maintain consistency between the spots list and the sorted structure.\nDecision The interviewer told us best-fit. So that\u0026rsquo;s the algorithm. Between B and C, I pick B.\nWhy? For interview-scope numbers (a few hundred spots per floor), iterating the full list is microseconds. The complexity of maintaining a sorted structure isn\u0026rsquo;t justified. If we later learn the lot has 100,000 spots and parking happens at 10,000/sec, we\u0026rsquo;d switch to C.\nThis is a tradeoff worth verbalizing in interview: \u0026ldquo;I\u0026rsquo;m using a linear scan for best-fit. It\u0026rsquo;s O(n) per parking event. For lots of even thousands of spots, that\u0026rsquo;s fast enough. If we needed millions of spots or extreme throughput, I\u0026rsquo;d maintain free-spot pools indexed by size — but that adds bookkeeping complexity that\u0026rsquo;s not worth it here.\u0026rdquo;\nDecision Point 2: What does Floor return? If a spot is found, return it. If not, return None.\nWe could raise an exception (\u0026ldquo;LotFullError\u0026rdquo;), but a full floor is not an error — it\u0026rsquo;s an expected normal outcome. Exceptions for control flow is an anti-pattern. Returning Optional[ParkingSpot] is the right choice.\nThe Code class Floor: \u0026#34;\u0026#34;\u0026#34; A single floor of the parking lot. Responsibilities: - Hold a fixed set of ParkingSpots. - Find a free spot that fits a given vehicle (best-fit algorithm). - Coordinate park/vacate calls to its spots. - Notify observers (e.g., DisplayBoard) when its state changes. Notice we delegate the actual occupancy management to ParkingSpot. Floor doesn\u0026#39;t directly mutate spot.vehicle — it asks the spot to park itself or vacate itself. This keeps the SRP boundary clean. \u0026#34;\u0026#34;\u0026#34; def __init__(self, floor_number: int, spots: list[ParkingSpot]): self.floor_number = floor_number self.spots = spots # Observers (DisplayBoards, alerters, etc.) registered with this floor. # We use a plain list for simplicity; a set would prevent duplicates # but we don\u0026#39;t expect duplicates in normal use. self._observers: list[\u0026#34;FloorObserver\u0026#34;] = [] def attach(self, observer: \u0026#34;FloorObserver\u0026#34;) -\u0026gt; None: \u0026#34;\u0026#34;\u0026#34;Register an observer to receive update notifications.\u0026#34;\u0026#34;\u0026#34; self._observers.append(observer) def detach(self, observer: \u0026#34;FloorObserver\u0026#34;) -\u0026gt; None: \u0026#34;\u0026#34;\u0026#34;Unregister an observer. Useful for tests and dynamic UIs.\u0026#34;\u0026#34;\u0026#34; self._observers.remove(observer) def _notify(self) -\u0026gt; None: \u0026#34;\u0026#34;\u0026#34; Internal: tell all observers this floor\u0026#39;s state changed. Called after every park/vacate. We don\u0026#39;t pass any data — observers can pull what they need from the floor. This is the \u0026#34;pull\u0026#34; variant of Observer (vs push). We chose pull because different observers want different things — display boards want counts; an alert system might want individual events. Pushing one common payload would be less flexible. \u0026#34;\u0026#34;\u0026#34; for observer in self._observers: observer.update(self) def find_free_spot(self, vehicle: Vehicle) -\u0026gt; Optional[ParkingSpot]: \u0026#34;\u0026#34;\u0026#34; Find the smallest free spot that can hold the vehicle. Algorithm: linear scan + min(). Time: O(n) where n is spots on this floor. Returns the spot if found, else None (floor full for this size). Design note: we don\u0026#39;t reserve or mutate anything here. Just find. Mutation happens in `park`. This separation makes the function side-effect-free and trivially testable. \u0026#34;\u0026#34;\u0026#34; # Filter to candidate spots. candidates = [s for s in self.spots if s.can_fit(vehicle)] if not candidates: return None # Best-fit: pick the smallest that fits. # `min` with a key is idiomatic and clear in Python. return min(candidates, key=lambda s: s.size.value) def park(self, vehicle: Vehicle) -\u0026gt; Optional[ParkingSpot]: \u0026#34;\u0026#34;\u0026#34; Find and occupy a spot for the vehicle. Returns the spot used, or None if no spot was available. Notice the two-step structure: find then park. This deliberately mirrors the way a real driver thinks (\u0026#34;I need to find a spot, then drive into it\u0026#34;). It also means find_free_spot is safely callable for \u0026#34;is there room?\u0026#34; queries without actually parking. \u0026#34;\u0026#34;\u0026#34; spot = self.find_free_spot(vehicle) if spot is None: return None spot.park(vehicle) # Spot does its own validation self._notify() # Tell observers state changed return spot def vacate(self, spot: ParkingSpot) -\u0026gt; Vehicle: \u0026#34;\u0026#34;\u0026#34; Free up a previously occupied spot. Returns the vehicle that was parked (handy for the caller to confirm it matches the ticket). We trust the caller to pass a spot that belongs to this floor. We could validate (`assert spot in self.spots`) but for an interview-scope design, the trust boundary is acceptable. \u0026#34;\u0026#34;\u0026#34; vehicle = spot.vacate() self._notify() return vehicle def free_spots_by_size(self) -\u0026gt; dict[SpotSize, int]: \u0026#34;\u0026#34;\u0026#34; Aggregate count of free spots per size category. This is what DisplayBoard renders. We compute on-demand rather than maintaining a counter, because: (a) For interview-scale numbers, the loop is negligible. (b) An incrementally-updated counter is one more state to keep consistent — and one more chance to bug. For very large floors, we\u0026#39;d cache and incrementally update. \u0026#34;\u0026#34;\u0026#34; # Initialize all sizes to 0 so the result has every key, # even if some sizes have no free spots. counts = {size: 0 for size in SpotSize} for spot in self.spots: if spot.is_free: counts[spot.size] += 1 return counts def __repr__(self): return f\u0026#34;Floor({self.floor_number}, {len(self.spots)} spots)\u0026#34; A note on the observer pattern here: I introduced _observers and _notify() even though we haven\u0026rsquo;t yet defined FloorObserver. That\u0026rsquo;s fine — we know it\u0026rsquo;s coming, and the structure is clean. In an interview, I\u0026rsquo;d code it this way and verbally say \u0026ldquo;I\u0026rsquo;ll define the FloorObserver interface next.\u0026rdquo;\n8. Stage 7 — Tickets, Gates, and the Park/Exit Flow Now we need the entry-exit flow. Let\u0026rsquo;s reason about who does what.\nWho creates the ticket? Three options:\nOption A: The Vehicle creates its own ticket. Option B: The EntryGate creates the ticket. Option C: The ParkingLot creates the ticket.\nA is wrong — the vehicle is just a thing in the world; it shouldn\u0026rsquo;t know about the lot\u0026rsquo;s data structures.\nB and C are both reasonable. C is more centralized (lot owns ticket creation). B is more distributed (gate handles the front-of-house).\nI\u0026rsquo;ll pick C — ParkingLot.park() creates and returns the ticket, and the gate calls into it.\nWhy: The lot is the authority on tickets — it tracks the active set and handles uniqueness. The gate is just the user-facing actor that triggers the operation. Putting ticket creation on the gate would force every gate instance to know how to construct unique IDs, which feels like leaking state.\nWhat goes on the ticket? A ticket needs enough information to:\nIdentify itself uniquely (so a customer can present it later). Refer to the vehicle (so we can validate at exit). Refer to the spot (so we know what to free). Record the entry time (so we can compute duration). Record the exit time + amount once paid (for the receipt). I\u0026rsquo;d model it as a dataclass — it\u0026rsquo;s mostly data, very little behavior.\nWhat does the gate do? In our scope, the EntryGate is a thin wrapper that takes a vehicle and asks the lot to park it. It exists mainly to model \u0026ldquo;where the user interacts.\u0026rdquo; The interesting logic lives in the lot.\nThe ExitGate is slightly more interesting: it takes a ticket, asks the lot to compute the bill (via the pricing strategy), accepts payment, and tells the lot to free the spot.\nThe Code from datetime import datetime import uuid from dataclasses import dataclass, field @dataclass class Ticket: \u0026#34;\u0026#34;\u0026#34; A receipt issued at entry, redeemed at exit. Why a dataclass? - This is mostly data with very little behavior. - @dataclass auto-generates __init__, __repr__, and __eq__, saving us boilerplate while keeping the class explicit. - Easy to extend later (add a `customer_phone` field, etc.). Fields: - id: unique identifier; we use a short UUID slice for readability in logs. In production we\u0026#39;d use a cryptographically unique ID to prevent ticket forgery. - vehicle, spot, floor_number: identifies what was parked where. - entry_time: set at issuance; used to compute duration. - exit_time, amount_paid: filled in at exit. \u0026#34;\u0026#34;\u0026#34; vehicle: Vehicle spot: ParkingSpot floor_number: int entry_time: datetime = field(default_factory=datetime.now) exit_time: Optional[datetime] = None amount_paid: Optional[float] = None # Use a default_factory so each ticket gets its own unique ID. # Generating it inline (e.g., id: str = str(uuid.uuid4())) would # share the same ID across all instances — a classic Python gotcha. id: str = field(default_factory=lambda: str(uuid.uuid4())[:8]) A subtle but important detail: field(default_factory=...) for the ID. If we\u0026rsquo;d written id: str = str(uuid.uuid4())[:8], Python would evaluate the expression once at class definition time and every ticket would get the same ID. Using default_factory ensures each instance gets a fresh value.\nNow the gates:\nclass EntryGate: \u0026#34;\u0026#34;\u0026#34; A physical entry point to the lot. Responsibility: take a vehicle, ask the lot to park it, and hand back a ticket (or signal \u0026#34;lot is full\u0026#34;). Why have a gate class at all (rather than calling lot.park directly)? - Models the real-world topology (multiple gates exist). - Provides a place to add gate-specific concerns later (e.g., gate-specific logging, \u0026#34;is this gate operational?\u0026#34;). - Lets us count entries per gate for analytics. Right now the class is tiny. That\u0026#39;s fine. Don\u0026#39;t add fields/methods speculatively; add when needed. \u0026#34;\u0026#34;\u0026#34; def __init__(self, gate_id: str, lot: \u0026#34;ParkingLot\u0026#34;): self.gate_id = gate_id self.lot = lot def admit(self, vehicle: Vehicle) -\u0026gt; Optional[Ticket]: \u0026#34;\u0026#34;\u0026#34; Admit a vehicle and issue a ticket. Returns None if the lot has no space for this vehicle. We deliberately let the lot decide. The gate doesn\u0026#39;t iterate floors itself — that\u0026#39;s not the gate\u0026#39;s job. \u0026#34;\u0026#34;\u0026#34; ticket = self.lot.park(vehicle) if ticket is None: print(f\u0026#34;[Gate {self.gate_id}] Sorry, no space for {vehicle}\u0026#34;) else: print(f\u0026#34;[Gate {self.gate_id}] Issued ticket {ticket.id} to {vehicle}\u0026#34;) return ticket class ExitGate: \u0026#34;\u0026#34;\u0026#34; A physical exit point. Responsibility: - Accept a ticket. - Compute the amount due (via pricing strategy). - Accept payment (in this scope, we just record it). - Tell the lot to free the spot. Why does the ExitGate hold a PricingStrategy reference rather than having the ParkingLot know about pricing? - In real lots, different exits could in principle apply different pricing (e.g., a VIP exit). Keeping pricing on the gate models that. - More importantly, it gives us a clean spot to demonstrate Strategy: gate.pricing is injected, easy to swap. A reasonable alternative: put pricing on the ParkingLot. Both work. The gate placement happens to be a bit more flexible. \u0026#34;\u0026#34;\u0026#34; def __init__(self, gate_id: str, lot: \u0026#34;ParkingLot\u0026#34;, pricing: \u0026#34;PricingStrategy\u0026#34;): self.gate_id = gate_id self.lot = lot self.pricing = pricing def process(self, ticket: Ticket) -\u0026gt; float: \u0026#34;\u0026#34;\u0026#34; Process an exit: 1. Stamp exit time. 2. Compute the bill via the injected pricing strategy. 3. Record payment (in this scope, just amount; gateway is out of scope). 4. Tell the lot to free the spot. Returns the amount charged. We\u0026#39;re combining \u0026#34;compute\u0026#34; and \u0026#34;process payment\u0026#34; into one method for simplicity. In a richer design you\u0026#39;d separate them so the customer could see the bill before confirming payment. \u0026#34;\u0026#34;\u0026#34; # Stamp the exit time so the pricing has the duration. ticket.exit_time = datetime.now() # Pricing is pluggable — could be HourlyPricing, FlatRatePricing, etc. amount = self.pricing.calculate(ticket) ticket.amount_paid = amount # Tell the lot to release the spot. We pass the whole ticket # because the lot needs the spot reference; and there\u0026#39;s value # in passing the full context for logging/auditing. self.lot.vacate(ticket) print(f\u0026#34;[Gate {self.gate_id}] Ticket {ticket.id} closed. \u0026#34; f\u0026#34;Amount: ₹{amount:.2f}\u0026#34;) return amount 9. Stage 8 — Pricing as a Strategy This is one of the most critical design moments. Pricing is exactly the kind of thing that varies — and it\u0026rsquo;s exactly the kind of thing that, if hardcoded, becomes the source of the worst kind of technical debt.\nWhy Strategy? Imagine we hardcode pricing inside ExitGate.process:\n# DON\u0026#39;T DO THIS def process(self, ticket): duration_hours = ... if isinstance(ticket.vehicle, Bike): amount = 10 * duration_hours elif isinstance(ticket.vehicle, Car): amount = 30 * duration_hours elif isinstance(ticket.vehicle, Truck): amount = 60 * duration_hours ... Now: what happens when marketing wants happy-hour pricing on Tuesdays? Or weekend rates? Or a flat fee for short stays? Or surge pricing? Or VIP discounts? Or membership rates?\nEach new requirement = modifying ExitGate. ExitGate becomes the source of all pricing change. Every change risks breaking existing pricing. Tests multiply. Bugs ship.\nInstead: extract pricing as a Strategy. The exit gate doesn\u0026rsquo;t know about pricing rules — it just calls self.pricing.calculate(ticket). The strategy implementation can be swapped, configured, decorated.\nDefining the Interface What does a pricing strategy need?\nInput: a Ticket (which contains entry time, exit time, vehicle, spot). Output: a float (the amount to charge). That\u0026rsquo;s the entire interface. Let\u0026rsquo;s not pollute it with extra params we might need someday. YAGNI.\nConcrete Strategies We were told: hourly pricing, different rates per vehicle type, round up to next hour.\nLet\u0026rsquo;s implement that, and one alternative (flat rate) to demonstrate the value of Strategy.\nThe Code from abc import ABC, abstractmethod import math class PricingStrategy(ABC): \u0026#34;\u0026#34;\u0026#34; The contract for all pricing algorithms. Why an abstract base class? - Enforces that all concrete strategies implement `calculate`. - Acts as the type for dependency injection (ExitGate depends on this abstraction, satisfying the Dependency Inversion Principle). - Documents the expected behavior in one place. Adding a new pricing rule = subclass + override calculate(). No existing strategy or any caller changes (Open/Closed Principle). \u0026#34;\u0026#34;\u0026#34; @abstractmethod def calculate(self, ticket: Ticket) -\u0026gt; float: \u0026#34;\u0026#34;\u0026#34; Compute the amount due for a ticket whose exit_time is set. Strategies are pure functions of the ticket — they don\u0026#39;t mutate anything. This makes them trivial to unit test (no setup, no DB). \u0026#34;\u0026#34;\u0026#34; ... class HourlyPricing(PricingStrategy): \u0026#34;\u0026#34;\u0026#34; Charge per started hour, with rates depending on vehicle type. \u0026#34;Per started hour\u0026#34; means we round duration UP to the nearest hour: a 2-hour-and-1-minute stay is charged for 3 hours. This is a common real-world policy (parking lots monetize partial hours). The rates table is a class attribute because: - It\u0026#39;s shared across all instances of HourlyPricing. - It\u0026#39;s effectively configuration — the algorithm is \u0026#34;multiply hours by per-vehicle rate,\u0026#34; and the rates are the parameters. To support per-lot rate variation, we could pass rates to __init__ instead. Showing the simplest version here. \u0026#34;\u0026#34;\u0026#34; # Rates in rupees per started hour, keyed by Vehicle subclass. # We use the class itself as the key (vs a string) so we get # type safety and IDE autocomplete. RATES: dict[type, float] = { Bike: 10.0, Car: 30.0, Truck: 60.0, } def calculate(self, ticket: Ticket) -\u0026gt; float: # Pricing requires the ticket to be stamped on exit. # If exit_time is None, we have a bug — fail loudly. if ticket.exit_time is None: raise ValueError(\u0026#34;Cannot price an unfinished ticket (no exit_time)\u0026#34;) # Compute duration in hours (as a float). duration_seconds = (ticket.exit_time - ticket.entry_time).total_seconds() duration_hours = duration_seconds / 3600 # Round UP to the next started hour. # math.ceil(0.0) is 0 — but a parking event always charges # at least 1 hour, so we max with 1 to enforce that floor. # (Alternative: charge nothing for stays under N minutes. # We can add that as a different strategy later.) billable_hours = max(1, math.ceil(duration_hours)) # Look up rate by the vehicle\u0026#39;s actual class. # If we ever add a Vehicle subtype without updating RATES, # this will KeyError — which is the right failure mode # (loud, immediate, easy to fix). vehicle_class = type(ticket.vehicle) rate = self.RATES[vehicle_class] return rate * billable_hours class FlatRatePricing(PricingStrategy): \u0026#34;\u0026#34;\u0026#34; Single flat fee per vehicle type, regardless of duration. Useful for special promotions (\u0026#34;flat ₹100 for any stay today\u0026#34;). Demonstrates the strategy pattern by being structurally different from HourlyPricing while honoring the same interface. \u0026#34;\u0026#34;\u0026#34; RATES: dict[type, float] = { Bike: 50.0, Car: 150.0, Truck: 300.0, } def calculate(self, ticket: Ticket) -\u0026gt; float: if ticket.exit_time is None: raise ValueError(\u0026#34;Cannot price an unfinished ticket\u0026#34;) return self.RATES[type(ticket.vehicle)] Now in the interview, I\u0026rsquo;d articulate:\n\u0026ldquo;I\u0026rsquo;m extracting pricing as a Strategy because pricing is the most volatile part of any commercial parking system. Hourly rates change, surge gets added, promotions come and go. By isolating it behind an interface, every new pricing rule is a new class — no existing code changes. The ExitGate depends on the abstract PricingStrategy, so I can swap implementations at construction time. This also makes pricing trivially unit-testable: it\u0026rsquo;s a pure function from ticket to float.\u0026rdquo;\nThat\u0026rsquo;s a Strategy justification with three angles: business motivation (pricing changes), technical motivation (OCP), and engineering payoff (testability).\n10. Stage 9 — Display Boards and the Observer Decision Customers driving in want to know \u0026ldquo;is the lot full? Where do I look?\u0026rdquo; — that\u0026rsquo;s what display boards solve.\nWhy Observer (and not just polling)? There are two ways to keep a display in sync with floor state:\nOption A — Polling: the display, on its own clock, asks \u0026ldquo;hey floor, what\u0026rsquo;s free?\u0026rdquo; every N seconds.\nOption B — Observer (push): the floor notifies the display whenever its state changes.\nBoth work. Observer is generally preferred because:\nUpdates are immediate, not delayed by the polling interval. No wasted work when nothing has changed. (Polling burns cycles even when state is static.) The floor is the source of truth and it knows when state changed — let it tell others. Polling has its place — when the source can\u0026rsquo;t notify (legacy system, polling-only API, distributed system without a message bus). For our in-process, single-threaded design, Observer is clearly better.\nWhat does the Observer interface look like? We have one signal — \u0026ldquo;floor state changed\u0026rdquo; — and we want to keep observers loosely coupled to the floor. So:\nclass FloorObserver(ABC): @abstractmethod def update(self, floor: Floor) -\u0026gt; None: ... The observer just gets the floor reference and pulls what it wants. We discussed earlier that this \u0026ldquo;pull\u0026rdquo; style is more flexible than push (where the floor would have to know what every observer needs).\nWhat goes on the DisplayBoard? The board\u0026rsquo;s job: when notified, render the current free counts.\nThat\u0026rsquo;s it. It doesn\u0026rsquo;t store history; it doesn\u0026rsquo;t decide layout. In a real system there\u0026rsquo;d be a UI layer; in our LLD scope, \u0026ldquo;render\u0026rdquo; means \u0026ldquo;print to console.\u0026rdquo;\nThe Code class FloorObserver(ABC): \u0026#34;\u0026#34;\u0026#34; Anything that wants to react to changes in a Floor\u0026#39;s occupancy. By depending on this interface (not on Floor or DisplayBoard concrete classes), Floor stays decoupled from its observers. We could add EmailAlertObserver, MetricsObserver, etc., without touching Floor. \u0026#34;\u0026#34;\u0026#34; @abstractmethod def update(self, floor: Floor) -\u0026gt; None: \u0026#34;\u0026#34;\u0026#34;Called by Floor whenever its occupancy state changes.\u0026#34;\u0026#34;\u0026#34; ... class DisplayBoard(FloorObserver): \u0026#34;\u0026#34;\u0026#34; Shows real-time free-spot counts for one floor, broken down by size. This is intentionally minimal — it just prints. In a real system this would push data to a physical LED display, a web dashboard, or a Kafka topic. The pattern is the same: react to the update. \u0026#34;\u0026#34;\u0026#34; def __init__(self, label: str = \u0026#34;\u0026#34;): # An optional label so a viewer (or a log reader) can tell # which board is which. self.label = label def update(self, floor: Floor) -\u0026gt; None: # Pull what we need from the floor. # We do this on every update; for a board that displays counts, # that\u0026#39;s exactly right. counts = floor.free_spots_by_size() # Render. The format string is chosen for readability in logs. # In production this would write to whatever display backend. prefix = f\u0026#34;[{self.label}] \u0026#34; if self.label else \u0026#34;\u0026#34; print( f\u0026#34;{prefix}Floor {floor.floor_number} free → \u0026#34; f\u0026#34;Small: {counts[SpotSize.SMALL]}, \u0026#34; f\u0026#34;Medium: {counts[SpotSize.MEDIUM]}, \u0026#34; f\u0026#34;Large: {counts[SpotSize.LARGE]}\u0026#34; ) Notice how small DisplayBoard is. That\u0026rsquo;s a good sign — it has a single, well-scoped responsibility. Its size says \u0026ldquo;this class does one thing.\u0026rdquo; If it grew to 200 lines, that would be a smell.\n11. Stage 10 — Putting It All Together Now we need the top-level coordinator: ParkingLot.\nWhat should ParkingLot know? Its floors (it owns them). The active tickets (so we can look them up at exit). What should ParkingLot not know? About display boards (those are observers attached to floors). About pricing (that lives in the exit gate\u0026rsquo;s strategy). About payment processing (out of scope). This restraint — knowing only what we need — is what keeps the lot from becoming a god class.\nThe Core Methods ParkingLot needs two main operations:\npark(vehicle) → Optional[Ticket] — find a floor with space, park there, issue a ticket. vacate(ticket) — free the spot referenced by the ticket. Both delegate the real work to Floor. The lot\u0026rsquo;s own logic is just orchestration.\nDecision Point: How do we pick which floor to try? When a vehicle arrives, we have multiple floors. Where do we send it?\nOption A: Try floors in order (0, 1, 2, \u0026hellip;). Simplest. Option B: Try the floor with the most free space first. Spreads load. Option C: Try the lowest floor first if vehicle is large; ground floor is easier to drive into. Encodes domain knowledge.\nFor our scope, Option A is fine. The interviewer didn\u0026rsquo;t specify. Verbalize this:\n\u0026ldquo;I\u0026rsquo;m trying floors in order. We could prefer floors with most free space, or have type-specific routing (heavy vehicles to ground floor), but the requirements didn\u0026rsquo;t call for that. I\u0026rsquo;ll mention it as an extension.\u0026rdquo;\nThe Code class ParkingLot: \u0026#34;\u0026#34;\u0026#34; The top-level entity that orchestrates parking and exit operations. Responsibilities: - Own the collection of floors. - Find a floor that can accommodate a vehicle (delegates to floors). - Track active tickets (so we can vacate by ticket). Explicitly NOT responsible for: - Pricing (handled by ExitGate\u0026#39;s PricingStrategy). - Display (handled by DisplayBoards observing floors). - Payment processing (out of scope). - User/customer modeling (no Customer entity in this design). By keeping the lot focused on orchestration, we avoid the classic \u0026#34;god class\u0026#34; anti-pattern. \u0026#34;\u0026#34;\u0026#34; def __init__(self, floors: list[Floor]): if not floors: raise ValueError(\u0026#34;ParkingLot must have at least one floor\u0026#34;) self.floors = floors # Map of ticket_id -\u0026gt; Ticket. Lets us look up a ticket # quickly at exit. dict gives O(1) lookup. self.active_tickets: dict[str, Ticket] = {} def park(self, vehicle: Vehicle) -\u0026gt; Optional[Ticket]: \u0026#34;\u0026#34;\u0026#34; Try each floor in order. The first one with space wins. Returns the issued Ticket, or None if every floor is full (for this vehicle\u0026#39;s required spot size). Why try floors in order rather than picking the \u0026#34;best\u0026#34; floor? - It\u0026#39;s simple and deterministic. - For interview scope, the requirements don\u0026#39;t dictate a preference. - Smarter routing (most-free-first, or size-aware routing) can be added later without changing this method\u0026#39;s signature. \u0026#34;\u0026#34;\u0026#34; for floor in self.floors: spot = floor.park(vehicle) if spot is not None: # Found space. Build and store the ticket. ticket = Ticket( vehicle=vehicle, spot=spot, floor_number=floor.floor_number, ) self.active_tickets[ticket.id] = ticket return ticket # All floors full for this vehicle\u0026#39;s size category. return None def vacate(self, ticket: Ticket) -\u0026gt; None: \u0026#34;\u0026#34;\u0026#34; Free the spot associated with this ticket and remove the ticket from the active set. We trust the caller has already computed payment and stamped exit_time on the ticket. The lot\u0026#39;s job is just spot release. \u0026#34;\u0026#34;\u0026#34; # Look up the floor by floor_number. # We use direct indexing because floor_number is the floor\u0026#39;s index. # (If floor numbers weren\u0026#39;t 0-based or sequential, we\u0026#39;d use a dict.) floor = self.floors[ticket.floor_number] floor.vacate(ticket.spot) # Remove from active set. .pop with a default avoids KeyError # if vacate is called twice for the same ticket — defensive # but cheap. self.active_tickets.pop(ticket.id, None) A Helper to Build a Demo Lot For the demo, we want a small but realistic lot. Let\u0026rsquo;s add a builder helper:\ndef build_sample_lot(num_floors: int = 3) -\u0026gt; ParkingLot: \u0026#34;\u0026#34;\u0026#34; Construct a sample lot: `num_floors` floors, each with 5 small, 10 medium, and 3 large spots, with a DisplayBoard attached. This is a convenience for demos and tests — not part of the core design. In a real system, lot configuration would come from a config file or database. \u0026#34;\u0026#34;\u0026#34; floors = [] for floor_num in range(num_floors): # Build the spots for this floor. spots = [] spots += [ ParkingSpot(f\u0026#34;F{floor_num}-S{i}\u0026#34;, SpotSize.SMALL) for i in range(5) ] spots += [ ParkingSpot(f\u0026#34;F{floor_num}-M{i}\u0026#34;, SpotSize.MEDIUM) for i in range(10) ] spots += [ ParkingSpot(f\u0026#34;F{floor_num}-L{i}\u0026#34;, SpotSize.LARGE) for i in range(3) ] floor = Floor(floor_num, spots) floor.attach(DisplayBoard(label=f\u0026#34;Display-F{floor_num}\u0026#34;)) floors.append(floor) return ParkingLot(floors) 12. Stage 11 — Validating with a Mental Walkthrough Before declaring victory, we run the design through a realistic scenario. This is not optional. Almost every interview design has a bug that surfaces only when you walk through it.\nScenario: A typical day A bike, a car, and a truck arrive in that order. They each park, stay for varying times, then leave.\nLet\u0026rsquo;s walk through it step by step.\n# Setup lot = build_sample_lot(num_floors=2) entry = EntryGate(\u0026#34;E1\u0026#34;, lot) exit_gate = ExitGate(\u0026#34;X1\u0026#34;, lot, HourlyPricing()) # 1. Bike arrives bike = Bike(\u0026#34;KA-01-B-1234\u0026#34;) ticket1 = entry.admit(bike) Trace through:\nentry.admit(bike) → lot.park(bike). lot.park(bike) loops floors. Floor 0: floor.park(bike). floor.park(bike) → find_free_spot(bike). Candidates: 5 SMALL, 10 MEDIUM, 3 LARGE all free, all can fit a bike. Best fit = SMALL. Picks F0-S0. spot.park(bike) succeeds. floor._notify() → DisplayBoard prints updated counts. Floor returns the spot. Lot creates a ticket, stores it, returns it. Gate prints \u0026ldquo;Issued ticket \u0026hellip; to Bike(\u0026hellip;)\u0026rdquo;. ✓ Works. Bike got a SMALL spot (best-fit), as required.\n# 2. Car arrives car = Car(\u0026#34;KA-01-C-5678\u0026#34;) ticket2 = entry.admit(car) Car needs MEDIUM. Candidates: 10 MEDIUM + 3 LARGE. Best fit = MEDIUM. Picks F0-M0. ✓ Works.\n# 3. Truck arrives truck = Truck(\u0026#34;KA-01-T-9999\u0026#34;) ticket3 = entry.admit(truck) Truck needs LARGE. Only 3 LARGE spots fit. Picks F0-L0. ✓ Works.\n# 4. Bike leaves after some time import time; time.sleep(0.05) # simulate elapsed time amount1 = exit_gate.process(ticket1) Ticket1\u0026rsquo;s exit_time set to now. pricing.calculate(ticket1) → duration ≈ 0.05s ≈ 0.0000139 hours. math.ceil(0.0000139) = 1. Rate for Bike = 10. Amount = 10. lot.vacate(ticket1) → floor 0 vacates spot F0-S0. DisplayBoard prints. Ticket removed from active set. ✓ Works. Amount is ₹10 (one hour minimum).\nEdge Cases — Where Designs Usually Break Let\u0026rsquo;s consider what happens in less happy scenarios.\nEdge case 1: Lot is full.\n# Fill all SMALL spots on floor 0. for i in range(5): entry.admit(Bike(f\u0026#34;BIKE-{i}\u0026#34;)) # Floor 0 has 0 small spots free. But MEDIUM is still free, and a bike # fits in MEDIUM. So our 6th bike should get a MEDIUM spot. ticket6 = entry.admit(Bike(\u0026#34;BIKE-6\u0026#34;)) # ticket6 is non-None — Floor 0 had MEDIUM spots; bike went into one. ✓ Best-fit means a bike gets a small spot when available, but happily takes a medium spot when small ones are full. Correct behavior.\nEdge case 2: A vehicle requests a size for which no spot exists anywhere.\nSay all LARGE spots across all floors are taken. A second truck arrives.\nFloor 0: find_free_spot(truck) → no candidates (no free LARGE). Returns None. Floor 1: same. Returns None. Lot.park returns None. Gate prints \u0026ldquo;Sorry, no space.\u0026rdquo; ✓ Correct.\nEdge case 3: Customer presents a ticket twice.\nAfter exiting, the customer (somehow) presents the same ticket at another exit gate.\nexit_gate.process(ticket1) — ticket1.exit_time is already set; pricing computes again, re-vacates a spot that\u0026rsquo;s no longer occupied. floor.vacate(spot) → spot.vacate() raises ValueError (\u0026ldquo;already empty\u0026rdquo;). So we\u0026rsquo;d get a noisy error. Is that OK? For our scope, yes — re-presenting a ticket is an error. We\u0026rsquo;d want explicit handling in production (e.g., reject already-paid tickets) but mentioning it as an extension is appropriate.\nIn an interview, you\u0026rsquo;d say:\n\u0026ldquo;I see one weakness: re-presenting a paid ticket would currently raise. In production, ExitGate.process should first check if ticket.exit_time is already set and refuse early with a clear message. I\u0026rsquo;d add that as a small refinement.\u0026rdquo;\nThat kind of self-critique scores points.\nEdge case 4: Display board observer was never attached.\nA floor with no observers — _notify iterates an empty list. No error. No notifications. Correct.\n✓ Defensive design holds.\nEdge case 5: ParkingLot constructed with no floors.\nWe added if not floors: raise ValueError(...) in __init__. Catches this immediately, with a clear message.\n✓ Defensive design holds.\nWalkthrough Summary We\u0026rsquo;ve validated the happy path and several edge cases. We found one minor issue (re-presenting a ticket) and noted it as a follow-up. The design behaves correctly for everything we\u0026rsquo;ve thrown at it.\nThis walkthrough is what separates \u0026ldquo;I wrote some classes\u0026rdquo; from \u0026ldquo;I designed a system.\u0026rdquo; Always do it in the interview, even briefly.\n13. Stage 12 — Anticipating Follow-Up Questions Interviewers love to extend the problem. Here are the questions they\u0026rsquo;re most likely to ask, with thought-out answers.\nQ: \u0026ldquo;How would you handle multiple gates parking simultaneously?\u0026rdquo; This is the classic concurrency question. With our current design:\nTwo threads could both call lot.park(vehicle) at the same time. Both could reach floor.find_free_spot(vehicle) and select the same spot. Both call spot.park(vehicle). The second one raises (\u0026ldquo;already occupied\u0026rdquo;) — but the first thread has already taken the spot, so we have a customer inside who got an error. Fix options:\nLock on the floor. Each Floor.park acquires a mutex. Simple, correct, but bottlenecks all parking on a floor. Fine for hundreds of cars; not for thousands per second.\nOptimistic locking on the spot. spot.park(vehicle) becomes atomic via a CAS (compare-and-swap). Each find_free_spot returns a candidate; the actual park retries on conflict. More throughput.\nPer-size queues. Maintain free-spot pools per size. Multiple parkers atomically pop. Highest throughput but most complex.\nFor an interview, I\u0026rsquo;d say:\n\u0026ldquo;I\u0026rsquo;d start with a per-floor lock. It\u0026rsquo;s simple and correct. If we needed higher concurrency, I\u0026rsquo;d move to atomic spot acquisition — but I wouldn\u0026rsquo;t add that complexity unless we measured contention as a real problem.\u0026rdquo;\nQ: \u0026ldquo;What about reserved spots — handicapped, electric, VIP?\u0026rdquo; Add an optional reserved_for attribute on ParkingSpot. can_fit extends to check this:\ndef can_fit(self, vehicle: Vehicle) -\u0026gt; bool: if not self.is_free: return False if self.size.value \u0026lt; vehicle.required_size.value: return False if self.reserved_for is not None and not self.reserved_for(vehicle): return False return True Where reserved_for is a callable like lambda v: v.is_electric or lambda v: v.has_handicap_permit. This is the Strategy pattern at the spot level — predicate-based reservation.\nQ: \u0026ldquo;How would you support different pricing on weekends or holidays?\u0026rdquo; Two clean approaches:\nApproach 1: Composite pricing. A WeekendMultiplier decorator wraps another pricing strategy and applies a multiplier on weekends.\nclass WeekendMultiplier(PricingStrategy): def __init__(self, base: PricingStrategy, multiplier: float = 1.5): self.base = base self.multiplier = multiplier def calculate(self, ticket): base_amount = self.base.calculate(ticket) if ticket.entry_time.weekday() \u0026gt;= 5: return base_amount * self.multiplier return base_amount Now ExitGate(lot, WeekendMultiplier(HourlyPricing())) does the right thing. Decorator pattern, layered on Strategy. Beautifully composable.\nApproach 2: A new strategy class. WeekendAwareHourlyPricing directly. Less flexible but simpler.\nI\u0026rsquo;d prefer Approach 1 because it composes — you can stack HolidayMultiplier, WeekendMultiplier, MemberDiscount, etc.\nQ: \u0026ldquo;What if the lot has 100,000 spots?\u0026rdquo; Our linear scans become noticeable. Mitigations:\nPer-size free-spot pools. Each floor maintains a heap or sorted set per size. find_free_spot becomes O(log n) instead of O(n). Per-floor counters. Maintain free_count_by_size incrementally on park/vacate. free_spots_by_size becomes O(1) instead of O(n). I wouldn\u0026rsquo;t pre-build these for an interview. I\u0026rsquo;d describe them as \u0026ldquo;if profiling showed contention here, this is how I\u0026rsquo;d evolve.\u0026rdquo;\nQ: \u0026ldquo;How would you persist this?\u0026rdquo; Introduce a Repository layer:\nclass TicketRepository(ABC): @abstractmethod def save(self, ticket: Ticket): ... @abstractmethod def find(self, ticket_id: str) -\u0026gt; Optional[Ticket]: ... class InMemoryTicketRepository(TicketRepository): ... class SQLTicketRepository(TicketRepository): ... ParkingLot depends on TicketRepository (DIP) instead of holding active_tickets directly. Now we can plug in real persistence without touching lot logic.\nQ: \u0026ldquo;How would you test this?\u0026rdquo; The design is very testable because of dependency injection:\nStrategies are pure functions of input. HourlyPricing().calculate(test_ticket) — no setup needed. Spot, Floor, Lot are independent units. Unit-test each in isolation. DisplayBoard is replaceable — write a RecordingObserver that captures updates as a list, assert on its contents. No external dependencies (DB, network). Pure Python, fast tests. A typical test:\ndef test_best_fit_picks_smallest_spot(): spots = [ ParkingSpot(\u0026#34;a\u0026#34;, SpotSize.LARGE), ParkingSpot(\u0026#34;b\u0026#34;, SpotSize.MEDIUM), # Best fit for a Car ParkingSpot(\u0026#34;c\u0026#34;, SpotSize.LARGE), ] floor = Floor(0, spots) spot = floor.park(Car(\u0026#34;X-1\u0026#34;)) assert spot.spot_id == \u0026#34;b\u0026#34; 14. Final Reflection — What This Problem Teaches If you\u0026rsquo;ve read this far, you\u0026rsquo;ve seen something more important than a parking lot design. You\u0026rsquo;ve seen the texture of how to think through an LLD problem.\nLet\u0026rsquo;s distill the meta-lessons.\nLesson 1: Design Decisions Have Reasons. State Them. At every junction, we considered alternatives and picked one with reasoning. This:\nDemonstrates engineering maturity. Catches bad ideas early (sometimes the discarded alternative reveals a flaw). Communicates clearly with the interviewer. The phrase \u0026ldquo;I\u0026rsquo;m doing X because Y, considered Z but rejected it because\u0026hellip;\u0026rdquo; is your most powerful interview tool.\nLesson 2: Defer, Don\u0026rsquo;t Pre-Solve We didn\u0026rsquo;t add Customer. We didn\u0026rsquo;t add reserved spots. We didn\u0026rsquo;t add concurrency. We didn\u0026rsquo;t add weekend pricing. We added them as we needed them, and not before.\nThe temptation to \u0026ldquo;design for everything\u0026rdquo; is the road to over-engineering. Solve what\u0026rsquo;s in scope; mention extensions; don\u0026rsquo;t build them.\nLesson 3: Boundaries Matter More Than Code The reason ParkingSpot knows nothing about Floor, and Floor knows nothing about DisplayBoard directly (only through the FloorObserver interface), is deliberate. These boundaries are what let us evolve the system later. Without them, every change ripples.\nThe boundaries are the design. The code is just notation.\nLesson 4: Patterns Are Tools, Not Trophies We used three classical patterns: Strategy (pricing), Observer (display boards), Factory (the helper builder, sort of). Each was chosen because it solved a specific problem we faced — not because we wanted to \u0026ldquo;demonstrate patterns.\u0026rdquo;\nThe interviewer wants to see you reach for the right pattern at the right moment, not pile patterns on. Our pricing-as-Strategy is a perfect example: the alternative (hardcoded if-elif) was clearly worse, so Strategy deserves to be there.\nLesson 5: Walk Through Your Design Before You Trust It The validation walkthrough caught the \u0026ldquo;ticket re-presentation\u0026rdquo; issue. That\u0026rsquo;s typical — designs always have one or two soft spots that surface only when you simulate use. Don\u0026rsquo;t skip this.\nLesson 6: Comments Earn Their Keep The code in this document is heavily commented — but every comment serves a purpose. They explain why, tradeoffs, alternatives considered, invariants. They don\u0026rsquo;t paraphrase the code (\u0026ldquo;this method parks a vehicle\u0026rdquo;); they justify it.\nThat level of code documentation isn\u0026rsquo;t always feasible in 45-minute interviews — but the thinking behind those comments is exactly what you should be verbalizing.\nWhat\u0026rsquo;s Next You now have one problem fully internalized — not just memorized, but understood at the level of why each decision was made. The remaining five problems in this series (Elevator, Library, Vending Machine, Splitwise, Ride-Hailing) follow the same structure.\nWhen you read the next ones, notice what changes and what stays the same. The framework — clarify, identify, decide-with-reasoning, code-with-comments, validate, anticipate — is universal. The specific decisions vary by problem.\nThat universality is what you want to internalize. The specific designs are bonuses.\n","permalink":"https://blogs.codeverra.com/blog/low_level_design/interview_questions/06a_lld_parking_lot_deep_dive/","summary":"\u003ch1 id=\"lld-deep-dive-1--designing-a-parking-lot\"\u003eLLD Deep Dive #1 — Designing a Parking Lot\u003c/h1\u003e\n\u003cblockquote\u003e\n\u003cp\u003e\u003cstrong\u003eWhat this document is:\u003c/strong\u003e A genuine, step-by-step walkthrough of how to \u003cem\u003ethink through\u003c/em\u003e a Parking Lot LLD problem in an interview. Not a finished design dropped on you — but the reasoning, the false starts, the alternatives weighed, and the justifications that lead to each design decision.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eTime investment:\u003c/strong\u003e Read this slowly. If you\u0026rsquo;re doing this for interview prep, it\u0026rsquo;s worth treating each section as its own thinking exercise — pause and ask yourself \u0026ldquo;what would I do here?\u0026rdquo; before reading on.\u003c/p\u003e","tags":null,"title":""},{"content":"LLD Deep Dive #2 — Designing an Elevator System What this document is: A genuine, step-by-step walkthrough of how to think through an Elevator LLD problem in an interview. We\u0026rsquo;ll reason from first principles, weigh alternatives, and document each design decision — building toward a clean, extensible design centered on the State pattern.\nTable of Contents Why This Problem Matters Stage 1 — Receiving the Problem Stage 2 — Clarifying Requirements Stage 3 — Identifying Entities Stage 4 — Modeling Requests (Internal vs External) Stage 5 — The State Pattern Decision Stage 6 — Implementing the States Stage 7 — The Elevator Class Stage 8 — The Dispatcher and Scheduling Strategy Stage 9 — The Building (Top-Level Coordinator) Stage 10 — Validating with a Mental Walkthrough Stage 11 — Anticipating Follow-Up Questions Final Reflection — What This Problem Teaches 1. Why This Problem Matters If Parking Lot is the canonical \u0026ldquo;model relationships\u0026rdquo; problem, Elevator is the canonical \u0026ldquo;model state transitions\u0026rdquo; problem. It\u0026rsquo;s specifically designed (whether interviewers know it or not) to make the State pattern shine.\nWhat makes the elevator interesting:\nAn elevator has distinct modes of operation — idle, moving up, moving down, stopped with doors open. Each mode allows different actions and produces different next moves. Multiple elevators must coordinate — when a request comes in, who handles it? Two kinds of input exist — pressing a button on a floor (external) vs pressing a destination inside the elevator (internal). Their routing is fundamentally different. Get this design right, and you\u0026rsquo;ve shown mastery of state machines, the Strategy pattern, and the discipline of separating coordination from execution. Get it wrong (one giant if chain on a state string), and the interviewer files you under \u0026ldquo;didn\u0026rsquo;t know about the State pattern.\u0026rdquo;\n2. Stage 1 — Receiving the Problem The interviewer says:\n\u0026ldquo;Design an elevator system.\u0026rdquo;\nThat\u0026rsquo;s it. Same as before — deliberately vague.\nPossible interpretations:\nA single elevator? Or multiple in a building? How many floors? Express elevators? Service elevators? Capacity limits? Weight limits? Real-time scheduling, or just FIFO? Emergency mode? Maintenance mode? What\u0026rsquo;s the simulation granularity — discrete time steps? Real time? We don\u0026rsquo;t know. So we ask.\n3. Stage 2 — Clarifying Requirements Let\u0026rsquo;s go through the questions structurally — what we\u0026rsquo;d ask, and why each question changes the design.\nQ1: One elevator or multiple? Why ask: With a single elevator, there\u0026rsquo;s no scheduling problem — every request goes to the only car. With multiple, we need a dispatcher to decide which elevator handles a given request. That\u0026rsquo;s a major architectural difference.\nSuppose: \u0026ldquo;Multiple — say, 3 elevators in a 10-floor building.\u0026rdquo;\nNow we know we need a dispatcher. We\u0026rsquo;ll likely also want a scheduling strategy (nearest-elevator, round-robin, etc.) — perfect setup for the Strategy pattern.\nQ2: How are requests made? Why ask: This shapes what kinds of input the system handles.\nIn a real elevator system, there are two kinds of requests:\nExternal (hall call): Someone on floor 5 presses \u0026ldquo;up.\u0026rdquo; They want some elevator to come pick them up. Internal (car call): Someone inside an elevator presses \u0026ldquo;8.\u0026rdquo; They want this elevator to go to floor 8. These are fundamentally different. External requests go through the dispatcher (it picks the elevator). Internal requests bypass the dispatcher (the rider is already inside a specific elevator).\nMany candidates miss this distinction and build a system that only handles one type. Naming this distinction up front is a strong signal of LLD experience.\nSuppose: \u0026ldquo;Yes, both types — external from a floor, and internal from inside the car.\u0026rdquo;\nGood. We now know we need at least two Request types (or one type with a flag — we\u0026rsquo;ll decide later).\nQ3: What states does an elevator have? Why ask: Some interviewers will tell you, others want you to enumerate them. Either way, naming the lifecycle is critical.\nA reasonable list:\nIdle — sitting at a floor, no pending requests. Moving up — between floors, heading up. Moving down — between floors, heading down. Door open — stopped at a floor with doors open. Door closed — stopped at a floor with doors closed (transitional). Some designs collapse \u0026ldquo;door open\u0026rdquo; and \u0026ldquo;door closed\u0026rdquo; into a single \u0026ldquo;stopped\u0026rdquo; state. We\u0026rsquo;ll discuss this when we model states.\nSuppose the interviewer agrees: \u0026ldquo;Yes, those states are fine.\u0026rdquo;\nQ4: How is a moving elevator scheduled? Why ask: Real elevators don\u0026rsquo;t reverse direction mid-trip. If an elevator is going up and there\u0026rsquo;s a request below, it usually finishes its upward run before turning around. This is sometimes called the SCAN/LOOK algorithm (analogous to disk scheduling).\nImplementing realistic scheduling is complex. For an interview, we usually simplify.\nSuppose: \u0026ldquo;Use a simple nearest-stop algorithm for now. Mention more sophisticated alternatives as extensions.\u0026rdquo;\nQ5: Dispatch strategy among elevators? Why ask: When an external request comes in, which elevator gets it?\nOptions:\nNearest idle elevator (or nearest of any state). Round-robin. Least-loaded (fewest pending stops). Trajectory-aware (prefer an elevator already moving toward the request). Suppose: \u0026ldquo;Nearest car. Also pluggable for future strategies.\u0026rdquo;\nExcellent — pluggable strategy = clear Strategy pattern signal.\nQ6: Capacity, weight limits, emergency? Why ask: Scope-bounding. These are real concerns but often beyond interview scope.\nSuppose: \u0026ldquo;Out of scope. Mention as extensions.\u0026rdquo;\nQ7: How does the simulation advance? Why ask: This is subtle but important. In a real system, elevators move continuously. In a simulation, we usually use discrete time steps (\u0026ldquo;ticks\u0026rdquo;). Each tick, every elevator advances its state by one unit (e.g., moves one floor, opens doors, etc.).\nThis ticking model is much simpler to implement than continuous time. We\u0026rsquo;ll use it.\nRecap \u0026ldquo;So we have: a building with 10 floors and 3 elevators. Each elevator has Idle / MovingUp / MovingDown / DoorOpen / DoorClosed states. Two request types — external (from a floor, handled by a dispatcher) and internal (from inside a car). Dispatcher uses nearest-car selection but is pluggable. Simulation advances in discrete ticks. Capacity, weight, and emergency are out of scope. Sound right?\u0026rdquo;\nThis recap gets confirmed, and now we design.\n4. Stage 3 — Identifying Entities Let\u0026rsquo;s pull nouns from the requirements.\nbuilding floor elevator request (external, internal) dispatcher scheduling strategy state (Idle, MovingUp, MovingDown, DoorOpen, DoorClosed) direction (Up, Down, Idle) button panel Let\u0026rsquo;s filter:\nNoun Candidate Class? Reasoning building Yes Top-level entity; owns floors and elevators floor Maybe — see below Does it have behavior of its own? elevator Yes Core entity with state and behavior request Yes First-class concept; benefits from inheritance for external/internal dispatcher Yes Distinct responsibility — routing requests scheduling strategy Yes (interface) Pluggable algorithm = Strategy pattern state Yes (hierarchy) The State pattern — one class per state direction Enum, not class Just a label button Reject UI artifact; we just receive resulting requests panel Reject Same reasoning Should Floor be a class? Floors don\u0026rsquo;t really have behavior in our scope — they\u0026rsquo;re just integer positions. We could make Floor a class to represent floor-specific things (per-floor buttons, displays, occupancy), but those are out of scope.\nDecision: Skip Floor as a class. Use integers for floor numbers. We can revisit if richer floor concerns enter scope.\nThis is a YAGNI decision, and it\u0026rsquo;s worth verbalizing:\n\u0026ldquo;I\u0026rsquo;m representing floors as integers rather than a Floor class. Floors don\u0026rsquo;t carry any behavior in our current scope — no per-floor buttons or displays we\u0026rsquo;re modeling. If we needed those, I\u0026rsquo;d promote Floor to a class.\u0026rdquo;\nFinal entity list:\nBuilding Elevator Request → ExternalRequest, InternalRequest Dispatcher SchedulingStrategy (abstract) ElevatorState → Idle, MovingUp, MovingDown, DoorOpen, DoorClosed Direction (enum) 5. Stage 4 — Modeling Requests (Internal vs External) The two request types are conceptually different:\nExternal: \u0026ldquo;I\u0026rsquo;m on floor 5, I want to go up.\u0026rdquo; Carries: source floor, desired direction. Internal: \u0026ldquo;I\u0026rsquo;m in elevator 2, take me to floor 8.\u0026rdquo; Carries: destination floor. How do we model them?\nApproach A — One Request class with flags class Request: def __init__(self, floor, request_type, direction=None): self.floor = floor self.request_type = request_type # \u0026#34;external\u0026#34; or \u0026#34;internal\u0026#34; self.direction = direction # only meaningful for external Pros: One class.\nCons:\nMixes concerns. direction is meaningful only for external — having a None field for internal is sloppy. Branches everywhere on request_type. Loses type safety. Approach B — Inheritance: ExternalRequest and InternalRequest class Request(ABC): @property @abstractmethod def floor(self) -\u0026gt; int: ... class ExternalRequest(Request): ... class InternalRequest(Request): ... Pros:\nEach class carries exactly the fields it needs. Type-checking can distinguish them (isinstance check). Adding a new request type (e.g., EmergencyRequest) is a clean extension. Cons: Two classes vs one.\nDecision Approach B — inheritance.\nThe two types differ in what data they carry and how they\u0026rsquo;re routed. Polymorphism here is justified: ExternalRequest goes through the dispatcher (which picks an elevator); InternalRequest goes directly to a specific elevator the rider is already inside. The shape of the data and the routing both differ — that\u0026rsquo;s exactly when inheritance pays off.\nIn the interview:\n\u0026ldquo;I\u0026rsquo;m modeling external and internal requests as separate subclasses of an abstract Request. They differ in fields (external has a direction, internal has destination only) and in routing (external goes through the dispatcher, internal goes directly to a specific elevator). A single class with flags would work but would push branching into the dispatcher; subclasses keep each type\u0026rsquo;s data clean and let the dispatcher use isinstance to make routing decisions explicitly.\u0026rdquo;\nThe Code from abc import ABC, abstractmethod from enum import Enum class Direction(Enum): \u0026#34;\u0026#34;\u0026#34; Direction of travel for an elevator. Why an Enum? - Type-safe: can\u0026#39;t accidentally pass \u0026#34;uo\u0026#34; instead of \u0026#34;up\u0026#34;. - Self-documenting in IDE autocomplete. The integer values (1, -1, 0) are deliberately chosen so that direction can be added to a floor number to compute the next floor: next_floor = current_floor + direction.value \u0026#34;\u0026#34;\u0026#34; UP = 1 DOWN = -1 IDLE = 0 class Request(ABC): \u0026#34;\u0026#34;\u0026#34; Abstract base for any kind of request that prompts elevator action. All requests have a target floor — that\u0026#39;s the common contract. Beyond that, subclasses add type-specific data. Why abstract a `floor` property instead of a regular attribute? 1. Each subclass might want its own internal name for the field (source_floor vs destination) but expose the same interface. 2. Marking it abstract makes the contract explicit: every Request MUST tell us which floor it concerns. \u0026#34;\u0026#34;\u0026#34; @property @abstractmethod def floor(self) -\u0026gt; int: \u0026#34;\u0026#34;\u0026#34;The floor this request concerns.\u0026#34;\u0026#34;\u0026#34; ... class ExternalRequest(Request): \u0026#34;\u0026#34;\u0026#34; A hall call: someone on a floor pressed an up/down button. Routing: external requests go through the Dispatcher because the requester doesn\u0026#39;t care which elevator picks them up. \u0026#34;\u0026#34;\u0026#34; def __init__(self, source_floor: int, direction: Direction): # Validate at construction time. Invariants are easier to # enforce here than scattered through downstream code. if direction == Direction.IDLE: raise ValueError(\u0026#34;External requests must specify UP or DOWN\u0026#34;) self._floor = source_floor self.direction = direction @property def floor(self) -\u0026gt; int: return self._floor def __repr__(self): return f\u0026#34;ExternalRequest(floor={self._floor}, dir={self.direction.name})\u0026#34; class InternalRequest(Request): \u0026#34;\u0026#34;\u0026#34; A car call: someone inside an elevator pressed a destination button. Routing: internal requests are submitted directly to the specific elevator the rider is in. They DO NOT go through the dispatcher. \u0026#34;\u0026#34;\u0026#34; def __init__(self, destination: int): self._floor = destination @property def floor(self) -\u0026gt; int: return self._floor def __repr__(self): return f\u0026#34;InternalRequest(dest={self._floor})\u0026#34; A small but important detail: I validate direction != IDLE for ExternalRequest in the constructor. An invalid request should be impossible to construct. Validating at construction time prevents bad data from spreading — it fails immediately, with a clear message, at the original site.\n6. Stage 5 — The State Pattern Decision Now the central design decision. How do we model elevator behavior over its lifecycle?\nThe naive approach uses a state field with branching logic:\nclass Elevator: def __init__(self): self.state = \u0026#34;idle\u0026#34; def step(self): if self.state == \u0026#34;idle\u0026#34;: ... elif self.state == \u0026#34;moving_up\u0026#34;: self.current_floor += 1 if self.current_floor in self.stops: self.state = \u0026#34;door_open\u0026#34; elif self.state == \u0026#34;moving_down\u0026#34;: ... This works. For a simple system, it\u0026rsquo;s perfectly readable.\nBut here\u0026rsquo;s what happens as the system grows:\nEvery method on Elevator gets the same if state == ... chain. Adding a new state (Maintenance, Emergency) requires touching every method. Logic for one state is interleaved with logic for all others — hard to read what \u0026ldquo;Idle behavior\u0026rdquo; actually is. Tests have to cover every state × every method combination. Now consider the State pattern:\nclass Elevator: def __init__(self): self.state = Idle() def step(self): self.state.step(self) Benefits:\nEach state\u0026rsquo;s behavior is in one place. Adding a new state = a new class, no existing code changes. Each state can be tested in isolation. Transitions are explicit (each state decides where to go next). Costs:\nMore classes (one per state). Slightly more indirection. For our problem (5 states × multiple methods, growing requirements), the State pattern is clearly worth it.\nVerbalizing the Decision \u0026ldquo;I\u0026rsquo;m using the State pattern. Each elevator state — Idle, MovingUp, MovingDown, DoorOpen, DoorClosed — gets its own class implementing a common ElevatorState interface. The alternative is a state-string-with-if-elif inside Elevator, which is fine for a sketch but quickly becomes unmaintainable as we add states or methods. Per-state classes give us cohesion (one state\u0026rsquo;s logic in one place), open-closed extensibility (new states are new classes), and isolated testability.\u0026rdquo;\n7. Stage 6 — Implementing the States A key question: what does the state interface look like?\nLooking at the operations we need:\nstep() — advance one tick. The state decides what to do (move, open doors, etc.) and what state to transition to. That\u0026rsquo;s actually the only operation we strictly need. Other operations on Elevator (accept_request, etc.) don\u0026rsquo;t need to be state-dependent in our scope — accepting a request is the same regardless of state.\nWe\u0026rsquo;ll keep our state interface narrow (just step) and add more methods later only if needed. This respects ISP — narrow interfaces are easier to implement and reason about.\nState Lifecycle How states flow:\n┌────────┐ │ Idle │ └───┬────┘ │ has request? up? ┌────┴────┐ down? ▼ ▼ ┌──────────┐ ┌────────────┐ │ MovingUp │ │ MovingDown │ └────┬─────┘ └──────┬─────┘ │ at stop? │ at stop? ▼ ▼ ┌──────────────────┐ │ DoorOpen │ └────────┬──────────┘ │ tick ▼ ┌──────────────────┐ │ DoorClosed │ └────────┬──────────┘ │ more requests? yes ┌────┴────┐ no ▼ ▼ (back to (back to MovingUp/Down) Idle) Each state knows the rules for transitioning to the next.\nThe Code class ElevatorState(ABC): \u0026#34;\u0026#34;\u0026#34; Abstract state for an elevator. Each concrete state encapsulates: - What the elevator does on a tick (one unit of simulated time). - When and to what state to transition. The state RECEIVES the elevator as an argument to step() rather than holding a reference to it. This keeps state objects stateless and therefore safely shareable across elevators (they\u0026#39;re effectively flyweights). It also makes the dependency direction explicit: the elevator owns the state, the state mutates the elevator. \u0026#34;\u0026#34;\u0026#34; @abstractmethod def step(self, elevator: \u0026#34;Elevator\u0026#34;) -\u0026gt; None: \u0026#34;\u0026#34;\u0026#34; Advance the elevator by one tick while in this state. May change the elevator\u0026#39;s current_floor, may transition to another state via elevator.set_state(...). \u0026#34;\u0026#34;\u0026#34; ... @abstractmethod def name(self) -\u0026gt; str: \u0026#34;\u0026#34;\u0026#34;Human-readable state name, used for logging and display.\u0026#34;\u0026#34;\u0026#34; ... class Idle(ElevatorState): \u0026#34;\u0026#34;\u0026#34; The elevator is sitting at a floor with no pending requests. On a tick: - If there\u0026#39;s a pending request, decide direction and transition. - If we\u0026#39;re already at a requested floor (rare edge case), open doors. - Otherwise, stay idle. \u0026#34;\u0026#34;\u0026#34; def step(self, elevator: \u0026#34;Elevator\u0026#34;) -\u0026gt; None: if not elevator.has_pending_requests(): # Nothing to do. Remain idle. return target = elevator.next_target() if target \u0026gt; elevator.current_floor: elevator.set_state(MovingUp()) elif target \u0026lt; elevator.current_floor: elevator.set_state(MovingDown()) else: # Already at the requested floor — open doors. elevator.set_state(DoorOpen()) def name(self) -\u0026gt; str: return \u0026#34;IDLE\u0026#34; class MovingUp(ElevatorState): \u0026#34;\u0026#34;\u0026#34; The elevator is moving up, one floor per tick. On a tick: - Increment current_floor. - If we\u0026#39;ve arrived at a requested stop, transition to DoorOpen. - Otherwise, keep moving (stay in MovingUp). Real elevators implement smarter behavior (LOOK algorithm: keep going up until no more upward stops, then reverse). We stick with simple nearest-stop tracking for now. \u0026#34;\u0026#34;\u0026#34; def step(self, elevator: \u0026#34;Elevator\u0026#34;) -\u0026gt; None: elevator.current_floor += 1 elevator.log(f\u0026#34;Moving UP → floor {elevator.current_floor}\u0026#34;) if elevator.current_floor in elevator.stops: # Arrived at a requested stop. Open doors. elevator.set_state(DoorOpen()) def name(self) -\u0026gt; str: return \u0026#34;MOVING_UP\u0026#34; class MovingDown(ElevatorState): \u0026#34;\u0026#34;\u0026#34;Mirror of MovingUp, in the other direction.\u0026#34;\u0026#34;\u0026#34; def step(self, elevator: \u0026#34;Elevator\u0026#34;) -\u0026gt; None: elevator.current_floor -= 1 elevator.log(f\u0026#34;Moving DOWN → floor {elevator.current_floor}\u0026#34;) if elevator.current_floor in elevator.stops: elevator.set_state(DoorOpen()) def name(self) -\u0026gt; str: return \u0026#34;MOVING_DOWN\u0026#34; class DoorOpen(ElevatorState): \u0026#34;\u0026#34;\u0026#34; Doors are open at a floor. Riders can board/disembark. On a tick: - Remove this floor from the pending stops (request fulfilled). - Transition to DoorClosed. We model door-open and door-closed as separate states (rather than one \u0026#34;stopped\u0026#34; state) because real elevators have these as physically distinct phases. Modeling them separately also gives us a clean place to add door-related behavior later (door obstruction sensors, etc.). \u0026#34;\u0026#34;\u0026#34; def step(self, elevator: \u0026#34;Elevator\u0026#34;) -\u0026gt; None: elevator.log(f\u0026#34;Doors OPEN at floor {elevator.current_floor}\u0026#34;) # Remove this stop from pending — we\u0026#39;re servicing it now. # discard() (vs remove()) doesn\u0026#39;t raise if the floor isn\u0026#39;t in stops. elevator.stops.discard(elevator.current_floor) elevator.set_state(DoorClosed()) def name(self) -\u0026gt; str: return \u0026#34;DOOR_OPEN\u0026#34; class DoorClosed(ElevatorState): \u0026#34;\u0026#34;\u0026#34; Doors are closed. About to either resume motion or go idle. On a tick: - If more pending requests, decide direction and start moving. - If no more requests, return to Idle. Note: this state\u0026#39;s logic mirrors Idle\u0026#39;s. We could collapse them, but keeping them separate is more honest about the elevator\u0026#39;s physical state and leaves room for state-specific behavior later (e.g., a brief delay after door closure before motion). \u0026#34;\u0026#34;\u0026#34; def step(self, elevator: \u0026#34;Elevator\u0026#34;) -\u0026gt; None: elevator.log(f\u0026#34;Doors CLOSED at floor {elevator.current_floor}\u0026#34;) if not elevator.has_pending_requests(): elevator.set_state(Idle()) return target = elevator.next_target() if target \u0026gt; elevator.current_floor: elevator.set_state(MovingUp()) elif target \u0026lt; elevator.current_floor: elevator.set_state(MovingDown()) else: # Another stop at the same floor — open doors again. elevator.set_state(DoorOpen()) def name(self) -\u0026gt; str: return \u0026#34;DOOR_CLOSED\u0026#34; A few important aspects:\n1. State objects hold no per-elevator data. They receive the elevator as argument and mutate it. We could share state instances across elevators (flyweight-style). We don\u0026rsquo;t bother, but it\u0026rsquo;s a nice property.\n2. Each state is small (10-30 lines). That\u0026rsquo;s a sign we\u0026rsquo;ve cut the right joints. Each state does one thing.\n3. Explicit transitions. Each state explicitly calls elevator.set_state(NextState()). The transition rules live with the behavior. There\u0026rsquo;s no central \u0026ldquo;state transition table\u0026rdquo; elsewhere.\n8. Stage 7 — The Elevator Class Now the elevator itself. Its responsibilities:\nTrack current floor. Hold pending stops (set of floor numbers). Hold its current state. Delegate step() to the current state. Accept new requests (add to stops). Provide query methods (has_pending_requests, next_target). What it does NOT do:\nDecide which request to handle next from a global perspective (dispatcher\u0026rsquo;s job). Compute pricing, manage occupancy. How does the elevator decide its next_target? This is the only \u0026ldquo;scheduling\u0026rdquo; decision the elevator makes for itself. Several approaches:\nApproach A — Nearest stop: pick the floor in stops closest to current_floor. Simple. Can lead to \u0026ldquo;thrashing\u0026rdquo; (going up, then down, then up).\nApproach B — Continue in current direction: if moving up, pick lowest stop above current; if no stops above, pick highest below.\nApproach C — Full LOOK algorithm: like B, but stops at every requested floor in current direction first, then reverses.\nFor interview scope, A is fine. We chose it earlier; let\u0026rsquo;s stick with it. We can mention B and C as extensions.\nThe Code from typing import Optional class Elevator: \u0026#34;\u0026#34;\u0026#34; A single elevator car within a building. State machine: the elevator\u0026#39;s behavior is governed by its current ElevatorState. Each tick, we delegate to state.step(self), which may move the elevator and/or transition to a new state. Pending stops are stored as a set (no duplicates, O(1) operations). The elevator chooses next_target() with a nearest-stop heuristic. Note: the Elevator does NOT decide WHICH external requests it should handle — that\u0026#39;s the Dispatcher\u0026#39;s job. The elevator just executes stops it\u0026#39;s been given (whether from internal requests or assigned by the dispatcher). \u0026#34;\u0026#34;\u0026#34; def __init__(self, elevator_id: int, num_floors: int): self.id = elevator_id self.num_floors = num_floors # Current floor (we start everyone at ground = 0). self.current_floor = 0 # Set of floors we need to visit. Set ensures no duplicates; # gives us O(1) add/remove/contains. self.stops: set[int] = set() # Current state. We start every elevator idle. self._state: ElevatorState = Idle() def set_state(self, state: ElevatorState) -\u0026gt; None: \u0026#34;\u0026#34;\u0026#34; Transition to a new state. Called by states themselves. Log the transition for visibility. \u0026#34;\u0026#34;\u0026#34; self.log(f\u0026#34;State {self._state.name()} → {state.name()}\u0026#34;) self._state = state @property def state_name(self) -\u0026gt; str: \u0026#34;\u0026#34;\u0026#34;Read-only access to current state name (for display, tests).\u0026#34;\u0026#34;\u0026#34; return self._state.name() @property def direction(self) -\u0026gt; Direction: \u0026#34;\u0026#34;\u0026#34; Current direction of motion, inferred from state. Used by the dispatcher\u0026#39;s scheduling strategy to decide which elevator best serves a new request (e.g., favor elevators already moving toward the requester). Derived rather than stored: the state already implies it. Keeping a separate field would risk drift between state and direction (always one source of truth). \u0026#34;\u0026#34;\u0026#34; if isinstance(self._state, MovingUp): return Direction.UP if isinstance(self._state, MovingDown): return Direction.DOWN return Direction.IDLE def step(self) -\u0026gt; None: \u0026#34;\u0026#34;\u0026#34;Advance the simulation by one tick. Delegates to current state.\u0026#34;\u0026#34;\u0026#34; self._state.step(self) def accept(self, request: Request) -\u0026gt; None: \u0026#34;\u0026#34;\u0026#34; Add a request\u0026#39;s target floor to our pending stops. Both internal and external requests result in the same action (add a stop). The DIFFERENCE between the two types is in HOW the request reached this elevator: - Internal: rider pressed a destination button while inside. - External: dispatcher assigned this elevator to a floor call. That difference is handled upstream (by the dispatcher); from the elevator\u0026#39;s perspective, all requests are just stops. \u0026#34;\u0026#34;\u0026#34; if not 0 \u0026lt;= request.floor \u0026lt; self.num_floors: raise ValueError( f\u0026#34;Floor {request.floor} out of range \u0026#34; f\u0026#34;(building has floors 0 to {self.num_floors - 1})\u0026#34; ) self.stops.add(request.floor) self.log(f\u0026#34;Accepted {request}; pending stops: {sorted(self.stops)}\u0026#34;) # If we\u0026#39;re idle, prod ourselves to start moving immediately # rather than waiting for the next external tick. This ensures # responsiveness: a request to an idle elevator gets processed # without an unnecessary one-tick delay. if isinstance(self._state, Idle): self._state.step(self) def has_pending_requests(self) -\u0026gt; bool: \u0026#34;\u0026#34;\u0026#34;True if there\u0026#39;s at least one floor we still need to visit.\u0026#34;\u0026#34;\u0026#34; return bool(self.stops) def next_target(self) -\u0026gt; int: \u0026#34;\u0026#34;\u0026#34; Pick the next floor to head toward. Strategy: nearest stop. We could be smarter (LOOK algorithm: finish current direction first), but nearest-stop is easy to understand and acceptable for interview scope. Precondition: stops is non-empty. Caller must check has_pending_requests() first. \u0026#34;\u0026#34;\u0026#34; if not self.stops: raise ValueError(\u0026#34;No pending stops to target\u0026#34;) return min(self.stops, key=lambda f: abs(f - self.current_floor)) def log(self, message: str) -\u0026gt; None: \u0026#34;\u0026#34;\u0026#34; Centralized logging. Keeps state-specific code from having to know about print formatting and lets us redirect logs (to file, in-memory list for tests, etc.) from one place. \u0026#34;\u0026#34;\u0026#34; print(f\u0026#34;[Elevator {self.id} @ floor {self.current_floor}] {message}\u0026#34;) def __repr__(self): return ( f\u0026#34;Elevator(id={self.id}, floor={self.current_floor}, \u0026#34; f\u0026#34;state={self.state_name}, stops={sorted(self.stops)})\u0026#34; ) A few things worth highlighting:\n1. direction is a derived property, not a field. The state already implies the direction; storing direction separately would invite the bug \u0026ldquo;I set state but forgot to update direction.\u0026rdquo; Always derive when you can.\n2. accept() proactively wakes up an idle elevator. Without this, a request to an idle elevator would only start being processed on the next external step() call. That one-tick delay is acceptable in some designs, but I prefer immediacy.\n3. The elevator\u0026rsquo;s logging is centralized. States call elevator.log(...), not print(...). We can route logs anywhere from one place.\n9. Stage 8 — The Dispatcher and Scheduling Strategy The dispatcher\u0026rsquo;s job: given an incoming request, decide which elevator handles it.\nDecision Point: What signature for the strategy? The strategy needs to pick one elevator from a list, given the request. So:\ndef select(self, request: Request, elevators: list[Elevator]) -\u0026gt; Optional[Elevator]: ... Returns Optional because in some scenarios no elevator may be suitable (e.g., all out of service in extensions).\nImplementing Nearest Car A reasonable scoring function:\nIdle elevators score by their distance to the request floor. Moving elevators score by distance, but penalized if their direction disagrees with the request. Why? An elevator moving up and a downward request below it isn\u0026rsquo;t a great match — it\u0026rsquo;d have to finish going up first. An idle nearby elevator is better.\nLet\u0026rsquo;s implement this cleanly.\nWhy Strategy at all? Because real elevator systems have evolved scheduling algorithms over decades. SCAN, LOOK, EDF (earliest deadline first), even ML-based dispatching exist. Hardcoding \u0026ldquo;nearest car\u0026rdquo; inside Dispatcher would lock us in. The Strategy pattern keeps it pluggable.\nThe Code class SchedulingStrategy(ABC): \u0026#34;\u0026#34;\u0026#34; The contract for any algorithm that picks an elevator to handle an external request. Receives the request and the list of elevators; returns the chosen one, or None if none are suitable. By depending on this abstraction (not a concrete algorithm), the Dispatcher follows the Dependency Inversion Principle. New algorithms (round-robin, least-loaded, ML-based) plug in without changing the dispatcher. \u0026#34;\u0026#34;\u0026#34; @abstractmethod def select( self, request: Request, elevators: list[Elevator], ) -\u0026gt; Optional[Elevator]: \u0026#34;\u0026#34;\u0026#34;Pick the best elevator to handle the request, or None.\u0026#34;\u0026#34;\u0026#34; ... class NearestCarStrategy(SchedulingStrategy): \u0026#34;\u0026#34;\u0026#34; Picks the elevator that can reach the request soonest. Scoring rule: - Distance from elevator\u0026#39;s current floor to request floor. - Plus a penalty if the elevator is moving in the OPPOSITE direction from the request (it has to finish its current run first). This is a heuristic, not optimal. Real systems use much more sophisticated scheduling. But it\u0026#39;s a sensible default for interview scope and demonstrates the right approach to extensibility. \u0026#34;\u0026#34;\u0026#34; # Penalty (in \u0026#34;virtual floors\u0026#34;) added to score when direction # disagrees. Tuned to be larger than typical distances so direction # mismatch dominates the choice. DIRECTION_MISMATCH_PENALTY = 1000 def select(self, request, elevators): if not elevators: return None return min(elevators, key=lambda e: self._score(e, request)) def _score(self, elevator: Elevator, request: Request) -\u0026gt; int: \u0026#34;\u0026#34;\u0026#34; Lower score = better choice. Computes: distance + (penalty if direction disagrees) \u0026#34;\u0026#34;\u0026#34; distance = abs(elevator.current_floor - request.floor) # Idle elevators have no direction issues. if elevator.direction == Direction.IDLE: return distance # For moving elevators, check if they\u0026#39;re heading toward the request. # If yes, no penalty. If they\u0026#39;re heading away, big penalty. moving_toward_request = ( (elevator.direction == Direction.UP and request.floor \u0026gt;= elevator.current_floor) or (elevator.direction == Direction.DOWN and request.floor \u0026lt;= elevator.current_floor) ) if moving_toward_request: return distance return distance + self.DIRECTION_MISMATCH_PENALTY class RoundRobinStrategy(SchedulingStrategy): \u0026#34;\u0026#34;\u0026#34; Cycles through elevators in order, ignoring distance entirely. Useful as a baseline / for showing the strategy interface really is pluggable. In practice it\u0026#39;s a poor scheduler — but it\u0026#39;s simple, fair (every elevator gets equal load over time), and predictable. \u0026#34;\u0026#34;\u0026#34; def __init__(self): self._next_index = 0 def select(self, request, elevators): if not elevators: return None chosen = elevators[self._next_index % len(elevators)] self._next_index += 1 return chosen The Dispatcher The dispatcher is small. It accepts requests and routes them — using the strategy for external requests, sending internal requests directly.\nclass Dispatcher: \u0026#34;\u0026#34;\u0026#34; Routes incoming requests to elevators. Two routing rules: - ExternalRequest: use the SchedulingStrategy to pick an elevator. - InternalRequest: route directly to a specific elevator (the one the rider is in). The dispatcher itself doesn\u0026#39;t decide HOW to pick an elevator — that\u0026#39;s delegated to the strategy. This separation lets us swap scheduling algorithms freely. \u0026#34;\u0026#34;\u0026#34; def __init__(self, elevators: list[Elevator], strategy: SchedulingStrategy): self.elevators = elevators self.strategy = strategy def submit( self, request: Request, for_elevator: Optional[Elevator] = None, ) -\u0026gt; Optional[Elevator]: \u0026#34;\u0026#34;\u0026#34; Submit a request for handling. For internal requests, the caller MUST pass `for_elevator` (the elevator the rider is in). We could try to infer it, but it\u0026#39;s cleaner to require it explicitly — the rider knows which elevator they\u0026#39;re in. For external requests, we ignore `for_elevator` and use the strategy to pick. Returns the elevator that was assigned, or None if no elevator could handle the request (rare; means strategy returned None). \u0026#34;\u0026#34;\u0026#34; if isinstance(request, InternalRequest): if for_elevator is None: raise ValueError( \u0026#34;Internal requests must specify for_elevator \u0026#34; \u0026#34;(the elevator the rider is in)\u0026#34; ) for_elevator.accept(request) return for_elevator # External request — let the strategy decide. chosen = self.strategy.select(request, self.elevators) if chosen is None: print(f\u0026#34;No elevator available for {request}\u0026#34;) return None print(f\u0026#34;Dispatcher: routing {request} to Elevator {chosen.id}\u0026#34;) chosen.accept(request) return chosen Notice the isinstance check. Some style guides discourage isinstance — but here it\u0026rsquo;s exactly the right tool. The dispatcher\u0026rsquo;s whole job is routing based on type. We could use polymorphism instead (e.g., request.route_via(dispatcher)), but that would put dispatcher knowledge into the request classes — leaking abstractions in the wrong direction. isinstance is honest about what\u0026rsquo;s happening.\n10. Stage 9 — The Building (Top-Level Coordinator) The Building is our top-level entity. Like ParkingLot in the previous deep-dive, it should be small — just hold the components and provide a tick() method to advance the simulation.\nclass Building: \u0026#34;\u0026#34;\u0026#34; Top-level entity holding floors, elevators, and the dispatcher. Responsibility: coordinate the simulation. Each tick(), every elevator advances. New requests are submitted via the dispatcher. Notice we don\u0026#39;t track floor objects (we use integer floor numbers). If we ever need per-floor entities (display boards, hall buttons), we\u0026#39;d add a Floor class here. \u0026#34;\u0026#34;\u0026#34; def __init__( self, num_floors: int, num_elevators: int, strategy: SchedulingStrategy, ): if num_floors \u0026lt; 2: raise ValueError(\u0026#34;Building needs at least 2 floors\u0026#34;) if num_elevators \u0026lt; 1: raise ValueError(\u0026#34;Building needs at least 1 elevator\u0026#34;) self.num_floors = num_floors # Build elevators with sequential IDs. self.elevators = [Elevator(i, num_floors) for i in range(num_elevators)] # The dispatcher orchestrates request routing using the strategy. self.dispatcher = Dispatcher(self.elevators, strategy) def tick(self) -\u0026gt; None: \u0026#34;\u0026#34;\u0026#34; Advance the simulation by one time unit. Every elevator gets one step. In a real (multi-threaded) system, each elevator would move on its own clock. For our discrete-time simulation, we drive them all together, which is simpler to reason about. \u0026#34;\u0026#34;\u0026#34; for elevator in self.elevators: elevator.step() def request( self, request: Request, for_elevator: Optional[Elevator] = None, ) -\u0026gt; Optional[Elevator]: \u0026#34;\u0026#34;\u0026#34;Convenience wrapper for submitting a request via the dispatcher.\u0026#34;\u0026#34;\u0026#34; return self.dispatcher.submit(request, for_elevator) 11. Stage 10 — Validating with a Mental Walkthrough Let\u0026rsquo;s run a realistic scenario.\nScenario: A 5-floor building with 2 elevators, both starting at floor 0. Someone on floor 3 presses \u0026ldquo;down.\u0026rdquo; Then someone on floor 1 presses \u0026ldquo;up.\u0026rdquo; Then the rider in the first-assigned elevator picks floor 0.\nbuilding = Building(num_floors=5, num_elevators=2, strategy=NearestCarStrategy()) e0, e1 = building.elevators Both elevators are at floor 0, idle. Direction = IDLE for both.\nbuilding.request(ExternalRequest(3, Direction.DOWN)) Trace:\nDispatcher receives ExternalRequest(3, DOWN). Strategy scores both elevators. Both at floor 0, both IDLE. Distance = 3 for both. Score tie. min() returns the first one (Python\u0026rsquo;s stable order). e0 is chosen. e0.accept(request) → e0.stops = {3}. e0 is Idle, so we proactively step. Idle.step(e0): target = 3, \u0026gt; current 0 → transition to MovingUp. building.tick() # tick 1 e0 is MovingUp. current_floor becomes 1. Not in stops, stays MovingUp. e1 is Idle, no requests. Stays Idle. building.request(ExternalRequest(1, Direction.UP)) Strategy scores both elevators for request to floor 1. e0: at floor 1, direction UP. Request to floor 1 in UP direction. Distance = 0, direction matches → score 0. e1: at floor 0, direction IDLE. Distance = 1. Score 1. e0 wins. But wait — e0 already had stops = {3}. Now stops = {1, 3}. e0 is in MovingUp; we don\u0026rsquo;t \u0026ldquo;wake it up\u0026rdquo; (only Idle elevators get woken). It\u0026rsquo;ll process the new stop on its next tick. Hmm. Let\u0026rsquo;s pause and think.\nIs this right? e0 is at floor 1, just having arrived during the previous tick. Stops = {1, 3}. On its next step (still in MovingUp), it\u0026rsquo;ll move to floor 2 — passing the new stop at floor 1!\nThis is a bug. When we accept a request whose floor we\u0026rsquo;re currently on, we should open doors, not move past it. Let me trace more carefully.\nActually wait — let me re-trace. When e0 stepped to floor 1 in the previous tick, it checked if self.current_floor in elevator.stops. At that point, stops was {3}, so 1 wasn\u0026rsquo;t in stops. It stayed MovingUp.\nNow we add 1 to stops. But we\u0026rsquo;ve already stepped past the check. Next tick, the state is still MovingUp, and step() will increment to floor 2.\nYes, this is a real bug. It\u0026rsquo;s a subtle one — the timing matters.\nHow to fix? When accept() is called and the new floor equals current_floor and we\u0026rsquo;re between transitions, we should probably open doors. Let me think about what\u0026rsquo;s cleanest.\nOption A — Special-case in accept(): if request.floor == current_floor, transition to DoorOpen. Adds a state-aware check to accept(), which we wanted to keep stateless.\nOption B — Always check at start of step(): before the state\u0026rsquo;s normal logic, check if current_floor is in stops; if so, ensure we open doors. Pushes the check into a wrapper.\nOption C — Acceptable simplification: acknowledge the bug in interview, note that real systems handle this with more nuanced state machines, and move on for time\u0026rsquo;s sake.\nFor the interview, I\u0026rsquo;d implement A as a small fix and explain it:\ndef accept(self, request: Request) -\u0026gt; None: ... self.stops.add(request.floor) # Edge case: if we\u0026#39;re currently AT the requested floor and not # already in DoorOpen, jump to DoorOpen so we don\u0026#39;t accidentally # pass our own stop. This handles the case where a new request # arrives for the floor we just reached but haven\u0026#39;t yet \u0026#34;noticed\u0026#34; # because the in-stops check happens during MovingUp/MovingDown. if (request.floor == self.current_floor and not isinstance(self._state, (DoorOpen, DoorClosed))): self.set_state(DoorOpen()) return # Otherwise, the existing wake-up-from-idle behavior: if isinstance(self._state, Idle): self._state.step(self) This kind of bug-finding-and-fixing is gold in an interview. It shows you actually trace through your design, not just write it.\nLet me continue the walkthrough with the fix in place.\nbuilding.request(ExternalRequest(1, Direction.UP)) e0 is at floor 1, in MovingUp. Request adds floor 1 to stops. Check: request.floor (1) == current_floor (1) and state isn\u0026rsquo;t Door*. → Transition to DoorOpen. The next tick will open the door, then close. building.tick() # tick 2 e0 is in DoorOpen. step(): logs \u0026ldquo;Doors OPEN at floor 1\u0026rdquo;, removes 1 from stops, transitions to DoorClosed. e1 still idle. building.tick() # tick 3 e0 is in DoorClosed. Has pending stops ({3}). Target = 3, \u0026gt; current 1. → MovingUp. e1 idle. building.tick() # tick 4 e0 in MovingUp. floor → 2. Not in stops. Continues MovingUp. building.tick() # tick 5 e0 in MovingUp. floor → 3. In stops! → DoorOpen. Now the rider inside presses floor 0:\nbuilding.request(InternalRequest(0), for_elevator=e0) Internal request. Routed directly to e0 (no strategy involved). e0.accept(InternalRequest(0)): adds 0 to stops. Not at current_floor (3 != 0). Not idle. So no wake-up needed. building.tick() # tick 6 e0 in DoorOpen. step(): logs, removes 3 from stops, transitions to DoorClosed. building.tick() # tick 7 e0 in DoorClosed. Has stops ({0}). Target 0, \u0026lt; current 3. → MovingDown. building.tick() # tick 8, 9, 10 floor → 2 → 1 → 0. At floor 0, in stops. → DoorOpen. building.tick() # tick 11 DoorOpen → removes 0 → DoorClosed. building.tick() # tick 12 DoorClosed → no stops → Idle. ✓ Full lifecycle works. The bug we found in tracing is fixed. Both internal and external requests are handled. The dispatcher used the strategy for external; bypassed it for internal.\nEdge cases worth mentioning Building with one elevator: the strategy still works (only one to pick). Logic uniform.\nRequest to a floor out of range: accept() raises ValueError. Caller (dispatcher) doesn\u0026rsquo;t catch — error propagates up. In production we\u0026rsquo;d want graceful handling at the API layer.\nMultiple requests for same floor: stops is a set, so deduplicated automatically.\nTied scores in strategy: Python\u0026rsquo;s min returns the first encountered — predictable but not always optimal. Could be improved with secondary keys (e.g., elevator with fewer pending stops as tiebreaker).\n12. Stage 11 — Anticipating Follow-Up Questions Q: \u0026ldquo;What if requests come in while the simulation is in the middle of a tick?\u0026rdquo; Currently, requests are atomic — they happen between ticks. In a multi-threaded system, requests might arrive concurrently with ticks. We\u0026rsquo;d need:\nLocking around elevator.stops modifications. Possibly a request queue that ticks drain at the start. For interview, I\u0026rsquo;d say:\n\u0026ldquo;Right now requests are processed synchronously between ticks. If we needed concurrency, I\u0026rsquo;d add a thread-safe request queue per elevator, and have step() drain it at the start of each tick. The state machine itself wouldn\u0026rsquo;t need changes.\u0026rdquo;\nQ: \u0026ldquo;How would you implement LOOK scheduling?\u0026rdquo; LOOK: keep going in current direction until no more stops in that direction; then reverse.\nChange next_target() to be direction-aware:\ndef next_target(self) -\u0026gt; int: if not self.stops: raise ValueError(\u0026#34;No pending stops\u0026#34;) if self.direction == Direction.UP: # Stops above us, then reverse if none. above = [s for s in self.stops if s \u0026gt; self.current_floor] if above: return min(above) # Lowest above (closest going up) return max(self.stops) # Reverse — highest below elif self.direction == Direction.DOWN: below = [s for s in self.stops if s \u0026lt; self.current_floor] if below: return max(below) return min(self.stops) else: # Idle: nearest is fine return min(self.stops, key=lambda f: abs(f - self.current_floor)) Note: this is just changing the strategy for one elevator\u0026rsquo;s choice of next stop. The dispatcher\u0026rsquo;s strategy is separate.\nQ: \u0026ldquo;How do you handle capacity?\u0026rdquo; Add capacity to Elevator. In accept(), reject (or queue) when capacity reached:\ndef __init__(self, ..., capacity: int = 10): self.capacity = capacity self.current_load = 0 # number of riders When doors open, riders get on/off. We\u0026rsquo;d need to track per-stop how many board and how many leave. This complicates the model considerably and probably isn\u0026rsquo;t worth doing in an interview unless asked.\nQ: \u0026ldquo;Maintenance or emergency state?\u0026rdquo; Trivial with State pattern: add Maintenance and Emergency classes. Maintenance might refuse all requests; Emergency might go to ground floor and stay there. No existing state class changes — that\u0026rsquo;s the OCP win.\nclass Maintenance(ElevatorState): def step(self, elevator): # Do nothing. Maintenance mode is inert. pass def name(self): return \u0026#34;MAINTENANCE\u0026#34; Then elevator.set_state(Maintenance()) puts it out of service. The dispatcher could filter it out:\nclass NearestCarStrategy(SchedulingStrategy): def select(self, request, elevators): # Filter out non-operational elevators. operational = [ e for e in elevators if not isinstance(e._state, Maintenance) # extension as needed ] if not operational: return None return min(operational, key=lambda e: self._score(e, request)) Q: \u0026ldquo;How would you test this?\u0026rdquo; Each piece is testable in isolation:\nStates: call state.step(mock_elevator) and assert what changed. No real elevator needed. Elevator: test transitions by stepping and checking state_name. Strategy: call strategy.select(request, [mock_elevators]) and assert the right one is chosen. Dispatcher: verify routing by submitting requests and checking which elevator received them. The State pattern especially helps here — each state\u0026rsquo;s logic is in one method, and you don\u0026rsquo;t need to set up a complex elevator history to test \u0026ldquo;what does Idle do given a pending request?\u0026rdquo;\nQ: \u0026ldquo;What about display boards showing each elevator\u0026rsquo;s floor?\u0026rdquo; Same Observer pattern as the parking lot — Elevator becomes observable; FloorIndicator observes and renders. We\u0026rsquo;d add attach() / _notify() to Elevator and notify on floor changes.\nI won\u0026rsquo;t code it in detail; the pattern from the parking lot deep-dive transfers directly.\n13. Final Reflection — What This Problem Teaches Lesson 1: When You See \u0026ldquo;Modes of Operation,\u0026rdquo; Reach for State Pattern The hallmark of a State-pattern problem: an entity has named operational modes that change over time, with different behavior per mode. Elevator, vending machine, order processing, document approval, traffic light, network connection — all classic examples.\nOnce you recognize the pattern, the design almost writes itself: identify the states, define the transition rules, write a class per state. The mechanical clarity of this approach is exactly what makes the State pattern beloved.\nLesson 2: Distinguish the Two Routing Surfaces External vs internal requests is the elevator-specific instance of a general principle: not all requests are alike. They may differ in source, in priority, in routing.\nWhen you spot this kind of asymmetry early — in any LLD problem — model it explicitly with separate types. The cost is an extra class or two; the benefit is dispatch logic that\u0026rsquo;s clear and extensible.\nLesson 3: Walking Through Designs Catches Real Bugs We found the \u0026ldquo;request for current floor while moving\u0026rdquo; bug only by tracing through the simulation step by step. No amount of staring at code reveals all the timing bugs. Always simulate.\nThe fact that we found the bug, named it, fixed it, and explained it — that sequence is exactly what interviewers are watching for. Real designers find bugs in their own designs. They don\u0026rsquo;t pretend bugs don\u0026rsquo;t exist.\nLesson 4: Derive, Don\u0026rsquo;t Duplicate Elevator.direction is derived from _state — not stored as a separate field. This avoids one of the most common bug categories: state and derived data drifting apart.\nIn any design, ask: \u0026ldquo;Is this attribute the source of truth, or can it be computed from something else?\u0026rdquo; If derivable, derive.\nLesson 5: Strategy Pattern Pays Off Multiple Times We have two strategies in this design: scheduling (which elevator handles a request) and next_target (which floor the elevator picks next). Each is independently swappable. Each has multiple reasonable implementations. Each is in a single, isolated file you can replace without touching the rest.\nThis is the State + Strategy combination at its best: one pattern handles the entity\u0026rsquo;s lifecycle; the other handles its decision-making algorithms.\nOnwards to the next problem: Library Management — where we\u0026rsquo;ll explore policy-driven design, the Book vs BookItem distinction, and how reservations introduce queues.\n","permalink":"https://blogs.codeverra.com/blog/low_level_design/interview_questions/06b_lld_elevator_deep_dive/","summary":"\u003ch1 id=\"lld-deep-dive-2--designing-an-elevator-system\"\u003eLLD Deep Dive #2 — Designing an Elevator System\u003c/h1\u003e\n\u003cblockquote\u003e\n\u003cp\u003e\u003cstrong\u003eWhat this document is:\u003c/strong\u003e A genuine, step-by-step walkthrough of how to think through an Elevator LLD problem in an interview. We\u0026rsquo;ll reason from first principles, weigh alternatives, and document each design decision — building toward a clean, extensible design centered on the State pattern.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003chr\u003e\n\u003ch2 id=\"table-of-contents\"\u003eTable of Contents\u003c/h2\u003e\n\u003col\u003e\n\u003cli\u003e\u003ca href=\"#1-why-this-problem-matters\"\u003eWhy This Problem Matters\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#2-stage-1--receiving-the-problem\"\u003eStage 1 — Receiving the Problem\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#3-stage-2--clarifying-requirements\"\u003eStage 2 — Clarifying Requirements\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#4-stage-3--identifying-entities\"\u003eStage 3 — Identifying Entities\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#5-stage-4--modeling-requests\"\u003eStage 4 — Modeling Requests (Internal vs External)\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#6-stage-5--the-state-pattern-decision\"\u003eStage 5 — The State Pattern Decision\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#7-stage-6--implementing-the-states\"\u003eStage 6 — Implementing the States\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#8-stage-7--the-elevator-class\"\u003eStage 7 — The Elevator Class\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#9-stage-8--the-dispatcher-and-scheduling-strategy\"\u003eStage 8 — The Dispatcher and Scheduling Strategy\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#10-stage-9--the-building-top-level-coordinator\"\u003eStage 9 — The Building (Top-Level Coordinator)\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#11-stage-10--validating-with-a-mental-walkthrough\"\u003eStage 10 — Validating with a Mental Walkthrough\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#12-stage-11--anticipating-follow-up-questions\"\u003eStage 11 — Anticipating Follow-Up Questions\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#13-final-reflection\"\u003eFinal Reflection — What This Problem Teaches\u003c/a\u003e\u003c/li\u003e\n\u003c/ol\u003e\n\u003chr\u003e\n\u003ch2 id=\"1-why-this-problem-matters\"\u003e1. Why This Problem Matters\u003c/h2\u003e\n\u003cp\u003eIf Parking Lot is the canonical \u0026ldquo;model relationships\u0026rdquo; problem, Elevator is the canonical \u0026ldquo;model state transitions\u0026rdquo; problem. It\u0026rsquo;s specifically designed (whether interviewers know it or not) to make the \u003cstrong\u003eState pattern\u003c/strong\u003e shine.\u003c/p\u003e","tags":null,"title":""},{"content":"LLD Deep Dive #3 — Designing a Library Management System What this document is: A genuine, step-by-step walkthrough of how to think through a Library Management LLD problem. We\u0026rsquo;ll reason from first principles, weigh alternatives, and document each design decision — building toward a clean, extensible system that handles books, members, loans, fines, and reservations.\nTable of Contents Why This Problem Matters Stage 1 — Receiving the Problem Stage 2 — Clarifying Requirements Stage 3 — Identifying Entities (And the Big Modeling Decision) Stage 4 — Modeling Books vs Book Copies Stage 5 — Modeling Members and Their Policies Stage 6 — The Loan Lifecycle Stage 7 — The Reservation Queue Stage 8 — Search Functionality Stage 9 — The Library (Top-Level Coordinator) Stage 10 — Validating with a Mental Walkthrough Stage 11 — Anticipating Follow-Up Questions Final Reflection — What This Problem Teaches 1. Why This Problem Matters The library system is the canonical \u0026ldquo;policy-driven domain modeling\u0026rdquo; interview problem. What it tests:\nThe Book vs BookItem distinction — a subtle modeling insight that separates designers who think in domain terms from those who blindly map \u0026ldquo;noun → class.\u0026rdquo; Policy variation across user types — students, faculty, and guests have different borrowing rules. Where does that variation live? Lifecycle management — books move through borrowed/available/reserved states; loans accrue fines; reservations queue up. Multiple \u0026ldquo;actors\u0026rdquo; — members, librarians, the system itself. Each has different responsibilities. Get this design right, and you\u0026rsquo;ve shown that you can model a domain as it actually exists rather than as it superficially appears.\n2. Stage 1 — Receiving the Problem The interviewer says:\n\u0026ldquo;Design a library management system.\u0026rdquo;\nVague again. What might be meant:\nA simple book-loan tracker? A multi-branch system? Digital books, audiobooks, ebooks? Member management with billing? Search engine for the catalog? We need to bound this with questions.\n3. Stage 2 — Clarifying Requirements Q1: What core operations must we support? Why ask: Sets the bare minimum scope. If they answer \u0026ldquo;just borrow and return,\u0026rdquo; the design is much simpler than if they say \u0026ldquo;borrow, return, reserve, search, fine.\u0026rdquo;\nSuppose: \u0026ldquo;Borrow, return, search, reserve, and assess fines for late returns.\u0026rdquo;\nComprehensive. We\u0026rsquo;re building a real (small) system.\nQ2: Single library or multiple branches? Why ask: Multiple branches means books can move between them, members might be branch-affiliated, transfers become a thing. Significantly more complex.\nSuppose: \u0026ldquo;Single library for now. Multi-branch is a possible extension.\u0026rdquo;\nGood — keeps scope manageable.\nQ3: What kinds of items can be borrowed? Why ask: \u0026ldquo;Just books\u0026rdquo;? Or also \u0026ldquo;ebooks, audiobooks, magazines, DVDs\u0026rdquo;? Each could have different rules (ebooks have license-based \u0026ldquo;loans\u0026rdquo; rather than physical copies).\nSuppose: \u0026ldquo;Just physical books for now. We might add ebooks later.\u0026rdquo;\nNote this — when designing, leave room for future item types but don\u0026rsquo;t over-engineer for them.\nQ4: How do we identify books? Why ask: This is where the Book vs BookItem distinction emerges. If the library has 5 copies of The Pragmatic Programmer, are those 5 separate things or one thing with quantity 5?\nThe right answer: 5 separate physical copies, each tracked individually, but they share metadata (title, author, ISBN, etc.) about the underlying title.\nWhy? Because each physical copy has its own state — one might be borrowed, one might be on hold, one might be lost. We can\u0026rsquo;t treat them as interchangeable items in a counter.\nSuppose: \u0026ldquo;Yes, distinguish title from physical copy. Each copy is uniquely identifiable.\u0026rdquo;\nExcellent — this is the key insight. Verbalize it:\n\u0026ldquo;I\u0026rsquo;ll model two distinct entities: Book (the title — author, ISBN, category — that\u0026rsquo;s shared across all copies) and BookItem (a specific physical copy — its location, status, condition). Many candidates conflate these and end up with awkward designs where \u0026lsquo;borrowing a book\u0026rsquo; is actually \u0026lsquo;decrementing a counter.\u0026rsquo; Modeling each copy as a first-class entity gives us per-copy state and a clean reservation model.\u0026rdquo;\nQ5: Different member types? Why ask: If all members are the same, life is simple. If students vs faculty have different rules, we need to model that variation.\nSuppose: \u0026ldquo;Yes — Students, Faculty, and Guests. Different borrowing limits and durations.\u0026rdquo;\nNow we know we need varied policy. Policy variation is a classic Strategy pattern application.\nMember Type Max Books Loan Duration Fine/Day Student 3 14 days ₹5 Faculty 10 30 days ₹2 Guest 1 7 days ₹10 Q6: How do reservations work? Why ask: Several possible behaviors:\n\u0026ldquo;I want this book sometime — notify me when available.\u0026rdquo; \u0026ldquo;I want this book at this date.\u0026rdquo; FIFO queue vs priority queue. Suppose: \u0026ldquo;FIFO. When a copy is returned, the next person in queue is notified.\u0026rdquo;\nSimple and standard. We\u0026rsquo;ll use a queue (deque).\nQ7: Fine handling? Why ask: What triggers fines? When are they computed? When are they paid? What if unpaid fines block borrowing?\nSuppose: \u0026ldquo;Fines are computed at return time based on days overdue. Track the amount; payment processing is out of scope. Unpaid fines don\u0026rsquo;t block borrowing in v1.\u0026rdquo;\nQ8: Concurrency and persistence? Suppose: \u0026ldquo;In-memory storage. Single-threaded.\u0026rdquo;\nRecap \u0026ldquo;To summarize: a single library with physical books (each title can have multiple copies, each copy tracked individually). Three member types — Student, Faculty, Guest — with different borrowing limits, durations, and fine rates. Operations: borrow, return, search (by title/author/category), reserve (FIFO queue), and fine assessment at return time. In-memory, single-threaded. Multi-branch and digital books are future extensions. Aligned?\u0026rdquo;\nConfirmed. Now we design.\n4. Stage 3 — Identifying Entities Pull nouns from the requirements:\nlibrary book copy / book item member (student, faculty, guest) loan reservation fine search librarian (administrative actor) queue Filter:\nNoun Class? Reasoning library Yes Top-level entity (facade) book Yes The title — shared metadata across copies book item / copy Yes Each physical copy with its own state member Yes (with subtypes? or single class?) We\u0026rsquo;ll decide soon loan Yes A borrowing record with its own lifecycle reservation Yes A queued request fine Maybe — see below Could be a class, or just a calculation search Yes A focused responsibility librarian Reject for now An actor outside the system; not modeled queue No Use a deque Should Fine be a class? A fine is just an amount of money associated with an overdue loan. It doesn\u0026rsquo;t have its own behavior beyond being computed. In our scope, treating it as loan.calculate_fine() returning a float is sufficient.\nIf the system grew to track fine payments, fine waivers, fine appeals, etc., then Fine would deserve a class. YAGNI — defer.\nMember subtypes — Inheritance or Strategy? This is the most interesting modeling decision in this problem. We have three member types with different behavior. Two natural approaches:\nApproach A — Inheritance: Student, Faculty, Guest are subclasses of Member. Each overrides methods like max_books(), loan_duration(), fine_per_day().\nApproach B — Composition (Strategy): A single Member class with a policy attribute. MembershipPolicy is a Strategy with implementations StudentPolicy, FacultyPolicy, GuestPolicy.\nBoth work. Let me weigh them.\nInheritance pros:\nSimple, intuitive — \u0026ldquo;a Student is a Member.\u0026rdquo; One class per type. Inheritance cons:\nA member\u0026rsquo;s type can change (a student graduates and becomes a faculty member, or vice versa). Inheritance makes this awkward — you\u0026rsquo;d have to construct a new object and migrate state. If we add new dimensions of variation (e.g., \u0026ldquo;premium member\u0026rdquo; status that adds extra benefits), we\u0026rsquo;d combinatorially explode subclasses. Composition (Strategy) pros:\nMember type can change at runtime — just swap the policy. Multiple orthogonal dimensions don\u0026rsquo;t combinatorially explode (e.g., we could have membership level and academic role separately). Easier to test policies in isolation. Composition cons:\nSlightly less natural to read — member.policy.max_books vs member.max_books. One more class to manage. Decision Composition / Strategy. Here\u0026rsquo;s why:\nThe policies are pure data + behavior, with no member-specific state. StudentPolicy doesn\u0026rsquo;t have a name or an email — it just has rules. Putting those rules on a separate object that the Member references gives us:\nThe ability to upgrade a member without recreating them (e.g., guest → student when they enroll). A single place to define each policy, which is easier for an administrator to read/edit. Easier extension — new policies plug in without subclassing Member. Verbalize:\n\u0026ldquo;I\u0026rsquo;m using composition: a single Member class that holds a MembershipPolicy reference. The alternative — Student/Faculty/Guest as Member subclasses — works, but member type can change in real life (a student graduates), and inheritance makes that awkward. With composition, a member\u0026rsquo;s policy can be swapped at runtime without recreating the member object. It also separates \u0026lsquo;who is the member\u0026rsquo; from \u0026lsquo;what rules apply to them\u0026rsquo; — two distinct concerns.\u0026rdquo;\nFinal entity list:\nLibrary (top-level facade) Book (the title) BookItem (a physical copy) Member MembershipPolicy (abstract) → StudentPolicy, FacultyPolicy, GuestPolicy Loan Reservation BookSearch (search functionality) BookStatus (enum) 5. Stage 4 — Modeling Books vs Book Copies The two-class split is the cornerstone of this design. Let\u0026rsquo;s model it carefully.\nWhat goes on Book? The title\u0026rsquo;s metadata — things shared by every copy of this title:\nISBN Title Author Category / genre Publication year (optional) Description (optional, often) What about the list of copies? Should Book hold its own copies?\nOption A: Book has a list of BookItem (copies). Option B: Copies are managed separately; book and copies are linked via ISBN.\nOption A is more cohesive — asking \u0026ldquo;what copies do we have of this book?\u0026rdquo; is natural. Let\u0026rsquo;s go with it.\nBut there\u0026rsquo;s a subtlety: who manages the list of all books? The Library. Who manages the copies of one specific book? That book itself.\nThis split puts ownership where it logically belongs: the library owns the book catalog; each book owns its copies.\nWhat goes on BookItem? The physical copy\u0026rsquo;s state:\nA reference back to its Book (so we can ask \u0026ldquo;what title is this?\u0026rdquo;) A unique copy ID (e.g., barcode) Current status: AVAILABLE / BORROWED / LOST / RESERVED (we can debate the last) Possibly: shelf location, condition Should \u0026ldquo;RESERVED\u0026rdquo; be a status of BookItem?\nThis is interesting. Let\u0026rsquo;s think.\nWhen a member reserves a title, they\u0026rsquo;re not reserving a specific copy — they\u0026rsquo;re queued for any copy. When a copy becomes available, the next person in queue gets it. So reservation is at the Book level, not the BookItem level.\nHence, BookItem.status doesn\u0026rsquo;t need a RESERVED value. Statuses are just AVAILABLE, BORROWED, LOST.\nThe Code from enum import Enum from typing import Optional class BookStatus(Enum): \u0026#34;\u0026#34;\u0026#34; Status of a single physical copy. AVAILABLE: in the library, ready to be borrowed. BORROWED: currently lent to a member. LOST: copy has been reported missing/destroyed. Notably absent: RESERVED. Reservations are at the Book (title) level, not the BookItem level — when a copy becomes AVAILABLE, the reservation queue is checked. The copy itself is just AVAILABLE; whoever asks first gets it (which is the reservation queue\u0026#39;s job to manage). \u0026#34;\u0026#34;\u0026#34; AVAILABLE = \u0026#34;available\u0026#34; BORROWED = \u0026#34;borrowed\u0026#34; LOST = \u0026#34;lost\u0026#34; class Book: \u0026#34;\u0026#34;\u0026#34; Represents a title in the catalog (e.g., \u0026#39;The Pragmatic Programmer\u0026#39;). Multiple physical copies (BookItem instances) can exist for one Book. The Book holds the metadata shared across all its copies. Why model Book and BookItem as distinct entities? - A title and a physical copy have different lifecycles. The title is added to the catalog once; copies come and go (lost, replaced). - Each physical copy has its own state — borrowed, available, lost. Conflating them into a single \u0026#39;Book with quantity\u0026#39; loses this. - Reservations queue against the title (next available copy wins), while loans target a specific copy. The two concepts demand separate entities. \u0026#34;\u0026#34;\u0026#34; def __init__( self, isbn: str, title: str, author: str, category: str, ): self.isbn = isbn # International Standard Book Number — unique key self.title = title self.author = author self.category = category # Copies of this title. We use a list because order doesn\u0026#39;t # really matter, and lookups are typically iterating to find # an available one. self.copies: list[BookItem] = [] def add_copy(self, copy: \u0026#34;BookItem\u0026#34;) -\u0026gt; None: \u0026#34;\u0026#34;\u0026#34; Add a physical copy. Called by the library when stocking new copies. We accept the BookItem (rather than constructing one here) because the caller may want to control how copies are created (with their own copy_ids, etc.). \u0026#34;\u0026#34;\u0026#34; self.copies.append(copy) def available_copy(self) -\u0026gt; Optional[\u0026#34;BookItem\u0026#34;]: \u0026#34;\u0026#34;\u0026#34; Return the first AVAILABLE copy, or None if no copies are free. Why \u0026#34;first\u0026#34; rather than some other policy? - We\u0026#39;re not optimizing for shelf location or anything subtle. - The list ordering doesn\u0026#39;t have meaning in our scope, so \u0026#34;first\u0026#34; is just a deterministic choice. - In a real system you might prefer the copy with the lowest shelf position, or the one in best condition. \u0026#34;\u0026#34;\u0026#34; return next( (c for c in self.copies if c.status == BookStatus.AVAILABLE), None, ) def total_copies(self) -\u0026gt; int: return len(self.copies) def available_copy_count(self) -\u0026gt; int: \u0026#34;\u0026#34;\u0026#34;Useful for displaying availability in search results.\u0026#34;\u0026#34;\u0026#34; return sum(1 for c in self.copies if c.status == BookStatus.AVAILABLE) def __repr__(self): return f\u0026#34;Book({self.isbn}, {self.title!r}, by {self.author})\u0026#34; class BookItem: \u0026#34;\u0026#34;\u0026#34; A single physical copy of a book. Has its own lifecycle (status), its own identifier, and a reference back to the Book metadata. The reference is one-way (item knows its book) rather than bidirectional, because Book already knows its copies via .copies — bidirectional links would be redundant and risk inconsistency. Conceptually a BookItem is owned by exactly one Book. We don\u0026#39;t enforce this with code (Python doesn\u0026#39;t make that easy), but we rely on the convention that you create BookItems via Book.add_copy. \u0026#34;\u0026#34;\u0026#34; def __init__(self, book: Book, copy_id: str): self.book = book self.copy_id = copy_id # Newly added copies start as AVAILABLE. self.status = BookStatus.AVAILABLE def __repr__(self): return f\u0026#34;BookItem({self.copy_id} of {self.book.title!r}, {self.status.name})\u0026#34; A few things worth noting:\n1. The Book ↔ BookItem relationship is one-way at the model level. Book has a list of copies, each copy has a back-reference to its book. We could maintain bidirectional integrity (when you remove a copy, also clear the back-ref) but for our scope, the simplicity of \u0026ldquo;Book owns copies\u0026rdquo; is fine.\n2. We deliberately did not expose mutation of BookItem.status through methods like borrow() or return_to_shelf(). Why? Because status changes are part of the loan flow, which is the Library\u0026rsquo;s responsibility, not the BookItem\u0026rsquo;s. The Library will set status directly. If we found that status changes were happening from many places and getting tangled, we\u0026rsquo;d refactor — but right now, exposing setters too early is premature encapsulation.\nA good interview-level critique of myself: \u0026ldquo;In production I\u0026rsquo;d probably wrap status changes in BookItem methods (mark_borrowed, mark_returned) to enforce valid transitions. For this scope, I\u0026rsquo;m letting the Library mutate the status directly because the loan flow is the only thing that does it.\u0026rdquo;\n6. Stage 5 — Modeling Members and Their Policies We decided: composition (Strategy). Let\u0026rsquo;s implement.\nDefining the Policy Interface What does a policy need to express?\nMaximum books a member can borrow at once. Loan duration in days. Fine per day for overdue returns. That\u0026rsquo;s the whole interface for now. Three values, exposed as properties.\nWhy properties (not just attributes)? Because the values might be computed in some implementations. E.g., a \u0026ldquo;premium\u0026rdquo; policy might compute fine_per_day as base_rate * (1 - discount). Properties give us that flexibility without changing callers.\nThe Code from abc import ABC, abstractmethod class MembershipPolicy(ABC): \u0026#34;\u0026#34;\u0026#34; Encapsulates the rules a member operates under: how many books they can borrow, how long they get to keep them, and what they pay per day for overdue returns. Implemented as Strategy: each concrete policy is a class, and a Member holds a reference to one. This decouples \u0026#39;who the member is\u0026#39; from \u0026#39;what rules they follow.\u0026#39; Adding a new policy (Premium, Alumni, etc.) = a new class. No changes to Member, Library, or any other policy. \u0026#34;\u0026#34;\u0026#34; @property @abstractmethod def max_books(self) -\u0026gt; int: \u0026#34;\u0026#34;\u0026#34;Maximum number of books this member can have on loan at one time.\u0026#34;\u0026#34;\u0026#34; ... @property @abstractmethod def loan_days(self) -\u0026gt; int: \u0026#34;\u0026#34;\u0026#34;How many days a single loan lasts before becoming overdue.\u0026#34;\u0026#34;\u0026#34; ... @property @abstractmethod def fine_per_day(self) -\u0026gt; float: \u0026#34;\u0026#34;\u0026#34;Fine amount per day for overdue returns.\u0026#34;\u0026#34;\u0026#34; ... def __repr__(self): return self.__class__.__name__ class StudentPolicy(MembershipPolicy): \u0026#34;\u0026#34;\u0026#34;Rules for student members: limited capacity, moderate fines.\u0026#34;\u0026#34;\u0026#34; @property def max_books(self) -\u0026gt; int: return 3 @property def loan_days(self) -\u0026gt; int: return 14 @property def fine_per_day(self) -\u0026gt; float: return 5.0 class FacultyPolicy(MembershipPolicy): \u0026#34;\u0026#34;\u0026#34; Rules for faculty members: generous limits and longer loans, with a lower fine rate (faculty are usually trusted to return on time). \u0026#34;\u0026#34;\u0026#34; @property def max_books(self) -\u0026gt; int: return 10 @property def loan_days(self) -\u0026gt; int: return 30 @property def fine_per_day(self) -\u0026gt; float: return 2.0 class GuestPolicy(MembershipPolicy): \u0026#34;\u0026#34;\u0026#34;Rules for guests: minimal access with strict consequences.\u0026#34;\u0026#34;\u0026#34; @property def max_books(self) -\u0026gt; int: return 1 @property def loan_days(self) -\u0026gt; int: return 7 @property def fine_per_day(self) -\u0026gt; float: return 10.0 A note on style: I\u0026rsquo;m using @property for each value. The alternative is class-level constants (max_books = 3). Class constants are slightly less flexible (can\u0026rsquo;t be overridden by subclasses with computation) but slightly cleaner-looking. I lean toward properties because it keeps the door open for computed values.\nThe Member Class class Member: \u0026#34;\u0026#34;\u0026#34; A library patron. Holds: - Identity (id, name, email — for contact and tracking). - Current policy (what rules apply to them). - Active loans (so we can enforce max_books). Note we DON\u0026#39;T hold a list of past loans here. Loan history belongs in the Library (or a dedicated repository) — we don\u0026#39;t want every Member to grow unboundedly with history. Why expose `policy` as a public attribute? Because it\u0026#39;s expected to be swappable (a guest becomes a student, etc.), and Python\u0026#39;s convention is direct attribute access for such cases. A setter wrapping it would add no value. \u0026#34;\u0026#34;\u0026#34; def __init__( self, member_id: str, name: str, email: str, policy: MembershipPolicy, ): self.member_id = member_id self.name = name self.email = email self.policy = policy # Loans currently in effect (not yet returned). # Used by the library to enforce max_books. self.active_loans: list[\u0026#34;Loan\u0026#34;] = [] def can_borrow_more(self) -\u0026gt; bool: \u0026#34;\u0026#34;\u0026#34; True if this member is below their borrowing limit. Used by the library before issuing a loan. \u0026#34;\u0026#34;\u0026#34; return len(self.active_loans) \u0026lt; self.policy.max_books def __repr__(self): return f\u0026#34;Member({self.name}, {self.policy})\u0026#34; The can_borrow_more() method is a small but important responsibility: the member knows how to check its own limit. We could put this logic in the Library (if len(member.active_loans) \u0026lt; member.policy.max_books), but that scatters policy knowledge. By putting can_borrow_more on Member, we keep the rule encapsulated.\nThis is the Tell-Don\u0026rsquo;t-Ask principle in action: ask the member if they can borrow more, rather than asking for their internals and computing externally.\n7. Stage 6 — The Loan Lifecycle A loan represents one specific instance of a member borrowing one specific copy. It has its own data and behavior.\nWhat does a loan need? The member. The book item being borrowed. Issue date (when the loan started). Due date (when it must be returned by). Return date (None until returned). An ID, for tracking. What about behavior? Determine if it\u0026rsquo;s overdue. Calculate the fine (based on overdue days × member\u0026rsquo;s policy). The fine calculation is interesting. Should it live on Loan, on a separate FineCalculator, or on the policy?\nOption A: loan.calculate_fine() — the loan knows its data, asks the policy for the rate. Option B: fine_calculator.compute(loan) — separate utility. Option C: policy.compute_fine(loan) — policy owns it.\nA is the most natural: the loan has all the data (issue date, return date, member\u0026rsquo;s policy), so it can compute. And it doesn\u0026rsquo;t grow much code — it\u0026rsquo;s a single method.\nIf fine logic became complex (rounding rules, grace periods, holiday handling), we\u0026rsquo;d extract a FineCalculator. For now, keep it on Loan.\nThe Code from datetime import datetime, timedelta from typing import Optional import uuid class Loan: \u0026#34;\u0026#34;\u0026#34; A single borrowing event: this member checked out this copy on this date. Lifecycle: 1. Created when the member borrows the copy. 2. Active until returned. 3. On return, return_date is stamped and the loan moves to \u0026#34;completed\u0026#34; (we don\u0026#39;t model this as an enum; we infer it from return_date being non-None). Holds enough data to compute fines, due dates, etc. \u0026#34;\u0026#34;\u0026#34; def __init__(self, member: Member, book_item: \u0026#34;BookItem\u0026#34;): self.id = str(uuid.uuid4())[:8] self.member = member self.book_item = book_item # Issue date is set at construction. Using datetime.now() means # loans created in the same millisecond would have the same time; # for our purposes that\u0026#39;s fine. self.issue_date = datetime.now() # Due date is computed from the member\u0026#39;s policy at the time of # borrowing. We store it (rather than recomputing each time) so # the loan terms don\u0026#39;t change retroactively if the member\u0026#39;s # policy is later upgraded. self.due_date = self.issue_date + timedelta(days=member.policy.loan_days) # Return date is None until the loan is closed. self.return_date: Optional[datetime] = None @property def is_returned(self) -\u0026gt; bool: \u0026#34;\u0026#34;\u0026#34;True if the loan has been completed.\u0026#34;\u0026#34;\u0026#34; return self.return_date is not None @property def is_overdue(self) -\u0026gt; bool: \u0026#34;\u0026#34;\u0026#34; True if the loan is past its due date AND not yet returned. Returned loans aren\u0026#39;t \u0026#39;overdue\u0026#39; — they\u0026#39;re done. \u0026#34;\u0026#34;\u0026#34; return not self.is_returned and datetime.now() \u0026gt; self.due_date def calculate_fine(self) -\u0026gt; float: \u0026#34;\u0026#34;\u0026#34; Compute the fine for this loan. Fine logic: - If returned on or before due date: no fine. - If returned after due date: (overdue days) × (member\u0026#39;s fine rate). - If not yet returned: compute as if returning right now (this is useful for displaying \u0026#39;fine if you return today\u0026#39; to members). We use the member\u0026#39;s CURRENT fine rate, not the rate at borrowing time. This is a policy choice — if the member\u0026#39;s fine rate changes mid-loan, the new rate applies. (Alternative: snapshot the rate on Loan creation. Both are defensible; we pick current for simplicity.) \u0026#34;\u0026#34;\u0026#34; end_date = self.return_date or datetime.now() if end_date \u0026lt;= self.due_date: return 0.0 # +1 to count partial days as full days (a common library policy). overdue_days = (end_date - self.due_date).days + 1 return overdue_days * self.member.policy.fine_per_day def __repr__(self): status = \u0026#34;returned\u0026#34; if self.is_returned else \u0026#34;active\u0026#34; return ( f\u0026#34;Loan({self.id}, {self.book_item.book.title!r} → \u0026#34; f\u0026#34;{self.member.name}, {status})\u0026#34; ) The decision to store due_date on the loan (rather than recomputing it) deserves attention. If we recomputed each time as issue_date + member.policy.loan_days, then a member upgrading their policy would suddenly have all their existing loans extended (or shortened). That\u0026rsquo;s almost certainly wrong — loan terms are a contract at the time of borrowing.\nWe DO use the current fine rate (not the rate at borrowing). That\u0026rsquo;s a choice — both are defensible. In a real interview, I\u0026rsquo;d note this tradeoff:\n\u0026ldquo;I store the due_date on the Loan so it\u0026rsquo;s a snapshot — if the member upgrades their policy, existing loans keep their original due dates. For fine_per_day, I use the current rate at calculation time. Both approaches are defensible; in production I\u0026rsquo;d ask product what behavior they want.\u0026rdquo;\n8. Stage 7 — The Reservation Queue Reservations let members \u0026ldquo;wait in line\u0026rdquo; for a borrowed book. When a copy is returned, the next person in queue is notified.\nWhere do queues live? The queues are per-title (per Book). We have three options:\nOption A: Each Book holds its own reservation queue. Option B: The Library holds a dict mapping ISBN → queue. Option C: A separate ReservationManager holds and manages all queues.\nA is most cohesive (Book owns its reservations). B is more centralized. C is most flexible (can grow into its own service).\nFor our scope, B is a reasonable middle ground — the library is the one orchestrating returns and reservations, so it makes sense for the library to hold the queues. If reservation logic grew (priority queues, time-based reservations, expiry), we\u0026rsquo;d extract C.\nWe could also do A — but it\u0026rsquo;d require Book to have notification logic, which feels like scope creep.\nDecision: B — Library holds dict[isbn] → deque[Reservation].\nThe Reservation class itself Pretty simple:\nThe member The book they reserved When they reserved it (for FIFO ordering and possible expiry) from collections import deque from dataclasses import dataclass, field from datetime import datetime @dataclass class Reservation: \u0026#34;\u0026#34;\u0026#34; A request to be notified when a book becomes available. Reservations are queued FIFO per title. The member at the front of the queue gets the next available copy. We use a dataclass because this is mostly data — there\u0026#39;s no interesting behavior on a Reservation itself. Note: we do NOT hold a reference to a specific BookItem. The member is queued for the title; whichever copy frees up first is theirs. \u0026#34;\u0026#34;\u0026#34; member: Member book: Book reserved_at: datetime = field(default_factory=datetime.now) The deque (from collections) is the right data structure for the queue: O(1) append (enqueue at the back) and O(1) popleft (dequeue from the front). A list would have O(n) pop(0).\nWe\u0026rsquo;ll integrate the queues into the Library next.\n9. Stage 8 — Search Functionality Members need to find books. By title, author, category, etc.\nShould search be a method on Library, or its own class? If search is just def search_by_title(...), it\u0026rsquo;s fine as Library methods. But:\nSearch will likely grow (filters, full-text, ranking). Search has very different concerns from the rest of the library (storage, retrieval, indexing). Separating search lets us swap implementations (e.g., add a real full-text engine like Elasticsearch later). I\u0026rsquo;ll make BookSearch a separate class. It\u0026rsquo;s a small but principled SRP boundary.\nWhat does search return? A list of Book objects matching the query. Should it return matches sorted by relevance? Filtered to those with available copies? Limited to N?\nFor our scope, let\u0026rsquo;s keep it simple: return all matches. Callers can filter further. YAGNI.\nThe Code class BookSearch: \u0026#34;\u0026#34;\u0026#34; Encapsulates search functionality over the catalog. Why a separate class (rather than methods on Library)? - Search has different lifecycle/scaling concerns from the core library. We might later replace the in-memory linear scan with an indexed engine — that swap is much easier with a separate class. - Library\u0026#39;s responsibility is orchestration; mixing search methods bloats it. Constructed with a list of Books — the caller (Library) provides the catalog. We don\u0026#39;t hold a reference to Library because we don\u0026#39;t need anything from it beyond the books. \u0026#34;\u0026#34;\u0026#34; def __init__(self, books: list[Book]): # We keep a reference to the list (not a copy). This means # if the caller\u0026#39;s list grows, our search sees new books too. # Trade-off: shared mutable state. For our purposes that\u0026#39;s # the desired behavior. self._books = books def by_title(self, query: str) -\u0026gt; list[Book]: \u0026#34;\u0026#34;\u0026#34; Find books whose title CONTAINS the query (case-insensitive). Linear scan; for thousands of books this is fine, for millions we\u0026#39;d want an index. \u0026#34;\u0026#34;\u0026#34; q = query.lower() return [b for b in self._books if q in b.title.lower()] def by_author(self, query: str) -\u0026gt; list[Book]: \u0026#34;\u0026#34;\u0026#34;Match on author substring (case-insensitive).\u0026#34;\u0026#34;\u0026#34; q = query.lower() return [b for b in self._books if q in b.author.lower()] def by_category(self, category: str) -\u0026gt; list[Book]: \u0026#34;\u0026#34;\u0026#34; Exact category match (case-insensitive). We use exact match for category because categories are typically a controlled vocabulary, not free text. \u0026#34;\u0026#34;\u0026#34; return [ b for b in self._books if b.category.lower() == category.lower() ] def by_isbn(self, isbn: str) -\u0026gt; Optional[Book]: \u0026#34;\u0026#34;\u0026#34; ISBN is unique — return at most one book. Returns None if no match. \u0026#34;\u0026#34;\u0026#34; return next((b for b in self._books if b.isbn == isbn), None) The decision to make by_title use contains (substring) but by_category use exact match reflects domain reality: titles are free text where you might remember a fragment (\u0026ldquo;pragmatic\u0026rdquo;); categories are an enumeration (you wouldn\u0026rsquo;t search \u0026ldquo;tec\u0026rdquo; hoping to find \u0026ldquo;Tech\u0026rdquo;).\nThis kind of small contextual decision — knowing the domain enough to make the right choice — is what shows real design thinking.\n10. Stage 9 — The Library (Top-Level Coordinator) Now we tie it together. The Library is the Facade — it exposes the operations users care about (borrow, return, reserve, search) and hides the internal machinery.\nWhat does Library hold? Catalog: dict[isbn → Book] Members: dict[member_id → Member] Active loans: dict[loan_id → Loan] (so we can look up loans by ID at return time) Reservation queues: dict[isbn → deque[Reservation]] The Core Operations Borrow:\nValidate member exists, book exists. Check member can borrow more. Find an available copy. If no copy available, return None (caller should consider reserving). Create the loan, update copy status, add to member\u0026rsquo;s active loans. Return:\nLook up the loan. Stamp return date, compute fine. Update copy status to AVAILABLE. Remove from member\u0026rsquo;s active loans. If there\u0026rsquo;s a reservation queue for this book, notify the next person. Return the fine amount. Reserve:\nValidate inputs. If a copy is currently available, no need to reserve — caller should just borrow. Add to the reservation queue. The Code class Library: \u0026#34;\u0026#34;\u0026#34; The top-level facade for the library system. Responsibilities: - Manage the catalog (books) and membership (members). - Orchestrate borrowing, returning, reserving. - Maintain the reservation queues. - Provide search. NOT responsible for: - Computing fines (that\u0026#39;s Loan\u0026#39;s job). - Knowing about borrowing limits (that\u0026#39;s the member\u0026#39;s policy). - Persistence (in scope: in-memory; production: would inject a repository). Library is intentionally a coordination layer — most actual work is delegated to the entities it owns. \u0026#34;\u0026#34;\u0026#34; def __init__(self): # Catalog: ISBN → Book. Dict for O(1) lookup. self.books: dict[str, Book] = {} # Membership directory: member_id → Member. self.members: dict[str, Member] = {} # Active loans: loan_id → Loan. Used to look up a loan at return. self.active_loans: dict[str, Loan] = {} # Reservation queues: ISBN → deque of Reservations. # We use deque for O(1) popleft (dequeue from the front). self.reservations: dict[str, deque[Reservation]] = {} # === Catalog management === def add_book(self, book: Book) -\u0026gt; None: \u0026#34;\u0026#34;\u0026#34; Add a book to the catalog. Idempotent — if the book already exists, just keep the existing entry. \u0026#34;\u0026#34;\u0026#34; if book.isbn in self.books: # Same ISBN means same title; ignore duplicates rather than # overwriting. Real systems might merge copies. return self.books[book.isbn] = book def add_member(self, member: Member) -\u0026gt; None: \u0026#34;\u0026#34;\u0026#34;Register a new member.\u0026#34;\u0026#34;\u0026#34; if member.member_id in self.members: raise ValueError(f\u0026#34;Member {member.member_id} already exists\u0026#34;) self.members[member.member_id] = member @property def search(self) -\u0026gt; BookSearch: \u0026#34;\u0026#34;\u0026#34; Convenience accessor for search. We construct a new BookSearch each access. This is cheap (BookSearch holds a reference, not a copy) but we could cache if it became a hotspot. \u0026#34;\u0026#34;\u0026#34; return BookSearch(list(self.books.values())) # === Borrow / Return / Reserve === def borrow(self, member_id: str, isbn: str) -\u0026gt; Optional[Loan]: \u0026#34;\u0026#34;\u0026#34; Attempt to issue a loan: this member borrows a copy of this title. Returns the Loan if successful, None if no copy is available or the member is at their limit. Order of checks matters: 1. First validate that the member and book exist (KeyError if not). 2. Then check the member\u0026#39;s borrowing limit (cheap check). 3. Then look for an available copy (potentially expensive). We do the cheap checks first to fail fast. \u0026#34;\u0026#34;\u0026#34; member = self.members[member_id] # raises KeyError if unknown book = self.books[isbn] if not member.can_borrow_more(): print( f\u0026#34;{member.name} cannot borrow more — \u0026#34; f\u0026#34;already at limit ({member.policy.max_books}).\u0026#34; ) return None copy = book.available_copy() if copy is None: print( f\u0026#34;No copies of {book.title!r} are available. \u0026#34; f\u0026#34;Consider reserving.\u0026#34; ) return None # All checks passed. Issue the loan. copy.status = BookStatus.BORROWED loan = Loan(member, copy) member.active_loans.append(loan) self.active_loans[loan.id] = loan print( f\u0026#34;Issued: {book.title!r} (copy {copy.copy_id}) → \u0026#34; f\u0026#34;{member.name}. Due: {loan.due_date.date()}\u0026#34; ) return loan def return_book(self, loan_id: str) -\u0026gt; float: \u0026#34;\u0026#34;\u0026#34; Process the return of a previously-issued loan. Steps: 1. Look up the loan (KeyError if unknown). 2. Stamp return date. 3. Compute the fine (uses loan\u0026#39;s own logic). 4. Update the copy back to AVAILABLE. 5. Remove the loan from active loans (it\u0026#39;s done). 6. Check the reservation queue for this title; if anyone\u0026#39;s waiting, notify the front of the queue. Returns the fine amount. We don\u0026#39;t actually charge the fine here — we just compute and return it. Payment processing is out of scope. A real system would track fine payment as a separate concern. \u0026#34;\u0026#34;\u0026#34; loan = self.active_loans[loan_id] # Stamp the loan as completed. loan.return_date = datetime.now() fine = loan.calculate_fine() # Free up the copy. loan.book_item.status = BookStatus.AVAILABLE # Remove the loan from active tracking. del self.active_loans[loan_id] loan.member.active_loans.remove(loan) title = loan.book_item.book.title print(f\u0026#34;Returned: {title!r}. Fine: ₹{fine:.2f}\u0026#34;) # Reservation queue handling. self._fulfill_next_reservation(loan.book_item.book) return fine def reserve(self, member_id: str, isbn: str) -\u0026gt; Optional[Reservation]: \u0026#34;\u0026#34;\u0026#34; Add a reservation for a member on a book. If a copy of this book is currently available, we don\u0026#39;t reserve — we tell the caller they should just borrow. This avoids useless reservations. Returns the Reservation if queued, None if a copy is available (suggesting borrow instead). \u0026#34;\u0026#34;\u0026#34; member = self.members[member_id] book = self.books[isbn] if book.available_copy() is not None: print( f\u0026#34;{book.title!r} has copies available. \u0026#34; f\u0026#34;{member.name} should just borrow it.\u0026#34; ) return None reservation = Reservation(member, book) # setdefault ensures the deque exists, even on first reservation # for this title. self.reservations.setdefault(isbn, deque()).append(reservation) position = len(self.reservations[isbn]) print( f\u0026#34;{member.name} reserved {book.title!r} \u0026#34; f\u0026#34;(position {position} in queue)\u0026#34; ) return reservation # === Internal helpers === def _fulfill_next_reservation(self, book: Book) -\u0026gt; None: \u0026#34;\u0026#34;\u0026#34; Called after a copy of `book` becomes available. If anyone has a reservation, notify the front of the queue. In our scope, \u0026#34;notify\u0026#34; is just printing. In a real system this would send an email/SMS, possibly start a hold timer (member has 24 hours to come pick up the book before the next person is notified). \u0026#34;\u0026#34;\u0026#34; queue = self.reservations.get(book.isbn) if not queue: return # No one waiting next_reservation = queue.popleft() print( f\u0026#34; → Notifying {next_reservation.member.name}: \u0026#34; f\u0026#34;{book.title!r} is now available for you.\u0026#34; ) A few details worth pointing out:\n1. borrow() does the cheap checks first (member exists, can_borrow_more) before the more expensive available_copy() check. Fast failure is better engineering.\n2. The reservation queue check happens inside return_book. This is a critical interaction point — the moment a book becomes available, we need to notify the next reservation. Easy to forget, but it\u0026rsquo;s the whole point of having reservations.\n3. reserve() checks for an available copy first. If a copy is available, queueing the reservation would be silly — the member should just borrow now. The method is honest with the caller about this.\n4. We use a deque for reservation queues. O(1) operations from both ends. A list would have O(n) pop(0).\n11. Stage 10 — Validating with a Mental Walkthrough Let\u0026rsquo;s run a realistic scenario.\nScenario: A library with 2 copies of \u0026ldquo;The Pragmatic Programmer.\u0026rdquo; Three members: Amit (student), Priya (faculty), Rohan (guest). They borrow, queue up, return.\nlibrary = Library() # Add a book with 2 copies. book = Book(\u0026#34;978-001\u0026#34;, \u0026#34;The Pragmatic Programmer\u0026#34;, \u0026#34;Hunt \u0026amp; Thomas\u0026#34;, \u0026#34;Tech\u0026#34;) book.add_copy(BookItem(book, \u0026#34;PP-001\u0026#34;)) book.add_copy(BookItem(book, \u0026#34;PP-002\u0026#34;)) library.add_book(book) # Add members. amit = Member(\u0026#34;M1\u0026#34;, \u0026#34;Amit\u0026#34;, \u0026#34;amit@x.com\u0026#34;, StudentPolicy()) priya = Member(\u0026#34;M2\u0026#34;, \u0026#34;Priya\u0026#34;, \u0026#34;priya@x.com\u0026#34;, FacultyPolicy()) rohan = Member(\u0026#34;M3\u0026#34;, \u0026#34;Rohan\u0026#34;, \u0026#34;rohan@x.com\u0026#34;, GuestPolicy()) for m in (amit, priya, rohan): library.add_member(m) State: 2 copies AVAILABLE. No loans. No reservations.\nloan_amit = library.borrow(\u0026#34;M1\u0026#34;, \u0026#34;978-001\u0026#34;) Trace:\nborrow(\u0026quot;M1\u0026quot;, \u0026quot;978-001\u0026quot;). Member found (Amit). Book found. amit.can_borrow_more() → 0 active loans, max 3. True. book.available_copy() → returns PP-001 (first AVAILABLE). Mark PP-001 BORROWED. Create Loan. Append to amit.active_loans. Add to library.active_loans. Print \u0026ldquo;Issued\u0026hellip;\u0026rdquo; State: 1 copy AVAILABLE (PP-002), 1 BORROWED (PP-001). Amit has 1 active loan.\nloan_priya = library.borrow(\u0026#34;M2\u0026#34;, \u0026#34;978-001\u0026#34;) Same flow. PP-002 selected. Now BORROWED. State: 0 copies available. Amit has 1, Priya has 1.\nloan_rohan = library.borrow(\u0026#34;M3\u0026#34;, \u0026#34;978-001\u0026#34;) Member, book found. Rohan.can_borrow_more → 0 \u0026lt; 1, True. book.available_copy() → None (both BORROWED). Returns None. Print \u0026ldquo;No copies available.\u0026rdquo; State: unchanged.\nres_rohan = library.reserve(\u0026#34;M3\u0026#34;, \u0026#34;978-001\u0026#34;) Member, book found. book.available_copy() → None. Good, queue the reservation. Reservation added to queue. Print \u0026ldquo;Rohan reserved\u0026hellip; (position 1)\u0026rdquo;. State: queue has [Rohan\u0026rsquo;s reservation].\nfine = library.return_book(loan_amit.id) Look up loan. Stamp return_date. Calculate fine. Loan was just created, so end_date - issue_date is tiny. 0.0 fine. Mark PP-001 AVAILABLE. Remove from active_loans. Check reservation queue for \u0026ldquo;978-001\u0026rdquo;: queue not empty! Pop Rohan from queue. Print \u0026ldquo;Notifying Rohan\u0026hellip;\u0026rdquo;. State: PP-001 AVAILABLE, queue empty. fine = 0.0.\nIn a richer system, the notification would actually trigger Rohan to come borrow it — and we\u0026rsquo;d want to \u0026ldquo;hold\u0026rdquo; the copy for him for some period. We don\u0026rsquo;t model that yet. Let\u0026rsquo;s note it as an extension.\n# Rohan now manually borrows. loan_rohan = library.borrow(\u0026#34;M3\u0026#34;, \u0026#34;978-001\u0026#34;) Member, book found. rohan.can_borrow_more() → 0 \u0026lt; 1, True. book.available_copy() → PP-001 AVAILABLE. Issue loan. State: PP-001 BORROWED by Rohan, PP-002 BORROWED by Priya, queue empty.\n✓ All transitions work. Reservation is correctly notified on return.\nEdge Cases to Consider Edge case 1: Member at borrowing limit tries to borrow.\nIf Rohan (max 1) already has a loan and tries to borrow another:\ncan_borrow_more() → False. Print \u0026ldquo;cannot borrow more\u0026rdquo;. Return None. ✓ Correct. Edge case 2: Returning a non-existent loan.\nlibrary.return_book(\u0026quot;nonexistent_id\u0026quot;) → KeyError from self.active_loans[...]. Loud failure. In production we\u0026rsquo;d catch and return a clean error. For our scope, KeyError is acceptable.\nEdge case 3: Reserving a book you can already borrow.\nbook.available_copy() is not None. Returns None with a message. ✓ Correct. Edge case 4: Multiple reservations for the same book.\nqueue has [A, B, C]. Copy returned. Notify A. Pop A. Next return: notify B. Pop B. ✓ FIFO works. Edge case 5: A reserved copy is \u0026ldquo;borrowed\u0026rdquo; by someone else before the reserved member gets to it.\nThis is a real bug. After we notify Rohan, the copy is AVAILABLE. If Amit walks in and borrows it, he gets it instead of Rohan.\nIn a real library, the notified person typically has a hold period (e.g., 24 hours) during which the copy is reserved for them. We don\u0026rsquo;t model this. Mention as an extension.\n\u0026ldquo;In a richer design, after notifying the front-of-queue member, the copy would be \u0026lsquo;on hold\u0026rsquo; for them for some period — neither borrowable by others nor returnable to general circulation. We\u0026rsquo;d model this with a HOLD status on BookItem and a hold expiration time. For interview scope I\u0026rsquo;ve kept it simpler.\u0026rdquo;\nThat\u0026rsquo;s a great self-critique to articulate.\n12. Stage 11 — Anticipating Follow-Up Questions Q: \u0026ldquo;How would you handle multiple branches?\u0026rdquo; Introduce Branch. Each branch has its own copies of books, its own members, its own queues. Books (titles) are shared across the system.\nclass Branch: def __init__(self, name): self.name = name self.copies: list[BookItem] = [] self.reservations: dict[str, deque[Reservation]] = {} class LibrarySystem: def __init__(self): self.branches: dict[str, Branch] = {} self.books: dict[str, Book] = {} # shared catalog self.members: dict[str, Member] = {} # shared membership Now operations specify which branch. Members can borrow from any branch. Inter-branch transfers become a thing.\nThis is significant complexity — easily another 200 lines. Don\u0026rsquo;t write it unless asked.\nQ: \u0026ldquo;How would you support digital books?\u0026rdquo; Abstract the borrowable concept:\nclass BorrowableItem(ABC): @abstractmethod def is_available(self): ... @abstractmethod def acquire(self): ... @abstractmethod def release(self): ... class BookItem(BorrowableItem): ... class EbookLicense(BorrowableItem): \u0026#34;\u0026#34;\u0026#34;An ebook license — N concurrent users can have it \u0026#39;checked out\u0026#39;.\u0026#34;\u0026#34;\u0026#34; Loan would now reference a BorrowableItem rather than specifically a BookItem. The logic generalizes.\nQ: \u0026ldquo;How would you implement notifications for real?\u0026rdquo; Use Observer. Library publishes events; consumers (email service, SMS service) subscribe.\nclass LibraryEvent(ABC): ... class ReservationFulfilled(LibraryEvent): member: Member; book: Book class LibraryEventListener(ABC): @abstractmethod def on_event(self, event: LibraryEvent): ... class EmailNotifier(LibraryEventListener): def on_event(self, event): if isinstance(event, ReservationFulfilled): self.send_email(event.member.email, ...) Library would have an event_listeners: list[LibraryEventListener] and call _notify(event) at appropriate points.\nQ: \u0026ldquo;How would you persist all this?\u0026rdquo; Repository pattern. Define BookRepository, MemberRepository, LoanRepository as interfaces. Provide in-memory and SQL implementations. Library depends on the interfaces.\nclass BookRepository(ABC): @abstractmethod def save(self, book): ... @abstractmethod def find_by_isbn(self, isbn): ... class Library: def __init__(self, books: BookRepository, members: MemberRepository, ...): self.books = books self.members = members This is straightforward DIP. Library logic doesn\u0026rsquo;t change; the storage backend does.\nQ: \u0026ldquo;What about concurrent borrowers?\u0026rdquo; Two members trying to borrow the last copy at the same time. With our current code:\nBoth call borrow(). Both find the same available copy. Both mark it BORROWED, both create loans. Bad. Fix: lock the book during the borrow operation. Or use compare-and-swap on the copy\u0026rsquo;s status (atomically transition AVAILABLE → BORROWED).\ndef borrow(self, member_id, isbn): with self._lock: # crude but works ... For higher concurrency, use per-book locks (multiple books can be borrowed in parallel; only racing for the same book matters).\nQ: \u0026ldquo;How would you test this?\u0026rdquo; Each component is testable:\nMembershipPolicy: trivial — assert returned values. Loan.calculate_fine: construct a Loan, manipulate dates, assert fines. Book.available_copy: add copies in various states, assert which is returned. Library.borrow: integration test — set up state, borrow, assert side effects (copy status, loan in dict, member\u0026rsquo;s active_loans). Tests run in milliseconds since everything is in-memory.\n13. Final Reflection — What This Problem Teaches Lesson 1: Distinguish Title from Copy The Book vs BookItem split is the most important decision in this design. It models reality: titles and physical copies are conceptually different things. Conflating them produces awkward APIs (\u0026ldquo;borrow this Book and decrement its quantity\u0026rdquo;) and loses per-copy state.\nThis pattern generalizes: anytime you have a \u0026ldquo;type\u0026rdquo; and many \u0026ldquo;instances of that type\u0026rdquo; with their own state, model both as separate entities.\nLesson 2: Policy Variation Belongs in Strategy, Not Subclasses We chose composition (Member has-a MembershipPolicy) over inheritance (Student extends Member). The deciding factors:\nType changes are normal. A guest enrolls → becomes a student. With composition, this is member.policy = StudentPolicy(). With inheritance, you\u0026rsquo;d have to construct a new Student and migrate state. Multiple variation dimensions. If we add another variation (e.g., \u0026ldquo;premium tier\u0026rdquo;), inheritance combinatorially explodes. Composition keeps each dimension orthogonal. The general principle: prefer composition when the \u0026ldquo;type\u0026rdquo; is mutable or when multiple varying dimensions exist.\nLesson 3: Tell, Don\u0026rsquo;t Ask member.can_borrow_more() is on the Member, not the Library. The library asks the member; it doesn\u0026rsquo;t reach into the member\u0026rsquo;s data and compute externally.\nThis Tell-Don\u0026rsquo;t-Ask discipline keeps logic close to data, which is what good encapsulation is about.\nLesson 4: Where Does Logic Go? A recurring question in this design: should X live on entity A or entity B?\nFine calculation: on Loan (it has the data). Borrowing limit check: on Member (it owns its policy and state). Reservation queue: on Library (it orchestrates returns and reservations). Search: in its own class (separation of concerns). The principle: logic belongs where the data lives, with separation when concerns truly differ.\nLesson 5: Walkthroughs Reveal Hidden Bugs We found the \u0026ldquo;notified-but-not-held copy\u0026rdquo; issue only by walking through the reservation flow. Without that walkthrough, we\u0026rsquo;d have missed it. Always trace through your design.\nEqually important: when we found the bug, we didn\u0026rsquo;t pretend it didn\u0026rsquo;t exist. We named it as a real-world concern, sketched the fix (HOLD status), and noted it as an extension. Self-critique scores points.\nLesson 6: YAGNI — Defer Until You Need It We didn\u0026rsquo;t model:\nMulti-branch (wasn\u0026rsquo;t required) Digital books (wasn\u0026rsquo;t required) Notifications (mentioned only) Fine payment (out of scope) Librarian as an entity (no behavior) Each of these is a real concern that could be added — but adding them speculatively would have made this design twice as long without making it better. The discipline of \u0026ldquo;we\u0026rsquo;re not solving that yet\u0026rdquo; is essential.\nOnwards to the next problem: Vending Machine — where the State pattern shines once more, and we\u0026rsquo;ll get into the weeds of inventory management and change-making logic.\n","permalink":"https://blogs.codeverra.com/blog/low_level_design/interview_questions/06c_lld_library_deep_dive/","summary":"\u003ch1 id=\"lld-deep-dive-3--designing-a-library-management-system\"\u003eLLD Deep Dive #3 — Designing a Library Management System\u003c/h1\u003e\n\u003cblockquote\u003e\n\u003cp\u003e\u003cstrong\u003eWhat this document is:\u003c/strong\u003e A genuine, step-by-step walkthrough of how to think through a Library Management LLD problem. We\u0026rsquo;ll reason from first principles, weigh alternatives, and document each design decision — building toward a clean, extensible system that handles books, members, loans, fines, and reservations.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003chr\u003e\n\u003ch2 id=\"table-of-contents\"\u003eTable of Contents\u003c/h2\u003e\n\u003col\u003e\n\u003cli\u003e\u003ca href=\"#1-why-this-problem-matters\"\u003eWhy This Problem Matters\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#2-stage-1--receiving-the-problem\"\u003eStage 1 — Receiving the Problem\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#3-stage-2--clarifying-requirements\"\u003eStage 2 — Clarifying Requirements\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#4-stage-3--identifying-entities\"\u003eStage 3 — Identifying Entities (And the Big Modeling Decision)\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#5-stage-4--modeling-books-vs-book-copies\"\u003eStage 4 — Modeling Books vs Book Copies\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#6-stage-5--modeling-members-and-their-policies\"\u003eStage 5 — Modeling Members and Their Policies\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#7-stage-6--the-loan-lifecycle\"\u003eStage 6 — The Loan Lifecycle\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#8-stage-7--the-reservation-queue\"\u003eStage 7 — The Reservation Queue\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#9-stage-8--search-functionality\"\u003eStage 8 — Search Functionality\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#10-stage-9--the-library-top-level-coordinator\"\u003eStage 9 — The Library (Top-Level Coordinator)\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#11-stage-10--validating-with-a-mental-walkthrough\"\u003eStage 10 — Validating with a Mental Walkthrough\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#12-stage-11--anticipating-follow-up-questions\"\u003eStage 11 — Anticipating Follow-Up Questions\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#13-final-reflection\"\u003eFinal Reflection — What This Problem Teaches\u003c/a\u003e\u003c/li\u003e\n\u003c/ol\u003e\n\u003chr\u003e\n\u003ch2 id=\"1-why-this-problem-matters\"\u003e1. Why This Problem Matters\u003c/h2\u003e\n\u003cp\u003eThe library system is the canonical \u0026ldquo;policy-driven domain modeling\u0026rdquo; interview problem. What it tests:\u003c/p\u003e","tags":null,"title":""},{"content":"LLD Deep Dive #4: Designing a Tic Tac Toe Game What this document covers: A rigorous, step-by-step examination of how to approach a Tic Tac Toe Low-Level Design (LLD) problem in a technical interview. The document walks through the reasoning process at each design juncture, evaluates competing alternatives, and provides justification for every decision made. The finished design is a consequence of the reasoning, not its starting point.\nHow to read this document: Read it slowly and deliberately. The problem appears trivial at the surface, since Tic Tac Toe requires only a three by three grid, two players, and a rule for detecting three marks in a row. That surface simplicity is precisely why the problem is used in interviews: it invites the unwary candidate to produce a short procedural script, when the actual expectation is a properly modelled, extensible, and testable object-oriented design.\nTable of Contents Why This Problem Is Deceptively Instructive Stage 1: Receiving the Problem Statement Stage 2: Clarifying the Requirements Stage 3: Identifying the Domain Entities Stage 4: Modelling the Board and Cells Stage 5: Modelling Symbols and Players Stage 6: Making a Move, Validation and Placement Stage 7: Win Detection as an Algorithmic Concern Stage 8: Modelling the Game State Stage 9: The Game Class as the Orchestrator Stage 10: Introducing an AI Player via the Strategy Pattern Stage 11: Observers for Game Events Stage 12: Validating the Design Through Mental Simulation Stage 13: Anticipating Follow-Up Questions Practice Questions with Detailed Solutions Summary Tables and Key Takeaways Final Reflection on the Design Process 1. Why This Problem Is Deceptively Instructive Before examining the mechanics of the design, it is worth establishing why an interviewer selects this specific problem. Low-Level Design (LLD) is the discipline of translating a set of functional requirements into a well-structured collection of classes, interfaces, and relationships that together implement the required behaviour. LLD sits between requirements analysis on one side and code implementation on the other, and its principal concerns are separation of responsibilities, extensibility, testability, and the correct use of design patterns.\nTic Tac Toe is chosen for interviews because it exhibits a specific combination of properties that make it useful as an assessment instrument. The problem is small enough to be completed within a typical interview window of forty-five to sixty minutes, yet it contains at least four independent design concerns: representation of the board, modelling of players and their behaviours, detection of terminal game conditions, and orchestration of the game loop. Each of these concerns admits multiple reasonable design choices, and the interviewer is interested in observing how the candidate reasons about those choices rather than simply which one is picked.\nThe following capabilities can be demonstrated through this problem:\nClass identification at appropriate granularity. Producing too few classes results in procedural code disguised as object-oriented code. Producing too many classes results in excessive ceremony and cognitive load. The candidate is expected to draw the line between what deserves to be a first-class abstraction and what is better represented as a value or attribute. State-machine modelling. A game progresses through a small number of well-defined states such as in-progress, won, and drawn. This provides an opportunity to demonstrate awareness of state modelling techniques and, importantly, the judgement to know when the full State pattern is warranted versus when a simpler enumeration and guard clauses will suffice. Application of the Strategy pattern. Two independent behaviours in the system can vary: how a player selects a move (human input versus algorithmic computation) and how the win condition is evaluated (three in a row on a three by three board versus more general variants). Both are canonical use cases for the Strategy pattern, which is defined as an object-oriented pattern that encapsulates interchangeable algorithms behind a common interface. Extensibility thinking. The interviewer will typically ask follow-up questions that require the design to accommodate a four by four board, a K-in-a-row rule, more than two players, or networked play. A rigid design fails these questions; a properly abstracted design accommodates them with minimal change. Complexity analysis of the win-detection algorithm. Naive implementations rescan the entire board after every move, incurring redundant work. More efficient implementations exploit the observation that only a bounded number of lines can be affected by the most recent move. The candidate is expected to articulate this trade-off and justify the choice made. A candidate who produces a short procedural script and stops has signalled that they do not distinguish coding from designing. A candidate who produces a properly abstracted design with clearly justified boundaries, pluggable strategies, and validated invariants has signalled senior-level thinking on a problem that junior candidates dismiss as an easy warm-up.\nDefinition of key terms used above:\nLow-Level Design (LLD): The activity of specifying the internal class structure, interfaces, method signatures, and object interactions that will realise a given set of requirements, typically expressed through class diagrams, sequence diagrams, and reference implementations. Design pattern: A named, general, reusable solution to a commonly occurring problem in software design, originally catalogued in the 1994 work commonly referred to as the Gang of Four book. Strategy pattern: A behavioural design pattern that defines a family of algorithms, encapsulates each one behind a common interface, and makes them interchangeable at runtime through composition. State pattern: A behavioural design pattern that allows an object to alter its behaviour when its internal state changes, by delegating state-specific behaviour to a separate class per state. Observer pattern: A behavioural design pattern that defines a one-to-many dependency between objects, so that when one object changes state, all its registered dependents are notified automatically. 2. Stage 1: Receiving the Problem Statement The interviewer opens with a deliberately terse instruction:\n\u0026ldquo;Design a Tic Tac Toe game.\u0026rdquo;\nNo specification is provided for board dimensions, number of players, presence of artificial intelligence, user interface expectations, or extensibility requirements. This vagueness is intentional and forms the first evaluation criterion: the candidate is being observed to determine whether they will proceed to code immediately or first establish scope.\nProceeding to code immediately, however familiar the problem may feel from childhood, is an error. The instruction admits several materially different interpretations:\nBoard dimensions. The classical three by three board is one possibility, but variants exist that use four by four, five by five, or even fifteen by fifteen grids as in the game of Gomoku. Number of players. Two players is the standard case, but nothing in the instruction rules out three or four player variants. Player types. Both players might be human, or one might be an artificial intelligence, or the game might need to support arbitrary combinations of human and machine participants. Presentation surface. The game might run on a command-line terminal, in a web browser, or as a graphical desktop application, or the interviewer may consider the user interface entirely out of scope. Win condition. Traditional Tic Tac Toe requires three collinear identical symbols. A generalisation to N-in-a-row on an N by N board is common, and further generalisations to K-in-a-row on an N by N board (where K may differ from N) are also possible. Auxiliary features. The design may or may not need to support undo of moves, saving and loading of games, replay of historical games, and score tracking across multiple sessions. Deployment context. The game might be a single-process console program, a networked multiplayer service, or an embedded component of a larger application. Making incorrect assumptions on any of these axes results in wasted design effort that cannot be recovered without discarding work. The disciplined response is therefore to pause, indicate that clarifying questions will follow, and then proceed methodically through those questions. A single introductory sentence establishes the correct posture: \u0026ldquo;Before I begin sketching classes, I would like to ask a series of clarifying questions to establish the scope and expected behaviours.\u0026rdquo; This sentence signals professional maturity and separates the candidate from those who begin typing immediately.\n3. Stage 2: Clarifying the Requirements Requirements clarification is a structured activity, not a random enumeration of questions. The recommended structure organises questions into three categories, each of which serves a distinct purpose in the design process.\nFunctional requirements describe what the system must do, expressed in terms of observable behaviours and outcomes. They answer questions of the form \u0026ldquo;what actions does the system perform?\u0026rdquo; and \u0026ldquo;what results does it produce?\u0026rdquo;\nNon-functional requirements describe qualities that the system must possess but that are not themselves behaviours. Common non-functional concerns include performance characteristics, concurrency behaviour, persistence requirements, security constraints, and scale expectations.\nOut-of-scope declarations enumerate concerns that are intentionally excluded from the design. Making these explicit prevents the candidate from over-engineering solutions to problems the interviewer did not ask them to solve, and it makes the assumed boundary of the design visible.\nThe following subsections walk through the actual questions the candidate should ask, along with the rationale for asking each and the design implications of the likely answers.\nQuestion 1: Should the board be fixed at three by three, or configurable to arbitrary N by N? The importance of this question stems from its effect on the win-detection algorithm. A fixed three by three board permits enumerating the eight winning lines as constants, whereas a configurable N by N board demands a general-purpose algorithm that generates winning lines dynamically. Designing initially for the fixed case and later discovering the requirement was general leads to substantial rework.\nSuppose the interviewer replies: \u0026ldquo;The board should support arbitrary N by N sizes, with three as the default.\u0026rdquo; This single sentence has significant design implications. The board dimension must become a parameter rather than a hard-coded constant, and the win-detection component must operate uniformly across all values of N.\nQuestion 2: What is the win condition for board sizes greater than three? On a three by three board, the condition of three identical marks in a row is unambiguous and coincides with the natural notion of N in a row where N equals the board size. On larger boards, however, two natural generalisations exist. The first requires N marks in a row, where N equals the board dimension. The second allows the winning line length K to differ from the board dimension N, enabling variants such as Gomoku, which requires five in a row on a fifteen by fifteen board.\nSuppose the interviewer specifies: \u0026ldquo;For an N by N board, the win condition is N in a row. Keep the design simple.\u0026rdquo; This answer permits a single algorithm that generates all lines of length equal to the board dimension. The more general K-in-a-row rule will be considered later as an extension point.\nQuestion 3: How many players participate, and what types of players are supported? The number of players affects the turn-rotation logic and the assignment of symbols. Supporting only two players permits certain simplifications, such as representing whose turn it is by a Boolean flag, whereas supporting a variable number of players requires index-based rotation. The type of player also matters: a human player receives moves through input from a console or user interface, while an artificial intelligence player computes moves through an algorithm. These two behaviours have almost nothing in common at the implementation level, though they share the same conceptual role in the game.\nSuppose the interviewer answers: \u0026ldquo;Support two players initially, both human. However, the design should accommodate the future addition of an artificial intelligence opponent without requiring modification of the game logic.\u0026rdquo; This is a direct invitation to apply the Strategy pattern to the player abstraction, defining a common interface that both human and machine players will implement.\nQuestion 4: Is the user interface in scope, and if so, what form should it take? User interface concerns can consume disproportionate amounts of design effort if not scoped explicitly. Interviewers typically expect the candidate to focus on the game logic and its abstractions, treating any rendering concerns as adjacent rather than central.\nSuppose the interviewer states: \u0026ldquo;Console rendering is acceptable for demonstration purposes, but the interview evaluation will concentrate on the game logic and its abstractions rather than on presentation.\u0026rdquo; This response permits a minimal string-based rendering method on the board and defers any more sophisticated presentation to a future concern.\nQuestion 5: Are features such as undo, save, load, or replay required? These features individually appear modest but each of them changes the design substantively. Undo requires the ability to reverse a move, which in turn requires either a history of moves or a snapshot mechanism. Save and load require serialization of game state. Replay requires either a move log or the ability to reconstruct intermediate states.\nSuppose the interviewer answers: \u0026ldquo;None of these are required for the initial version. Mention them briefly as potential extensions.\u0026rdquo; The design should therefore not implement these features but should be structured in a way that permits their later addition. Maintaining a move history at negligible cost is a defensible pre-emptive investment because it enables undo, replay, and audit features without redesign.\nQuestion 6: Is the design a single game session, or is it expected to manage multiple games such as a tournament? The distinction between a single game and a series of games is architecturally significant because tournament management introduces new concerns such as bracketing, score aggregation, and match scheduling. Assuming the wrong scope leads to either an incomplete design or an over-engineered one.\nSuppose the interviewer states: \u0026ldquo;One game per session is sufficient. Tournament features are out of scope.\u0026rdquo; This confirms a simpler design without a Tournament class or associated scaffolding.\nQuestion 7: What are the concurrency and persistence requirements? Two humans playing on a single console produce a strictly sequential interaction that requires no concurrency management. Networked multiplayer games, in contrast, require careful attention to concurrent state modification, message ordering, and recovery from disconnected clients. Persistence requirements determine whether game state must survive process termination.\nSuppose the interviewer confirms: \u0026ldquo;The design is a single-process, in-memory game with no networking or persistence requirements.\u0026rdquo; This clearance permits a straightforward synchronous design without introducing locks, transactions, or external storage.\nConsolidating the Scope Once the clarification round is complete, the candidate should restate the confirmed scope back to the interviewer in a single consolidated summary. Restating serves two purposes: it forces the interviewer to confirm or correct the candidate\u0026rsquo;s understanding before design work begins, and it visibly documents the assumptions on which the subsequent design will rest.\nA suitable restatement is: \u0026ldquo;The design will support a two-player game on an N by N board with a default size of three, using the N-in-a-row win condition across rows, columns, and diagonals. Both players are human initially, but the player abstraction should allow an artificial intelligence player to be added later without modification of the game logic. Rendering will be minimal and console-based. There is one game per session, no undo or persistence, no networking, and no concurrency requirements. Please confirm if any of these assumptions is incorrect or incomplete.\u0026rdquo;\nThis confirmation gate is the natural transition point from analysis into design.\n4. Stage 3: Identifying the Domain Entities With the requirements fixed, the next activity is to identify the classes that will populate the design. The recommended technique is noun extraction: enumerate the significant nouns in the requirements specification and evaluate each one as a candidate class. Not every noun deserves its own class, however, and the discipline lies in the filtering step rather than the enumeration step.\nThree questions guide the filter. First, does the noun have distinct identity, meaning that different instances of it need to be tracked separately? Second, does the noun possess its own state or behaviour, distinguishing it from a mere data value? Third, is the noun genuinely part of the problem domain, or is it a piece of technical vocabulary or a linguistic filler that should not be reified into a class?\nThe nouns present in the clarified requirements include the following: game, board, cell, player, human player, artificial intelligence player, symbol, mark, move, position, turn, win condition, score, and console. Applying the three-question filter yields the following assessment:\nNoun Distinct Identity Own State or Behaviour Verdict Game Yes, one game per session Yes, orchestrates play Class Board Yes, one per game Yes, stores cells and validates moves Class Cell Weak, referenced by coordinates None beyond holding a symbol Not a class (represented as a symbol value inside the board grid) Player Yes, one per participant Yes, holds identity and generates moves Class hierarchy Symbol / Mark Small fixed set of values None beyond identity of the value Enumeration Move Yes, each move is a distinct event Small, primarily data Immutable data class Position Weak, referenced by row and column None beyond value semantics Immutable data class Turn Not an entity, merely a piece of state None Attribute of Game Win condition Not an entity, but an algorithm Yes, defines behaviour Strategy interface Score Not tracked in single-game scope None Rejected for the current scope Console Rendering surface Yes, produces output Small helper class if needed Two of the entries in this table involve judgement calls that warrant explicit justification, since the wrong decision in either case would degrade the design.\nJustification for Not Modelling Cell as a Class A cell in this problem holds exactly one value from the fixed set {X, O, EMPTY}. Introducing a Cell class would create a wrapper around a single enumeration value, contributing indirection without adding either state or behaviour. The Single Responsibility Principle, which states that a class should have one and only one reason to change, cannot be satisfied by a class that has no responsibility beyond storing a single enumeration value. Consequently, the board grid will be represented as a two-dimensional array of Symbol values, and any per-cell operations will be expressed as methods on the Board class using coordinate pairs.\nThis decision is reversible if requirements later demand richer per-cell behaviour, such as cells that award scoring multipliers or cells that are highlighted for animation. The principle of You Aren\u0026rsquo;t Gonna Need It (YAGNI) counsels against introducing such structures speculatively, and the noun filter correctly excludes them at this stage.\nJustification for Modelling Position as a Frozen Data Class A position denotes the coordinates of a cell on the board and is used pervasively throughout the design: in method signatures, in the move history, and potentially as keys in dictionaries for indexed lookup. Four representations are available: a raw tuple, a namedtuple, a frozen dataclass, and a full class.\nA raw tuple of the form (row, column) provides no attribute names and no type documentation, which reduces code readability and eliminates the possibility of static type checking on individual fields. A namedtuple resolves the attribute-naming concern but retains the implementation constraints of a tuple. A frozen dataclass, introduced in Python 3.7, provides named typed attributes, immutability, automatic implementation of equality and hashing, and the ability to add methods later without changing the calling code. A full class introduces ceremony without corresponding benefit at this scale.\nThe frozen dataclass is selected because it provides the best combination of type safety, immutability, hashability (which permits use as dictionary keys), and extensibility. The term \u0026ldquo;frozen\u0026rdquo; here means that instances cannot be modified after construction; any attempt to assign to a field raises a FrozenInstanceError. Immutability is desirable for Position because a coordinate pair is conceptually a value, and values do not change over time.\nFinal Set of Classes and Supporting Types Applying the noun filter yields the following components:\nClasses:\nGame, which coordinates the overall game loop Board, which maintains the grid state and enforces move legality Player, the abstract base for all player types HumanPlayer, which reads moves from console input RandomAIPlayer and SmartAIPlayer, which will be introduced in Stage 10 Interfaces (abstract base classes):\nWinChecker, which encapsulates the algorithm for determining a winner GameObserver, which will be introduced in Stage 11 for event notification Enumerations:\nSymbol, holding the values X, O, and EMPTY GameStatus, holding the values IN_PROGRESS, WON, and DRAWN Immutable data classes:\nPosition, denoting a coordinate pair Move, recording a completed move as a historical fact Exceptions:\nGameOverError, raised when a move is attempted after the game has ended The classes deliberately omitted include Cell, Turn, Score, Rules as an entity, and any Tournament-related structures. Each omission is a design decision, not an oversight, and each is justified by the current scope. This restraint is central to sound design; a class that is not needed is a class that will not need to be maintained, tested, or explained.\n5. Stage 4: Modelling the Board and Cells The Board class occupies a central position in the design because it holds the ground-truth state of the game. Design choices at this stage propagate through the rest of the system, so each choice warrants explicit justification.\nChoice of Grid Representation Three plausible representations exist for the grid.\nRepresentation A: two-dimensional list. The grid is stored as a list of lists, where each inner list represents one row. Access is performed via grid[row][column]. This representation matches the conventional mental model of a two-dimensional array, and it produces code that reads naturally in relation to the problem statement. Memory overhead includes one list object per row plus one list object for the outer container, which is negligible for the board sizes under consideration.\nRepresentation B: one-dimensional list with index arithmetic. The grid is stored as a single flat list of length N times N, and coordinate access is computed as grid[row * N + column]. This representation uses slightly less memory than the two-dimensional list because it involves one list object rather than N plus one. The associated cost is that all coordinate access requires explicit index arithmetic, which introduces opportunities for off-by-one errors and reduces readability.\nRepresentation C: dictionary keyed by Position. The grid is stored as a dictionary from Position to Symbol, with occupied cells present as keys and unoccupied cells either absent or explicitly mapped to EMPTY. This representation is efficient in memory when the board is sparsely populated, and it generalises naturally to non-rectangular grids such as hexagonal boards. The associated cost is hashing overhead on every access and the loss of natural ordering when iterating over cells.\nThe two-dimensional list representation is selected for this design. The reasoning is that the board is small and dense in every scenario the design must support, so the memory arguments for Representation B do not apply. The readability argument in favour of Representation A is significant, since the code will read almost identically to the mathematical description of the problem. The generalisation argument for Representation C does not apply because the problem specifies a rectangular board.\nLocation of Move Validation Logic When a player attempts to place a mark at a coordinate pair, two conditions must be verified: the coordinate pair must lie within the bounds of the board, and the cell at that coordinate must currently be empty. Three plausible locations exist for this validation logic.\nThe first option locates validation on the top-level Game class, treating validation as a coordination responsibility. The second option locates validation on the Board class, treating validation as a natural consequence of the fact that the Board holds the state being validated. The third option introduces a separate MoveValidator class, treating validation as a distinct concern worthy of its own abstraction.\nThe second option is selected. The Board class is the sole holder of the state that determines move legality, so placing the validation on the Board minimises the amount of state that must be exposed to other classes and honours the principle that logic belongs with the data it operates on. Introducing a MoveValidator class would separate the validation logic from the state it validates without corresponding benefit, since the validation rules under consideration are simple and are unlikely to grow substantially. The Game class will invoke Board methods to perform validation rather than duplicating the logic.\nHandling of Invalid Move Attempts Two error-signalling conventions are available: return a Boolean indicating success or failure, or raise an exception when an invalid operation is attempted. The choice determines the ergonomics and safety of the calling code.\nThe convention adopted here is to expose a query method is_valid_move that returns a Boolean without raising, and a mutating method place that raises ValueError if invoked with an invalid argument. The rationale is that the query method is expected to be used defensively by callers who wish to check legality before acting, whereas the mutating method should not be reached with an invalid argument in correctly written code. Silent failure of a mutating method would obscure programming errors and defer their detection, whereas an exception surfaces the error immediately at its source. This convention mirrors the design of Python\u0026rsquo;s built-in data structures, where methods such as dict.pop provide both a mutating variant and a variant with a default value for defensive use.\nThe Board Class Implementation The implementation that follows realises the design decisions justified above.\nfrom dataclasses import dataclass from enum import Enum from typing import Optional class Symbol(Enum): \u0026#34;\u0026#34;\u0026#34; Enumeration of the possible values a cell may hold. Definition: An enumeration is a finite, named set of related values. Python\u0026#39;s Enum class provides type safety by restricting the set of admissible values at both runtime and static analysis time. Design rationale for including EMPTY as an enum member rather than representing empty cells with None: 1. Uniform typing. Every cell holds a value of type Symbol, which eliminates the need to write Optional[Symbol] throughout the codebase and simplifies type annotations for method signatures that operate on cell values. 2. Explicit rendering. An EMPTY value carries a display glyph (the dot character below), which the rendering method can emit uniformly without a conditional branch for None. 3. Semantic clarity. An empty cell is a distinct game-state concept, and representing it as a first-class enumeration value acknowledges its status as such. \u0026#34;\u0026#34;\u0026#34; EMPTY = \u0026#34;.\u0026#34; X = \u0026#34;X\u0026#34; O = \u0026#34;O\u0026#34; @dataclass(frozen=True) class Position: \u0026#34;\u0026#34;\u0026#34; Immutable coordinate pair identifying a cell on the board. Definition: A dataclass is a Python construct, introduced in Python 3.7 via PEP 557, that generates boilerplate methods (__init__, __repr__, __eq__, and optionally __hash__) from a class definition annotated with type-hinted attributes. The frozen=True parameter marks instances as immutable: any attempt to assign to an attribute after construction raises FrozenInstanceError. Immutability is desirable here because a Position is conceptually a mathematical value, and values do not change over time. Immutability also enables use of Position as a dictionary key, since frozen dataclasses receive a working __hash__ implementation automatically. A Position could alternatively be represented as a plain tuple, but a dataclass provides attribute names (pos.row instead of pos[0]) and allows methods to be attached in future revisions without changing the calling convention. \u0026#34;\u0026#34;\u0026#34; row: int col: int class Board: \u0026#34;\u0026#34;\u0026#34; The N-by-N game grid, responsible for storing cell state and enforcing the rules governing move legality. Responsibilities, aligned with the Single Responsibility Principle: 1. Store the current value of every cell. 2. Report the current value at any given coordinate. 3. Determine whether a proposed move is legal. 4. Apply a legal move by placing a symbol at a coordinate. 5. Report whether the board is fully occupied. 6. Produce a human-readable rendering of the board state. Responsibilities explicitly excluded from this class: 1. Determining whether the game has been won. Win detection is encapsulated separately in a WinChecker implementation so that alternative win rules can be introduced without modifying the Board class. 2. Managing turn rotation or player identity. The Board operates solely in terms of Symbol values and has no awareness of which player holds which symbol. 3. Rendering to any surface beyond returning a string. Producing graphical or networked output is the concern of a separate presentation component. \u0026#34;\u0026#34;\u0026#34; def __init__(self, size: int = 3): \u0026#34;\u0026#34;\u0026#34; Construct an N-by-N board with every cell initialised to EMPTY. The minimum admissible size is three. Boards of size one and two are excluded because they are degenerate: a one by one board is won by any move, and a two by two board admits only trivial or unwinnable configurations. Rejecting these sizes at construction time prevents downstream code from having to handle these anomalous cases. \u0026#34;\u0026#34;\u0026#34; if size \u0026lt; 3: raise ValueError( f\u0026#34;Board size must be at least 3, received {size}\u0026#34; ) self.size = size # The grid is constructed using a nested list comprehension # rather than the more concise but incorrect expression # [[Symbol.EMPTY] * size] * size. The latter form creates a # single inner list and repeats the reference to it size # times, which means that mutating one row would mutate all # rows. The nested comprehension creates a distinct inner # list for each row and thereby avoids this aliasing hazard. self._grid: list[list[Symbol]] = [ [Symbol.EMPTY for _ in range(size)] for _ in range(size) ] # An occupied-cell counter is maintained so that the is_full # query can be answered in constant time. Recomputing the # count by iteration would require O(N squared) work per # query, which is inconsequential for small boards but # unnecessary given that the counter is trivially maintained # by incrementing on each successful placement. self._filled_count = 0 # ------------------------------------------------------------------ # Query methods, none of which mutate the board state. # ------------------------------------------------------------------ def get(self, position: Position) -\u0026gt; Symbol: \u0026#34;\u0026#34;\u0026#34; Return the Symbol currently occupying the given position. The caller is expected to have verified that the position lies within bounds before invoking this method. \u0026#34;\u0026#34;\u0026#34; return self._grid[position.row][position.col] def is_in_bounds(self, position: Position) -\u0026gt; bool: \u0026#34;\u0026#34;\u0026#34; Return True if and only if the given coordinate pair falls within the valid index range of the board along both axes. \u0026#34;\u0026#34;\u0026#34; return ( 0 \u0026lt;= position.row \u0026lt; self.size and 0 \u0026lt;= position.col \u0026lt; self.size ) def is_empty_at(self, position: Position) -\u0026gt; bool: \u0026#34;\u0026#34;\u0026#34; Return True if the position is both within bounds and currently occupied by the EMPTY symbol. This is a defensive combination that ensures the bounds check precedes the cell access, since indexing an out-of-range coordinate would raise IndexError. \u0026#34;\u0026#34;\u0026#34; return ( self.is_in_bounds(position) and self._grid[position.row][position.col] == Symbol.EMPTY ) def is_valid_move(self, position: Position) -\u0026gt; bool: \u0026#34;\u0026#34;\u0026#34; Return True if the given position is a legal target for a move, which requires that it be within bounds and currently empty. Callers use this method to determine whether it is safe to invoke the place method with the given position. \u0026#34;\u0026#34;\u0026#34; return self.is_empty_at(position) @property def is_full(self) -\u0026gt; bool: \u0026#34;\u0026#34;\u0026#34; Return True if every cell on the board has been played. The computation is constant time thanks to the maintained occupancy counter, in contrast to a full iteration over the grid. \u0026#34;\u0026#34;\u0026#34; return self._filled_count == self.size * self.size # ------------------------------------------------------------------ # Mutating methods. # ------------------------------------------------------------------ def place(self, symbol: Symbol, position: Position) -\u0026gt; None: \u0026#34;\u0026#34;\u0026#34; Place the given symbol at the given position, subject to validation. Three conditions must be satisfied for the placement to proceed. First, the symbol must be a real playing symbol rather than EMPTY, since EMPTY does not represent a move that any player would make. Second, the position must lie within the bounds of the board. Third, the target cell must currently be empty. If any of the three conditions is violated, the method raises ValueError with a descriptive message identifying the specific violation. Raising rather than returning False is appropriate here because reaching this method with invalid arguments indicates a programming defect in the caller, and defects should surface immediately at their source rather than being silently absorbed. \u0026#34;\u0026#34;\u0026#34; if symbol == Symbol.EMPTY: raise ValueError( \u0026#34;Cannot place EMPTY as a move; EMPTY is a state, not an action\u0026#34; ) if not self.is_in_bounds(position): raise ValueError( f\u0026#34;Position {position} is out of bounds for a board \u0026#34; f\u0026#34;of size {self.size}\u0026#34; ) if not self.is_empty_at(position): raise ValueError( f\u0026#34;Position {position} is already occupied by \u0026#34; f\u0026#34;{self.get(position).value}\u0026#34; ) self._grid[position.row][position.col] = symbol self._filled_count += 1 # ------------------------------------------------------------------ # Rendering. This is deliberately minimal since presentation # concerns are outside the scope of the game logic. # ------------------------------------------------------------------ def render(self) -\u0026gt; str: \u0026#34;\u0026#34;\u0026#34; Return a human-readable multi-line string representation of the board suitable for printing to a text terminal. The method returns a string rather than printing directly so that callers may redirect the output to a log file, a test assertion, or an alternative presentation surface without modification of this method. \u0026#34;\u0026#34;\u0026#34; header = \u0026#34; \u0026#34; + \u0026#34; \u0026#34;.join(str(c) for c in range(self.size)) separator = \u0026#34; \u0026#34; + \u0026#34;-\u0026#34; * (4 * self.size - 1) rows = [] for r in range(self.size): cells = \u0026#34; \u0026#34;.join( self._grid[r][c].value for c in range(self.size) ) rows.append(f\u0026#34; {r} | {cells}\u0026#34;) return \u0026#34;\\n\u0026#34;.join([header, separator] + rows) Several aspects of this implementation warrant additional commentary.\nThe decision to include EMPTY as a Symbol enumeration value rather than using None simplifies typing throughout the codebase. Every cell holds a value of type Symbol, which permits methods that operate on cell values to be typed without the overhead of Optional wrapping, and which permits equality comparisons to be expressed uniformly.\nThe occupancy counter _filled_count is a form of derived state that is maintained incrementally on each successful placement. The alternative of recomputing the count by iteration on demand would be functionally equivalent but computationally wasteful. The tradeoff of maintaining derived state is the requirement to keep it consistent with the primary state, which is why placement is the only method that mutates the counter and does so atomically alongside the grid modification.\nThe list comprehension used to construct the initial grid deliberately avoids the concise but hazardous form that multiplies a single row list. The multiplied form produces a list containing multiple references to the same inner list, so mutating any single row through indexing would mutate every row. The nested comprehension creates an independent inner list for each row and thereby avoids this aliasing hazard entirely.\nWin-detection logic is deliberately absent from this class. The reason is that win detection is a candidate variation point in the design, and locating variable behaviour behind a dedicated abstraction is a core discipline of extensible design. A separate WinChecker class will be introduced in Stage 7.\n6. Stage 5: Modelling Symbols and Players The Symbol enumeration was defined in the previous section. This section addresses the Player abstraction, which is the mechanism by which different kinds of move-generating actors are represented uniformly within the game.\nChoice Between a Single Class and a Class Hierarchy Two competing designs are available for representing players.\nThe first design uses a single Player class with a callable attribute that provides moves. The callable is injected at construction time and may point to a function that reads console input, a function that computes a move algorithmically, or any other move-producing function. The advantages of this design are its brevity and its functional character. The disadvantages are that different kinds of players may require different auxiliary state, such as game-tree search parameters for an artificial intelligence, and packing all such state into a single class produces a class with an ill-defined responsibility.\nThe second design uses an abstract Player class as the root of a hierarchy, with concrete subclasses HumanPlayer, RandomAIPlayer, and SmartAIPlayer implementing the abstract interface. The advantages of this design are that each subclass may carry its own state and behaviour without polluting the others, and that the abstract interface documents the contract that every player must honour. The disadvantages are the initial ceremony of defining a class hierarchy for what could be a single class.\nThe class hierarchy design is selected for two principal reasons. The first reason is that the different kinds of players in this design have materially different state and behaviour: a human player interacts with the console, an artificial intelligence player evaluates game trees, and a hypothetical networked player would communicate over a socket. Encapsulating each of these in its own class localises the associated concerns. The second reason is that the class hierarchy provides a natural place to formally state, using the abstract method mechanism, the contract that all players must satisfy. This aligns with the Interface Segregation Principle, which states that clients should not be forced to depend on interfaces they do not use, and with the general principle of programming to an interface rather than an implementation.\nAssignment of Symbols to Players A player is associated with a specific playing symbol for the duration of a game. Two designs are available for making this association.\nThe first design constructs a Player with its symbol supplied as a constructor argument. The player\u0026rsquo;s symbol is intrinsic to its identity from the moment of construction, and any code that examines the player can immediately determine its symbol without consulting external state.\nThe second design constructs a Player without a symbol and assigns the symbol later, typically when the player is added to a game. This design permits the same player object to participate in multiple games with different symbols, at the cost of an intermediate state in which the player exists but has no assigned symbol.\nThe first design is selected. The player and its symbol are conceptually inseparable during the game, and constructing them together avoids the intermediate unassigned state that the second design would produce. The consequence is that a player object is bound to a single game session; if the same human were to play in multiple games, a new Player object would be constructed for each. This trade-off is acceptable given the scope of the design.\nThe Player Class Hierarchy Implementation from abc import ABC, abstractmethod class Player(ABC): \u0026#34;\u0026#34;\u0026#34; Abstract base class defining the interface that all player types must implement. Definition: An abstract base class (ABC) is a class that cannot be instantiated directly and that declares one or more abstract methods that concrete subclasses must implement. Python provides the abstract base class mechanism through the abc module. Design rationale for defining a class hierarchy rather than a single Player class with a move-provider callback: 1. Different player types have materially different auxiliary state. A human player has no computation-related state, a Monte Carlo AI has iteration counts and random seeds, a minimax AI has search depth limits and evaluation function parameters, and a networked player has communication channels and message-serialisation logic. Packing all of this into a single class would violate the Single Responsibility Principle. 2. The abstract method make_move formally states the contract that every player must honour. Static analysis tools and the Python runtime both enforce that concrete subclasses provide an implementation, preventing the accidental instantiation of a player that cannot produce moves. 3. Introducing a new player type consists of adding a new subclass and requires no modification of the existing code, which is the essence of the Open-Closed Principle. The abstract make_move method is the strategy pattern realised through inheritance: the interface is fixed by the abstract base class, and each concrete subclass provides its own strategy for the same interface. \u0026#34;\u0026#34;\u0026#34; def __init__(self, name: str, symbol: Symbol): \u0026#34;\u0026#34;\u0026#34; Construct a player with the given display name and playing symbol. The name is used for logging and display purposes and does not affect the game logic. The symbol determines which mark this player places on the board and is intrinsic to the player\u0026#39;s identity within a game session. The EMPTY symbol is rejected because it does not represent a valid playing symbol; assigning it to a player would produce a player who cannot make legal moves. Detecting this at construction time prevents the error from surfacing later during gameplay. \u0026#34;\u0026#34;\u0026#34; if symbol == Symbol.EMPTY: raise ValueError( \u0026#34;A player cannot be assigned the EMPTY symbol\u0026#34; ) self.name = name self.symbol = symbol @abstractmethod def make_move(self, board: Board) -\u0026gt; Position: \u0026#34;\u0026#34;\u0026#34; Return the Position at which this player wishes to place its next symbol. The board is passed as an argument so that the player may inspect the current state of play in reaching its decision. Although the board is technically mutable, the contract of this method requires that implementations treat it as read-only; any mutation of the board during move selection would corrupt the game state. Implementations are responsible for returning a legal move. The Game class re-validates the returned move as a defence against implementation errors, but a well-behaved player implementation will only ever return a position that satisfies board.is_valid_move at the time of invocation. \u0026#34;\u0026#34;\u0026#34; ... def __repr__(self) -\u0026gt; str: \u0026#34;\u0026#34;\u0026#34;Return a diagnostic representation for logging and debugging.\u0026#34;\u0026#34;\u0026#34; return f\u0026#34;{self.__class__.__name__}({self.name}, {self.symbol.name})\u0026#34; class HumanPlayer(Player): \u0026#34;\u0026#34;\u0026#34; A player whose move decisions are supplied via console input. The implementation is intentionally tolerant of ill-formed input: a typographical error or an illegal move causes the method to prompt again rather than to raise, since these represent expected user behaviour rather than exceptional conditions. Input format: two whitespace-separated integers denoting the row index and column index respectively, with both indices being zero-based. Design note: input handling is localised to this class rather than the Game class because the Game class should have no dependency on any particular input mechanism. A different concrete Player subclass, such as one reading input from a graphical user interface, would replace this class without any change to the Game orchestration logic. \u0026#34;\u0026#34;\u0026#34; def make_move(self, board: Board) -\u0026gt; Position: \u0026#34;\u0026#34;\u0026#34; Prompt the human user for a move, re-prompting on ill-formed or illegal input, and return the resulting Position. \u0026#34;\u0026#34;\u0026#34; while True: # Display the current board state so that the human has # the information needed to select a move. In a design # with a proper presentation layer, this rendering would # be triggered by an observer rather than by the player # itself, but the current scope permits this simpler # arrangement. print(board.render()) raw_input_text = input( f\u0026#34;{self.name} playing {self.symbol.value}, \u0026#34; f\u0026#34;enter row and column indices in the range \u0026#34; f\u0026#34;0 to {board.size - 1}, separated by a space \u0026#34; f\u0026#34;(for example, \u0026#39;1 2\u0026#39;): \u0026#34; ).strip() tokens = raw_input_text.split() if len(tokens) != 2: print( \u0026#34;Input must consist of exactly two integers \u0026#34; \u0026#34;separated by whitespace. Please try again.\u0026#34; ) continue try: row = int(tokens[0]) col = int(tokens[1]) except ValueError: print( \u0026#34;Both inputs must be integers. Please try again.\u0026#34; ) continue position = Position(row, col) if not board.is_valid_move(position): print( f\u0026#34;The move {position} is not legal: the position \u0026#34; f\u0026#34;is either out of bounds or the cell is already \u0026#34; f\u0026#34;occupied. Please try again.\u0026#34; ) continue return position Two aspects of this implementation deserve additional commentary.\nThe abstract Player class deliberately provides no default implementation of make_move. This forces every concrete subclass to make an explicit choice about how moves are generated, and it prevents the accidental creation of a partially-formed player. Python\u0026rsquo;s abstract base class mechanism enforces this at instantiation time: attempting to instantiate a subclass that has not implemented all abstract methods raises TypeError, which surfaces the error at the earliest possible moment.\nThe HumanPlayer.make_move implementation encapsulates all interactive input handling within a single method. The loop structure permits the method to keep prompting until a valid move is supplied, treating typographical errors and illegal moves as expected user behaviour that should be recovered from gracefully. The alternative of raising an exception on invalid input would push the recovery burden onto the caller, which would in turn need to know about console input mechanics, thereby violating the encapsulation this class is intended to provide.\nConcrete artificial intelligence implementations are deferred to Stage 10, after the win-checker and Game class have been defined, since the intelligence implementations depend on both.\n7. Stage 6: Making a Move, Validation and Placement Before designing the Game class, it is worth examining the sequence of operations that constitutes a single move. Making this sequence explicit clarifies the responsibilities of the participating classes and reveals whether a dedicated Move class is warranted.\nThe Move Sequence A single move consists of the following operations executed in order:\nThe Game identifies the current player by consulting its turn-tracking state. The Game invokes the current player\u0026rsquo;s make_move method, passing the board as an argument, and receives a Position in return. The Game validates the returned Position against the current board state, treating an invalid Position as evidence of a defect in the player implementation. The Game invokes board.place with the player\u0026rsquo;s symbol and the returned Position, updating the board\u0026rsquo;s state. The Game records the move in its history for potential future features such as undo or replay. The Game invokes the WinChecker to determine whether the completed move resulted in a win. If so, the game enters the WON state and terminates further play. If no win occurred, the Game checks whether the board is now full. If so, the game enters the DRAWN state and terminates. Otherwise, the Game advances the turn-tracking state to the next player. Every one of these operations is conceptually part of a single \u0026ldquo;make a move\u0026rdquo; action, and packaging them together within the Game class ensures that the invariants of the game are maintained atomically.\nJustification for a Move Data Class A Move records a completed move as a historical fact, capturing the identity of the player, the position played, and a timestamp. Three of the eventual features that this history enables are undo, replay, and audit logging.\nThe alternative to a Move data class is to not track history at all, discarding move information as soon as the move has been applied to the board. This alternative has the advantage of simplicity but forecloses the future features that history enables. Given that the cost of maintaining a history is a single list append per move, which is negligible in both time and space for a game of this size, and given that the presence of history opens the door to substantially richer functionality with minimal further effort, the preemptive investment in a Move class is justified even though the current scope does not require it.\nThis is a rare case in which the principle of You Aren\u0026rsquo;t Gonna Need It is overridden. The overriding factor is the very low cost of the investment combined with the very high value of the future capabilities it enables. In general, the principle should be observed unless a specific and compelling counter-argument can be articulated.\nThe Move Class Implementation from datetime import datetime from dataclasses import field @dataclass(frozen=True) class Move: \u0026#34;\u0026#34;\u0026#34; An immutable record of a completed move, comprising the identity of the player who made the move, the position on the board where the mark was placed, and the timestamp at which the move occurred. Design rationale for making the class frozen (immutable): 1. A completed move is a historical fact and, by definition, cannot be revised. Enforcing immutability at the language level prevents accidental modification. 2. Frozen dataclasses are hashable, which permits Move instances to be stored in sets and used as dictionary keys if future features require it. Design rationale for including the timestamp: 1. The cost is negligible: a single additional field with a default factory. 2. The benefit is substantial: features such as move-time analysis, replay pacing, and timeout enforcement (as used in timed variants of chess) all require timestamps, and retrofitting them later would require modifying every call site that constructs a Move. Design rationale for storing the player rather than only the symbol: the player object contains the player\u0026#39;s display name, which is useful for logging and replay. The symbol can always be recovered from move.player.symbol, so storing the player is strictly more informative. \u0026#34;\u0026#34;\u0026#34; player: Player position: Position timestamp: datetime = field(default_factory=datetime.now) The field(default_factory=datetime.now) construction merits explanation. If the timestamp field were initialised with the expression datetime.now() as its default value, Python would evaluate that expression exactly once at the time the class was defined, and every subsequent Move instance would receive the same timestamp. Using field(default_factory=datetime.now) supplies a callable that Python invokes at each Move construction, producing a fresh timestamp per instance. This same idiom applies to any mutable or freshly-computed default value in a dataclass and is one of the more common subtleties of the dataclass API.\n8. Stage 7: Win Detection as an Algorithmic Concern Win detection is the most algorithmically substantive part of the design. This section examines the range of possible implementations, justifies the selected approach, and explains why the win-detection logic is separated into its own class rather than embedded within the Board.\nThe Naive Constant Approach and Its Limitations The simplest possible implementation for a three by three board enumerates the eight winning lines as constants and checks each of them after every move.\nWINNING_LINES = [ [(0, 0), (0, 1), (0, 2)], # top row [(1, 0), (1, 1), (1, 2)], # middle row [(2, 0), (2, 1), (2, 2)], # bottom row [(0, 0), (1, 0), (2, 0)], # left column [(0, 1), (1, 1), (2, 1)], # middle column [(0, 2), (1, 2), (2, 2)], # right column [(0, 0), (1, 1), (2, 2)], # main diagonal [(0, 2), (1, 1), (2, 0)], # anti-diagonal ] This approach is correct for a three by three board and, taken in isolation, requires no substantial explanation. However, it fails immediately when the board size is generalised, because the set of winning lines depends on the board dimension and cannot be hard-coded once for all sizes. Since the confirmed requirements specify support for arbitrary N by N boards, this approach is inadmissible.\nThe Generalised Algorithm For an N by N board with the N-in-a-row win condition, the winning lines are precisely the following:\nThe N rows, each of which is a sequence of N cells sharing a row index. The N columns, each of which is a sequence of N cells sharing a column index. The main diagonal, consisting of the N cells whose row and column indices are equal. The anti-diagonal, consisting of the N cells whose row and column indices sum to N minus 1. The total number of lines is therefore 2N + 2. For each line, the algorithm inspects the N cells along the line and determines whether they all contain the same non-EMPTY symbol. If any line satisfies this condition, the symbol common to that line is the winner. If no line satisfies the condition, no winner exists.\nThe time complexity of a single win check under this algorithm is O(N squared), since the algorithm inspects 2N + 2 lines of N cells each. For the sizes under consideration, this is trivially fast: at N equal to three, the total cell inspection count is at most 24; at N equal to fifteen, the count is at most 480. These figures are well below any practical performance threshold.\nThe Incremental Alternative A more sophisticated alternative observes that only a small number of lines can be affected by any single move: specifically, the row containing the move, the column containing the move, and at most two diagonals passing through the move. All other lines are unchanged from their state after the previous move, so re-examining them is wasted work.\nUnder the incremental approach, the win checker inspects only the lines passing through the most recent move, reducing the per-move complexity to O(N) rather than O(N squared). This is a factor-of-N improvement that becomes significant at very large board sizes but is inconsequential at the sizes under consideration.\nThe trade-off is that the incremental approach requires more code and admits more opportunities for error, particularly in the diagonal cases where the geometry is less symmetric than for rows and columns. For a design at the current scope, the full-scan approach is preferred for its simplicity and obvious correctness. If the design were subsequently deployed to a context with very large boards or very high move rates, the incremental approach could be introduced without disturbing the rest of the system, since win detection is encapsulated behind an interface.\nJustification for Separating Win Detection from the Board The Board class could plausibly include a winner method that performs win detection directly against its own state. However, this would tightly couple the win rule to the Board, making it difficult to accommodate variant win rules such as K-in-a-row on an N-by-N board or the corner-configuration wins found in some Tic Tac Toe variants.\nThe Strategy pattern provides the appropriate solution. Win detection is encapsulated behind an abstract WinChecker interface, and multiple concrete implementations can coexist. The Game class holds a reference to whichever WinChecker implementation is active for the current game, and it invokes the checker after each move without concerning itself with the details of the check. Introducing a new win rule consists of writing a new WinChecker subclass and instantiating the Game with an instance of that subclass, requiring no modification of the Game, Board, or Player classes.\nThis separation also improves testability. A WinChecker is a pure function of the board state: it accepts a board and returns either a winning symbol or None, with no side effects and no dependence on external state. Pure functions are the easiest kind of code to unit-test because their behaviour is fully determined by their inputs.\nThe WinChecker Interface and NInARowChecker Implementation from typing import Iterator, Optional class WinChecker(ABC): \u0026#34;\u0026#34;\u0026#34; Abstract base class defining the interface for all win-detection algorithms. Every concrete implementation of this interface is a pure function of the board state, meaning that its behaviour depends only on the board passed as an argument and produces no observable side effects. This property makes WinChecker implementations trivially unit-testable. Introducing a new win rule (for example, K-in-a-row, or a corner-configuration win rule) consists of writing a new subclass of this base class and injecting an instance of it into the Game at construction time. No modification of the Game, Board, or Player classes is required. This is a direct application of the Open-Closed Principle: the system is open for extension via new subclasses but closed for modification of existing classes. \u0026#34;\u0026#34;\u0026#34; @abstractmethod def check(self, board: Board) -\u0026gt; Optional[Symbol]: \u0026#34;\u0026#34;\u0026#34; Return the Symbol that has won the game, or None if no player has yet achieved the win condition. The method does not distinguish between an in-progress game and a drawn game; that distinction is the responsibility of the Game class, which combines the WinChecker result with the board\u0026#39;s is_full property to reach a final determination. \u0026#34;\u0026#34;\u0026#34; ... class NInARowChecker(WinChecker): \u0026#34;\u0026#34;\u0026#34; Concrete WinChecker implementation for the standard N-in-a-row win rule, where N is equal to the board dimension. Algorithm: enumerate every winning line on the board (there are 2N + 2 such lines), and for each line determine whether all N cells contain the same non-EMPTY symbol. If any line satisfies this condition, return the symbol common to that line. If no line satisfies the condition, return None. Time complexity: O(N squared) per invocation. For the board sizes supported by this design, this is well below any performance threshold that would justify a more sophisticated implementation. \u0026#34;\u0026#34;\u0026#34; def check(self, board: Board) -\u0026gt; Optional[Symbol]: \u0026#34;\u0026#34;\u0026#34; Iterate over every winning line and return the winning symbol of the first line that constitutes a win, or None if no such line exists. \u0026#34;\u0026#34;\u0026#34; for line in self._all_lines(board.size): winner = self._line_winner(board, line) if winner is not None: return winner return None def _all_lines(self, n: int) -\u0026gt; Iterator[list[Position]]: \u0026#34;\u0026#34;\u0026#34; Generate every winning line on an N-by-N board. The method is implemented as a generator, meaning that lines are produced lazily one at a time rather than being accumulated into a list. Laziness has two advantages: memory consumption is bounded by a single line rather than by all lines simultaneously, and short-circuit evaluation in the caller can terminate iteration as soon as a winning line is found. The four categories of winning lines are generated in the following order: rows, then columns, then the main diagonal, then the anti-diagonal. \u0026#34;\u0026#34;\u0026#34; # Rows: for each row index r, the line consists of the cells # at (r, 0), (r, 1), ..., (r, n - 1). for r in range(n): yield [Position(r, c) for c in range(n)] # Columns: for each column index c, the line consists of the # cells at (0, c), (1, c), ..., (n - 1, c). for c in range(n): yield [Position(r, c) for r in range(n)] # Main diagonal: the cells at (0, 0), (1, 1), ..., (n - 1, n - 1). yield [Position(i, i) for i in range(n)] # Anti-diagonal: the cells at (0, n - 1), (1, n - 2), ..., (n - 1, 0). yield [Position(i, n - 1 - i) for i in range(n)] def _line_winner( self, board: Board, line: list[Position] ) -\u0026gt; Optional[Symbol]: \u0026#34;\u0026#34;\u0026#34; Determine whether the given line constitutes a win. A line constitutes a win if and only if all of its cells contain the same non-EMPTY symbol. The method extracts the symbol of the first cell in the line, immediately rejects the line if that symbol is EMPTY (since an empty first cell cannot begin a winning line), and otherwise verifies that every subsequent cell contains the same symbol. The all() built-in short-circuits on the first False value, so the method returns as soon as any mismatch is detected. \u0026#34;\u0026#34;\u0026#34; first = board.get(line[0]) if first == Symbol.EMPTY: return None if all(board.get(p) == first for p in line[1:]): return first return None The design of this implementation supports future extension in a straightforward manner. A K-in-a-row variant, which generalises the win condition to permit winning lines shorter than the board dimension, is implemented by writing a new subclass of WinChecker with a different line-generation and inspection strategy that accepts K as a parameter. No modification of the existing code is required. This extension is demonstrated in the Practice Questions section.\nThe use of a generator for _all_lines is worth explaining in more detail. A generator function in Python is one that contains one or more yield statements. When called, a generator function returns a generator object, which is an iterator that produces values on demand. Each call to the iterator\u0026rsquo;s __next__ method resumes the function from its most recent yield, executes until the next yield, and produces the yielded value. Memory consumption is bounded by a single value at a time, rather than by the full sequence of values. For small board sizes the difference is negligible, but the generator idiom is a good habit because it scales to arbitrarily large sequences without changing the calling convention.\n9. Stage 8: Modelling the Game State A game progresses through a small number of well-defined states. The initial state is in-progress, during which players alternate moves. The game may terminate in either of two ways: a player achieves the win condition (the won state) or the board fills without any player winning (the drawn state). Both terminal states share the property that no further moves are permitted; any attempt to play after termination should be rejected as an error.\nThis section addresses how the state should be represented in the design and, in particular, whether the full State pattern is warranted or whether a simpler representation will suffice.\nThe State Pattern Versus an Enumeration The State pattern is a behavioural design pattern in which each state is represented as a separate class, and the object whose state is being modelled delegates state-dependent operations to an instance of the current state class. State transitions are effected by replacing the current state instance with an instance of the destination state class. The pattern is valuable when different states admit substantially different behaviour, because the branching that would otherwise be required (typically in the form of conditional dispatch on a state-identifying field) is replaced by polymorphism.\nThe Elevator design in an earlier document of this series used the State pattern precisely because the Idle, MovingUp, MovingDown, and DoorsOpen states admit substantially different behaviour: each state accepts different transitions, produces different next actions, and has different rules governing valid inputs. The pattern\u0026rsquo;s overhead was justified by the richness of the per-state behaviour.\nIn the Tic Tac Toe design, the state-dependent behaviour is markedly less rich. The IN_PROGRESS state accepts moves and applies them to the board; the WON and DRAWN states both reject moves with an identical response. The transitions are also very simple: only two are possible, both from IN_PROGRESS, and neither has any special behaviour beyond updating the status field.\nGiven this simplicity, the full State pattern would introduce structural complexity disproportionate to the behavioural variation it captures. A single enumeration with three values, combined with a guard clause at the entry of the play_turn method, expresses the same behaviour in fewer lines and with lower cognitive load. This is the correct choice, and articulating the reason for it demonstrates awareness that pattern selection is a matter of judgement rather than a matter of applying patterns wherever they fit syntactically.\nThe GameStatus Enumeration Implementation class GameStatus(Enum): \u0026#34;\u0026#34;\u0026#34; Enumeration of the three possible game states. Design rationale for using an enumeration rather than the State pattern: The State pattern is valuable when different states admit substantially different behaviour that would otherwise be expressed as conditional dispatch. In this design, the state dependent behaviour is limited to a single conditional at the entry of the play_turn method, which rejects moves in terminal states. This is insufficient behavioural variation to justify the structural overhead of a class per state. Should future requirements introduce richer per-state behaviour (for example, a PAUSED state that accepts unpause commands but not moves, or an AWAITING_CONFIRMATION state that accepts move confirmations), the design could be refactored to use the full State pattern without disturbing the rest of the system. \u0026#34;\u0026#34;\u0026#34; IN_PROGRESS = \u0026#34;in_progress\u0026#34; WON = \u0026#34;won\u0026#34; DRAWN = \u0026#34;drawn\u0026#34; class GameOverError(Exception): \u0026#34;\u0026#34;\u0026#34; Exception raised when a move is attempted on a game that has already reached a terminal state (either WON or DRAWN). Defining a distinct exception type, rather than raising a generic ValueError, allows callers to distinguish this specific error condition from other invalid-argument errors and to handle it separately if desired. \u0026#34;\u0026#34;\u0026#34; pass The choice to define a custom exception class rather than reuse ValueError is intentional. Custom exception classes carry semantic meaning that generic exceptions do not: catching GameOverError in a caller signals a very specific error condition, whereas catching ValueError catches a broader range of unrelated errors. The cost of defining a custom exception is minimal (three lines including docstring), and the payoff is more precise error handling.\n10. Stage 9: The Game Class as the Orchestrator The Game class occupies the top of the composition hierarchy in this design. It holds references to the Board, the collection of Players, the WinChecker, and the current game state, and it defines the game loop that drives play forward. The design goal for this class is to make it an orchestrator rather than a repository of logic: each of the other classes contains its own domain-specific behaviour, and the Game class merely coordinates their interaction.\nThe Boundaries of the Game Class The following responsibilities belong to the Game class:\nOwning the Board, the collection of Players, and the WinChecker. Tracking whose turn it currently is. Enforcing the invariant that no moves may be played after the game has reached a terminal state. Recording the sequence of moves in a history for potential future use. Coordinating the sequence of operations that constitute a single turn. The following responsibilities do not belong to the Game class and are delegated elsewhere:\nDetermining how a player selects a move; this is the responsibility of the Player subclass. Managing the internal representation of the board grid; this is the responsibility of the Board class. Determining whether a completed board configuration is a win; this is the responsibility of the WinChecker. Producing user-facing output or presentation; this is the responsibility of a separate presentation component (or, in this scope, of console print statements localised to the driver code). This distribution of responsibilities keeps the Game class small and focused, preventing the anti-pattern colloquially known as the god class, in which a single class accumulates so many responsibilities that it becomes difficult to modify, test, or reason about.\nTurn Rotation Strategy Two representations of turn state are available. The first uses a Boolean flag indicating whether the current player is player one or player two, which is compact but does not generalise beyond two players. The second uses an integer index into the players list, which is trivially generalisable to any number of players by advancing the index modulo the list length.\nThe integer-index representation is selected because it generalises without additional code. Advancing the index is a single expression: index = (index + 1) % len(players). This expression is correct for any number of players and requires no special-casing for the two-player scenario.\nThe Order of Terminal-State Checks After each move, the Game must check whether the game has reached a terminal state. Two terminal conditions exist: a win, indicated by the WinChecker returning a non-None symbol, and a draw, indicated by the board being full without a winner. The order in which these checks are performed matters.\nConsider a scenario in which the last empty cell on the board is filled with a mark that completes a winning line. If the draw check is performed first, the check succeeds (because the board is now full) and the game is incorrectly declared a draw. The win check must be performed first, so that a winning move on the final cell is recognised as a win rather than misclassified as a draw. This ordering is not obvious from a superficial reading of the code, and it is exactly the kind of subtle correctness issue that a mental walkthrough of the design (covered in Stage 12) is intended to surface.\nThe Game Class Implementation from typing import Optional class Game: \u0026#34;\u0026#34;\u0026#34; Orchestrator class that composes the Board, Players, WinChecker, and game state into a functioning game. Responsibilities of this class: 1. Own the Board, the ordered collection of Players, and the WinChecker. 2. Track whose turn it is via an index into the players list. 3. Enforce that moves may not be played after termination. 4. Coordinate the sequence of operations that constitute a single turn: solicit the move, validate it, apply it, record it, check terminal conditions, and advance the turn. 5. Maintain a history of moves for future use. Responsibilities delegated to other classes: 1. Move selection is delegated to Player.make_move. 2. Grid state and move validation are delegated to Board. 3. Win detection is delegated to WinChecker. 4. User interface concerns are outside the scope of this class. Composition over inheritance: this class does not inherit from any other class; its behaviour arises from the composition of its collaborators. This composition-based design is more flexible than an inheritance-based alternative because collaborators can be substituted independently. \u0026#34;\u0026#34;\u0026#34; def __init__( self, players: list[Player], board_size: int = 3, win_checker: Optional[WinChecker] = None, ): \u0026#34;\u0026#34;\u0026#34; Construct a new Game with the specified players, board size, and win checker. The players argument must contain at least two players, and every player must have a distinct symbol. Both invariants are checked at construction time so that violations are detected before any gameplay begins. The win_checker argument is optional; if omitted, an NInARowChecker is instantiated by default. Making the default explicit (rather than hiding it behind a None check elsewhere) makes the behaviour of the class transparent to callers. \u0026#34;\u0026#34;\u0026#34; if len(players) \u0026lt; 2: raise ValueError( f\u0026#34;A game requires at least two players, received {len(players)}\u0026#34; ) symbols = [p.symbol for p in players] if len(set(symbols)) != len(symbols): raise ValueError( f\u0026#34;Players must have distinct symbols, but received \u0026#34; f\u0026#34;the following symbols: {[s.name for s in symbols]}\u0026#34; ) self.board = Board(size=board_size) self.players = players self.win_checker = win_checker or NInARowChecker() # Turn tracking is implemented as an integer index into the # players list. Advancing the turn is a modulo operation, # which generalises to any number of players without # additional code. self._current_player_index = 0 # The game begins in the IN_PROGRESS state and transitions # to either WON or DRAWN upon reaching a terminal condition. self.status = GameStatus.IN_PROGRESS # The winner attribute is None until the game reaches the # WON state, at which point it is set to the winning player. # A drawn game has no winner and this attribute remains None. self.winner: Optional[Player] = None # The move history is a list of Move records in chronological # order. The cost of maintaining this list is a single list # append per move, which is negligible, and the presence of # the history enables features such as undo, replay, and # audit logging to be added later without redesign. self.history: list[Move] = [] # ------------------------------------------------------------------ # Public interface. # ------------------------------------------------------------------ @property def current_player(self) -\u0026gt; Player: \u0026#34;\u0026#34;\u0026#34; Return the Player whose turn it is currently. This property is defined for the convenience of callers who wish to identify the current player without knowing about the internal index representation. \u0026#34;\u0026#34;\u0026#34; return self.players[self._current_player_index] @property def is_over(self) -\u0026gt; bool: \u0026#34;\u0026#34;\u0026#34; Return True if the game has reached a terminal state, whether WON or DRAWN. This is the convenience predicate used by the game loop to determine whether further turns should be played. \u0026#34;\u0026#34;\u0026#34; return self.status != GameStatus.IN_PROGRESS def play_turn(self) -\u0026gt; None: \u0026#34;\u0026#34;\u0026#34; Execute a single complete turn. The sequence of operations is as follows: 1. Verify that the game is still in progress, raising GameOverError otherwise. 2. Identify the current player and solicit their move. 3. Re-validate the returned move as a defensive check. 4. Apply the move to the board. 5. Record the move in the history. 6. Check for a winning line via the WinChecker. If a winner is found, transition to the WON state and return. 7. Otherwise, check whether the board is now full. If it is, transition to the DRAWN state and return. 8. Otherwise, advance the turn to the next player. The check for a winning line must precede the check for a full board, because a move that both fills the final cell and completes a winning line should be classified as a win rather than as a draw. \u0026#34;\u0026#34;\u0026#34; if self.is_over: raise GameOverError( f\u0026#34;Cannot play a turn because the game has already \u0026#34; f\u0026#34;terminated in state {self.status.name}\u0026#34; ) player = self.current_player # Solicit the move from the player. A well-behaved player # implementation returns a legal Position; a defective one # might return an illegal Position, and the following # validation catches that case. position = player.make_move(self.board) # Re-validate the returned position against the board. If # the position is illegal, this indicates a defect in the # player implementation, and the resulting exception carries # sufficient information to identify the offending player. if not self.board.is_valid_move(position): raise ValueError( f\u0026#34;Player {player} returned an illegal move at \u0026#34; f\u0026#34;position {position}\u0026#34; ) # Apply the move to the board. self.board.place(player.symbol, position) # Record the move in the history. move = Move(player=player, position=position) self.history.append(move) # Check for a winning line. If a winner is found, transition # to the WON state, identify the winning player, and return # without advancing the turn. winner_symbol = self.win_checker.check(self.board) if winner_symbol is not None: self.status = GameStatus.WON # Locate the player whose symbol matches the winning # symbol. A linear search over the players list is used # because the list is small; a dictionary lookup would # be marginally faster but is not warranted at this scale. self.winner = next( p for p in self.players if p.symbol == winner_symbol ) return # No winner. Check for a drawn game, indicated by a full # board with no winner. if self.board.is_full: self.status = GameStatus.DRAWN return # The game continues. Advance the turn to the next player. self._advance_turn() def play(self) -\u0026gt; None: \u0026#34;\u0026#34;\u0026#34; Execute the game loop, repeatedly calling play_turn until the game reaches a terminal state, and then produce a summary of the outcome. This method is a convenience wrapper around play_turn for callers that wish to run the game to completion in a synchronous, blocking fashion. Callers that require turn-by turn control (for example, a graphical user interface driven by an event loop) should invoke play_turn directly and manage the loop themselves. \u0026#34;\u0026#34;\u0026#34; while not self.is_over: self.play_turn() print(self.board.render()) if self.status == GameStatus.WON: print( f\u0026#34;The game has ended. \u0026#34; f\u0026#34;{self.winner.name} playing {self.winner.symbol.value} \u0026#34; f\u0026#34;is the winner.\u0026#34; ) else: print(\u0026#34;The game has ended in a draw.\u0026#34;) # ------------------------------------------------------------------ # Internal helpers. # ------------------------------------------------------------------ def _advance_turn(self) -\u0026gt; None: \u0026#34;\u0026#34;\u0026#34; Rotate the turn to the next player. The modulo operation makes this method correct for any number of players. The method is separated from play_turn so that the turn-advancement rule is expressed in one place and can be modified independently if the rotation rule ever needs to change (for example, to skip a player who has resigned). \u0026#34;\u0026#34;\u0026#34; self._current_player_index = ( self._current_player_index + 1 ) % len(self.players) Several aspects of this implementation deserve additional commentary.\nThe re-validation of the move returned by the player is a defence in depth. A well-behaved player implementation should never return an illegal move, and the HumanPlayer implementation guarantees this by re-prompting until a legal move is supplied. However, an artificial intelligence player with a bug in its move-selection logic might return an illegal move, and detecting this condition with an informative error message is more helpful than allowing the bug to manifest as an obscure error deep inside the Board class.\nThe winner lookup uses a linear search over the players list rather than maintaining a symbol-to-player dictionary. The reasoning is that the list has at most a handful of entries in any realistic scenario, and the constant factors of dictionary lookup exceed the constant factors of linear search at these sizes. A dictionary would become preferable only if the list grew to hundreds or thousands of entries, which is far outside the scope of any reasonable Tic Tac Toe variant.\nThe move history is populated on every successful move even though no current feature consumes it. This preemptive investment is defensible because the cost is negligible (one list append per move) and because it enables a family of future features (undo, replay, save-and-load, audit) to be added without redesign. Preemptive investment of this kind should be uncommon rather than routine; it is warranted here because the cost-benefit ratio is exceptionally favourable.\nThe separation between play_turn (which executes a single turn) and play (which runs the loop to completion) is a general design discipline: primitive operations are separated from convenience wrappers so that callers with different needs can select the appropriate level of granularity. A caller driving the game from a graphical event loop, for example, would invoke play_turn from an event handler and would not use play at all.\n11. Stage 10: Introducing an AI Player via the Strategy Pattern With the core game infrastructure in place, this section demonstrates how the Player abstraction accommodates artificial intelligence implementations without any modification to the existing code. Two AI implementations are provided: a trivial random player and a perfect-play minimax player. The two implementations differ dramatically in their internal complexity but share the same external interface, illustrating the Strategy pattern\u0026rsquo;s principal benefit.\nThe Random AI Implementation The random AI selects uniformly at random from the set of currently legal moves. This implementation is useful for three purposes: as a stand-in for a human player during automated testing, as a beginner-level opponent in real gameplay, and as a baseline for measuring the strength of more sophisticated implementations.\nimport random class RandomAIPlayer(Player): \u0026#34;\u0026#34;\u0026#34; A player that selects uniformly at random from the currently legal moves. This implementation is suitable for the following purposes: 1. Automated testing that requires a player capable of playing a full game without console input. 2. Beginner-level opposition in interactive gameplay. 3. A baseline against which more sophisticated artificial intelligence implementations can be measured. Time complexity per move: O(N squared) to enumerate the legal moves, plus O(1) for the random selection. \u0026#34;\u0026#34;\u0026#34; def make_move(self, board: Board) -\u0026gt; Position: \u0026#34;\u0026#34;\u0026#34; Enumerate the currently legal positions and return one chosen uniformly at random. \u0026#34;\u0026#34;\u0026#34; # Enumerate the legal positions. A position is legal if the # cell at that position is currently empty; the is_empty_at # method encapsulates both the bounds check and the emptiness # check. legal_positions = [ Position(row, col) for row in range(board.size) for col in range(board.size) if board.is_empty_at(Position(row, col)) ] # The Game class should never invoke make_move when no legal # moves exist, since the game would have terminated as a # draw before that point. A defensive check is included # nonetheless, since the cost is negligible and the benefit # is a clear error message in the event of a defect elsewhere. if not legal_positions: raise RuntimeError( \u0026#34;make_move was invoked but no legal moves are available\u0026#34; ) return random.choice(legal_positions) The Minimax AI Implementation The minimax algorithm is a recursive decision procedure for two-player zero-sum games with perfect information. Tic Tac Toe satisfies all three conditions: it has exactly two players, one player\u0026rsquo;s win is the other\u0026rsquo;s loss (zero sum), and the game state is fully visible to both players (perfect information).\nDefinition of minimax: The algorithm assigns a numerical score to each possible game state, representing the value of that state from the perspective of one player, referred to as the maximiser. Terminal states are scored directly: a win for the maximiser scores +1, a loss scores -1, and a draw scores 0. Non-terminal states are scored recursively: if it is the maximiser\u0026rsquo;s turn, the score of the state is the maximum of the scores of the states reachable by any legal move; if it is the opponent\u0026rsquo;s (the minimiser\u0026rsquo;s) turn, the score is the minimum of those reachable states\u0026rsquo; scores. The maximiser plays optimally by selecting the move that maximises the score, under the assumption that the opponent will also play optimally.\nDefinition of alpha-beta pruning: Alpha-beta pruning is a refinement of minimax that eliminates the evaluation of moves that cannot affect the final decision. The refinement maintains two bounds, conventionally called alpha (the highest score the maximiser is guaranteed to achieve so far) and beta (the lowest score the minimiser is guaranteed to achieve so far). When the score of a partially-evaluated subtree is proven to fall outside the range (alpha, beta), the remainder of the subtree can be skipped without changing the final result. This pruning does not change the correctness of the algorithm; it only reduces the number of states examined.\nFor a three by three Tic Tac Toe board, the minimax algorithm with alpha-beta pruning is fast enough to compute the optimal move in a small fraction of a second, and it guarantees that the AI player will never lose (the AI wins or draws every game). For larger board sizes, minimax becomes computationally infeasible without additional refinements such as depth limits and heuristic evaluation functions, and Monte Carlo tree search or reinforcement learning approaches become preferable.\nclass SmartAIPlayer(Player): \u0026#34;\u0026#34;\u0026#34; A player that selects moves via the minimax algorithm with alpha-beta pruning, achieving perfect play on the three by three board. For the three by three board, the entire game tree is small enough to be exhaustively searched within a fraction of a second. The resulting player never loses: it wins whenever a winning strategy exists, and draws otherwise. For larger boards, minimax becomes computationally infeasible without additional refinements such as depth limits, transposition tables, and heuristic evaluation functions. This implementation is therefore suitable for demonstration purposes on small boards but should not be relied upon for larger variants. \u0026#34;\u0026#34;\u0026#34; def __init__(self, name: str, symbol: Symbol): super().__init__(name, symbol) # The AI uses an internal WinChecker to evaluate hypothetical # board states during the recursive search. Instantiating this # once at construction time and reusing it avoids the cost of # repeated instantiation during the search. self._checker = NInARowChecker() def make_move(self, board: Board) -\u0026gt; Position: \u0026#34;\u0026#34;\u0026#34; Select the move that maximises the score under minimax evaluation. The implementation iterates over every currently legal position, tentatively applies the move, evaluates the resulting board state via minimax, undoes the move, and retains the position that produced the highest score. \u0026#34;\u0026#34;\u0026#34; # Identify the opponent\u0026#39;s symbol. The design assumes exactly # two players; if the design were extended to more players, # the opponent identification would need to be revisited. opponent = Symbol.O if self.symbol == Symbol.X else Symbol.X best_score = float(\u0026#34;-inf\u0026#34;) best_position: Optional[Position] = None for row in range(board.size): for col in range(board.size): position = Position(row, col) if not board.is_empty_at(position): continue # Tentatively apply the move to the board. board.place(self.symbol, position) # Evaluate the resulting position via minimax. The # score reflects the outcome assuming both players # play optimally from this point forward. score = self._minimax( board=board, maximiser_symbol=self.symbol, to_move=opponent, alpha=float(\u0026#34;-inf\u0026#34;), beta=float(\u0026#34;inf\u0026#34;), ) # Undo the tentative move. The Board class does not # currently expose a public undo method, so the AI # manipulates the private state directly. In a # production design, an unplace method should be # added to Board to provide a clean interface. board._grid[row][col] = Symbol.EMPTY board._filled_count -= 1 if score \u0026gt; best_score: best_score = score best_position = position # A well-behaved caller (the Game class) ensures that # make_move is invoked only when at least one legal move # exists, so best_position should always be non-None. The # assertion documents this expected invariant. assert best_position is not None, ( \u0026#34;make_move was invoked with no legal moves available\u0026#34; ) return best_position def _minimax( self, board: Board, maximiser_symbol: Symbol, to_move: Symbol, alpha: float, beta: float, ) -\u0026gt; float: \u0026#34;\u0026#34;\u0026#34; Recursively evaluate the given board position via minimax with alpha-beta pruning. Return value semantics: +1 indicates that the maximiser will win under optimal play. -1 indicates that the maximiser will lose under optimal play. 0 indicates that the game will end in a draw. Parameters: board: the current board state, mutated during recursion. maximiser_symbol: the symbol whose perspective is being optimised. to_move: the symbol of the player whose turn it currently is. alpha: the highest score the maximiser is guaranteed so far. beta: the lowest score the minimiser is guaranteed so far. The alpha and beta bounds are updated as the search progresses, and any subtree whose score cannot fall within the (alpha, beta) range is skipped without further evaluation. \u0026#34;\u0026#34;\u0026#34; # Base case: terminal position. winner = self._checker.check(board) if winner == maximiser_symbol: return 1 if winner is not None: return -1 if board.is_full: return 0 # Recursive case: enumerate legal moves for the player to move # and evaluate each recursively. opponent = Symbol.O if to_move == Symbol.X else Symbol.X is_maximising = (to_move == maximiser_symbol) if is_maximising: best = float(\u0026#34;-inf\u0026#34;) for row in range(board.size): for col in range(board.size): position = Position(row, col) if not board.is_empty_at(position): continue board.place(to_move, position) score = self._minimax( board, maximiser_symbol, opponent, alpha, beta ) board._grid[row][col] = Symbol.EMPTY board._filled_count -= 1 best = max(best, score) alpha = max(alpha, best) if beta \u0026lt;= alpha: # Beta cutoff: no further exploration of this # subtree can improve the maximiser\u0026#39;s outcome # given the minimiser\u0026#39;s guaranteed response. return best return best else: best = float(\u0026#34;inf\u0026#34;) for row in range(board.size): for col in range(board.size): position = Position(row, col) if not board.is_empty_at(position): continue board.place(to_move, position) score = self._minimax( board, maximiser_symbol, opponent, alpha, beta ) board._grid[row][col] = Symbol.EMPTY board._filled_count -= 1 best = min(best, score) beta = min(beta, best) if beta \u0026lt;= alpha: # Alpha cutoff: no further exploration of this # subtree can degrade the minimiser\u0026#39;s outcome # given the maximiser\u0026#39;s guaranteed response. return best return best Two design compromises in this implementation deserve honest acknowledgement.\nThe first compromise is that the AI accesses the private _grid and _filled_count attributes of the Board directly in order to undo a tentative move. This is an encapsulation violation: the Board\u0026rsquo;s private attributes should not be reached across the class boundary. The proper resolution is to add an unplace method to the Board class that reverses a placement in a controlled manner, and to have the AI invoke that method. Practice Question 1 asks the reader to make this improvement.\nThe second compromise is that the opponent identification assumes exactly two players and hard-codes the mapping from X to O and vice versa. Generalising to more than two players would require the AI to hold a reference to the players list and identify the opponent contextually. The current implementation documents this limitation in its docstring.\nThe critical observation is that neither the Game class nor any other existing class requires any modification to accommodate these AI implementations. Both AI subclasses satisfy the abstract Player interface, and constructing a Game with one or two AI players is a matter of passing different Player instances into the Game constructor. This is the Strategy pattern producing its intended benefit: variability of behaviour behind a stable interface.\n12. Stage 11: Observers for Game Events The final major concern of the design is the ability of external components to react to game events. Examples of components that might wish to be notified include logging systems that record moves for audit purposes, user interface components that update in response to game state changes, network broadcasters that relay events to spectators, and statistics collectors that aggregate data across many games.\nThe naive approach is to embed calls to each such component directly within the Game class, but this approach quickly produces a god class as each new listener requires modifications to the Game code. The Observer pattern provides a principled alternative.\nThe Observer Pattern Definition: The Observer pattern is a behavioural design pattern that defines a one-to-many dependency between a subject and its observers, so that when the subject changes state, all registered observers are notified automatically. The pattern decouples the subject from the concrete observer classes by requiring only that observers implement a common notification interface.\nIn this design, the Game class plays the role of the subject, and any class that wishes to react to game events plays the role of an observer. Observers register themselves with the Game via an attach method and are notified via callback methods when events occur.\nThe Design of the Observer Interface Two general variants of the Observer pattern are commonly encountered. The first variant, sometimes called the pull model, defines a single generic notification method (typically update) that observers use to pull whatever information they need from the subject. The second variant, sometimes called the push model, defines multiple event-specific notification methods, each of which receives the relevant event data as arguments.\nThe push model is selected here because games have several distinct events (a move was made, the game ended) that different observers may care about to different degrees. Providing event-specific methods with default no-op implementations allows each concrete observer to implement only the events it cares about, in accordance with the Interface Segregation Principle.\nThe Observer Implementation class GameObserver(ABC): \u0026#34;\u0026#34;\u0026#34; Abstract base class defining the notification interface for components that wish to react to game events. Design rationale: the interface exposes multiple event-specific methods rather than a single generic method, so that concrete observers may implement only those events they care about. Each method is provided with a default no-op implementation, so that subclassing does not require overriding methods that the observer does not use. This design aligns with the Interface Segregation Principle, which holds that clients should not be forced to depend on interfaces they do not use. The Game class does not know the concrete types of its observers; it interacts with them solely through this abstract interface. This decoupling means that new observer types can be added without any modification to the Game class, in accordance with the Open-Closed Principle. \u0026#34;\u0026#34;\u0026#34; def on_move(self, game: \u0026#34;Game\u0026#34;, move: Move) -\u0026gt; None: \u0026#34;\u0026#34;\u0026#34; Called after a legal move has been applied to the board and recorded in the history, but before the terminal-state checks. Default behaviour is a no-op. \u0026#34;\u0026#34;\u0026#34; pass def on_game_end(self, game: \u0026#34;Game\u0026#34;) -\u0026gt; None: \u0026#34;\u0026#34;\u0026#34; Called when the game transitions to a terminal state (WON or DRAWN). Observers may consult game.status to determine which terminal state was reached and game.winner to identify the winning player if applicable. Default behaviour is a no-op. \u0026#34;\u0026#34;\u0026#34; pass class ConsoleLogger(GameObserver): \u0026#34;\u0026#34;\u0026#34; Concrete observer that prints move and outcome information to the console. Suitable for demonstrations and for debugging games in which the players do not themselves produce visible output. \u0026#34;\u0026#34;\u0026#34; def on_move(self, game: \u0026#34;Game\u0026#34;, move: Move) -\u0026gt; None: turn_number = len(game.history) print( f\u0026#34;[log] Turn {turn_number}: \u0026#34; f\u0026#34;{move.player.name} placed {move.player.symbol.value} \u0026#34; f\u0026#34;at {move.position}\u0026#34; ) def on_game_end(self, game: \u0026#34;Game\u0026#34;) -\u0026gt; None: if game.status == GameStatus.WON: print( f\u0026#34;[log] The game has ended. Winner: {game.winner.name}\u0026#34; ) else: print(\u0026#34;[log] The game has ended in a draw.\u0026#34;) Integrating Observers with the Game Class The Game class requires small modifications to support observer registration and notification. The modifications add an observers collection, methods to attach and detach observers, and internal notification helpers that iterate over the observers and invoke the appropriate callback methods.\nclass Game: # ... previously defined content ... def __init__( self, players: list[Player], board_size: int = 3, win_checker: Optional[WinChecker] = None, ): # ... previously defined initialisation ... # The observers collection holds registered listeners. A plain # list is used rather than a set because duplicate registrations # are unlikely in practice and, if they occur, produce # duplicate notifications rather than errors. self._observers: list[GameObserver] = [] def attach(self, observer: GameObserver) -\u0026gt; None: \u0026#34;\u0026#34;\u0026#34; Register the given observer to receive event notifications. Observers registered after game events have already occurred will not receive retroactive notifications for those events. \u0026#34;\u0026#34;\u0026#34; self._observers.append(observer) def detach(self, observer: GameObserver) -\u0026gt; None: \u0026#34;\u0026#34;\u0026#34; Unregister the given observer. If the observer was not previously registered, the method raises ValueError, mirroring the behaviour of list.remove. \u0026#34;\u0026#34;\u0026#34; self._observers.remove(observer) def _notify_move(self, move: Move) -\u0026gt; None: \u0026#34;\u0026#34;\u0026#34; Invoke the on_move callback on every registered observer, passing the current game and the move that was just applied. \u0026#34;\u0026#34;\u0026#34; for observer in self._observers: observer.on_move(self, move) def _notify_end(self) -\u0026gt; None: \u0026#34;\u0026#34;\u0026#34; Invoke the on_game_end callback on every registered observer, passing the current game. \u0026#34;\u0026#34;\u0026#34; for observer in self._observers: observer.on_game_end(self) def play_turn(self) -\u0026gt; None: # ... previously defined content up to and including # history.append(move) ... # Notify observers that a move has been made. self._notify_move(move) # ... win-check block, with the addition of _notify_end() call # before the return statement ... # ... draw-check block, with the addition of _notify_end() call # before the return statement ... The result of these modifications is that any number of observers can be attached to a game without any code change in the Game class, and each observer can react to whichever events it cares about. Testing becomes straightforward because a test can attach a recording observer that captures all events into a list, permitting assertions about the sequence and content of events.\n13. Stage 12: Validating the Design Through Mental Simulation Before declaring a design complete, the designer should walk through the design mentally against a set of representative scenarios. This exercise, sometimes called a design walk-through or a paper simulation, exposes correctness issues that are not apparent from reading the code in isolation. Almost every design has one or more subtle issues that surface only when the design is executed in the mind against realistic inputs.\nScenario 1: A Standard Human Versus Human Game on a Three by Three Board The setup constructs two human players and a game with the default board size, attaches a console logger, and enters the game loop.\nalice = HumanPlayer(\u0026#34;Alice\u0026#34;, Symbol.X) bob = HumanPlayer(\u0026#34;Bob\u0026#34;, Symbol.O) game = Game(players=[alice, bob], board_size=3) game.attach(ConsoleLogger()) game.play() The trace of a single turn proceeds as follows. The Game confirms that the game is in progress. The current player is Alice (index 0). Alice\u0026rsquo;s make_move method displays the board, prompts for input, parses the input as (0, 0), validates the move, and returns Position(0, 0). The Game re-validates the move successfully. The Board places Symbol.X at (0, 0), incrementing the fill count to 1. A Move record is appended to the history. The logger is notified of the move and prints an appropriate line. The WinChecker runs and finds no winning line, since the first move cannot possibly complete a line of three. The board is not full. The turn advances to index 1 (Bob). Control returns to the outer loop, which invokes play_turn again.\nThe trace continues through subsequent turns in the same manner. Eventually, either Alice or Bob completes a winning line, at which point the WinChecker returns the winning symbol, the Game transitions to the WON state, the winner is identified, the logger is notified, and the loop terminates. Alternatively, all nine cells are filled without any winning line being completed, at which point the WinChecker returns None, the fullness check succeeds, the Game transitions to the DRAWN state, the logger is notified, and the loop terminates.\nThis scenario passes.\nScenario 2: A Game That Ends in a Draw Consider a terminal board configuration in which none of the eight winning lines contain three identical marks and every cell is occupied. After the ninth move produces such a configuration, the WinChecker inspects each of the eight lines and finds no line whose three cells all share the same non-EMPTY symbol. The check returns None. The fullness check succeeds because the fill count equals nine, which equals three squared. The Game transitions to the DRAWN state and the loop terminates with the drawn-game message.\nThis scenario passes.\nScenario 3: A Game Won on the Final Move Consider a board state in which eight cells are filled without any winner, and the final available cell is filled with a mark that completes a winning line. The critical concern in this scenario is the order in which terminal conditions are checked.\nAfter the ninth move is applied, the board is now full. If the design were to check fullness first, the fullness check would succeed and the Game would incorrectly transition to the DRAWN state, silently discarding the fact that the same move also completed a winning line. If the design instead checks for a winner first, the WinChecker returns the winning symbol, the Game correctly transitions to the WON state, and the early return prevents the fullness check from executing.\nThe Game class as designed checks the winner first, so this scenario passes. This is a subtle correctness property that would not be visible from a casual inspection of the code, and it is precisely the kind of issue that mental simulation is intended to surface. Documenting this ordering explicitly in the docstring of play_turn serves as a signpost for future maintainers who might otherwise be tempted to reorder the checks for stylistic reasons.\nScenario 4: A Player Implementation Returns an Illegal Move Suppose a defective AI player implementation returns a Position at which the cell is already occupied. The Game\u0026rsquo;s re-validation via board.is_valid_move detects this and raises ValueError with a message identifying the offending player and the illegal position. The message provides sufficient context for the defect to be located and corrected. This scenario passes with the expected behaviour of surfacing the defect rather than hiding it.\nScenario 5: A Move Is Attempted After the Game Has Terminated Suppose the game has already transitioned to the WON state and a caller invokes play_turn again. The guard clause at the entry of play_turn detects that is_over is True and raises GameOverError with a message identifying the terminal status. This scenario passes with the expected behaviour of preventing invalid operations on a terminated game.\nScenario 6: Two Players Are Constructed with the Same Symbol Suppose the Game constructor receives two Player instances both holding Symbol.X. The constructor detects that the distinct-symbols invariant is violated and raises ValueError before any gameplay begins. This scenario passes because the invariant is checked at the earliest possible moment.\nScenario 7: A Board of Size Less Than Three Is Requested Suppose the Board constructor receives a size of two. The constructor detects that the minimum-size invariant is violated and raises ValueError. This scenario passes.\nSummary of the Validation Exercise The design passes all seven scenarios, and the exercise has surfaced one non-obvious correctness property (the ordering of the winner check and the fullness check) that the design was constructed to honour deliberately. The exercise has also confirmed that the design\u0026rsquo;s error-handling strategy consistently surfaces defects at their source rather than concealing them.\nOne trust boundary is worth noting explicitly: the design assumes that a Player implementation does not mutate the board during its make_move invocation. A rogue implementation that placed marks directly on the board would corrupt the game state, since the Game class would subsequently apply the returned move on top of the already-modified board. Enforcing this invariant would require passing an immutable view of the board to the player rather than the mutable Board instance itself. This hardening is not implemented in the current design but is noted as a potential future improvement.\n14. Stage 13: Anticipating Follow-Up Questions Interviewers commonly extend the initial problem with follow-up questions that probe the design\u0026rsquo;s extensibility. This section addresses the follow-up questions most likely to arise and sketches the design responses to each.\nQuestion: How would the design support undo of the most recent move? The design already maintains a move history, so undo requires only the ability to reverse a placement on the board and to restore the previous turn state. A conceptual implementation of an undo method is as follows:\ndef undo(self) -\u0026gt; None: \u0026#34;\u0026#34;\u0026#34; Reverse the most recent move, restoring the board and turn state to their previous values. If no move has been made, raise ValueError. \u0026#34;\u0026#34;\u0026#34; if not self.history: raise ValueError(\u0026#34;There is no move to undo\u0026#34;) last_move = self.history.pop() # Reverse the placement on the board. In the current design this # requires accessing private attributes of the Board, which is an # encapsulation violation. Adding a public Board.unplace method # would resolve this cleanly. self.board._grid[last_move.position.row][last_move.position.col] = Symbol.EMPTY self.board._filled_count -= 1 # Roll back the turn indicator so that the player whose move was # undone is now the current player again. self._current_player_index = ( self._current_player_index - 1 ) % len(self.players) # Reset the terminal state, since the undone move may have been # the one that ended the game. self.status = GameStatus.IN_PROGRESS self.winner = None A more principled implementation would use the Command pattern, in which each move is represented as a Command object with execute and undo methods. The Game would maintain a stack of executed commands, and undo would pop the most recent command and invoke its undo method. This design generalises to arbitrary undo depth and to redo operations.\nQuestion: How would the design support saving and loading of games? Two design approaches are available. The first serialises the current game state (board grid, turn index, players, status, and history) to a persistent format such as JSON. Loading reconstructs the game from the serialised state. The advantage is that saving and loading are constant-time operations regardless of game length. The disadvantage is that the serialised state includes all derived state, which can become inconsistent if the format changes across versions.\nThe second approach serialises only the sequence of moves. Loading reconstructs the game by replaying the moves from an empty starting position. The advantage is that the serialised representation is minimal and self-consistent by construction. The additional benefit is that the replay mechanism required for loading is also useful for demonstrating games to spectators. The disadvantage is that loading takes time proportional to the number of moves, though this is negligible in practice.\nThe second approach is preferred for its elegance and additional capability.\nQuestion: How would the design generalise to K-in-a-row on an N-by-N board? The K-in-a-row variant permits the winning line length K to differ from the board dimension N. This variant enables games such as Gomoku, which uses K equal to five on an N by N board of any size at least five.\nThe generalisation is implemented by writing a new WinChecker subclass named KInARowChecker that accepts K as a constructor parameter. The algorithm for this variant iterates over every cell as a potential starting point, and for each starting point examines the K cells extending in each of four forward directions (right, down, down-right, down-left). If any such K-cell sequence lies entirely within the board and contains the same non-EMPTY symbol in every cell, that symbol is the winner. The full implementation appears as Practice Question 3.\nThe Game constructor accepts the new WinChecker via its existing win_checker parameter. No modification of the Game, Board, or Player classes is required. This is the Strategy pattern demonstrating its intended benefit of accommodating new algorithms without disturbing the surrounding code.\nQuestion: How would the design support more than two players? The Game class already supports any number of players via its index-based turn rotation. Three modifications are required to complete the generalisation. First, the Symbol enumeration must be extended to include additional values (for example, TRIANGLE, STAR) so that each additional player has a distinct symbol. Second, the AI implementations must be revised so that opponent identification does not hard-code the two-player X-versus-O assumption; instead, the AI must consult the players list to identify all opponents. Third, the WinChecker interface is unchanged, since the win condition remains that any player\u0026rsquo;s symbol form a winning line.\nA middle-ground approach that avoids extending the Symbol enumeration is to promote Symbol from an enumeration to a proper class that permits construction of new symbol values at runtime with arbitrary labels. This approach loses the type-safety benefits of enumerations but gains flexibility. The choice depends on how many players the extended design must support and whether the set of players is known at compile time.\nQuestion: How would the design support networked multiplayer? Networked multiplayer is implemented by introducing a new Player subclass called NetworkedPlayer, whose make_move method blocks on a socket read for the remote client\u0026rsquo;s move. From the Game class\u0026rsquo;s perspective, this player is indistinguishable from a HumanPlayer or an AIPlayer: the Player abstraction handles the variation.\nOn the server side, the Game runs to completion, and each move triggers a notification to registered observers. A NetworkBroadcastObserver would be attached to the Game to relay events to all connected spectator clients. The Observer pattern accommodates this addition without any change to the Game class.\nAdditional concerns arise in a networked context that are outside the scope of the LLD as presented: session management, authentication, disconnection handling, and consistency guarantees. These are properly addressed at a higher level of the system architecture.\nQuestion: How would the design be tested? The design is highly amenable to unit testing because of its dependency injection and its separation of concerns. The following testing strategies are available.\nThe Board class is tested by constructing a board, applying a sequence of placements, and asserting on the resulting state via the query methods. Invalid operations are tested by asserting that the appropriate exceptions are raised.\nThe WinChecker implementations are tested as pure functions of board state. A test constructs a board with specific cell values, invokes the checker\u0026rsquo;s check method, and asserts on the returned value. No mock objects are required because the checker has no external dependencies.\nThe Game class is tested with a helper class named ScriptedPlayer, which returns a pre-programmed sequence of moves rather than reading from console input. The full implementation appears as Practice Question 2. A test constructs a game with two ScriptedPlayers, plays the game to completion, and asserts on the final status and winner.\nObserver behaviour is tested with a RecordingObserver that captures every callback into a list. A test attaches the recorder, plays a game, and asserts on the sequence of recorded events.\nQuestion: How would the AI difficulty be made configurable? Configurable difficulty is implemented by nesting a Strategy pattern within the AI player. The SmartAIPlayer accepts a MoveSelectionStrategy at construction time, and the strategy determines how moves are computed. Possible strategies include random selection, heuristic evaluation, minimax with a specified depth limit, and Monte Carlo tree search with a specified iteration count. Difficulty corresponds to the choice of strategy.\nThis design has the interesting property that Strategy is applied at two levels: the Player is a strategy for how moves are chosen, and within an AI player the MoveSelectionStrategy is a further strategy for how the AI decides. This nesting is not gratuitous; each level of Strategy addresses a distinct axis of variation.\n15. Practice Questions with Detailed Solutions The following exercises are provided to deepen understanding of the concepts covered in this document. Attempt each question independently before consulting the accompanying solution.\nPractice Question 1: Introduce a Public unplace Method on Board Task: The SmartAIPlayer.make_move method as currently implemented accesses the private _grid and _filled_count attributes of the Board to reverse a tentative placement. This is an encapsulation violation. Introduce a public unplace method on the Board class that reverses a placement in a controlled manner, and refactor the AI implementation to use this method instead of accessing private state directly. The unplace method should validate its argument and raise ValueError on invalid input.\nSolution The unplace method is symmetric to the place method in its validation discipline: it verifies that the position is within bounds and currently occupied, and it raises ValueError if either condition is violated. The method returns the symbol that was removed, which is useful for undo stacks that may need to know what was removed.\ndef unplace(self, position: Position) -\u0026gt; Symbol: \u0026#34;\u0026#34;\u0026#34; Reverse a previous placement, restoring the cell at the given position to EMPTY. Return the symbol that was removed. The method validates its argument in the same manner as place: the position must be within bounds, and the cell must currently be occupied. Raising ValueError on invalid input maintains symmetry with place and ensures that programming defects surface at their source. \u0026#34;\u0026#34;\u0026#34; if not self.is_in_bounds(position): raise ValueError( f\u0026#34;Position {position} is out of bounds for a board \u0026#34; f\u0026#34;of size {self.size}\u0026#34; ) if self.is_empty_at(position): raise ValueError( f\u0026#34;Position {position} is already empty and cannot be \u0026#34; f\u0026#34;unplaced\u0026#34; ) removed_symbol = self._grid[position.row][position.col] self._grid[position.row][position.col] = Symbol.EMPTY self._filled_count -= 1 return removed_symbol The AI implementation is refactored to use this method, eliminating the direct access to private attributes:\n# Within SmartAIPlayer.make_move: board.place(self.symbol, position) score = self._minimax(...) board.unplace(position) # replaces direct manipulation of _grid and _filled_count The benefit of this refactoring extends beyond the immediate encapsulation improvement. The Board\u0026rsquo;s private attributes are now genuinely private, meaning that any change to the internal representation of the grid does not require corresponding changes in the AI. If a future revision were to replace the two-dimensional list representation with a one-dimensional list or a dictionary, the AI would continue to function without modification because it interacts with the Board solely through the public interface.\nPractice Question 2: Implement a ScriptedPlayer for Testing Task: Implement a ScriptedPlayer class that plays a pre-programmed sequence of moves rather than reading from console input or computing moves. The class is intended for use in deterministic unit tests, where reproducibility requires that the sequence of moves be fully specified in advance. If the player is asked to produce more moves than were specified at construction time, the class should raise a clear error.\nSolution The ScriptedPlayer accepts the move sequence as a constructor parameter and returns moves in order on successive invocations of make_move. Running out of scripted moves indicates that the test scenario is misconfigured, so the method raises RuntimeError with a descriptive message.\nclass ScriptedPlayer(Player): \u0026#34;\u0026#34;\u0026#34; A player that returns moves from a pre-programmed sequence, in order. Suitable for unit tests that require deterministic and reproducible gameplay. If the player is asked to produce more moves than were specified at construction time, the make_move method raises RuntimeError. This condition indicates either a test scenario that runs longer than expected (which may reveal a bug in the code under test) or a script that is too short (which is a defect in the test itself). \u0026#34;\u0026#34;\u0026#34; def __init__( self, name: str, symbol: Symbol, moves: list[Position], ): super().__init__(name, symbol) # A shallow copy of the moves list is stored so that # subsequent modification of the caller\u0026#39;s list does not # affect the player. Position instances are frozen dataclasses, # so a shallow copy is safe. self._moves = list(moves) self._index = 0 def make_move(self, board: Board) -\u0026gt; Position: if self._index \u0026gt;= len(self._moves): raise RuntimeError( f\u0026#34;ScriptedPlayer {self.name} was asked for move number \u0026#34; f\u0026#34;{self._index + 1} but only {len(self._moves)} moves \u0026#34; f\u0026#34;were provided at construction time\u0026#34; ) next_move = self._moves[self._index] self._index += 1 return next_move A representative unit test using this class demonstrates how the design\u0026rsquo;s separation of concerns enables straightforward deterministic testing:\ndef test_x_wins_via_top_row(): \u0026#34;\u0026#34;\u0026#34; Verify that a game in which X completes the top row results in the correct final status and winner. \u0026#34;\u0026#34;\u0026#34; x_moves = [Position(0, 0), Position(0, 1), Position(0, 2)] o_moves = [Position(1, 0), Position(1, 1)] alice = ScriptedPlayer(\u0026#34;Alice\u0026#34;, Symbol.X, x_moves) bob = ScriptedPlayer(\u0026#34;Bob\u0026#34;, Symbol.O, o_moves) game = Game(players=[alice, bob]) game.play() assert game.status == GameStatus.WON, ( f\u0026#34;Expected WON, got {game.status}\u0026#34; ) assert game.winner is alice, ( f\u0026#34;Expected alice as winner, got {game.winner}\u0026#34; ) The test exercises the entire game loop end-to-end without any human interaction and without any randomness, producing the same outcome on every run.\nPractice Question 3: Implement KInARowChecker for Gomoku Variants Task: Implement a KInARowChecker class that generalises the win condition to permit winning lines of length K on a board of dimension N, where K may be less than or equal to N. This class enables variants such as Gomoku, which uses K equal to five on a fifteen by fifteen board. The class should be a drop-in replacement for NInARowChecker and should require no modification of any other class in the design.\nSolution The algorithm iterates over every cell as a potential starting point for a winning line. For each starting point, it examines the K cells extending in each of four forward directions: right (0, 1), down (1, 0), down-right (1, 1), and down-left (1, -1). The four backward directions (left, up, up-left, up-right) do not need to be examined separately, because any line extending backward from cell A also extends forward from some other cell B, and B will be examined as its own starting point in the outer iteration.\nclass KInARowChecker(WinChecker): \u0026#34;\u0026#34;\u0026#34; Concrete WinChecker for the generalised K-in-a-row win rule, where K may be less than or equal to the board dimension. Example uses: Standard three by three Tic Tac Toe: KInARowChecker(k=3) on a board of size 3. Gomoku: KInARowChecker(k=5) on a board of size 15. Algorithm: iterate over every cell as a potential starting point for a winning line. For each starting cell that is not EMPTY, examine the K cells extending in each of four forward directions (right, down, down-right, down-left). If any such K-cell sequence lies entirely within the board and contains the same non-EMPTY symbol in every cell, that symbol is the winner. The four backward directions do not need to be examined separately because they are covered by starting from a different cell. Time complexity: O(N squared * 4 * K) per invocation, which is equal to O(N squared * K). \u0026#34;\u0026#34;\u0026#34; # The four forward directions, expressed as (row-delta, col-delta) # tuples. Backward directions are implicitly covered by starting # from the other end of the line. _DIRECTIONS = [(0, 1), (1, 0), (1, 1), (1, -1)] def __init__(self, k: int): if k \u0026lt; 2: raise ValueError( f\u0026#34;The win length K must be at least 2, received {k}\u0026#34; ) self.k = k def check(self, board: Board) -\u0026gt; Optional[Symbol]: n = board.size # If K exceeds the board dimension, no winning line of length # K can fit on the board and no winner is possible. if self.k \u0026gt; n: return None for row in range(n): for col in range(n): starting_symbol = board.get(Position(row, col)) if starting_symbol == Symbol.EMPTY: continue for direction_row, direction_col in self._DIRECTIONS: if self._line_matches( board, row, col, direction_row, direction_col, starting_symbol, ): return starting_symbol return None def _line_matches( self, board: Board, start_row: int, start_col: int, direction_row: int, direction_col: int, target_symbol: Symbol, ) -\u0026gt; bool: \u0026#34;\u0026#34;\u0026#34; Return True if the K cells starting at (start_row, start_col) and stepping by (direction_row, direction_col) all contain target_symbol and all lie within the board. \u0026#34;\u0026#34;\u0026#34; n = board.size for step in range(self.k): current_row = start_row + step * direction_row current_col = start_col + step * direction_col if not (0 \u0026lt;= current_row \u0026lt; n and 0 \u0026lt;= current_col \u0026lt; n): return False if board.get(Position(current_row, current_col)) != target_symbol: return False return True Constructing a Gomoku game with this checker requires no modification of the Game, Board, or Player classes:\ngame = Game( players=[HumanPlayer(\u0026#34;Alice\u0026#34;, Symbol.X), HumanPlayer(\u0026#34;Bob\u0026#34;, Symbol.O)], board_size=15, win_checker=KInARowChecker(k=5), ) The Game class accepts the new WinChecker via its existing constructor parameter. This exercise concretely demonstrates the value of designing WinChecker as a Strategy interface from the outset: a substantial new game variant is accommodated by writing a single new class, with zero modification of the surrounding code.\nPractice Question 4: Add an on_invalid_move Observer Event Task: Extend the GameObserver interface with a new event method called on_invalid_move that is invoked when a player returns an illegal move. Observers should receive the game, the offending player, and the illegal position as arguments. Wire this event into Game.play_turn so that observers are notified before the Game raises its ValueError.\nSolution The new event method is added to the abstract base class with a default no-op implementation, so that existing observer subclasses continue to function without modification. The Game\u0026rsquo;s play_turn method is amended to notify observers of the invalid-move event before raising the exception.\nclass GameObserver(ABC): def on_move(self, game: \u0026#34;Game\u0026#34;, move: Move) -\u0026gt; None: pass def on_game_end(self, game: \u0026#34;Game\u0026#34;) -\u0026gt; None: pass def on_invalid_move( self, game: \u0026#34;Game\u0026#34;, player: Player, position: Position, ) -\u0026gt; None: \u0026#34;\u0026#34;\u0026#34; Called when a player\u0026#39;s make_move method returns an illegal move. Observers may use this event for logging, metrics, or alternative recovery strategies. Default behaviour is a no-op. \u0026#34;\u0026#34;\u0026#34; pass The Game\u0026rsquo;s play_turn method is amended as follows:\ndef play_turn(self) -\u0026gt; None: if self.is_over: raise GameOverError( f\u0026#34;Cannot play a turn because the game has already \u0026#34; f\u0026#34;terminated in state {self.status.name}\u0026#34; ) player = self.current_player position = player.make_move(self.board) if not self.board.is_valid_move(position): # Notify observers before raising, so that observers may log # the event, update statistics, or trigger alternative # recovery logic before the exception propagates. for observer in self._observers: observer.on_invalid_move(self, player, position) raise ValueError( f\u0026#34;Player {player} returned an illegal move at \u0026#34; f\u0026#34;position {position}\u0026#34; ) # ... remainder of the method is unchanged ... This extension illustrates how the Observer pattern shades into event-driven policy. A concrete observer might, for example, log the invalid-move event for later analysis, or it might implement a retry policy that solicits a corrected move from the offending player before allowing the exception to propagate. Either policy can be introduced without any further modification to the Game class, because the point of extension is the Observer interface.\nPractice Question 5: Critique the Current Design Task: Identify three specific weaknesses in the design as it stands, and for each, describe how the weakness could be addressed. Consider issues of encapsulation, extensibility, testability, and general software engineering discipline.\nSolution Weakness 1: Encapsulation violation in the SmartAIPlayer minimax implementation.\nThe SmartAIPlayer accesses the private _grid and _filled_count attributes of the Board directly in order to undo tentative placements during the minimax search. This coupling is fragile because any future change to the Board\u0026rsquo;s internal representation would break the AI without any warning at the Board interface level. The correct resolution is to introduce a public unplace method on the Board class, as detailed in Practice Question 1, and refactor the AI to use that method exclusively.\nWeakness 2: The Symbol enumeration limits the design to two or three players.\nThe Symbol enumeration currently contains three values: EMPTY, X, and O. Adding support for more players would require either extending the enumeration (which is a modification of the type and therefore affects all code that depends on it) or promoting Symbol from an enumeration to a class that permits construction of new symbol values at runtime. The first approach is simpler but less flexible; the second approach is more flexible but loses the compile-time type safety that enumerations provide.\nA pragmatic middle ground is to extend the enumeration with a modest set of additional values (for example, up to eight symbols) and to document that the design supports at most that many players. This approach preserves type safety while accommodating small variations. A design that must support arbitrary numbers of dynamically-created player symbols should adopt the class-based approach.\nWeakness 3: The Game.play method performs input and output directly.\nThe Game.play method contains print statements that render the final board and announce the outcome. This directly couples the Game class to the console output channel, which is inconsistent with the general design principle of keeping input and output at the edges of the system rather than embedded in the core logic.\nThe resolution is to remove the print statements from Game.play and to introduce a ConsoleRenderer observer that reacts to on_game_end events by producing the final rendering. Callers who wish to render the game to the console attach a ConsoleRenderer; callers who wish to render it to a graphical user interface or a network channel attach a different observer. The Game class becomes independent of the output surface, and the same Game logic can be used with any presentation mechanism.\nA related weakness that could be listed as a fourth item is that the history attribute is exposed as a mutable list rather than through a getter method that returns a copy. External code that iterates over the history could inadvertently modify it, corrupting the game\u0026rsquo;s audit trail. Introducing a get_history method that returns a defensive copy of the list resolves this issue while preserving the ability of external code to inspect the history.\n16. Summary Tables and Key Takeaways This section consolidates the design into reference tables and summarises the principal lessons of the exercise.\nClass Inventory Component Kind Role Symbol Enumeration The three possible cell values: X, O, EMPTY Position Frozen dataclass A coordinate pair identifying a cell Move Frozen dataclass A historical record of one completed move GameStatus Enumeration The three possible game states GameOverError Exception Raised when a move is attempted on a terminated game Board Class Owner of the grid state and enforcer of move legality Player Abstract base class The contract that all player types must satisfy HumanPlayer Concrete subclass A player driven by console input RandomAIPlayer Concrete subclass A player that selects moves uniformly at random SmartAIPlayer Concrete subclass A player that selects moves via minimax WinChecker Abstract base class The contract for win-detection algorithms NInARowChecker Concrete subclass The standard N-in-a-row win rule KInARowChecker Concrete subclass (Practice Q3) The generalised K-in-a-row win rule GameObserver Abstract base class The contract for components that react to game events ConsoleLogger Concrete subclass An observer that logs events to the console Game Class The orchestrator that composes all of the above Design Patterns Applied Pattern Location in the Design Justification Strategy Player hierarchy, WinChecker hierarchy Player behaviour and win rules are both axes of variation that benefit from being encapsulated behind stable interfaces. Observer GameObserver interface and Game.attach mechanism External reactions to game events (logging, presentation, networking) should be decoupled from the Game class to avoid the accumulation of responsibilities. Enumeration with guard clauses GameStatus and play_turn guard The state-dependent behaviour of the game is limited enough that a full State pattern would introduce disproportionate structural overhead. SOLID Principles Applied Principle Where It Applies Single Responsibility Principle Each class has one focused responsibility: Board manages the grid, Player selects moves, WinChecker evaluates win conditions, Game coordinates the loop. Open-Closed Principle New player types and new win rules are added by writing new subclasses rather than by modifying existing classes. Liskov Substitution Principle Any Player subclass may be substituted wherever a Player is expected without altering the correctness of the Game. The same holds for WinChecker and GameObserver. Interface Segregation Principle The GameObserver interface provides multiple event-specific methods with default no-op implementations, so that observers implement only the events they care about. Dependency Inversion Principle The Game class depends on the abstract Player and WinChecker interfaces rather than on their concrete implementations, permitting substitution at construction time. Key Design Decisions and Justifications Decision Justification Two-dimensional list for the board grid Matches the natural mental model of the problem, produces readable code, and performs adequately for the scales under consideration. Symbol enumeration includes EMPTY Uniform typing across all cell operations, avoiding the overhead of Optional wrapping. Position as a frozen dataclass Provides named attributes, immutability, hashability, and extensibility with negligible ceremony. Player as an abstract class hierarchy Different player types have materially different auxiliary state and behaviour, justifying separate classes rather than a single class with a callback. WinChecker as a separate Strategy Win rules are an anticipated axis of variation; separating the algorithm behind an interface permits new rules without modification of existing code. GameStatus as an enumeration rather than State pattern The state-dependent behaviour is limited to a single guard clause, insufficient to justify the structural overhead of a class per state. Move history maintained despite the absence of a current consumer The cost is negligible and the presence of history enables a family of future features without redesign. Observer for event notification, Strategy for behaviour variation Clear separation between two different extension mechanisms: Observer for reactions to events, Strategy for pluggable algorithms. The Universal LLD Framework The framework applied in this document is universal across LLD problems. It consists of the following stages, executed in order:\nClarify. Never assume; ask structured questions to establish functional requirements, non-functional requirements, and out-of-scope declarations. Enumerate entities. Extract nouns from the requirements and filter them through the identity-behaviour-domain test to identify candidate classes. Decide with alternatives. For each significant design choice, enumerate the plausible alternatives and articulate the reasoning that leads to the selected option. Code with intent. Write implementations whose comments explain the reasons for the design decisions rather than merely describing what the code does. Validate by simulation. Walk through the design mentally against representative scenarios, including edge cases, and document any issues found. Anticipate extensions. Describe how the design would accommodate likely follow-up requirements and identify the extension points that make those accommodations possible. Internalising this framework is more valuable than memorising any specific design, because the framework applies to arbitrary future problems whereas the specific design applies only to Tic Tac Toe.\n17. Final Reflection on the Design Process The exercise of designing Tic Tac Toe well is instructive precisely because the problem sounds too small to reward significant design effort. The temptation to produce a short procedural implementation is strong, and the candidate who succumbs to that temptation has communicated something specific about their engineering discipline. The candidate who resists the temptation and produces a properly abstracted design has communicated something quite different.\nSeveral general lessons emerge from the exercise.\nRestraint Is a Design Skill Throughout the design, decisions were made to omit classes that would not have contributed value at the current scope. Cell was omitted because it would have been a wrapper around a single enumeration value. Turn was omitted because it is a piece of state rather than an entity. Score was omitted because the scope specified only a single game. Tournament-related structures were omitted for the same reason. Each of these omissions is a design decision, not an oversight, and articulating the reason for each demonstrates that the designer is exercising judgement rather than accumulating classes reflexively.\nOver-design and under-design are both failures. The correct middle path is characterised by adding structure only when its value can be articulated, and by removing structure when its cost exceeds its benefit. The phrase \u0026ldquo;I considered adding class X but decided against it because Y\u0026rdquo; carries as much evidential weight as the phrase \u0026ldquo;I chose to add class X because Y.\u0026rdquo;\nStrategy Pattern Rewards Foresight The Player and WinChecker abstractions were introduced as abstract classes from the beginning of the design, well before any concrete need for variation existed. The value of that foresight materialised in Stage 10, when two AI player implementations were introduced without any modification to the surrounding code, and in Practice Question 3, when a K-in-a-row win rule was introduced without any modification to the surrounding code.\nThe trigger for applying the Strategy pattern is not the presence of current variation but the anticipation of future variation. Recognising the likelihood of future variation is a distinct skill from responding to current variation, and it is the more valuable of the two skills because it prevents rework rather than merely accommodating it.\nPattern Application Is a Matter of Judgement The State pattern was explicitly rejected in favour of an enumeration with a guard clause. This decision would have been the wrong one for the Elevator design, where the states admit substantially different behaviour, but it is the right decision for the Tic Tac Toe design, where the state-dependent behaviour is limited to a single conditional check. The ability to make this distinction is more valuable than the ability to apply a pattern mechanically wherever it fits syntactically.\nDesign patterns are tools that solve specific classes of problems. Applying a pattern where its problem does not exist introduces overhead without benefit, and it clutters the code with structure that will need to be maintained and explained despite serving no purpose. The candidate who declines to apply a pattern, and articulates the reason for the declining, demonstrates deeper understanding than the candidate who applies every pattern that could plausibly fit.\nMental Simulation Surfaces Non-Obvious Correctness Issues The Stage 12 validation exercise surfaced a correctness property of the design that is not obvious from reading the code: the winner check must precede the fullness check, or a game won on the final move will be misclassified as a draw. This property is documented in the docstring of play_turn, but the property itself was surfaced only by mentally simulating a specific scenario. Correctness issues of this kind are common in designs that have not been walked through, and they are often responsible for the most embarrassing production defects.\nMental simulation is inexpensive relative to the debugging cost of the defects it prevents. The discipline of walking through the design against representative scenarios should be routine, not an afterthought.\nCross-Problem Fluency Is the Ultimate Goal The design decisions in this document mirror the decisions made in earlier problems in the series. The Player hierarchy is structurally similar to the Vehicle hierarchy in the Parking Lot design. The WinChecker Strategy is structurally similar to the PricingStrategy in the same design. The GameObserver interface is structurally similar to the FloorObserver interface used with the Parking Lot\u0026rsquo;s display boards. The pattern of validation via exceptions, the pattern of Enum-based status representation, and the pattern of composition-based orchestration all recur across the four designs that have been developed so far.\nThis recurrence is not a coincidence. It is the essence of design fluency: the ability to recognise the same structural patterns across superficially different problems and to apply the same principles to their solutions. A candidate who has internalised this fluency does not need to prepare for LLD interviews as such; the candidate simply does LLD, and each new problem is a variation on themes already understood.\nThe remaining problems in this series (Vending Machine, Splitwise, Ride-Hailing, and others) will add further concepts to the accumulated body of understanding: inventory management with concurrent access, graph-based relationship modelling, distributed state management, and real-time matching. As each new problem is worked through, the guiding question should be: what is new here, and what is a recurrence of a pattern already understood? The framework remains constant; the specific applications vary. Fluency is the ability to distinguish the invariant from the variable.\nThis content is part of Codeverra, a platform for learning coding, data science, DSA, and AI from scratch. Explore more at: https://codeverra.com\n","permalink":"https://blogs.codeverra.com/blog/low_level_design/interview_questions/06d_lld_tic_tac_toe_deep_dive/","summary":"\u003ch1 id=\"lld-deep-dive-4-designing-a-tic-tac-toe-game\"\u003eLLD Deep Dive #4: Designing a Tic Tac Toe Game\u003c/h1\u003e\n\u003cblockquote\u003e\n\u003cp\u003e\u003cstrong\u003eWhat this document covers:\u003c/strong\u003e A rigorous, step-by-step examination of how to approach a Tic Tac Toe Low-Level Design (LLD) problem in a technical interview. The document walks through the reasoning process at each design juncture, evaluates competing alternatives, and provides justification for every decision made. The finished design is a consequence of the reasoning, not its starting point.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eHow to read this document:\u003c/strong\u003e Read it slowly and deliberately. The problem appears trivial at the surface, since Tic Tac Toe requires only a three by three grid, two players, and a rule for detecting three marks in a row. That surface simplicity is precisely why the problem is used in interviews: it invites the unwary candidate to produce a short procedural script, when the actual expectation is a properly modelled, extensible, and testable object-oriented design.\u003c/p\u003e","tags":null,"title":""},{"content":"LLD Deep Dive #4: Designing a Grocery / E-Commerce Management System What this document is: A step-by-step walkthrough of the reasoning process required to design an e-commerce LLD problem in an interview setting. The specific flavour is a grocery store, though the design generalises to any online retail system. Every design decision is derived from first principles, alternatives are evaluated at each fork, and the resulting system is one whose components can be explained, defended, and extended.\nWhy this problem is different: The Parking Lot case study addressed containment hierarchies. The Elevator case study addressed state machines. The Library case study addressed policy-driven modelling. E-commerce combines all three approaches while adding payments, notifications, and cross-cutting concerns such as inventory. It is a system in the true sense of the term, not a single mechanism.\nTime investment: Substantial. Read the document across two sessions if necessary. When preparing for interviews, treat every \u0026ldquo;Option A / B / C\u0026rdquo; decision point as your own decision; pause and select an option before proceeding to the analysis.\nTable of Contents Why This Problem Matters Stage 1: Receiving the Problem Stage 2: Clarifying Requirements Stage 3: Identifying Entities and Filtering Rigorously Stage 4: Modeling Products and the Catalog Stage 5: Inventory and the Placement of Stock Stage 6: The Shopping Cart Stage 7: Pricing, Discounts, and Promotions Stage 8: The Order Lifecycle Stage 9: Payment Processing Stage 10: Shipping and Delivery Stage 11: Notifications Stage 12: The Top-Level Coordinator Stage 13: The Complete Class Diagram Stage 14: Sequence Diagram for the Checkout Flow Stage 15: Modular Code Structure Stage 16: Validating the Design Through a Mental Walkthrough Stage 17: Anticipating Follow-Up Questions Final Reflection: What This Problem Teaches Practice Questions Summary and Key Takeaways 1. Why This Problem Matters E-commerce ranks among the most frequently posed large-scope LLD problems because a single design exercises nearly every classical design pattern. The problem tests domain modelling through products, categories, and inventory, where the designer must determine which class owns which fact and where state resides. The cart introduces mutable, per-user, transient state that is small in scope but frequently designed incorrectly. Pricing, discounts, and promotions require variation without modification of the core, which makes them natural candidates for the Strategy and Decorator patterns.\nThe order lifecycle, comprising the sequence of statuses (placed, confirmed, packed, shipped, delivered, returned), is a textbook application of the State pattern. Payment methods and shipping methods each demand independent variation and consequently constitute separate Strategy families. Notifications for order updates delivered via email, SMS, or push channels represent the canonical use case for the Observer pattern. The interconnection of all these concerns tests whether responsibilities can remain cleanly separated across many collaborators.\nA well-executed design of this problem demonstrates every core skill an LLD interviewer expects to see. A poorly executed design, typified by a single god class named OrderService that absorbs every responsibility, demonstrates the opposite.\nThe grocery variant introduces two constraints worth noting. First, weight-based products (for example, 1 kg of tomatoes) coexist with unit-based products (such as one packet of biscuits), which tests whether the Product and CartItem abstractions model quantities cleanly across both cases. Second, perishability and shelf-life considerations exist, though they are largely outside the scope of LLD; the topic serves as a useful clarifying question rather than a required design element.\nDefinitions used throughout this document:\nEntity: A domain concept with identity that persists across states, such as Customer or Order. Value object: A domain concept defined entirely by its attributes, without independent identity, such as Address or Money. Strategy pattern: A behavioural pattern in which a family of algorithms is defined behind a common interface, enabling the algorithm to vary independently of the clients that use it. State pattern: A behavioural pattern in which an object\u0026rsquo;s behaviour changes when its internal state changes, achieved by delegating state-dependent behaviour to separate state classes. Observer pattern: A behavioural pattern in which one object (the subject) maintains a list of dependents (observers) and notifies them of state changes. YAGNI: \u0026ldquo;You Aren\u0026rsquo;t Gonna Need It\u0026rdquo;, a principle advising against implementing functionality until it is actually required. DIP (Dependency Inversion Principle): High-level modules should depend on abstractions rather than on concrete implementations. OCP (Open/Closed Principle): Software entities should be open for extension but closed for modification. 2. Stage 1: Receiving the Problem The interviewer presents the following statement:\n\u0026ldquo;Design an online grocery store.\u0026rdquo;\nThe statement provides no information regarding scale, team composition, or scope boundaries. Multiple interpretations are consistent with this prompt. The system could be a single-store site or a marketplace hosting many independent sellers. Inventory could be owned directly or aggregated from partners. Delivery could operate on a real-time model (as with Blinkit) or a next-day model (as with BigBasket). Payments could be handled internally or delegated to an external gateway. Access could be limited to a web interface or extended across web, mobile, and point-of-sale terminals.\nSince none of these variables is known, none should be assumed. The correct first action, consistent across every LLD problem, is to pause and clarify the requirements before committing to any design decision.\n3. Stage 2: Clarifying Requirements A structured clarification round organises questions into three categories: functional requirements (what the system does), non-functional requirements (which qualities the system must exhibit), and out-of-scope items (which capabilities the system will explicitly not provide).\nQuestion 1: Single store or marketplace? Rationale for asking: A marketplace architecture introduces Seller as a first-class entity, requires per-seller inventory tracking, split payment flows, and per-seller shipping calculations. A single-store architecture is substantially simpler.\nAssumed answer: \u0026ldquo;Single store; the platform owns the inventory.\u0026rdquo;\nThis answer eliminates the Seller entity and split-payment logic from the design scope.\nQuestion 2: What product types must be supported? Rationale for asking: If every product is unit-priced, a single quantity shape suffices. If weight-priced groceries are also supported, quantity ceases to be strictly integral.\nAssumed answer: \u0026ldquo;Both unit-based items (biscuits, shampoo) and weight-based items (tomatoes, rice). Weight-priced items are billed per kilogram, and customers can order any quantity in grams.\u0026rdquo;\nThis answer directly shapes the CartItem and Product abstractions.\nQuestion 3: Does the system manage inventory and stock? Rationale for asking: Some designs assume infinite stock; realistic designs do not. The answer also determines whether stock must be reserved at the point of adding to cart or only checked at the point of checkout.\nAssumed answer: \u0026ldquo;Yes, stock must be tracked. A simple check at checkout is sufficient; cart-level reservation is not required.\u0026rdquo;\nThis assumption eliminates an entire class of concurrency concerns from the current scope.\nQuestion 4: Which discounts and promotions must be supported? Rationale for asking: Discounts constitute the most volatile part of any commercial system, since marketing teams routinely request new promotion types. Hardcoded discount logic is therefore a recognised anti-pattern.\nAssumed answer: \u0026ldquo;Coupon codes (10% off, ₹100 off), buy-one-get-one, and a member discount (5% for members).\u0026rdquo;\nThree distinct promotion types is a sufficient basis for demonstrating Strategy and Composite pattern composition.\nQuestion 5: Which payment methods must be supported? Rationale for asking: Each payment method exhibits a distinct flow. Card and UPI payments require a payment gateway, while cash-on-delivery does not. Gateway integration itself is an implementation concern that would consume disproportionate time.\nAssumed answer: \u0026ldquo;Card, UPI, wallet, and cash-on-delivery. Gateway internals are out of scope; the design must model the flow rather than the plumbing.\u0026rdquo;\nThis scoping allows the design to focus on architectural seams rather than integration details.\nQuestion 6: Which shipping and delivery options must be supported? Rationale for asking: Shipping constitutes another Strategy candidate. Each method exhibits distinct cost and estimated-time-of-arrival logic.\nAssumed answer: \u0026ldquo;Standard (next-day, free above ₹500), Express (same-day, ₹99), Scheduled (customer picks a delivery slot, ₹49).\u0026rdquo;\nThree options with divergent cost logic justify a Strategy-based design.\nQuestion 7: Are notifications required? Rationale for asking: Notifications are inexpensive to add and provide a substantial signal of design quality. Omitting the question forecloses the opportunity to apply the Observer pattern.\nAssumed answer: \u0026ldquo;Email and SMS on the events order-placed, order-shipped, and order-delivered.\u0026rdquo;\nObserver pattern is confirmed as necessary.\nQuestion 8: Must returns and refunds be modelled? Rationale for asking: Return workflows are frequently as complex as ordering workflows and warrant explicit confirmation.\nAssumed answer: \u0026ldquo;Model return as an order state; the refund flow itself is out of scope.\u0026rdquo;\nThe scope permits extending OrderStatus with a RETURNED state while deferring refund logic.\nQuestion 9: What are the non-functional constraints regarding concurrency, persistence, and scale? Rationale for asking: These non-functional concerns are best budgeted for once, then largely deferred, to avoid distraction from the primary design task.\nAssumed answer: \u0026ldquo;An in-memory, single-threaded design is acceptable. Concurrency and database persistence should be discussed at the end as extensions.\u0026rdquo;\nThe design can therefore focus on clean object composition and defer infrastructure concerns.\nRecap to the Interviewer Before writing any code, the requirements should be restated for confirmation:\n\u0026ldquo;To confirm the requirements: a single-store grocery site with owned inventory, supporting both unit-priced and weight-priced items, with stock checked at checkout, three promotion types (coupon, BOGO, member discount), four payment methods (card, UPI, wallet, COD) with gateway internals out of scope, three delivery options (standard, express, scheduled), email and SMS notifications on key events, returns modelled as an order state with refund flow out of scope. The design will be single-threaded and in-memory, treating concurrency, database persistence, and scale as extensions. Please confirm.\u0026rdquo;\nAny omissions can be corrected at this point, and any over-scoping can be trimmed before implementation begins.\n4. Stage 3: Identifying Entities and Filtering Rigorously The next step involves extracting nouns from the requirements and filtering them, since not every noun corresponds to a class.\nInitial noun list:\nproduct, category, weight, unit, price, stock, inventory, warehouse, customer, address, cart, cart item, order, order item, order status, payment, payment method, coupon, discount, promotion, shipping, delivery method, notification, email, SMS, member, wallet.\nEach noun is then evaluated against three criteria:\nDoes it possess identity, meaning that specific instances must be distinguished from one another? Does it possess behaviour or state independent of being a mere value? Is it a domain concept rather than infrastructure vocabulary? Noun Identity? Behaviour or state? Verdict product Yes Yes (has price, linked to stock) Class category Yes Weak (primarily a label) Class (retained for tree structure) weight / unit No No Enum (ProductUnit) price No No Field on Product stock Yes (per product) Yes (mutates) Class (InventoryItem) inventory Yes (singular) Yes (orchestrates) Class (Inventory service) warehouse Contextual Contextual Deferred (out of scope) customer Yes Yes (owns cart, orders, addresses) Class address Yes Value-like Class (small dataclass) cart Yes (per customer) Yes Class cart item Yes Small; carries quantity and product Class (compact) order Yes Yes (transitions through states) Class order item Yes Snapshot of price and quantity Class order status No Yes (behavioural) State pattern classes payment Yes Yes Class payment method No identity Yes (algorithmic variation) Strategy coupon / promotion Yes Yes Class combined with Strategy discount No identity Yes (algorithmic variation) Strategy shipping method No identity Yes (algorithmic variation) Strategy notification Ephemeral Small Class (event object) email / SMS Delivery channels Yes Class hierarchy (Observer) member Type of customer Not distinct Field on Customer (is_member: bool) wallet Would be its own subsystem Not required Deferred; treated as a payment method only The disciplined rejections merit explicit attention. The Warehouse entity is omitted because no requirement calls for multi-warehouse operation. Wallet balance and transaction ledgers are not modelled because the requirements treat wallet purely as a payment method. Weight-versus-unit distinctions are represented as a property of Product rather than as a class hierarchy, since the behaviour is uniform across both cases.\nThe resulting skeleton consists of the following classes and enumerations:\nProduct, Category, InventoryItem, Inventory, Customer, Address, Cart, CartItem, Order, OrderItem, OrderState (with concrete state subclasses), Payment, PaymentStrategy (with concrete method subclasses), Promotion / DiscountStrategy (with concrete promotion subclasses), ShippingStrategy (with concrete method subclasses), OrderObserver (with EmailObserver, SmsObserver subclasses), OnlineStore (the top-level orchestrator) Enumerations include ProductUnit; further enumerations may emerge as the design develops.\nThis is a substantial component list, but each component will be developed incrementally with a small, defensible scope.\n5. Stage 4: Modeling Products and the Catalog The design begins where every e-commerce design begins: defining what constitutes a product.\nDecision Point 1: Single class or class hierarchy? Option A: A single Product class containing fields for every attribute.\nclass Product: sku: str name: str price: float unit: str # \u0026#34;piece\u0026#34; or \u0026#34;kg\u0026#34; stock: int perishable: bool This option is simple but conflates concerns, since pricing, unit, and stock all reside on the same class.\nOption B: Inheritance, with Product as a base class and UnitProduct and WeightProduct as subclasses.\nType-specific behaviour would reside on the type. This structure feels natural at first glance. The question, however, is what actually differs between the subclasses. The answer is only the unit of measure, which constitutes a thin justification for a full class hierarchy.\nOption C: Composition, with a single Product class parameterised by a unit enumeration and a per-unit price.\nclass Product: unit: ProductUnit # PIECE or KG price_per_unit: float This option is simple, avoids a hierarchy, and handles both cases uniformly.\nDecision Option C is selected. Inheritance is only justified when subclasses exhibit substantially different behaviour, not merely different data. In this case, both unit and weight products flow through identical cart, order, and payment logic. The sole difference lies in the interpretation of quantity and the arithmetic used to total items, which can be handled with a small enum and consistent numerical operations.\nAn interviewer explanation would proceed as follows:\n\u0026ldquo;I will use a single Product class parameterised by a unit enum with values PIECE and KG. A hierarchy would be tempting, but since the only real difference between subclasses is the interpretation of quantity while the rest of the behaviour is identical, an enum-based design is simpler and remains open to new units.\u0026rdquo;\nDecision Point 2: Where does stock reside? Three placements are possible:\nOption A: Stock as a field on Product, accessed as product.stock. This is the simplest option but mixes catalog data with inventory state. A change in stock would conceptually mutate the product itself, which conflicts with the intuition that a product\u0026rsquo;s identity does not include its current stock level.\nOption B: A separate InventoryItem per product, managed by an Inventory service. This option separates catalog data (read-heavy) from inventory state (write-heavy).\nOption C: Both, with stock cached on Product and the authoritative value in Inventory. This constitutes a performance optimisation rather than a design decision.\nOption B is selected on the basis of a domain-driven design principle: the catalog is a read model and the inventory is a write model, so the two should be separable to permit independent evolution and, in a future implementation, storage in different persistence stores.\nThe Code from __future__ import annotations from abc import ABC, abstractmethod from dataclasses import dataclass, field from datetime import datetime from enum import Enum from typing import Optional import uuid class ProductUnit(Enum): \u0026#34;\u0026#34;\u0026#34; Represents the unit in which a product is priced and counted. PIECE denotes items sold in whole units, such as packs of biscuits or bottles. Quantity is always integral. KG denotes items sold by weight, with price expressed per kilogram. Quantity is a float value; 0.25 corresponds to 250 grams. An enum is preferred over free-form strings for three reasons: type safety at call sites, IDE autocompletion support, and trivial extensibility to additional units such as LITRE or DOZEN. \u0026#34;\u0026#34;\u0026#34; PIECE = \u0026#34;piece\u0026#34; KG = \u0026#34;kg\u0026#34; @dataclass class Category: \u0026#34;\u0026#34;\u0026#34; Groups related products and supports nesting via the `parent` field. The class is intentionally minimal; additional fields are introduced only when a specific behaviour requires them. \u0026#34;\u0026#34;\u0026#34; name: str parent: Optional[\u0026#34;Category\u0026#34;] = None @dataclass class Product: \u0026#34;\u0026#34;\u0026#34; Represents a single sellable item in the catalog. Identity is established by the `sku` field (Stock Keeping Unit), which is unique across the entire catalog. Attributes deliberately excluded from this class: - stock, since it resides in Inventory for separation of concerns. - description, images, and reviews, since these are out of scope and would be added when concrete requirements demand them. The practice of adding fields speculatively is the most common cause of domain model bloat. \u0026#34;\u0026#34;\u0026#34; sku: str name: str price_per_unit: float unit: ProductUnit category: Category def display_price(self) -\u0026gt; str: \u0026#34;\u0026#34;\u0026#34;Returns a formatted price string. Pure formatting with no domain logic.\u0026#34;\u0026#34;\u0026#34; return f\u0026#34;₹{self.price_per_unit:.2f} / {self.unit.value}\u0026#34; First Class Diagram: Catalog classDiagram class ProductUnit { \u0026lt;\u0026lt;enum\u0026gt;\u0026gt; PIECE KG } class Category { +String name +Category parent } class Product { +String sku +String name +float price_per_unit +ProductUnit unit +Category category +display_price() String } Product --\u0026gt; Category : belongs to Product --\u0026gt; ProductUnit : uses Category --\u0026gt; Category : parent The diagram remains small and focused, without leaks into other concerns. No class here has any awareness of carts, orders, or inventory.\n6. Stage 5: Inventory and the Placement of Stock The prior decision established that stock resides outside Product. The next task is to design the class that owns it.\nThe Shape An InventoryItem tracks the available quantity of one product. The Inventory class is a service that manages many InventoryItem instances.\nThe rationale for making InventoryItem a class rather than a simple dict[sku, quantity] mapping is twofold. First, real inventories accumulate behaviour over time: they track reorder thresholds (\u0026ldquo;reorder when quantity is at or below 10\u0026rdquo;), reserved quantities during checkout, and lot identifiers for perishable batch tracking. Even when these behaviours are not implemented initially, the class provides a stable location for them. Second, encapsulation is preserved. External callers should not manipulate stock through direct dictionary access such as inventory[sku] -= 3, but rather through methods such as inventory.reduce(sku, 3) that permit the service to validate the operation.\nFields are not added speculatively; the design remains minimal until requirements dictate otherwise.\nDecision Point: What operations does Inventory expose? The minimum useful operation set is:\nadd(product, quantity): restocking or initial stocking. has(product, quantity): non-mutating availability check. reduce(product, quantity): deduction on order confirmation. available(product): read-only quantity query for display purposes. A reserve operation is intentionally excluded. Reservation is required when checkout takes significant time and stock levels can become stale, which is both a concurrency concern and a user-experience concern. The requirements permit checking stock at checkout only, so reservation is out of scope and will be noted as a possible extension.\nThe Code class OutOfStockError(Exception): \u0026#34;\u0026#34;\u0026#34;Raised when a reduce operation would take stock below zero.\u0026#34;\u0026#34;\u0026#34; @dataclass class InventoryItem: \u0026#34;\u0026#34;\u0026#34; Stock record for a single Product. Maintained as a dedicated class so that future extensions such as reorder_threshold, reserved_quantity for stock holds during checkout, or lot_id for perishable batch tracking can be added without modifying every caller. \u0026#34;\u0026#34;\u0026#34; product: Product quantity: float # float type accommodates partial quantities for KG products class Inventory: \u0026#34;\u0026#34;\u0026#34; In-memory stock management service. Responsibilities: - Track available stock for every product. - Provide availability queries. - Mutate stock on restocking or consumption. Deliberately excluded responsibilities: - Knowledge of which products exist, which is the catalog\u0026#39;s role. - Reservations across long-lived transactions, treated as an extension. - Multi-warehouse routing, also treated as an extension. \u0026#34;\u0026#34;\u0026#34; def __init__(self): # Keyed by SKU for O(1) lookup and to avoid retaining stale Product # references, since SKU is the stable identity. self._items: dict[str, InventoryItem] = {} def add(self, product: Product, quantity: float) -\u0026gt; None: \u0026#34;\u0026#34;\u0026#34;Add stock. Creates the record if it does not yet exist.\u0026#34;\u0026#34;\u0026#34; if quantity \u0026lt;= 0: raise ValueError(\u0026#34;Quantity to add must be positive\u0026#34;) if product.sku not in self._items: self._items[product.sku] = InventoryItem(product, 0.0) self._items[product.sku].quantity += quantity def available(self, product: Product) -\u0026gt; float: \u0026#34;\u0026#34;\u0026#34;Returns current available quantity; returns 0 for an unknown SKU.\u0026#34;\u0026#34;\u0026#34; item = self._items.get(product.sku) return item.quantity if item else 0.0 def has(self, product: Product, quantity: float) -\u0026gt; bool: \u0026#34;\u0026#34;\u0026#34;Non-mutating availability check.\u0026#34;\u0026#34;\u0026#34; return self.available(product) \u0026gt;= quantity def reduce(self, product: Product, quantity: float) -\u0026gt; None: \u0026#34;\u0026#34;\u0026#34; Deducts stock. Raises OutOfStockError when quantity is insufficient. Rationale for raising rather than returning False: reducing more stock than exists constitutes a caller-side bug, since callers are expected to invoke `has` first. Loud failure is preferable to silent corruption. \u0026#34;\u0026#34;\u0026#34; if not self.has(product, quantity): raise OutOfStockError( f\u0026#34;Only {self.available(product)} {product.unit.value} \u0026#34; f\u0026#34;of {product.name} available; requested quantity was {quantity}\u0026#34; ) self._items[product.sku].quantity -= quantity Observation on Dependency Direction Inventory depends on Product, but Product has no awareness of Inventory. The dependency direction is correct. If, at a later stage, the catalog must be published without exposing stock information (for example, to an external search engine), Product remains independent and requires no modification.\n7. Stage 6: The Shopping Cart The cart is per-customer, ephemeral, and mutable. The following decisions govern its design.\nDecision Point 1: How is quantity represented in CartItem? For unit-priced products, quantity is naturally an integer. For weight-priced products, quantity is a floating-point value such as 0.5 kilograms.\nOption A: Two CartItem subclasses, one per unit type. This option is excessive, since the arithmetic operations are identical across both types.\nOption B: A single CartItem class with quantity: float. This option is simple.\nOption B is selected. Validation occurs at add-time to ensure that unit-based products receive integer quantities:\nif product.unit == ProductUnit.PIECE and quantity != int(quantity): raise ValueError(\u0026#34;Piece-based products require whole-number quantities\u0026#34;) Decision Point 2: What does the cart expose? The cart\u0026rsquo;s public interface consists of the following operations:\nadd(product, quantity): merges quantity with an existing line if the product is already present. remove(product): removes the line entirely. update(product, new_quantity): sets the line to an exact quantity. clear(): empties the cart. subtotal(): computes the sum of line totals. items(): returns a snapshot of the cart contents for pricing or checkout consumption. Decision Point 3: Does the cart check stock at add-time? Option A: Yes. Immediately reject an \u0026ldquo;add 5 apples\u0026rdquo; operation when only 3 units are in stock. This option provides responsive feedback.\nOption B: No. Carts function as wishlists, and validation occurs at checkout.\nOption B was selected during requirements clarification (\u0026ldquo;checking at checkout is sufficient\u0026rdquo;). The cart therefore functions as a pure container without stock validation.\nThe Code @dataclass class CartItem: \u0026#34;\u0026#34;\u0026#34; Represents a single line in the cart. Adopts snapshot semantics: `unit_price_at_add` captures the price at the time the item was added, so that mid-shopping catalog price changes do not silently reprice the cart. This is a decision grounded in user trust; many production sites reprice at checkout instead, which represents a legitimate alternative to be articulated during interview discussion. \u0026#34;\u0026#34;\u0026#34; product: Product quantity: float unit_price_at_add: float def line_total(self) -\u0026gt; float: return self.quantity * self.unit_price_at_add class Cart: \u0026#34;\u0026#34;\u0026#34; A customer\u0026#39;s active shopping cart. Instances are one-per-customer. The cart resides in memory in this design; in a production system it would be persisted, for example in Redis or a database, to survive page reloads and session breaks. \u0026#34;\u0026#34;\u0026#34; def __init__(self, customer: \u0026#34;Customer\u0026#34;): self.customer = customer # Keyed by SKU so that add() correctly merges quantities. self._items: dict[str, CartItem] = {} def add(self, product: Product, quantity: float) -\u0026gt; None: if quantity \u0026lt;= 0: raise ValueError(\u0026#34;Quantity must be positive\u0026#34;) # Prevent fractional additions of unit-based products. if product.unit == ProductUnit.PIECE and quantity != int(quantity): raise ValueError(\u0026#34;Piece-based products require whole-number quantities\u0026#34;) if product.sku in self._items: # Merge with the existing line, preserving the original price snapshot. self._items[product.sku].quantity += quantity else: self._items[product.sku] = CartItem( product=product, quantity=quantity, unit_price_at_add=product.price_per_unit, ) def update(self, product: Product, quantity: float) -\u0026gt; None: \u0026#34;\u0026#34;\u0026#34;Sets quantity to an exact value; a value of 0 removes the item.\u0026#34;\u0026#34;\u0026#34; if quantity == 0: self.remove(product) return if product.sku not in self._items: # Behaviour resembles add() when the item is not already present. self.add(product, quantity) return self._items[product.sku].quantity = quantity def remove(self, product: Product) -\u0026gt; None: self._items.pop(product.sku, None) # silent no-op when the SKU is absent def clear(self) -\u0026gt; None: self._items.clear() def items(self) -\u0026gt; list[CartItem]: \u0026#34;\u0026#34;\u0026#34;Returns a snapshot copy; callers must not mutate internal state.\u0026#34;\u0026#34;\u0026#34; return list(self._items.values()) def subtotal(self) -\u0026gt; float: \u0026#34;\u0026#34;\u0026#34;Sum of all line totals, before promotions and shipping.\u0026#34;\u0026#34;\u0026#34; return sum(item.line_total() for item in self._items.values()) def is_empty(self) -\u0026gt; bool: return len(self._items) == 0 Class Diagram: Cart Cluster classDiagram class Customer { +String customer_id +String name +String email +bool is_member } class Cart { -dict~str,CartItem~ _items +add(product, quantity) +update(product, quantity) +remove(product) +clear() +items() list~CartItem~ +subtotal() float +is_empty() bool } class CartItem { +Product product +float quantity +float unit_price_at_add +line_total() float } Customer \u0026#34;1\u0026#34; --\u0026gt; \u0026#34;1\u0026#34; Cart : owns Cart \u0026#34;1\u0026#34; *-- \u0026#34;many\u0026#34; CartItem : contains CartItem --\u0026gt; Product : refers to 8. Stage 7: Pricing, Discounts, and Promotions The requirements specify three promotion types: coupon codes, buy-one-get-one, and member discount. This variability is a clear signal for the Strategy pattern. The layering of multiple promotions on a single order (member discount combined with coupon) additionally motivates a Composite or Decorator arrangement over the Strategy foundation.\nJustification for Strategy A hardcoded discount implementation would appear as follows:\n# Anti-pattern; do not implement def total(cart, coupon, is_member): total = cart.subtotal() if coupon == \u0026#34;SAVE10\u0026#34;: total *= 0.9 if is_member: total *= 0.95 if some_bogo_rule: ... return total Every new promotion type requires editing this function, which risks breaking every existing promotion. Such code is untestable and unmaintainable. The Strategy pattern eliminates precisely this risk.\nThe Interface A discount takes the cart\u0026rsquo;s context and returns the amount to subtract from the subtotal:\nclass DiscountStrategy(ABC): @abstractmethod def compute(self, cart: Cart, customer: Customer) -\u0026gt; float: \u0026#34;\u0026#34;\u0026#34;Returns the ₹ discount to apply. The return value is never negative.\u0026#34;\u0026#34;\u0026#34; Every promotion implements this interface. Layering promotions is achieved by composing invocations, which is where the Composite pattern (or a decorator arrangement) applies.\nConcrete Strategies The design requires four concrete strategies:\nPercentageCoupon: for example, \u0026ldquo;SAVE10\u0026rdquo; applies 10% off. FlatCoupon: for example, \u0026ldquo;FLAT100\u0026rdquo; applies ₹100 off. MemberDiscount: 5% off when customer.is_member is true. BuyOneGetOne: for a specific product, every second unit is free. The Code class DiscountStrategy(ABC): \u0026#34;\u0026#34;\u0026#34; Interface for any pricing adjustment. Contract: - Returns the ₹ amount to subtract from the current total. - Never returns a value that would make the total negative; callers may enforce this floor. - Pure computation with no side effects and no input or output, making it trivially testable. \u0026#34;\u0026#34;\u0026#34; @abstractmethod def compute(self, cart: Cart, customer: Customer) -\u0026gt; float: ... class NoDiscount(DiscountStrategy): \u0026#34;\u0026#34;\u0026#34;Null Object implementation that simplifies callers requiring a discount strategy in every path.\u0026#34;\u0026#34;\u0026#34; def compute(self, cart, customer) -\u0026gt; float: return 0.0 class PercentageCoupon(DiscountStrategy): \u0026#34;\u0026#34;\u0026#34; Percentage-based coupon, for example SAVE10 corresponding to 10% off. The optional `max_discount` parameter caps the discount, supporting the common commercial constraint of \u0026#34;up to ₹500 off\u0026#34; promotions. \u0026#34;\u0026#34;\u0026#34; def __init__(self, code: str, percent: float, max_discount: Optional[float] = None): if not 0 \u0026lt; percent \u0026lt;= 100: raise ValueError(\u0026#34;percent must be in (0, 100]\u0026#34;) self.code = code self.percent = percent self.max_discount = max_discount def compute(self, cart, customer) -\u0026gt; float: raw = cart.subtotal() * (self.percent / 100) return min(raw, self.max_discount) if self.max_discount else raw class FlatCoupon(DiscountStrategy): \u0026#34;\u0026#34;\u0026#34;Flat ₹ discount, for example FLAT100 corresponding to ₹100 off. Capped at the subtotal to prevent negative totals.\u0026#34;\u0026#34;\u0026#34; def __init__(self, code: str, amount: float, min_order: float = 0): self.code = code self.amount = amount self.min_order = min_order def compute(self, cart, customer) -\u0026gt; float: if cart.subtotal() \u0026lt; self.min_order: return 0.0 return min(self.amount, cart.subtotal()) class MemberDiscount(DiscountStrategy): \u0026#34;\u0026#34;\u0026#34;5% discount for members, gated by the `customer.is_member` flag.\u0026#34;\u0026#34;\u0026#34; def __init__(self, percent: float = 5.0): self.percent = percent def compute(self, cart, customer) -\u0026gt; float: if not customer.is_member: return 0.0 return cart.subtotal() * (self.percent / 100) class BuyOneGetOne(DiscountStrategy): \u0026#34;\u0026#34;\u0026#34; For a specified product, every second unit is free. Illustrates that strategies can be product-scoped rather than exclusively cart-wide. \u0026#34;\u0026#34;\u0026#34; def __init__(self, target_sku: str): self.target_sku = target_sku def compute(self, cart, customer) -\u0026gt; float: for item in cart.items(): if item.product.sku == self.target_sku: free_units = int(item.quantity) // 2 return free_units * item.unit_price_at_add return 0.0 Composing Multiple Discounts Actual orders often combine discounts: a member discount combined with a coupon combined with a BOGO offer. Two structurally distinct approaches exist.\nApproach 1: Composite by summation.\nclass CompositeDiscount(DiscountStrategy): \u0026#34;\u0026#34;\u0026#34;Sums multiple discounts. Each strategy sees the original subtotal.\u0026#34;\u0026#34;\u0026#34; def __init__(self, strategies: list[DiscountStrategy]): self.strategies = strategies def compute(self, cart, customer) -\u0026gt; float: total = sum(s.compute(cart, customer) for s in self.strategies) return min(total, cart.subtotal()) # never below zero Approach 2: Sequential (Decorator-like) chaining. Each discount computes against the total already reduced by prior discounts. This resembles real \u0026ldquo;apply coupon after member discount\u0026rdquo; behaviour.\nThe Composite approach is simpler and matches the model in which each promotion applies its percentage to the original subtotal, which is the model most commercial sites adopt. The Composite approach is therefore selected.\nInterview Explanation \u0026ldquo;Every discount implements the same DiscountStrategy interface, so the checkout is unaware of which specific discount type is being applied. Multiple promotions are combined by a Composite that sums individual discounts and floors the total at zero. Adding a new promotion, such as a Diwali surge discount, requires only a single new class, and stacking it with existing promotions requires only one additional line at construction time. This is a direct application of the Open/Closed Principle.\u0026rdquo;\nClass Diagram: Discounts classDiagram class DiscountStrategy { \u0026lt;\u0026lt;abstract\u0026gt;\u0026gt; +compute(cart, customer) float } class NoDiscount class PercentageCoupon { +String code +float percent +float max_discount } class FlatCoupon { +String code +float amount +float min_order } class MemberDiscount { +float percent } class BuyOneGetOne { +String target_sku } class CompositeDiscount { -list~DiscountStrategy~ strategies } DiscountStrategy \u0026lt;|-- NoDiscount DiscountStrategy \u0026lt;|-- PercentageCoupon DiscountStrategy \u0026lt;|-- FlatCoupon DiscountStrategy \u0026lt;|-- MemberDiscount DiscountStrategy \u0026lt;|-- BuyOneGetOne DiscountStrategy \u0026lt;|-- CompositeDiscount CompositeDiscount o-- DiscountStrategy : composes 9. Stage 8: The Order Lifecycle An order is not merely data; it possesses a lifecycle governed by explicit transitions:\nPENDING -\u0026gt; CONFIRMED -\u0026gt; PACKED -\u0026gt; SHIPPED -\u0026gt; DELIVERED | | | CANCELLED CANCELLED (rarely) -\u0026gt; RETURNED Each state permits a distinct set of operations. In PENDING, the order can be paid for or cancelled. In CONFIRMED, it awaits packing and can still be cancelled. In SHIPPED, cancellation is no longer possible; only delivery or return-on-arrival is valid. In DELIVERED, a return is possible within a defined window.\nThe naive approach adopts status: str as a field and scatters large if status == \u0026quot;shipped\u0026quot;: chains throughout the codebase. This pattern is precisely the pain that the State pattern is designed to resolve.\nJustification for the State Pattern The trade-off between an enumeration-based approach and the State pattern deserves explicit consideration.\nThe enumeration approach is simple, but every operation on Order becomes an if-elif ladder over the status. Adding a new status requires modifying every operation.\nThe State pattern approach models each state as a distinct class with its own behaviour. Adding a status requires only a new class and minor changes to transition edges. Operations dispatch polymorphically to the current state.\nFor an order with approximately 5 states and approximately 3 state-dependent operations, the State pattern justifies its complexity. For simpler lifecycles with 2 to 3 states, the pattern may be excessive.\nThe State pattern is selected for this design. The number of states justifies it, and the pattern is a strong signal of design maturity in interview contexts.\nThe Interface class OrderState(ABC): @abstractmethod def name(self) -\u0026gt; str: ... def confirm_payment(self, order): raise InvalidTransition(f\u0026#34;Cannot confirm payment in state {self.name()}\u0026#34;) def pack(self, order): raise InvalidTransition(f\u0026#34;Cannot pack in state {self.name()}\u0026#34;) def ship(self, order): raise InvalidTransition(f\u0026#34;Cannot ship in state {self.name()}\u0026#34;) def deliver(self, order): raise InvalidTransition(f\u0026#34;Cannot deliver in state {self.name()}\u0026#34;) def cancel(self, order): raise InvalidTransition(f\u0026#34;Cannot cancel in state {self.name()}\u0026#34;) def return_order(self, order): raise InvalidTransition(f\u0026#34;Cannot return in state {self.name()}\u0026#34;) The central design idea is that the base class raises an exception for every operation by default. Concrete states override only those operations they permit. Three consequences follow. First, invalid transitions fail loudly by default, everywhere. Second, adding an operation requires one method on OrderState plus overrides in states where it is legal. Third, adding a new state requires a single new subclass.\nThe Code class InvalidTransition(Exception): \u0026#34;\u0026#34;\u0026#34;Raised when an operation is attempted that the current state forbids.\u0026#34;\u0026#34;\u0026#34; class OrderState(ABC): @abstractmethod def name(self) -\u0026gt; str: ... # Every operation is forbidden by default; subclasses permit what applies. def confirm_payment(self, order): raise InvalidTransition(f\u0026#34;confirm_payment invalid in {self.name()}\u0026#34;) def pack(self, order): raise InvalidTransition(f\u0026#34;pack invalid in {self.name()}\u0026#34;) def ship(self, order): raise InvalidTransition(f\u0026#34;ship invalid in {self.name()}\u0026#34;) def deliver(self, order): raise InvalidTransition(f\u0026#34;deliver invalid in {self.name()}\u0026#34;) def cancel(self, order): raise InvalidTransition(f\u0026#34;cancel invalid in {self.name()}\u0026#34;) def return_order(self, order): raise InvalidTransition(f\u0026#34;return invalid in {self.name()}\u0026#34;) class PendingState(OrderState): def name(self) -\u0026gt; str: return \u0026#34;PENDING\u0026#34; def confirm_payment(self, order): # Successful payment causes the order to become Confirmed. order._transition_to(ConfirmedState()) def cancel(self, order): order._transition_to(CancelledState()) class ConfirmedState(OrderState): def name(self) -\u0026gt; str: return \u0026#34;CONFIRMED\u0026#34; def pack(self, order): order._transition_to(PackedState()) def cancel(self, order): # Cancellation remains possible prior to packing. order._transition_to(CancelledState()) class PackedState(OrderState): def name(self) -\u0026gt; str: return \u0026#34;PACKED\u0026#34; def ship(self, order): order._transition_to(ShippedState()) class ShippedState(OrderState): def name(self) -\u0026gt; str: return \u0026#34;SHIPPED\u0026#34; def deliver(self, order): order._transition_to(DeliveredState()) class DeliveredState(OrderState): def name(self) -\u0026gt; str: return \u0026#34;DELIVERED\u0026#34; def return_order(self, order): order._transition_to(ReturnedState()) class CancelledState(OrderState): def name(self) -\u0026gt; str: return \u0026#34;CANCELLED\u0026#34; # Terminal state; no transitions are permitted. class ReturnedState(OrderState): def name(self) -\u0026gt; str: return \u0026#34;RETURNED\u0026#34; # Terminal state; no transitions are permitted. The Order class itself becomes structurally thin under this design:\n@dataclass class OrderItem: \u0026#34;\u0026#34;\u0026#34;Snapshot of a purchased line, decoupled from the cart after checkout.\u0026#34;\u0026#34;\u0026#34; product: Product quantity: float unit_price: float def line_total(self) -\u0026gt; float: return self.quantity * self.unit_price class Order: \u0026#34;\u0026#34;\u0026#34; An order is a snapshot of a checkout event: the items purchased, the prices at which they were purchased, the customer making the purchase, the shipping destination, and the current lifecycle state. All state transitions occur through the current state object\u0026#39;s method, so `Order` contains no if-elif chains on state, only delegation. \u0026#34;\u0026#34;\u0026#34; def __init__(self, order_id: str, customer: Customer, items: list[OrderItem], shipping_address: \u0026#34;Address\u0026#34;, subtotal: float, discount: float, shipping_cost: float): self.order_id = order_id self.customer = customer self.items = items self.shipping_address = shipping_address self.subtotal = subtotal self.discount = discount self.shipping_cost = shipping_cost self.total = subtotal - discount + shipping_cost self.placed_at = datetime.now() # Initial state is PENDING, awaiting payment confirmation. self._state: OrderState = PendingState() # Observers subscribing to state changes (email, SMS, etc.). self._observers: list[\u0026#34;OrderObserver\u0026#34;] = [] @property def state(self) -\u0026gt; str: \u0026#34;\u0026#34;\u0026#34;Public read-only view of the current state\u0026#39;s name.\u0026#34;\u0026#34;\u0026#34; return self._state.name() # Observer attachment and detachment def attach(self, observer: \u0026#34;OrderObserver\u0026#34;) -\u0026gt; None: self._observers.append(observer) def detach(self, observer: \u0026#34;OrderObserver\u0026#34;) -\u0026gt; None: self._observers.remove(observer) def _notify(self) -\u0026gt; None: for observer in self._observers: observer.on_state_change(self) # Called by state classes def _transition_to(self, new_state: OrderState) -\u0026gt; None: \u0026#34;\u0026#34;\u0026#34;Internal method; invoked only by state classes.\u0026#34;\u0026#34;\u0026#34; print(f\u0026#34;[Order {self.order_id}] {self._state.name()} -\u0026gt; {new_state.name()}\u0026#34;) self._state = new_state self._notify() # Public API, delegating to the current state def confirm_payment(self): self._state.confirm_payment(self) def pack(self): self._state.pack(self) def ship(self): self._state.ship(self) def deliver(self): self._state.deliver(self) def cancel(self): self._state.cancel(self) def return_order(self): self._state.return_order(self) Advantages of the Approach Every operation on Order reduces to a one-line delegate expression: self._state.method(self). Adding a new state, such as REFUND_INITIATED, requires one new class and one line of change in DeliveredState. Illegal transitions raise clear exceptions rather than causing silent corruption. Order is straightforwardly testable, since it can be constructed in any state and each transition can be verified in isolation.\nState Diagram stateDiagram-v2 [*] --\u0026gt; PENDING PENDING --\u0026gt; CONFIRMED : confirm_payment PENDING --\u0026gt; CANCELLED : cancel CONFIRMED --\u0026gt; PACKED : pack CONFIRMED --\u0026gt; CANCELLED : cancel PACKED --\u0026gt; SHIPPED : ship SHIPPED --\u0026gt; DELIVERED : deliver DELIVERED --\u0026gt; RETURNED : return_order CANCELLED --\u0026gt; [*] RETURNED --\u0026gt; [*] 10. Stage 9: Payment Processing The requirements specify four payment methods (card, UPI, wallet, and cash-on-delivery), with gateway internals excluded from scope. This is another application of the Strategy pattern, in a different domain but with the same structural characteristics.\nThe Interface class PaymentStrategy(ABC): @abstractmethod def pay(self, amount: float) -\u0026gt; \u0026#34;PaymentResult\u0026#34;: ... This single method constitutes the entire contract.\nConcrete Strategies @dataclass class PaymentResult: \u0026#34;\u0026#34;\u0026#34;Return value from `pay()`, containing a success flag and a message or transaction identifier.\u0026#34;\u0026#34;\u0026#34; success: bool message: str transaction_id: Optional[str] = None class CardPayment(PaymentStrategy): \u0026#34;\u0026#34;\u0026#34;Card payment. In a production system, this class would delegate to a payment gateway such as Stripe or Razorpay.\u0026#34;\u0026#34;\u0026#34; def __init__(self, card_number: str, cvv: str): # In production, card details would not be stored in this form; # tokenisation via the gateway would be mandatory. This is a # model-only representation. self.card_number = card_number self.cvv = cvv def pay(self, amount: float) -\u0026gt; PaymentResult: # Stub implementation that simulates gateway approval. return PaymentResult(True, f\u0026#34;Charged ₹{amount:.2f} to card \u0026#34; f\u0026#34;****{self.card_number[-4:]}\u0026#34;, transaction_id=str(uuid.uuid4())[:8]) class UpiPayment(PaymentStrategy): def __init__(self, upi_id: str): self.upi_id = upi_id def pay(self, amount: float) -\u0026gt; PaymentResult: return PaymentResult(True, f\u0026#34;₹{amount:.2f} paid via UPI ({self.upi_id})\u0026#34;, transaction_id=str(uuid.uuid4())[:8]) class WalletPayment(PaymentStrategy): \u0026#34;\u0026#34;\u0026#34;Uses the customer\u0026#39;s in-app wallet balance. This is a simplified representation; production wallets maintain their own transaction logs and balance holds.\u0026#34;\u0026#34;\u0026#34; def __init__(self, wallet_balance: float): self.wallet_balance = wallet_balance def pay(self, amount: float) -\u0026gt; PaymentResult: if amount \u0026gt; self.wallet_balance: return PaymentResult(False, f\u0026#34;Insufficient wallet balance \u0026#34; f\u0026#34;(₹{self.wallet_balance:.2f})\u0026#34;) self.wallet_balance -= amount return PaymentResult(True, f\u0026#34;₹{amount:.2f} deducted from wallet\u0026#34;, transaction_id=str(uuid.uuid4())[:8]) class CashOnDelivery(PaymentStrategy): \u0026#34;\u0026#34;\u0026#34;No immediate payment; funds are collected on delivery. Always reports success to the checkout flow.\u0026#34;\u0026#34;\u0026#34; def pay(self, amount: float) -\u0026gt; PaymentResult: return PaymentResult(True, f\u0026#34;COD selected: pay ₹{amount:.2f} on delivery\u0026#34;, transaction_id=\u0026#34;COD\u0026#34;) Design Nuance Worth Articulating Cash-on-delivery does not constitute immediate payment; it is a deferred commitment. Nevertheless, it conforms to the same interface as the immediate payment methods. This uniformity is a feature rather than a defect: the checkout logic remains ignorant of whether payment is immediate or deferred, since the strategy encapsulates that distinction. This is precisely the value that the Strategy pattern provides, since the caller writes a single code path regardless of the underlying variation.\n11. Stage 10: Shipping and Delivery The requirements specify three shipping options (Standard, Express, and Scheduled), each with its own cost calculation and estimated time of arrival. This is another instance of the Strategy pattern.\n@dataclass class ShippingQuote: \u0026#34;\u0026#34;\u0026#34;Cost and estimated time of arrival for a shipping method applied to a specific order.\u0026#34;\u0026#34;\u0026#34; cost: float eta_days: int method_name: str class ShippingStrategy(ABC): @abstractmethod def quote(self, cart: Cart, address: \u0026#34;Address\u0026#34;) -\u0026gt; ShippingQuote: ... class StandardShipping(ShippingStrategy): \u0026#34;\u0026#34;\u0026#34;Free above ₹500; otherwise ₹49 flat. Estimated delivery in 1 day.\u0026#34;\u0026#34;\u0026#34; def quote(self, cart, address): cost = 0.0 if cart.subtotal() \u0026gt;= 500 else 49.0 return ShippingQuote(cost, 1, \u0026#34;Standard\u0026#34;) class ExpressShipping(ShippingStrategy): \u0026#34;\u0026#34;\u0026#34;Same-day delivery at a flat rate of ₹99.\u0026#34;\u0026#34;\u0026#34; def quote(self, cart, address): return ShippingQuote(99.0, 0, \u0026#34;Express\u0026#34;) class ScheduledShipping(ShippingStrategy): \u0026#34;\u0026#34;\u0026#34;Customer-selected delivery slot at ₹49 flat.\u0026#34;\u0026#34;\u0026#34; def __init__(self, slot: str): self.slot = slot # for example, \u0026#34;Tomorrow 6-8pm\u0026#34; def quote(self, cart, address): return ShippingQuote(49.0, 1, f\u0026#34;Scheduled ({self.slot})\u0026#34;) The interface takes a Cart rather than an Order, because shipping quotes must be calculated before the order exists as a domain object. This temporal constraint influenced the interface design.\n12. Stage 11: Notifications Customers require updates on their orders, which is the canonical use case for the Observer pattern. The Order class already includes the scaffolding for attach, detach, and _notify operations. The remaining work involves defining the observer side.\nPush versus Pull Notification Model The design adopts a push model in which the observer receives the full order object. The rationale is that observers typically require the same information (state, order identifier, items), which makes pushing simpler. The pull model is preferable when observers differ significantly in their information requirements, which is not the case here.\nThe Code class OrderObserver(ABC): \u0026#34;\u0026#34;\u0026#34; Reacts to order state changes. The design depends on this interface rather than on concrete observers, which permits adding new notification channels (push notification, WhatsApp, in-app toast) without modifying `Order` or the checkout flow. \u0026#34;\u0026#34;\u0026#34; @abstractmethod def on_state_change(self, order: Order) -\u0026gt; None: ... class EmailObserver(OrderObserver): \u0026#34;\u0026#34;\u0026#34;Sends templated emails on state changes. Simulated with print output.\u0026#34;\u0026#34;\u0026#34; def __init__(self, to_address: str): self.to_address = to_address def on_state_change(self, order: Order) -\u0026gt; None: subjects = { \u0026#34;CONFIRMED\u0026#34;: \u0026#34;Your order is confirmed!\u0026#34;, \u0026#34;SHIPPED\u0026#34;: \u0026#34;Your order has shipped\u0026#34;, \u0026#34;DELIVERED\u0026#34;: \u0026#34;Your order was delivered\u0026#34;, \u0026#34;CANCELLED\u0026#34;: \u0026#34;Your order was cancelled\u0026#34;, \u0026#34;RETURNED\u0026#34;: \u0026#34;Your return is being processed\u0026#34;, } subject = subjects.get(order.state) if subject: print(f\u0026#34;[Email -\u0026gt; {self.to_address}] {subject} (Order {order.order_id})\u0026#34;) class SmsObserver(OrderObserver): \u0026#34;\u0026#34;\u0026#34;Sends short SMS updates.\u0026#34;\u0026#34;\u0026#34; def __init__(self, phone: str): self.phone = phone def on_state_change(self, order: Order) -\u0026gt; None: # Only a subset of states receive SMS notifications, since SMS # is expensive and should be reserved for high-signal events. if order.state in (\u0026#34;SHIPPED\u0026#34;, \u0026#34;DELIVERED\u0026#34;, \u0026#34;CANCELLED\u0026#34;): print(f\u0026#34;[SMS -\u0026gt; {self.phone}] Order {order.order_id} is {order.state}\u0026#34;) A Subtle Design Point EmailObserver selects what to send based on the state. This logic could alternatively reside in Order, in which case Order would invoke distinct observer methods for distinct transitions. This alternative was rejected for three reasons. First, it would expand the observer interface into methods such as on_confirmed, on_shipped, and so on. Second, adding a new state would require updating every observer. Third, the single-method Observer keeps the pattern clean, and observers can filter events themselves.\nThis trade-off is worth articulating explicitly during an interview.\n13. Stage 12: The Top-Level Coordinator Every design developed so far requires an orchestrator at the top of the composition. In this design, that role is filled by OnlineStore.\nResponsibilities of OnlineStore The class holds the following domain state:\nThe catalog (products and categories). The inventory. Customers and their carts. Orders that have been placed. Deliberately Excluded Responsibilities The class remains ignorant of the following, all of which are delegated to Strategy or Observer collaborators:\nHow discounts are computed. How payments are processed. How shipping is quoted. How notifications are delivered. This restraint keeps the top-level class thin, which is the entire point of orchestration.\nThe Address Value Object @dataclass class Address: line1: str city: str pincode: str line2: str = \u0026#34;\u0026#34; state: str = \u0026#34;\u0026#34; def formatted(self) -\u0026gt; str: parts = [self.line1, self.line2, self.city, self.state, self.pincode] return \u0026#34;, \u0026#34;.join(p for p in parts if p) Customer class Customer: def __init__(self, customer_id: str, name: str, email: str, phone: str, is_member: bool = False): self.customer_id = customer_id self.name = name self.email = email self.phone = phone self.is_member = is_member self.addresses: list[Address] = [] # Cart is created lazily by the store to keep initialisation simple. The OnlineStore class OnlineStore: \u0026#34;\u0026#34;\u0026#34; Top-level orchestrator. Owns: - The catalog (products and categories). - The inventory. - Customers and their carts. - Orders. The two headline methods are: - add_to_cart(customer, product, quantity) - checkout(customer, ...) All other concerns (discounts, payment, shipping, notifications) are delegated to injected strategies and observers, which prevents this class from becoming a god object. \u0026#34;\u0026#34;\u0026#34; def __init__(self, inventory: Inventory): self.inventory = inventory self.products: dict[str, Product] = {} # sku -\u0026gt; Product self.customers: dict[str, Customer] = {} self.carts: dict[str, Cart] = {} # customer_id -\u0026gt; Cart self.orders: dict[str, Order] = {} # order_id -\u0026gt; Order # Catalog operations def add_product(self, product: Product, initial_stock: float = 0): self.products[product.sku] = product if initial_stock \u0026gt; 0: self.inventory.add(product, initial_stock) def register_customer(self, customer: Customer): self.customers[customer.customer_id] = customer self.carts[customer.customer_id] = Cart(customer) def cart_of(self, customer: Customer) -\u0026gt; Cart: return self.carts[customer.customer_id] # Cart operation def add_to_cart(self, customer: Customer, product: Product, quantity: float): # Inventory is deliberately not checked at this point, consistent # with the earlier decision that stock is checked at checkout only. self.cart_of(customer).add(product, quantity) # Checkout def checkout(self, customer: Customer, address: Address, discount: DiscountStrategy, payment: PaymentStrategy, shipping: ShippingStrategy, observers: Optional[list[OrderObserver]] = None) -\u0026gt; Optional[Order]: \u0026#34;\u0026#34;\u0026#34; The critical flow. Steps: 1. Snapshot the cart to prevent mid-validation changes. 2. Validate stock for every line. 3. Compute the price (subtotal, discount, shipping, total). 4. Charge the payment. 5. On payment success: - Reduce inventory (which is now committed). - Build the Order and attach observers. - Transition the Order to CONFIRMED (which fires notifications). - Clear the cart. 6. On failure: return None; the cart remains untouched. The order of these steps is deliberate; the rationale is detailed in the inline comments below. \u0026#34;\u0026#34;\u0026#34; cart = self.cart_of(customer) if cart.is_empty(): raise ValueError(\u0026#34;Cannot checkout an empty cart\u0026#34;) # Step 1. Snapshot. Build OrderItems now so that any subsequent # cart mutation cannot silently corrupt the order contents. snapshot = [ OrderItem(item.product, item.quantity, item.unit_price_at_add) for item in cart.items() ] # Step 2. Validate stock BEFORE charging. There is no benefit to # taking money that cannot be fulfilled. for item in snapshot: if not self.inventory.has(item.product, item.quantity): raise OutOfStockError( f\u0026#34;{item.product.name}: insufficient stock at checkout\u0026#34; ) # Step 3. Pricing. subtotal = sum(i.line_total() for i in snapshot) discount_amount = discount.compute(cart, customer) shipping_quote = shipping.quote(cart, address) # Guard against negative totals from overly-aggressive discounts. discount_amount = min(discount_amount, subtotal) total = subtotal - discount_amount + shipping_quote.cost # Step 4. Charge. result = payment.pay(total) if not result.success: print(f\u0026#34;[Checkout] Payment failed: {result.message}\u0026#34;) return None # Step 5a. Reduce inventory now that payment has been confirmed. for item in snapshot: self.inventory.reduce(item.product, item.quantity) # Step 5b. Build the order. order = Order( order_id=str(uuid.uuid4())[:8], customer=customer, items=snapshot, shipping_address=address, subtotal=subtotal, discount=discount_amount, shipping_cost=shipping_quote.cost, ) for obs in (observers or []): order.attach(obs) # Step 5c. Transition to CONFIRMED, which triggers notifications. order.confirm_payment() # Step 5d. Clear the cart, but only after all prior steps have # succeeded. If any earlier step raised, the cart remains # available for a retry. cart.clear() self.orders[order.order_id] = order print(f\u0026#34;[Checkout] Order {order.order_id} placed. Total ₹{total:.2f}. \u0026#34; f\u0026#34;{result.message}\u0026#34;) return order Key Design Properties Each dependency (discount, payment, shipping, observers) is injected at checkout call time. OnlineStore hardcodes none of them, in strict adherence to the Dependency Inversion Principle. The step order within checkout is deliberate, with inline comments explaining the rationale (validate first, then charge, then commit inventory, then clear cart). No reference to email, SMS, or specific payment brands appears anywhere in this class.\n14. Stage 13: The Complete Class Diagram The following diagram presents the complete design. The interfaces (represented by dashed inheritance lines in UML) are the only abstractions on which OnlineStore and Order depend. They never depend on concrete implementations, which is the property that makes the design extensible.\nclassDiagram direction LR class OnlineStore { -Inventory inventory -dict products -dict customers -dict carts -dict orders +add_product(product, stock) +register_customer(customer) +add_to_cart(customer, product, quantity) +checkout(customer, address, discount, payment, shipping, observers) Order } class Inventory { -dict _items +add(product, quantity) +has(product, quantity) bool +reduce(product, quantity) +available(product) float } class InventoryItem { +Product product +float quantity } class Product { +String sku +String name +float price_per_unit +ProductUnit unit +Category category } class Category { +String name +Category parent } class Customer { +String customer_id +String name +String email +String phone +bool is_member +list~Address~ addresses } class Address { +String line1 +String city +String pincode } class Cart { -dict _items +add(product, quantity) +remove(product) +subtotal() float } class CartItem { +Product product +float quantity +float unit_price_at_add } class Order { +String order_id +Customer customer +list~OrderItem~ items +Address shipping_address +float total -OrderState _state -list~OrderObserver~ _observers +confirm_payment() +pack() +ship() +deliver() +cancel() +return_order() } class OrderItem { +Product product +float quantity +float unit_price } class OrderState { \u0026lt;\u0026lt;abstract\u0026gt;\u0026gt; +name() String } class PendingState class ConfirmedState class PackedState class ShippedState class DeliveredState class CancelledState class ReturnedState class DiscountStrategy { \u0026lt;\u0026lt;abstract\u0026gt;\u0026gt; +compute(cart, customer) float } class PercentageCoupon class FlatCoupon class MemberDiscount class BuyOneGetOne class CompositeDiscount class PaymentStrategy { \u0026lt;\u0026lt;abstract\u0026gt;\u0026gt; +pay(amount) PaymentResult } class CardPayment class UpiPayment class WalletPayment class CashOnDelivery class ShippingStrategy { \u0026lt;\u0026lt;abstract\u0026gt;\u0026gt; +quote(cart, address) ShippingQuote } class StandardShipping class ExpressShipping class ScheduledShipping class OrderObserver { \u0026lt;\u0026lt;abstract\u0026gt;\u0026gt; +on_state_change(order) } class EmailObserver class SmsObserver OnlineStore o-- Inventory OnlineStore o-- Product OnlineStore o-- Customer OnlineStore o-- Cart OnlineStore o-- Order Inventory *-- InventoryItem InventoryItem --\u0026gt; Product Product --\u0026gt; Category Customer *-- Address Cart --\u0026gt; Customer Cart *-- CartItem CartItem --\u0026gt; Product Order --\u0026gt; Customer Order *-- OrderItem OrderItem --\u0026gt; Product Order --\u0026gt; Address Order o-- OrderState Order o-- OrderObserver OrderState \u0026lt;|-- PendingState OrderState \u0026lt;|-- ConfirmedState OrderState \u0026lt;|-- PackedState OrderState \u0026lt;|-- ShippedState OrderState \u0026lt;|-- DeliveredState OrderState \u0026lt;|-- CancelledState OrderState \u0026lt;|-- ReturnedState DiscountStrategy \u0026lt;|-- PercentageCoupon DiscountStrategy \u0026lt;|-- FlatCoupon DiscountStrategy \u0026lt;|-- MemberDiscount DiscountStrategy \u0026lt;|-- BuyOneGetOne DiscountStrategy \u0026lt;|-- CompositeDiscount CompositeDiscount o-- DiscountStrategy PaymentStrategy \u0026lt;|-- CardPayment PaymentStrategy \u0026lt;|-- UpiPayment PaymentStrategy \u0026lt;|-- WalletPayment PaymentStrategy \u0026lt;|-- CashOnDelivery ShippingStrategy \u0026lt;|-- StandardShipping ShippingStrategy \u0026lt;|-- ExpressShipping ShippingStrategy \u0026lt;|-- ScheduledShipping OrderObserver \u0026lt;|-- EmailObserver OrderObserver \u0026lt;|-- SmsObserver Reading the Diagram in an Interview Context Pointing to the five interfaces (OrderState, DiscountStrategy, PaymentStrategy, ShippingStrategy, and OrderObserver), the design can be summarised as follows:\n\u0026ldquo;These five abstractions constitute the seams of the design. All classes positioned above them represent the core domain (cart, order, inventory). All classes positioned below them represent variation, specifically the strategies. Adding a new discount, payment method, shipping method, notification channel, or order state entails writing a new subclass in one of these slots. OnlineStore and Order remain unchanged. This is the Open/Closed Principle in practical application.\u0026rdquo;\n15. Stage 14: Sequence Diagram for the Checkout Flow The class diagram documents which classes exist and how they are related. It does not document what occurs at runtime when operations execute. The sequence diagram fills that gap and is arguably the most important UML diagram for reviewing a design\u0026rsquo;s runtime behaviour.\nCheckout is the most instructive flow in this design because it engages nearly every collaborator. A clean sequence diagram at this point suggests that the class boundaries are correct. A tangled diagram suggests coupling defects.\nThe Diagram sequenceDiagram autonumber actor Customer participant Store as OnlineStore participant Cart participant Inv as Inventory participant Disc as DiscountStrategy participant Ship as ShippingStrategy participant Pay as PaymentStrategy participant Order participant State as OrderState participant Obs as OrderObserver Customer-\u0026gt;\u0026gt;Store: checkout(customer, address, discount, payment, shipping, observers) Store-\u0026gt;\u0026gt;Cart: items() Cart--\u0026gt;\u0026gt;Store: [CartItem...] Store-\u0026gt;\u0026gt;Store: build OrderItem snapshot loop for each item Store-\u0026gt;\u0026gt;Inv: has(product, qty) Inv--\u0026gt;\u0026gt;Store: True or raise OutOfStockError end Store-\u0026gt;\u0026gt;Disc: compute(cart, customer) Disc--\u0026gt;\u0026gt;Store: discount_amount Store-\u0026gt;\u0026gt;Ship: quote(cart, address) Ship--\u0026gt;\u0026gt;Store: ShippingQuote(cost, eta) Store-\u0026gt;\u0026gt;Store: total = subtotal - discount + shipping Store-\u0026gt;\u0026gt;Pay: pay(total) Pay--\u0026gt;\u0026gt;Store: PaymentResult(success, txn_id) alt Payment success loop for each item Store-\u0026gt;\u0026gt;Inv: reduce(product, qty) end Store-\u0026gt;\u0026gt;Order: new Order(...) [starts in PendingState] Store-\u0026gt;\u0026gt;Order: attach(observer) for each obs Store-\u0026gt;\u0026gt;Order: confirm_payment() Order-\u0026gt;\u0026gt;State: confirm_payment(order) State-\u0026gt;\u0026gt;Order: _transition_to(ConfirmedState) Order-\u0026gt;\u0026gt;Obs: on_state_change(order) Obs--\u0026gt;\u0026gt;Customer: Email or SMS notification Store-\u0026gt;\u0026gt;Cart: clear() Store--\u0026gt;\u0026gt;Customer: Order else Payment failure Store--\u0026gt;\u0026gt;Customer: None [inventory and cart untouched] end Reading the Diagram Several properties of this diagram merit attention. First, no arrow crosses the OnlineStore boundary more than once for the same concern. OnlineStore invokes each collaborator once, receives an answer, and moves on, without back-and-forth chattiness. Second, every collaborator that OnlineStore invokes is either an interface (Discount, Shipping, Payment, Observer) or its own owned data (Cart, Inventory, Order), which is precisely the visual signature of the Dependency Inversion Principle. Third, the alt block cleanly separates success and failure paths, with the failure path consisting of a single return arrow because the step ordering ensures nothing needs to be undone. Fourth, observers are notified through the state transition rather than directly by OnlineStore, which demonstrates the Observer pattern\u0026rsquo;s value: OnlineStore is not even aware that observers exist during notification, since only Order maintains that awareness.\nThe Lifecycle Sequence Diagram (Post-Checkout) Once the order exists, fulfilment is a short and structurally uniform sequence:\nsequenceDiagram autonumber actor Ops as Ops Staff participant Order participant State as OrderState participant Obs as OrderObserver Ops-\u0026gt;\u0026gt;Order: pack() Order-\u0026gt;\u0026gt;State: pack(order) State-\u0026gt;\u0026gt;Order: _transition_to(PackedState) Order-\u0026gt;\u0026gt;Obs: on_state_change(order) Ops-\u0026gt;\u0026gt;Order: ship() Order-\u0026gt;\u0026gt;State: ship(order) State-\u0026gt;\u0026gt;Order: _transition_to(ShippedState) Order-\u0026gt;\u0026gt;Obs: on_state_change(order) Ops-\u0026gt;\u0026gt;Order: deliver() Order-\u0026gt;\u0026gt;State: deliver(order) State-\u0026gt;\u0026gt;Order: _transition_to(DeliveredState) Order-\u0026gt;\u0026gt;Obs: on_state_change(order) Every transition follows the same three-step pattern: a user invokes an operation on Order, Order delegates to the current state, and the state effects the transition and triggers notification. This uniformity is precisely the value that the State pattern provides.\nInterview Explanation \u0026ldquo;I would draw the checkout sequence diagram because that is the flow in which every collaborator interacts. If my class boundaries are incorrect, the sequence diagram becomes cluttered with arrows crossing in multiple directions and multiple round-trips. The diagram here is clean: OnlineStore invokes each collaborator once and proceeds. This constitutes a sanity check on the underlying class design.\u0026rdquo;\n16. Stage 15: Modular Code Structure The code developed throughout this deep dive resides in a production codebase as separate files rather than a single monolithic module. The manner in which the code is partitioned is not decorative; it is itself part of the design. File boundaries determine where compilation, imports, and cognitive load reside.\nThe organising principle followed here is: one file per meaningful design decision.\nThe Layout ecommerce_code/ ├── README.md # scope, structure, and instructions to run ├── demo.py # end-to-end runnable example │ ├── ecommerce/ # the package │ ├── __init__.py # re-exports for convenient imports │ │ │ ├── enums.py # ProductUnit │ ├── exceptions.py # OutOfStockError, InvalidTransition │ │ │ ├── product.py # Product, Category │ ├── customer.py # Customer, Address │ ├── inventory.py # Inventory, InventoryItem │ ├── cart.py # Cart, CartItem │ ├── order.py # Order, OrderItem │ │ │ ├── order_states.py # State pattern: 7 OrderState subclasses │ │ │ ├── discounts.py # Strategy: DiscountStrategy + 6 implementations │ ├── payments.py # Strategy: PaymentStrategy + 4 implementations │ ├── shipping.py # Strategy: ShippingStrategy + 3 implementations │ ├── observers.py # Observer: OrderObserver + 2 implementations │ │ │ └── store.py # OnlineStore, the top-level orchestrator │ └── tests/ ├── test_inventory.py ├── test_order_states.py └── test_checkout.py Rationale for the Layout Three reasonable approaches to organising this code exist. Understanding the choice among them is itself a design lesson.\nAlternative A: Flat layout with everything in one ecommerce.py. This approach is simple but unreadable at scale, since every change forces the reader to reprocess the entire file. It is rejected on those grounds.\nAlternative B: Layered layout with directories such as models/, services/, strategies/, observers/. This approach appears architecturally pure but impedes navigation. Understanding Order would require jumping between models/order.py, services/order_service.py, and other locations. Cohesion should take priority over abstraction hierarchies.\nAlternative C: Domain-grouped layout with one file per concept. This is the selected approach. Each file corresponds to one design decision, with related items grouped (all State classes in order_states.py) and unrelated items separated (payment strategies distinct from shipping strategies).\nThe heuristic applied is: if two files always change together, they should probably be a single file; if one file has multiple reasons to change, it should be split. The order-state classes always change together and therefore reside in one file. Discounts, payments, and shipping vary independently and therefore reside in three separate files.\nFile-to-Decision Mapping Each file crystallises one decision made earlier in this deep dive:\nFile Decision it embodies enums.py Use of an enum for unit rather than a class hierarchy product.py Composition over inheritance for product types inventory.py Separation of stock from catalog as read versus write model cart.py Snapshot of price at add-time to protect the customer order.py Order as a snapshot; transitions delegate to state classes order_states.py Base class forbids every operation; subclasses permit selectively discounts.py Strategy combined with Composite; new promotions are new classes only payments.py Strategy hides gateway details; the caller has one code path shipping.py Strategy accepts Cart rather than Order; quotes precede order creation observers.py Observers self-filter to keep the interface a single method store.py Thin orchestrator; every variable dependency is injected When every file in a real codebase can be mapped back to a design decision in this manner, the codebase is teachable, reviewable, and, most importantly, evolvable. When a developer asks \u0026ldquo;where do I add a new payment method?\u0026rdquo;, there is a single, obvious, correct answer.\nImport Graph A healthy modular design maintains an acyclic import graph. The graph for this design is:\ngraph LR enums[enums.py] exceptions[exceptions.py] product[product.py] customer[customer.py] inventory[inventory.py] cart[cart.py] order[order.py] order_states[order_states.py] discounts[discounts.py] payments[payments.py] shipping[shipping.py] observers[observers.py] store[store.py] product --\u0026gt; enums inventory --\u0026gt; product inventory --\u0026gt; exceptions cart --\u0026gt; product cart --\u0026gt; customer cart --\u0026gt; enums order --\u0026gt; product order --\u0026gt; customer order_states --\u0026gt; exceptions order --\u0026gt; order_states observers --\u0026gt; order discounts --\u0026gt; cart discounts --\u0026gt; customer shipping --\u0026gt; cart shipping --\u0026gt; customer store --\u0026gt; product store --\u0026gt; customer store --\u0026gt; inventory store --\u0026gt; cart store --\u0026gt; order store --\u0026gt; discounts store --\u0026gt; payments store --\u0026gt; shipping store --\u0026gt; observers store --\u0026gt; exceptions Every dependency flows upward toward store.py; nothing lower depends on anything higher. This property yields two practical benefits. First, the code becomes straightforwardly testable in isolation, since unit tests for discounts.py require no dependency on OnlineStore. Second, the code becomes easier to reason about, since each file operates within a small, bounded universe.\nOne subtlety warrants mention: order.py imports order_states.py, since Order requires PendingState as its initial state, and order_states.py type-hints Order. This natural pairing is resolved cleanly using TYPE_CHECKING-guarded imports and a deferred import inside Order.__init__, so that no runtime cycle exists.\nRunning the Code python demo.py # runs the end-to-end scenario python -m unittest discover tests -v # runs all 19 tests The code has no external dependencies and relies solely on the Python standard library. This constraint is intentional rather than incidental: the design is not propped up by a framework. If Django, FastAPI, or SQLAlchemy were to become unavailable, the design would still stand.\n17. Stage 16: Validating the Design Through a Mental Walkthrough A design should not be trusted until a realistic scenario has been traced through it. This validation step catches defects that inspection alone cannot detect.\nScenario: A Member Purchases Groceries with a Coupon The scenario is as follows. Priya is a member. She browses the catalog, adds items to her cart, applies a coupon, checks out using UPI with Standard shipping, and the resulting order proceeds through the lifecycle until it is delivered.\nStep 1: Setup of catalog, inventory, and store.\n# Catalog and inventory inv = Inventory() store = OnlineStore(inv) groceries = Category(\u0026#34;Groceries\u0026#34;) tomatoes = Product(\u0026#34;SKU-TOM\u0026#34;, \u0026#34;Tomatoes\u0026#34;, 40.0, ProductUnit.KG, groceries) biscuits = Product(\u0026#34;SKU-BIS\u0026#34;, \u0026#34;Biscuits\u0026#34;, 30.0, ProductUnit.PIECE, groceries) store.add_product(tomatoes, initial_stock=100) # 100 kg store.add_product(biscuits, initial_stock=200) # 200 packs # Customer priya = Customer(\u0026#34;C-001\u0026#34;, \u0026#34;Priya\u0026#34;, \u0026#34;priya@example.com\u0026#34;, \u0026#34;+91-9xxxxxxxxx\u0026#34;, is_member=True) priya.addresses.append(Address(line1=\u0026#34;14 MG Road\u0026#34;, city=\u0026#34;Bengaluru\u0026#34;, pincode=\u0026#34;560001\u0026#34;)) store.register_customer(priya) Step 2: Add to cart.\nstore.add_to_cart(priya, tomatoes, 1.5) # 1.5 kg store.add_to_cart(priya, biscuits, 3) # 3 packs # subtotal = 1.5 * 40 + 3 * 30 = 60 + 90 = 150 Step 3: Checkout.\ndiscounts = CompositeDiscount([ MemberDiscount(percent=5), # 5% off subtotal = 7.5 PercentageCoupon(\u0026#34;SAVE10\u0026#34;, percent=10), # 10% off = 15 ]) # Total discount = 22.5 payment = UpiPayment(\u0026#34;priya@upi\u0026#34;) shipping = StandardShipping() # subtotal 150 \u0026lt; 500, so shipping is ₹49 observers = [ EmailObserver(priya.email), SmsObserver(priya.phone), ] order = store.checkout( customer=priya, address=priya.addresses[0], discount=discounts, payment=payment, shipping=shipping, observers=observers, ) # Expected total = 150 - 22.5 + 49 = ₹176.50 # The Order should be in CONFIRMED state, having transitioned from # Pending to Confirmed upon successful payment. Tracing the execution: checkout produces a snapshot of the cart. Stock validation confirms 100 kg is at least 1.5 kg and 200 units is at least 3 units. Pricing yields subtotal=150, discount=22.5, shipping=49, and total=176.50. Payment succeeds. Inventory is reduced: tomatoes from 100 to 98.5, biscuits from 200 to 197. The Order is created in the PENDING state, and observers are attached. Calling order.confirm_payment() transitions the state to CONFIRMED and triggers notification. The EmailObserver sends \u0026ldquo;Your order is confirmed!\u0026rdquo;, while the SmsObserver does not send, since CONFIRMED is not in its notification set (consistent with the filter policy). The cart is then cleared.\nStep 4: Fulfilment.\norder.pack() # CONFIRMED -\u0026gt; PACKED (no email or SMS trigger) order.ship() # PACKED -\u0026gt; SHIPPED (both email and SMS trigger) order.deliver() # SHIPPED -\u0026gt; DELIVERED (both email and SMS trigger) Each transition prints its state change and triggers notification, producing clean output.\nStep 5: An attempted illegal transition.\ntry: order.cancel() # DELIVERED cannot be cancelled except InvalidTransition as e: print(f\u0026#34;Rejected as expected: {e}\u0026#34;) The State pattern converts this attempt into an immediate and loud error, rather than silent corruption.\nA Potential Defect Worth Considering Reconsidering the checkout method, one edge case merits attention. If payment.pay(...) raises an unhandled exception rather than returning success=False, inventory has not yet been reduced and the cart has not been cleared, so those are safe. However, the payment attempt has neither been logged nor cleaned up.\nFor LLD scope this is acceptable, and would be noted as follows: in production, the payment call should be wrapped in a try/except, the failure logged, and a retry potentially enqueued. For interview scope, this observation should be raised as a defensive-programming note rather than a design defect.\nA More Substantial Concern: The Ordering Bug If inventory had been reduced before charging payment, and payment then failed, stock would have been consumed for an order that never materialised. In a multi-tenant environment with concurrent operations and no reservation mechanism, one customer\u0026rsquo;s failed payment could block another customer\u0026rsquo;s successful one. The chosen ordering (validate, then charge, then reduce) avoids this class of bug. This point should be raised explicitly during the interview:\n\u0026ldquo;I deliberately validate stock first, then charge payment, then reduce inventory. If inventory were reduced first and payment then failed, a compensation mechanism would be required, which is essentially a full rollback problem. Charging first keeps the failure path simple.\u0026rdquo;\n18. Stage 17: Anticipating Follow-Up Questions Question: How would concurrency be handled at checkout? Consider two customers attempting to purchase the last unit of a product simultaneously. The has followed by reduce sequence constitutes a check-then-act pattern, which is unsafe under concurrent execution. Both threads observe one unit available, both proceed to reduce, and the second reduce fails with OutOfStockError, but only after both customers may have been charged.\nThree approaches address this concern:\nLocking within Inventory: wrap the has and reduce operations per SKU in a mutex. This approach is simple and correct but serialises all writes on a given SKU. Atomic decrement with rollback: the reduce operation returns success or failure atomically; on failure, the just-completed payment is refunded. This approach is more complex. Reservation at add-to-cart: stock is held for a defined duration such as N minutes. This approach eliminates the race upfront but complicates cart user experience. The recommended starting point is option 1, with options 2 and 3 mentioned as evolutionary paths.\nQuestion: How would multiple warehouses be supported? Introduce a Warehouse entity. Each product then has stock per warehouse, and Inventory becomes multi-warehouse-aware:\nclass Inventory: def has(self, product, quantity, warehouse=None): ... def reduce(self, product, quantity, warehouse=None): ... The checkout logic then selects a warehouse, either the one closest to address.pincode or splitting the order across multiple warehouses. The order-splitting requirement is its own subproblem and would be treated as a separate extension.\nQuestion: How would persistence be implemented? Repository classes would be introduced:\nclass OrderRepository(ABC): @abstractmethod def save(self, order: Order): ... @abstractmethod def find(self, order_id: str) -\u0026gt; Optional[Order]: ... OnlineStore would depend on the interface rather than on any specific storage mechanism. Concrete implementations would include InMemoryOrderRepo, SqlOrderRepo, and MongoOrderRepo. This is the same Dependency Inversion approach used for the strategy classes.\nQuestion: How would cart persistence across sessions be implemented? Move carts to a persistent store such as Redis, keyed by customer_id. The cart would be serialised to JSON and restored on login. The Cart class itself requires no change; only its storage location changes.\nQuestion: How would product recommendations be supported? Recommendations are outside LLD scope but can be structured cleanly. A RecommendationEngine observes orders (an application of the Observer pattern) and updates its internal model. The product page queries the engine at render time. This should be noted as a follow-on system rather than a feature to design within the current scope.\nQuestion: How would flash sales with per-customer quantity limits be supported? Two constraints apply. A total quantity limit across all customers requires an inventory-level cap. A per-customer limit requires a check in the checkout flow against that customer\u0026rsquo;s order history.\nBoth constraints are cross-cutting rules. A clean approach involves introducing a CheckoutValidator chain implementing the Chain of Responsibility pattern. Each validator checks one rule. Adding a new rule requires only a new validator without modification of existing code.\nQuestion: Where should input validation reside? Validation is best distributed across three layers. The API layer (outside the scope of this design) validates types, presence, and format. The domain layer (as implemented here) validates business rules such as positive quantity and non-empty cart. The data layer enforces constraints at the database level. Duplication across layers is intentional and constitutes defence in depth.\nQuestion: How would this design be tested? Because dependencies are injected, testing becomes straightforward:\nEach PaymentStrategy is effectively a pure function and is trivially unit-testable. Inventory has no dependencies, so tests reduce to pure state assertions. Order transitions can be tested per state, without scaffolding. OnlineStore.checkout can be tested with mock strategies such as AlwaysSucceedPayment, AlwaysFailPayment, and RecordingObserver. A representative test:\ndef test_checkout_reduces_inventory_only_on_payment_success(): inv = Inventory() store = OnlineStore(inv) # ... set up product, customer, cart ... class FailingPayment(PaymentStrategy): def pay(self, amount): return PaymentResult(False, \u0026#34;declined\u0026#34;) result = store.checkout(customer, address, NoDiscount(), FailingPayment(), StandardShipping()) assert result is None assert inv.available(product) == initial_stock # nothing reduced assert not store.cart_of(customer).is_empty() # cart intact The test requires no mocking framework and no monkey-patching, which is a direct payoff of the design.\n19. Final Reflection: What This Problem Teaches The reader who has followed the full derivation has now designed a system with the following properties: five Strategy families (Discount, Payment, Shipping, and two implicit ones through the pricing Composite and the Observer\u0026rsquo;s message routing); one state machine with seven states and correct transition guards; one Observer implementation supporting clean multi-channel notifications; and a top-level orchestrator that remains under 100 lines of code.\nThis is substantial territory covered in a single problem. The meta-lessons deserve explicit articulation.\nLesson 1: The Same Patterns, Applied Repeatedly Strategy appears four times in this design. The repetition is not gratuitous; rather, four distinct concerns (pricing, payment, shipping, and notification routing) each vary independently. When a real concern varies, Strategy is the appropriate answer, without exception.\nThe valuable skill to develop is the recognition of the shape: \u0026ldquo;this concern will change independently, with multiple valid implementations.\u0026rdquo; This recognition is more valuable than memorisation of the pattern itself.\nLesson 2: State Pattern Justifies Itself at Four or More States The Order class has 7 states and 6 operations. Under a status: str design, 42 potential (state, operation) combinations would require manual verification within a single large if-ladder. The State pattern reduces 42 combinations to 42 default raise InvalidTransition invocations, with the legal transitions overriding to permit. This constitutes a substantial maintainability improvement.\nFor 2 to 3 states, an enumeration remains appropriate. For 5 or more states, the State pattern earns its complexity.\nLesson 3: Checkout Step Ordering Is Design The step order in OnlineStore.checkout (snapshot, validate, price, charge, reduce, build, notify, clear) is designed so that failure at any step leaves the system in a consistent state. Failure before charging: the cart is untouched and there are no side effects. Failure after charging: the order is still created and the charge is real, which is the commitment made to the customer. Failure after reduction: the order is created and paid, and inventory is correctly decremented.\nSequencing is design. Careless sequencing is one of the leading causes of production bugs in real systems.\nLesson 4: Snapshots Prevent Silent Corruption OrderItem captures unit_price at checkout time. If the catalog price changes subsequently, historical orders remain unaffected. Similarly, CartItem.unit_price_at_add protects the customer\u0026rsquo;s experience. The general principle is: design for time, since data at the time of a transaction is often different from data now.\nLesson 5: Interfaces Are the Real Design Every diagram in this document contains dashed inheritance arrows (--|\u0026gt;) pointing to abstractions. The seams (DiscountStrategy, PaymentStrategy, ShippingStrategy, OrderObserver, OrderState) are the locations at which the system flexes. Every other class serves as connective structure.\nWhen reviewing any LLD design, the primary question is not \u0026ldquo;does this execute correctly?\u0026rdquo; but rather \u0026ldquo;where can this bend?\u0026rdquo; If bending the design requires editing many concrete classes, the seams are in the wrong location.\nLesson 6: The Class Diagram Is Not Cosmetic Drawing the class diagram prior to writing code catches boundary defects that would otherwise be detected mid-implementation. Drawing the diagram after writing code (as done here) serves as a quality check: a tangled diagram implies tangled code.\nAn interviewer observing a candidate sketch a class diagram observes an engineer who thinks structurally rather than syntactically. This capability is worth practising deliberately.\nPractice Questions The reader should attempt the following questions before consulting the sketch answers.\nQuestion 1: Add \u0026ldquo;Gift Wrapping\u0026rdquo; The requirement: for an additional ₹49, the order is gift-wrapped. Where does this feature integrate into the design?\nSketch answer: Either introduce a new ShippingStrategy decorator such as GiftWrap(base_shipping) that adds ₹49 to any wrapped shipping\u0026rsquo;s quote, or introduce a separate Addon concept charged as a distinct line item if wrapping is independent of shipping. Both approaches are viable. The Decorator approach reuses the shipping interface; the line-item approach is more accurate for reporting purposes. Both should be discussed with the interviewer.\nQuestion 2: Add \u0026ldquo;Scheduled Auto-Repeat Orders\u0026rdquo; The requirement: customers can specify \u0026ldquo;deliver this cart every Monday\u0026rdquo;. Where does this feature reside?\nSketch answer: Introduce a Subscription entity, separate from Order, that stores a cart template combined with a schedule. A SubscriptionScheduler (which would be a background job in production and an explicit call in this design) triggers checkout(...) according to the schedule. Order and all classes below it require no modification, which constitutes a design win.\nQuestion 3: Reserve stock at add-to-cart rather than only at checkout Sketch answer: Add a reserved: float field to InventoryItem. Compute available = quantity - reserved. Cart.add now invokes inventory.reserve(product, quantity), and Cart.remove invokes inventory.release(...). Reservations expire after N minutes, which requires a scheduler. The trade-off is that stock cannot be \u0026ldquo;sniped\u0026rdquo; by another customer, but carts become more expensive to maintain and abandoned-cart handling becomes more complex.\nQuestion 4: Coupon and BOGO cannot be combined on the same order Sketch answer: The CompositeDiscount becomes more sophisticated. Rather than a simple summation, it applies conflict rules, possibly sorting by priority with mutually-exclusive tags. A cleaner approach introduces DiscountRule objects that describe when a discount applies, and a DiscountEngine selects the optimal combination. The trade-off is between structural cleanliness and additional complexity.\nQuestion 5: Return and Refund Flow Sketch answer: Add states RETURN_REQUESTED, RETURN_APPROVED, RETURNED, and REFUNDED. The transitions form a mini state machine within the DeliveredState. Refund becomes a reverse payment operation; PaymentStrategy acquires a refund(amount) method. This arguably violates the Interface Segregation Principle for COD, which has no refund pathway, so a split into a RefundablePayment interface would be worth discussing.\nSummary and Key Takeaways The following points capture the essential lessons of this case study:\nE-commerce is a \u0026ldquo;system of systems\u0026rdquo;. The top-level orchestrator (OnlineStore) remains thin only because every collaborator sits behind an interface. Five clean seams (DiscountStrategy, PaymentStrategy, ShippingStrategy, OrderState, OrderObserver) are the locations where all future change will occur. Every other component remains stable. The State pattern is the appropriate tool once a lifecycle contains four or more meaningful states with multiple state-dependent operations. The Strategy pattern answers the question \u0026ldquo;how do I vary this behaviour without modifying callers?\u0026rdquo; The pattern was applied four times in this design, each application justified independently. The Observer pattern decouples \u0026ldquo;an event has occurred\u0026rdquo; from \u0026ldquo;who is interested\u0026rdquo;. Adding a new channel requires only a new observer, with no other changes. Class diagrams reveal design quality. A tangled diagram implies tangled code; a clean diagram with abstractions occupying central positions implies a design that bends where it needs to. Sequencing is a design decision. The checkout step order was chosen so that failure at any step leaves the system in a consistent state. The YAGNI principle governs scope throughout. Warehouses, product images, reviews, and wallet ledgers are all absent from the design because none was requested, with clear extension points documented for each. Summary Table of Design Patterns Used Pattern Location Purpose Strategy DiscountStrategy, PaymentStrategy, ShippingStrategy Vary algorithm without modifying caller State OrderState hierarchy Encapsulate state-dependent behaviour Observer OrderObserver hierarchy Decouple event source from event consumers Composite CompositeDiscount Combine multiple discounts uniformly Null Object NoDiscount Simplify caller code by removing null checks Dependency Injection OnlineStore.checkout parameters Depend on abstractions, not concretions Snapshot / Value Object OrderItem, CartItem.unit_price_at_add Preserve historical data against future changes This content is part of Codeverra, a platform for learning coding, data science, DSA, and AI from scratch. Explore more: https://codeverra.com\n","permalink":"https://blogs.codeverra.com/blog/low_level_design/interview_questions/06e_lld_ecommerce_deep_dive/","summary":"\u003ch1 id=\"lld-deep-dive-4-designing-a-grocery--e-commerce-management-system\"\u003eLLD Deep Dive #4: Designing a Grocery / E-Commerce Management System\u003c/h1\u003e\n\u003cblockquote\u003e\n\u003cp\u003e\u003cstrong\u003eWhat this document is:\u003c/strong\u003e A step-by-step walkthrough of the reasoning process required to design an e-commerce LLD problem in an interview setting. The specific flavour is a grocery store, though the design generalises to any online retail system. Every design decision is derived from first principles, alternatives are evaluated at each fork, and the resulting system is one whose components can be explained, defended, and extended.\u003c/p\u003e","tags":null,"title":""},{"content":"LLD Deep Dive #4: Designing a Ride-Sharing System What this document is. This is a comprehensive, first-principles walkthrough of how to design a ride-sharing system (of the Uber or Ola variety) as a low-level design (LLD) interview problem. It reasons from the vague problem statement through requirement clarification, entity identification, class hierarchy decisions, behavioural modelling with design patterns, and finally a fully composed system. Every design choice is stated together with the alternatives that were considered and the criteria that decided between them, so that a learner reading this can not only reproduce the design but also defend it and adapt it.\nTable of Contents Learning Objectives and How to Use This Document Why the Ride-Sharing Problem Matters Prerequisite Vocabulary Stage 1: Receiving the Problem Statement Stage 2: Clarifying Requirements Stage 3: Identifying Entities and Weighing the Class Hierarchy Decision Stage 4: Modelling Users, Riders, and Drivers Stage 5: Locations, Vehicles, and the Composition Choice Stage 6: The Trip Lifecycle and the State Pattern Stage 7: Matching a Rider to a Driver (Strategy Application 1) Stage 8: Pricing, Surge, and Layered Strategies (Strategy Application 2) Stage 9: Payment Methods (Strategy Application 3 with Factory) Stage 10: Cancellation Encoded as a Policy Object Stage 11: The RideManager as a Facade Stage 12: Validating the Design with Mental Walkthroughs Stage 13: Anticipating Follow-Up Questions Summary of Design Decisions Key Takeaways Practice Questions and Solutions Further Reading and Cross-References 1. Learning Objectives and How to Use This Document After completing this deep dive, a learner should be able to:\nReproduce the entire ride-sharing design from a blank sheet of paper, including class relationships, state transitions, and pattern choices. Defend every non-trivial choice by naming the alternatives that were rejected and the criterion that eliminated each one. Explain how the three separate Strategy applications (matching, pricing, payment) coexist without one leaking into another. Identify which SOLID principle each modelling decision preserves. Extend the design to accommodate common follow-up requirements such as pool rides, scheduled rides, real-time notifications, and geospatial scaling of driver lookup. The document is intended to be read linearly. Each stage assumes the previous stage has been understood. If a reader is comfortable with the general pattern of LLD interviews and wants only the design rationale, the summary in section 17 together with the code listings inside sections 6 through 14 will be sufficient. Everyone else is advised to read from the beginning.\n2. Why the Ride-Sharing Problem Matters The ride-sharing problem occupies a distinguished place in the standard low-level design curriculum. Where the parking-lot problem is largely a test of class hierarchies, the elevator problem is largely a test of state machines, and the library management problem is largely a test of domain modelling, ride-sharing exercises all three of these skills simultaneously and adds a further dimension of pluggable algorithms. The following characteristics make it uniquely valuable as a diagnostic problem for evaluating design maturity.\nThe system contains several pluggable algorithms that must coexist. Selecting a driver for a given rider is one pluggable choice: today the platform may prefer proximity, tomorrow it may prefer driver rating, next quarter it may prefer some blend of both. Computing the fare for a completed trip is a second pluggable choice: distance-and-time-based pricing is standard, but flat-rate airport pricing and subscription-based pricing are all common in real deployments. Applying surge to a base fare is a third concern that varies independently of the base pricing algorithm. Handling payment is a fourth axis of variation, because cash, card, UPI, and wallet all involve different flows and different failure modes. A design that fails to isolate these axes will produce a monolithic manager class in which every pricing tweak requires touching payment code, and every matching change risks breaking cancellation logic.\nA trip has a rich lifecycle. A trip is not a static record. It is a stateful object that passes through several qualitatively distinct phases, namely request, driver assignment, in-progress travel, completion, and cancellation. Each phase permits a different set of operations. A ride that has already been completed cannot be cancelled; a ride whose driver has not yet been assigned should not be startable; and the cost of cancellation depends on which phase the trip was in when the cancellation was requested. This is a textbook motivation for the State design pattern.\nTwo roles interact through a shared substrate. A rider requests a ride; a driver accepts one. Both are people with accounts on the platform, and both have profile data, ratings, and notification preferences in common. The question of whether to model them as a single class with a role attribute, as two entirely separate classes, or as subclasses of a common base is a genuine design decision that reveals a candidate\u0026rsquo;s understanding of substitutability and single responsibility.\nThe system is naturally event-driven. State changes in a trip should notify both parties. New requests should be broadcast to nearby drivers. A driver going online should update matching availability. In a full production system these are handled through an event bus, a publish-subscribe queue, and push notification services. Even at the LLD scope, the design should be structured such that adding an Observer layer later requires no changes to existing classes.\nScale considerations are latent in every method. A naive nearest-driver scan over the entire driver population is O(N) per request. In production, the pool of drivers is partitioned by a geospatial index (a geohash grid, an H3 cell, or an R-tree). The design must at least allow such an index to be dropped in without disturbing the rest of the system. An LLD design that hardcodes the linear scan into the manager will require a rewrite when the scaling question is posed; a well-decomposed design will require only a new implementation of an existing interface.\nFor all of these reasons the ride-sharing problem tests the single most important skill in LLD, which is the ability to decompose a system into independent concerns and compose them back through well-defined interfaces. A candidate who cannot do this will produce a god class named RideManager that contains matching logic, pricing logic, surge logic, payment logic, and state transitions, all held together with conditional statements. A candidate who can do this will produce a lean manager that owns nothing except references to strategy objects, each of which can be replaced without touching any other.\nThe remainder of this document works through the problem stage by stage, in the order that these stages would be encountered during an actual interview.\n3. Prerequisite Vocabulary Before proceeding, several terms need precise definitions. These terms will recur throughout the document and precision here saves confusion later.\nTerm Definition Entity An object with identity that persists through changes to its attributes. Two entities are equal only if they share the same identity, not merely the same attribute values. Value object An object without identity, defined entirely by its attributes. Two value objects with equal attributes are considered equal. Value objects are commonly immutable. Aggregate root An entity that owns a cluster of related objects and mediates all external access to them. In this design, the Trip acts as an aggregate root over its state, driver reference, and payment record. Strategy pattern A behavioural design pattern that encapsulates an algorithm behind an interface so that different algorithms can be substituted at runtime without altering the client that uses them. State pattern A behavioural design pattern in which an object appears to change its class as its internal state changes, achieved by delegating state-dependent behaviour to a separate state object. Facade pattern A structural design pattern that provides a simplified interface to a subsystem of related classes, hiding the internal complexity from external callers. Factory pattern A creational design pattern in which the construction of an object is deferred to a dedicated function or class, isolating callers from the details of instantiation. Open/Closed Principle (OCP) The SOLID principle stating that software entities should be open for extension but closed for modification. New behaviour should be added by writing new code, not by editing existing code. Single Responsibility Principle (SRP) The SOLID principle stating that a class should have exactly one reason to change, meaning it should encapsulate only one axis of variation. Liskov Substitution Principle (LSP) The SOLID principle stating that objects of a subtype must be usable wherever objects of the supertype are expected, without altering the correctness of the program. Interface Segregation Principle (ISP) The SOLID principle stating that clients should not be forced to depend on interfaces they do not use. Dependency Inversion Principle (DIP) The SOLID principle stating that high-level modules should depend on abstractions, not on concrete implementations. Geospatial index A data structure optimised for querying spatial data. Examples include geohash-based grids, quadtrees, R-trees, and Uber\u0026rsquo;s H3 hexagonal grid. In production ride-sharing systems, drivers are stored in such an index so that nearest-neighbour queries run in sub-linear time. Surge multiplier A dynamic scalar (typically between 1.0 and about 5.0) applied to a base fare to reflect current supply and demand conditions in the pickup area. With these terms established, the actual design work begins.\n4. Stage 1: Receiving the Problem Statement In a real interview, the problem statement is typically brief and deliberately underspecified. A representative opening is:\n\u0026ldquo;Design a ride-sharing system, something like Uber or Ola.\u0026rdquo;\nThis single sentence is the entirety of the initial specification. Nothing else is provided. The candidate must interpret this as the starting point of a conversation, not as a complete requirement document. Several qualitatively different systems are compatible with this description, and the design that emerges will depend heavily on which interpretation is adopted. The main axes along which interpretations vary are:\nScope of user roles. Is the design confined to the rider-facing experience, or does it include the driver-facing side as well? Realistically both sides are needed for the trip lifecycle to make sense, so both are typically in scope. Geographic scope. Is this a single-city deployment, a national one, or a global one? Geographic scope affects assumptions about time zones, currency, regulatory constraints, and geospatial partitioning. Vehicle categories. Is the platform limited to sedans, or does it include a wider set (mini, sedan, SUV, auto-rickshaw, bike, and possibly food-delivery two-wheelers)? The presence of multiple vehicle types affects both pricing and matching. Trip types. Are only point-to-point on-demand trips supported, or are scheduled trips, outstation trips, and shared (pool) trips also in scope? Each additional trip type changes the lifecycle and introduces new concerns. Auxiliary concerns. Is payment in scope, or is it treated as external? Are ratings supported? Cancellations? Surge pricing? Loyalty programmes? Matching model. Is matching centralised (the platform assigns the driver) or auction-style (drivers bid on requests)? These are architecturally very different systems. Deployment model. Is this an in-memory single-instance design suitable for an LLD interview, or is it distributed across multiple servers with all the coordination that implies? The vagueness of the problem statement is intentional on the interviewer\u0026rsquo;s part. It is a test of whether the candidate will begin by writing classes (which is the wrong instinct) or begin by asking questions (which is the correct instinct). The correct response is always the second: seek clarification before committing to design decisions that will constrain everything that follows.\n5. Stage 2: Clarifying Requirements Clarification is organised into three categories: functional requirements (what the system must do), non-functional requirements (what qualities the system must have), and explicit non-goals (what the system will deliberately not do within the scope of this design). Working through these categories in an ordered fashion prevents important considerations from being forgotten and demonstrates methodical thinking to the interviewer.\nEach question below is stated together with the rationale for asking it, so that the reader can generalise the pattern to problems outside this specific one.\n5.1 Question: What is the end-to-end flow of a typical trip? Without an agreed flow, it is impossible to distinguish essential entities from optional ones. A flow that involves pre-booking will need a scheduler; a flow that involves bidding will need an auction model; a flow that is purely on-demand will need neither. The candidate should ask for a description of the happy path.\nA representative interviewer response is:\n\u0026ldquo;A rider opens the app, requests a ride from a current location to a chosen destination, the system selects a suitable nearby driver, the driver accepts, drives to the pickup point, starts the trip on arrival, drives to the destination, ends the trip, the rider pays, and both parties rate each other.\u0026rdquo;\nThis description commits the design to three things implicitly. First, matching is assignment-based rather than auction-based, because the system selects the driver rather than the driver bidding. Second, trips are on-demand rather than scheduled, because no booking-for-later step appears in the flow. Third, each trip has one driver rather than several, because the description mentions a driver rather than drivers. Each of these commitments is important and should be noted.\n5.2 Question: What vehicle categories are supported? The presence or absence of multiple categories determines whether vehicle type is a first-class concept in the design. A single-vehicle system requires no dispatch on type; a multi-vehicle system requires that pricing, matching, and possibly cancellation all be aware of type.\nA representative response is:\n\u0026ldquo;Three types are supported, namely mini, sedan, and SUV. Each has different base fares and per-kilometre rates.\u0026rdquo;\nThis confirms that VehicleType will be a real enumeration used at least by the pricing component (to look up rates) and by the matching component (to filter candidates by requested type). The question of whether vehicle types justify a full class hierarchy is deferred to the entity-identification stage, where it will be considered explicitly.\n5.3 Question: How is matching performed? Matching is the single most substantive algorithmic component of the system and therefore the most important to design for variability. If the interviewer says \u0026ldquo;nearest driver,\u0026rdquo; the candidate should probe whether that is the only policy that will ever be used, because the answer heavily influences whether matching should be extracted as a Strategy.\nA representative response is:\n\u0026ldquo;For now, the nearest available driver of the requested vehicle type. Ranking by driver rating may be added later.\u0026rdquo;\nThe phrase \u0026ldquo;for now\u0026rdquo; is a strong signal that matching logic must be pluggable. The candidate should treat this as an explicit request to design for extension, which is exactly what the Strategy pattern is intended to enable.\n5.4 Question: Is surge pricing supported? Surge pricing is a dynamic scalar applied to fares based on current demand and supply. If it is in scope, it must integrate cleanly with base fare computation; if it is not, an entire concern can be omitted from the design.\nA representative response is:\n\u0026ldquo;Yes. During peak hours, prices are multiplied by an area-specific factor.\u0026rdquo;\nSurge is confirmed as an in-scope concern that varies independently of base pricing. This suggests that surge should be modelled as its own component that composes with pricing rather than being embedded within it.\n5.5 Question: What payment methods are supported? Payment method affects the design in two ways. First, each method has different behaviour and different failure modes, which motivates polymorphism. Second, the integration with an external gateway is often out of scope for LLD interviews, and confirming this saves time.\nA representative response is:\n\u0026ldquo;Card, UPI, wallet, and cash. The actual payment gateway integration is out of scope; simply record that a payment was made.\u0026rdquo;\nThis confirms four payment methods, which is enough variety to justify the Strategy pattern, and simultaneously scopes out the external integration concern.\n5.6 Question: Are ratings supported? Ratings, if present, must be recorded per trip and aggregated per user. They may also influence future matching. If ratings are out of scope, an entire attribute and its associated update logic can be removed from the User model.\nA representative response is:\n\u0026ldquo;Yes, in both directions. The rider rates the driver, the driver rates the rider, and this happens after each completed trip.\u0026rdquo;\nRatings are bidirectional and per-trip. A running average lives on the User entity so that a driver\u0026rsquo;s overall rating is available at matching time without recomputing from history.\n5.7 Question: Is cancellation supported, and if so with what fee structure? Cancellation is a common source of subtle design bugs, particularly around state validation and fee computation. Establishing the fee structure early makes the state design cleaner.\nA representative response is:\n\u0026ldquo;Yes. Cancellation is free before a driver has been assigned, incurs a small fee if cancelled after driver acceptance but before pickup, and incurs a larger fee if cancelled after the trip has started.\u0026rdquo;\nThis response confirms two important design consequences. First, the fee depends on the lifecycle phase at cancellation time, which is direct evidence that the trip has a genuine lifecycle deserving of a State pattern. Second, the fee schedule itself is a policy that could change independently of the state machine, which suggests it should be extracted as a separate policy object.\n5.8 Question: What are the non-functional expectations regarding concurrency, persistence, and scale? These questions distinguish an LLD interview (which focuses on object-oriented design) from a high-level design interview (which focuses on distributed systems). The interviewer typically expects the candidate to note these concerns and set them aside for the follow-up discussion.\nA representative response is:\n\u0026ldquo;Assume an in-memory, single-threaded, single-instance design. Concurrency, persistence, and geospatial scaling should be mentioned at the end as extensions rather than built into the initial design.\u0026rdquo;\nThis response gives permission to build a clean in-memory design and defer the difficult distributed concerns to a follow-up discussion. This is the standard scope for an LLD interview and should be accepted gratefully.\n5.9 Summary of Clarified Scope After the clarification round, the candidate should restate the agreed scope explicitly to confirm alignment with the interviewer. A concise restatement follows.\nConcern Decision Region Single region, in-memory, single-instance Trip type On-demand, single-driver-per-trip Vehicle types Mini, sedan, SUV (each with distinct pricing) Matching Nearest available driver of the requested type, with future support for other policies Surge Peak-hour multiplier, area-aware in interface but time-based by default Payment methods Cash, card, UPI, wallet (gateway integration out of scope) Ratings Bidirectional, one per completed trip, aggregated into a running average Cancellation Allowed at all pre-completion states, with state-dependent fees Concurrency Out of scope for the initial design, discussed as an extension Persistence Out of scope for the initial design, discussed as an extension Geospatial indexing Linear scan for the initial design, discussed as an extension This summary functions as the contract between the candidate and the interviewer. Any correction or addition made at this point will affect the entire subsequent design, so it is worth the minute or two required to state it explicitly.\n6. Stage 3: Identifying Entities and Weighing the Class Hierarchy Decision With the scope agreed, the next task is to identify the entities that will populate the design. This is done by extracting the significant nouns from the requirements and evaluating each one for genuine classhood. Not every noun deserves to become a class; equally, some concepts that are not explicitly named in the requirements must be introduced as classes because they represent axes of variation that the design needs to accommodate.\n6.1 Concrete Entities from the Domain The following entities are directly implied by the requirements. Each represents something that exists in the real world of the system and has a persistent identity or a clear representational role.\nEntity Role Justification User Base for anyone with an account Both riders and drivers share identity, contact, and rating attributes. Rider A user who requests trips Distinct role with distinct operations, warrants a separate type. Driver A user who offers trips Distinct role, plus additional state (vehicle, current location, availability). Vehicle The physical car (or auto or bike) driven by a driver Has identity (licence plate) and metadata (type, model). Trip The core lifecycle object of the system Central aggregate; owns state, references to driver and rider, fare, payment, and ratings. Location A geographic coordinate Value object with equality by attributes, used ubiquitously. Rating A score with an optional comment Recorded per trip; may be a full class or a scalar depending on requirements. Payment The record of a completed monetary transaction Sometimes a full entity, sometimes a value; see section 12 for the reasoning. 6.2 Behavioural Components Not Directly Named in the Requirements The following components are not nouns in the problem statement but are essential to the design because they encapsulate axes of variation. Each one exists to satisfy the Open/Closed Principle for a specific concern.\nComponent Purpose Pattern TripState Formal representation of a trip\u0026rsquo;s lifecycle phase State pattern MatchingStrategy Pluggable algorithm for choosing a driver Strategy pattern PricingStrategy Pluggable algorithm for computing base fare Strategy pattern SurgeCalculator Pluggable component that returns a surge multiplier Strategy pattern (informally) PaymentMethod Pluggable mechanism for money movement Strategy pattern CancellationPolicy Pluggable rule for cancellation-fee computation Policy object (a strategy by another name) RideManager Facade coordinating all of the above Facade pattern 6.3 Nouns That Are Deliberately Not Modelled as Classes An equally important part of entity identification is recognising which nouns from the problem statement should not become classes. Every unnecessary class adds cognitive load, testing burden, and coupling. The following nouns are rejected as classes in this design, with explicit reasons.\nNoun Reason for Rejection App The user-facing application is the caller of this design, not part of it. Notification Without a concrete requirement involving delivery guarantees or history, a class is speculative; the concept is deferred to an Observer extension. Map Geospatial and routing services are external. The design uses Location and distance_to; anything richer is an external dependency. Bank or Gateway The external payment gateway is out of scope by explicit interviewer statement. Route A route with turn-by-turn directions is a real thing in production but has no operational role in the LLD; a trip is adequately described by its source and destination. The discipline of not modelling things is at least as important as the discipline of modelling them. A class that exists without a genuine role is not a neutral cost; it becomes something future maintainers must read, understand, and preserve. Rejecting speculative classes is a demonstration of design restraint.\n6.4 Informal Class Relationships Before writing any code, the relationships between the identified entities are sketched in prose. This informal class diagram guides the subsequent implementation.\nA Trip has-one Rider, has-one Driver (once assigned), has-one Vehicle (populated at driver assignment), has-one Location for its source, has-one Location for its destination, has-one TripState, and eventually has-one final fare and has-one Payment record. A Driver has-one Vehicle and has-one current Location, and carries an is_available flag. The RideManager owns a collection of registered Driver instances and a registry of all Trip instances, and holds references (not copies) to the MatchingStrategy, PricingStrategy, SurgeCalculator, and CancellationPolicy currently configured. TripState is an abstract class with concrete subclasses Requested, DriverAssigned, Started, Completed, and Cancelled. A Trip moves through these states via method calls that are delegated to the current state object. This informal model is a working hypothesis. If a walkthrough later reveals a contradiction or a missing relationship, the model is revised. The next several sections take this hypothesis and translate it into concrete code, one concern at a time.\n7. Stage 4: Modelling Users, Riders, and Drivers The first substantive modelling decision concerns how to represent the two user roles. This is worth treating carefully because it is a design decision that gets made silently and badly by many candidates, when in fact the alternatives have distinctly different consequences.\n7.1 The Three Defensible Approaches There are three internally consistent ways to model users in a ride-sharing system. Each is discussed below with its advantages and disadvantages.\nApproach 1: A single User class with a role attribute. In this design there is exactly one class, and instances differentiate themselves by an attribute such as role == \u0026quot;rider\u0026quot; or role == \u0026quot;driver\u0026quot;. The apparent economy is misleading. Drivers require attributes that riders do not, most notably a vehicle and a current location. If those attributes live on the shared class, riders will carry them as permanently null fields, and every access to a driver-specific attribute will require a guard against the null case. Furthermore, driver-specific and rider-specific methods will accumulate on the same class, forcing the class to become a repository of role-conditional logic. This is the very situation that object-oriented modelling was designed to eliminate.\nApproach 2: Two entirely independent classes, Rider and Driver, that share nothing. This design avoids the null-attribute problem of Approach 1 by giving each role its own class. However, it discards the fact that both are users of the same platform. Any behaviour or attribute that is genuinely shared (identity, contact information, rating, notification preferences, authentication) must be duplicated. If a third role is ever added (a support agent, a fleet operator, an admin), the duplication triples. This design violates the Don\u0026rsquo;t Repeat Yourself principle in a way that grows worse with time.\nApproach 3: A base User class with Rider and Driver as subclasses. This design places the shared attributes and behaviour on the base class and adds role-specific attributes and behaviour on the subclasses. When a real difference in behaviour or state exists, the class hierarchy exposes it plainly rather than hiding it behind runtime checks. This is the approach adopted here.\nThe three approaches are summarised in the following table.\nCriterion Role Flag Independent Classes Base Class with Subclasses Null-attribute pollution High (drivers\u0026rsquo; fields on riders) None None Duplication of shared behaviour None High None Type safety at API boundaries Low (dispatch on flag) High High Extensibility for new roles Low (adds branches everywhere) Low (more duplication) High (new subclass) Adherence to SRP Violated (mixed responsibilities) Satisfied Satisfied Adherence to LSP Violated (behaviour depends on flag) N/A (no shared supertype) Satisfied Approach 3 is chosen for this design because it satisfies the greatest number of criteria and imposes the fewest constraints on future evolution.\n7.2 Implementation of the User Hierarchy The base class captures the shared identity and rating attributes.\nfrom dataclasses import dataclass, field from typing import Optional class User: \u0026#34;\u0026#34;\u0026#34; Base class for anyone with an account in the system. Holds identity and contact information along with a running rating. Both riders and drivers accumulate ratings from the other party after each completed trip, so the rating attribute is placed on the base class rather than on either subclass. \u0026#34;\u0026#34;\u0026#34; def __init__(self, user_id: str, name: str, phone: str): self.user_id = user_id self.name = name self.phone = phone # Every new user begins at a neutral 5.0 rating so that # no user is systematically disadvantaged by matching # strategies that prefer highly-rated participants. In a # production system this would be more nuanced, for # example using a provisional flag or a Bayesian prior, # but a plain float is sufficient for the design. self.rating: float = 5.0 self.total_ratings: int = 0 def update_rating(self, new_score: int) -\u0026gt; None: \u0026#34;\u0026#34;\u0026#34; Incorporate a new rating (an integer between 1 and 5) into the running average. The running total count is maintained explicitly so that the average is a true cumulative mean rather than a naive blending of the current average with the new score. A naive blend of the form (rating + new_score) / 2 would over-weight recent ratings, which is incorrect. \u0026#34;\u0026#34;\u0026#34; if not 1 \u0026lt;= new_score \u0026lt;= 5: raise ValueError(\u0026#34;Rating must be an integer between 1 and 5\u0026#34;) total = self.rating * self.total_ratings + new_score self.total_ratings += 1 self.rating = total / self.total_ratings The Rider subclass is intentionally empty in the current design.\nclass Rider(User): \u0026#34;\u0026#34;\u0026#34; A user who requests rides. Riders have no persistent state beyond what a User carries. The subclass is retained for two reasons even in the absence of additional state. First, it expresses design intent: a Rider instance cannot be accidentally used where a Driver is expected, which is a form of type safety that catches errors at development time. Second, it provides a natural home for future rider-specific behaviour such as saved addresses, payment preferences, or ride history views. \u0026#34;\u0026#34;\u0026#34; pass A frequent objection is that an empty subclass is a waste of a class. This objection misses the point of subclasses. A class does not need to earn its place by adding fields or methods; it can earn its place by adding type identity. In a strongly-typed function signature, def start_trip(rider: Rider, ...) makes it a compile-time (or in Python, a lint-time) error to pass a driver where a rider is expected. That guarantee is worth the cost of a two-line subclass.\nThe Driver subclass adds the driver-specific attributes.\nclass Driver(User): \u0026#34;\u0026#34;\u0026#34; A user who provides rides. Adds vehicle ownership, current location, and an availability flag. The is_available flag is the piece of driver state that matching strategies inspect when filtering candidates. In a real system this would be more nuanced (on-duty but out of area, on-duty but currently in a fare, off-duty entirely), but a boolean is sufficient for the LLD scope. \u0026#34;\u0026#34;\u0026#34; def __init__( self, user_id: str, name: str, phone: str, vehicle: \u0026#34;Vehicle\u0026#34;, location: \u0026#34;Location\u0026#34;, ): super().__init__(user_id, name, phone) self.vehicle = vehicle self.location = location self.is_available = True def go_online(self) -\u0026gt; None: self.is_available = True def go_offline(self) -\u0026gt; None: self.is_available = False Two design commitments embedded in this code deserve explicit mention.\nFirst, is_available lives on the Driver, not on the Trip. This is a deliberate placement. Availability is a property of the driver as an actor in the world, not a property of any particular trip. A trip does not know whether its driver is available; a driver knows whether they are. The RideManager toggles this flag as part of assignment and release.\nSecond, the Driver composes a Vehicle rather than inheriting from one. A driver has-a vehicle; a driver is not a vehicle. This is the classical composition-over-inheritance judgment, and in this case the correct answer is unambiguous because a driver and a vehicle are qualitatively different kinds of things.\n8. Stage 5: Locations, Vehicles, and the Composition Choice Two supporting types now need to be modelled. Both are structurally simple but conceptually important because they force decisions about value semantics and about when a class hierarchy is or is not appropriate.\n8.1 Location as an Immutable Value Object A Location is a geographic coordinate. It has no identity of its own; two Location(12.97, 77.59) instances are the same location. This is the defining characteristic of a value object.\nThe natural Python implementation is a frozen dataclass, which provides value-based equality, hashability, and immutability with almost no code.\nimport math from dataclasses import dataclass @dataclass(frozen=True) class Location: \u0026#34;\u0026#34;\u0026#34; A geographic point defined by latitude and longitude. Frozen for two reasons. First, value semantics: two locations with the same coordinates should be considered equal, and freezing gives automatic equality by value along with hashability. Second, immutability: a location, once created, should not mutate. When a driver moves, the driver\u0026#39;s location reference is reassigned to a new Location instance rather than mutating the existing one. This eliminates a category of aliasing bugs in which shared references are unexpectedly modified. \u0026#34;\u0026#34;\u0026#34; lat: float lng: float def distance_to(self, other: \u0026#34;Location\u0026#34;) -\u0026gt; float: \u0026#34;\u0026#34;\u0026#34; Compute the approximate straight-line distance in kilometres to another location. The implementation uses the equirectangular approximation, which is accurate over city-scale distances (up to approximately a few dozen kilometres) and computationally cheaper than the Haversine formula. Over larger distances or at extreme latitudes, Haversine or a full geodesic calculation would be required. For the LLD scope, the equirectangular approximation is more than sufficient. For production use, the relevant distance is road distance rather than crow-flight distance, which would be obtained from an external routing service such as Google Maps Directions API or OSRM. The design does not depend on which distance function is used, only on the signature. \u0026#34;\u0026#34;\u0026#34; # Approximately 111 kilometres per degree of latitude. # The longitude conversion is less accurate at higher # latitudes because meridians converge, but for LLD it # is more than sufficient. dlat = self.lat - other.lat dlng = self.lng - other.lng return math.sqrt(dlat * dlat + dlng * dlng) * 111 Placing distance_to as a method of Location rather than as a free function is a stylistic choice with a real basis. At the call site, driver.location.distance_to(trip.source) reads more naturally than distance(driver.location, trip.source), and the method form makes it plain which object owns the operation.\n8.2 Vehicle as an Entity with an Enum Type A Vehicle is not purely a value object because two cars with the same licence plate are the same car; the plate serves as identity. However, a vehicle is small and rarely modified, so the full weight of a mutable entity is not required.\nThe more consequential decision is whether to model vehicle types (mini, sedan, SUV) as an enumeration or as a class hierarchy. This is the decision point at which many novice designs go wrong by defaulting to a hierarchy when an enum would be clearer.\nThe criterion for choosing between an enum and a class hierarchy is whether the different types require different behaviour. If the types differ only in their metadata (base fare, capacity, name), an enum suffices. If the types differ in the methods they support or in how they behave, a class hierarchy is warranted.\nIn the ride-sharing domain, vehicle types are used for two things: pricing looks up rates by type, and matching filters candidates by type. Neither of these is polymorphic behaviour dispatched to the type itself; both are external components that consume the type as data. There is no Mini.start() that differs meaningfully from Sedan.start() in this code. Therefore an enum is the correct choice.\nThis is worth contrasting explicitly with the parking-lot problem, where Bike, Car, and Truck each declared their own required spot size, and where spot-fitting logic was dispatched to the vehicle type. In that domain, type-specific behaviour existed and a class hierarchy was appropriate. In the ride-sharing domain, no such type-specific behaviour exists, and the same modelling choice would be inappropriate.\nfrom enum import Enum from dataclasses import dataclass class VehicleType(Enum): \u0026#34;\u0026#34;\u0026#34; Enumeration of supported vehicle categories. Ordered by capacity from smallest to largest for readability, although the code does not depend on the enumeration order. \u0026#34;\u0026#34;\u0026#34; MINI = \u0026#34;mini\u0026#34; SEDAN = \u0026#34;sedan\u0026#34; SUV = \u0026#34;suv\u0026#34; @dataclass class Vehicle: \u0026#34;\u0026#34;\u0026#34; A physical vehicle owned by a driver. Identity is by licence plate; type is used by pricing (to look up rates) and by matching (to filter candidates by the rider\u0026#39;s requested type). The model field is metadata for display purposes only. \u0026#34;\u0026#34;\u0026#34; plate: str type: VehicleType model: str = \u0026#34;\u0026#34; The Vehicle dataclass is not frozen because some attributes such as colour or model may legitimately change over the life of the vehicle. A driver may switch to a different car under the same account, or an administrator may correct a typographical error. Freezing the dataclass would prevent these legitimate operations without providing any real value.\n9. Stage 6: The Trip Lifecycle and the State Pattern The Trip is the central entity of the system, and its lifecycle is the single most important modelling concern. The design must express clearly which operations are legal at which times and must make it impossible for callers to trigger an operation from a state that does not permit it.\n9.1 The Lifecycle in Prose A trip progresses through the following states in the following order.\nRequested. The rider has issued a ride request; no driver has been assigned yet. The system may transition this trip to DriverAssigned when a suitable driver is found, or to Cancelled if the rider cancels before any driver is assigned. DriverAssigned. A driver has accepted the request and is en route to the pickup point. The trip may transition to Started when the driver reaches the pickup and initiates the ride, or to Cancelled if either party cancels. Started. The trip is under way; the driver is transporting the rider toward the destination. The trip may transition to Completed when the driver reaches the destination and ends the ride, or (less commonly but legally) to Cancelled if the ride is aborted mid-way. Completed. The trip has finished successfully; fare has been computed and payment has been attempted. This is a terminal state; no further transitions are legal, although ratings may still be recorded. Cancelled. The trip has been aborted; a cancellation fee may or may not apply depending on the state at cancellation time. This is a terminal state. The transitions permitted from each state are summarised in the following table.\nFrom State Permitted Transitions Requested DriverAssigned (via assign_driver), Cancelled (via cancel) DriverAssigned Started (via start), Cancelled (via cancel) Started Completed (via complete), Cancelled (via cancel) Completed None (terminal); rating recording is still allowed as a non-state-changing operation Cancelled None (terminal) 9.2 Why the State Pattern Is Appropriate The naive way to model a lifecycle is with a string or enum attribute (trip.state == \u0026quot;requested\u0026quot;) combined with conditional statements in every trip method. This approach works at very small scale but degrades quickly. Two problems appear.\nFirst, every operation must inspect the state and branch. Adding a new operation requires adding a new conditional to every state in the state\u0026rsquo;s own switch, and adding a new state requires editing every operation. This is a matrix of conditionals whose complexity grows as the product of states and operations, not as their sum.\nSecond, the string-based approach provides no compile-time or lint-time guarantee that all legal transitions are handled and that all illegal transitions are rejected. Bugs of the form \u0026ldquo;we forgot to handle cancellation from the Started state\u0026rdquo; become common and are only caught in testing.\nThe State design pattern addresses both problems. It defines an abstract TripState class with methods for every possible transition, and creates a concrete subclass for each state. The default implementation of every method on the abstract class raises an exception, so any transition not explicitly overridden by a concrete state is rejected. The Trip delegates all state-dependent operations to its current TripState instance. Adding a new state requires only adding a new subclass; adding a new operation requires only adding a new method to each state that permits it.\nThe State pattern brings three concrete benefits to this design.\nTransition rules are localised. Each state class knows what it can do; no external code needs to check the state before acting. Illegal transitions are impossible. The default raising behaviour of the abstract class ensures that any attempt to perform an operation from a state that does not override it fails loudly rather than silently. Extension is additive. Adding a Paused state (for a mid-trip stop, for example) requires one new class and no modifications to existing states or operations. 9.3 The Abstract State from abc import ABC, abstractmethod from typing import Optional class TripState(ABC): \u0026#34;\u0026#34;\u0026#34; Abstract base for all trip states. Every state exposes the same set of transition methods so that the Trip can invoke them polymorphically without knowing the concrete state. The default behaviour of each transition is to raise, because most transitions are invalid from most states. Concrete states override only the transitions they permit. The design was considered against two alternatives: 1. Defining each concrete state with only its legal transitions. This appears tidier but breaks polymorphism because callers would need to know which methods a state supports. It also complicates static type checking. 2. Defining a single generic method such as handle(event) that dispatches on event type inside each state. This is cleaner when there are many small events but becomes verbose for a small number of well-defined transitions. The chosen shape names each transition explicitly and relies on the base-class defaults to raise for illegal transitions. Callers can trust that trip.state.assign_driver(...) either performs the correct transition or fails visibly. \u0026#34;\u0026#34;\u0026#34; @abstractmethod def name(self) -\u0026gt; str: ... def assign_driver(self, trip: \u0026#34;Trip\u0026#34;, driver: \u0026#34;Driver\u0026#34;) -\u0026gt; None: raise InvalidStateTransition( f\u0026#34;Cannot assign a driver in state {self.name()}\u0026#34; ) def start(self, trip: \u0026#34;Trip\u0026#34;) -\u0026gt; None: raise InvalidStateTransition( f\u0026#34;Cannot start a trip in state {self.name()}\u0026#34; ) def complete(self, trip: \u0026#34;Trip\u0026#34;, fare: float) -\u0026gt; None: raise InvalidStateTransition( f\u0026#34;Cannot complete a trip in state {self.name()}\u0026#34; ) def cancel(self, trip: \u0026#34;Trip\u0026#34;) -\u0026gt; None: raise InvalidStateTransition( f\u0026#34;Cannot cancel a trip in state {self.name()}\u0026#34; ) class InvalidStateTransition(Exception): \u0026#34;\u0026#34;\u0026#34;Raised when an operation is attempted from an illegal state.\u0026#34;\u0026#34;\u0026#34; The design commitment embedded here is that the state itself is the guard. There is no if trip.state == ... anywhere in the trip class or in the manager. The state object enforces which transitions are legal. This is the single largest reason to use the State pattern over an enum.\n9.4 The Concrete States class Requested(TripState): def name(self) -\u0026gt; str: return \u0026#34;REQUESTED\u0026#34; def assign_driver(self, trip: \u0026#34;Trip\u0026#34;, driver: \u0026#34;Driver\u0026#34;) -\u0026gt; None: trip.driver = driver trip.vehicle = driver.vehicle trip.state = DriverAssigned() def cancel(self, trip: \u0026#34;Trip\u0026#34;) -\u0026gt; None: # No driver has been assigned yet, so cancellation is free. # The fee itself is computed by the CancellationPolicy, # not by the state. trip.state = Cancelled() class DriverAssigned(TripState): def name(self) -\u0026gt; str: return \u0026#34;DRIVER_ASSIGNED\u0026#34; def start(self, trip: \u0026#34;Trip\u0026#34;) -\u0026gt; None: from datetime import datetime trip.started_at = datetime.now() trip.state = Started() def cancel(self, trip: \u0026#34;Trip\u0026#34;) -\u0026gt; None: # A driver was assigned but the ride has not started, so # a small cancellation fee is incurred. The fee amount is # determined by the CancellationPolicy at the time it is # invoked; the state itself simply knows that this # transition is legal. trip.state = Cancelled() class Started(TripState): def name(self) -\u0026gt; str: return \u0026#34;STARTED\u0026#34; def complete(self, trip: \u0026#34;Trip\u0026#34;, fare: float) -\u0026gt; None: from datetime import datetime trip.completed_at = datetime.now() trip.fare = fare trip.state = Completed() def cancel(self, trip: \u0026#34;Trip\u0026#34;) -\u0026gt; None: # The trip has already started, so the largest # cancellation fee applies. trip.state = Cancelled() class Completed(TripState): def name(self) -\u0026gt; str: return \u0026#34;COMPLETED\u0026#34; # All transitions from Completed use the base-class defaults, # which raise InvalidStateTransition. A completed trip is # terminal with respect to lifecycle transitions, although # ratings may still be recorded through a separate # non-state-changing method on the manager. class Cancelled(TripState): def name(self) -\u0026gt; str: return \u0026#34;CANCELLED\u0026#34; # Terminal state. No further transitions are legal. A subtle but important point is that the states know about the transition graph but do not know about fees, ratings, or fare computation. Each state is responsible for exactly one thing, namely enforcing which lifecycle transitions are legal. Fee computation, ratings, and pricing are separate concerns handled by separate components. This separation is what allows each concern to evolve independently.\n9.5 The Trip Class from datetime import datetime from typing import Optional import uuid class Trip: \u0026#34;\u0026#34;\u0026#34; The central entity of the system, representing one ride from request through to completion or cancellation. A Trip carries the data associated with a ride but delegates all state-dependent behaviour to its current TripState instance. Callers should almost always interact with trips through the RideManager rather than by invoking Trip methods directly, because the manager owns cross-cutting concerns such as driver-pool management and fare computation. \u0026#34;\u0026#34;\u0026#34; def __init__( self, rider: \u0026#34;Rider\u0026#34;, source: \u0026#34;Location\u0026#34;, destination: \u0026#34;Location\u0026#34;, vehicle_type: \u0026#34;VehicleType\u0026#34;, ): self.id: str = str(uuid.uuid4()) self.rider = rider self.source = source self.destination = destination self.requested_vehicle_type = vehicle_type # Populated when a driver is assigned. self.driver: Optional[\u0026#34;Driver\u0026#34;] = None self.vehicle: Optional[\u0026#34;Vehicle\u0026#34;] = None # Populated as the lifecycle progresses. self.started_at: Optional[datetime] = None self.completed_at: Optional[datetime] = None self.fare: Optional[float] = None self.cancellation_fee: Optional[float] = None # Ratings are recorded once the trip has completed and # both parties have submitted their scores. self.rider_rating_of_driver: Optional[int] = None self.driver_rating_of_rider: Optional[int] = None # Every new trip begins in the Requested state. self.state: TripState = Requested() # The following methods are thin pass-throughs that delegate # to the current state. They exist for two reasons: first, # they read more naturally at the call site than # trip.state.start(trip); second, they provide a single choke # point where cross-cutting concerns such as logging or audit # trails could be added later without modifying the state # classes. def assign_driver(self, driver: \u0026#34;Driver\u0026#34;) -\u0026gt; None: self.state.assign_driver(self, driver) def start(self) -\u0026gt; None: self.state.start(self) def complete(self, fare: float) -\u0026gt; None: self.state.complete(self, fare) def cancel(self) -\u0026gt; None: self.state.cancel(self) def distance_km(self) -\u0026gt; float: return self.source.distance_to(self.destination) The Trip class is deliberately thin. It owns the data that describes one ride and delegates behaviour to its state. It does not compute fares, does not select drivers, and does not process payments. Those responsibilities belong to the components that specialise in each of them.\n10. Stage 7: Matching a Rider to a Driver (Strategy Application 1) The task of matching a rider to a driver is the first of three places in this system where the Strategy pattern is applied. It is worth understanding not just what the Strategy pattern is, but why it is the right choice here and what would go wrong if a simpler design were used instead.\n10.1 Why Matching Is a Strategy During requirement clarification, the interviewer stated that matching would use the nearest available driver \u0026ldquo;for now\u0026rdquo; and that ranking by driver rating might be added later. This is an explicit statement of variability along one axis, and it is exactly the situation for which the Strategy pattern was designed.\nConsider what happens if the nearest-driver logic is hardcoded directly inside the RideManager. The manager acquires a large, specific responsibility (choosing drivers) in addition to its coordination responsibility. When a second matching policy is later added, the manager either grows a conditional (if self.policy == \u0026quot;nearest\u0026quot;: ... elif self.policy == \u0026quot;highest_rated\u0026quot;: ...) or is edited directly to replace one algorithm with another. Both outcomes violate the Open/Closed Principle. The correct alternative is to extract matching as an abstract interface with concrete implementations, and to have the manager hold a reference to one such implementation. Adding a new matching policy then requires writing a new class rather than editing an existing one.\n10.2 The Abstract Strategy from abc import ABC, abstractmethod from typing import Optional class MatchingStrategy(ABC): \u0026#34;\u0026#34;\u0026#34; Chooses one driver for a given trip request from a pool of candidate drivers. Concrete subclasses encapsulate specific matching policies. The method returns None if no suitable driver exists. The caller must handle the None case explicitly; the strategy does not raise for the ordinary case of no available driver because that case is expected and recoverable, not exceptional. \u0026#34;\u0026#34;\u0026#34; @abstractmethod def select( self, trip: \u0026#34;Trip\u0026#34;, drivers: list[\u0026#34;Driver\u0026#34;], ) -\u0026gt; Optional[\u0026#34;Driver\u0026#34;]: ... The method signature is deliberately narrow. The strategy takes only the trip (which conveys the pickup location and the requested vehicle type) and the pool of drivers. It does not take a reference to the RideManager or to any other component. This narrow interface prevents the strategy from developing coupling to concerns outside its own responsibility and keeps it easy to test in isolation.\n10.3 Two Concrete Strategies The first concrete strategy implements the nearest-driver policy that satisfies the initial requirement.\nclass NearestDriverStrategy(MatchingStrategy): \u0026#34;\u0026#34;\u0026#34; Selects the driver closest to the pickup point, from those who are available and driving a vehicle of the requested type. The time complexity is O(N) in the size of the driver pool. This is acceptable for an LLD-scope design because the pool is treated as small. For production use at scale, drivers would be pre-indexed by a geospatial data structure such as a geohash grid or an H3 hexagonal grid, and the strategy would receive only the drivers within the relevant cells. That indexing lives outside the strategy; the strategy itself remains a simple minimum-selection over whatever pool is passed in. \u0026#34;\u0026#34;\u0026#34; def select(self, trip, drivers): candidates = [ d for d in drivers if d.is_available and d.vehicle.type == trip.requested_vehicle_type ] if not candidates: return None return min( candidates, key=lambda d: d.location.distance_to(trip.source), ) The second strategy implements the ranking-by-rating policy that the interviewer mentioned as a possible future addition. Its inclusion here serves two purposes: it demonstrates that the abstraction is real (not vacuous), and it provides a second data point that clarifies what varies between strategies.\nclass HighestRatedNearbyStrategy(MatchingStrategy): \u0026#34;\u0026#34;\u0026#34; Selects, from the K nearest candidates, the one with the highest rating. This policy is genuinely different from the nearest-driver policy. A slightly-farther driver with a noticeably better rating will be chosen in preference to a nearer driver with an average rating. The K parameter is a tunable value: a smaller K biases the policy toward proximity, and a larger K biases it toward rating quality. Setting K to 1 recovers the pure nearest-driver policy exactly. \u0026#34;\u0026#34;\u0026#34; def __init__(self, k: int = 5): self.k = k def select(self, trip, drivers): candidates = [ d for d in drivers if d.is_available and d.vehicle.type == trip.requested_vehicle_type ] if not candidates: return None nearest_k = sorted( candidates, key=lambda d: d.location.distance_to(trip.source), )[: self.k] return max(nearest_k, key=lambda d: d.rating) The RideManager will hold a reference to whichever matching strategy is configured at construction time. Substituting one for another requires no changes to the manager, no changes to the trip class, and no changes to any other strategy. This is the property that the Strategy pattern was designed to provide, and this is why the extraction is worth the small additional structural cost.\n11. Stage 8: Pricing, Surge, and Layered Strategies (Strategy Application 2) The computation of a trip\u0026rsquo;s fare is the second application of the Strategy pattern, and it introduces a subtlety that the matching component did not: the fare is composed of two independently varying parts, namely a base fare and a surge multiplier. This section discusses why these two parts must be modelled as separate components, and how they are composed by the manager without either component becoming aware of the other.\n11.1 Why Pricing and Surge Must Be Separate Both pricing and surge produce numerical outputs that end up multiplied together to yield the final fare. A tempting simplification would be to combine them into a single FareStrategy that returns the final number directly. This simplification is a mistake, for the following reasons.\nPricing and surge vary along different axes. Pricing depends on distance, time, and vehicle type. Surge depends on time of day and area of the city. Combining them means that any change to the base-fare formula would require editing surge-aware code, and any change to the surge calculation would require editing pricing code.\nPricing is stable; surge is dynamic. Base pricing schedules change infrequently, typically once per quarter as part of formal rate revisions. Surge multipliers change continuously as supply and demand fluctuate. Coupling them produces a component that is simultaneously slow-changing and fast-changing, which is a signal that two concerns have been conflated.\nTesting is simpler when the two are separate. A test of the pricing strategy can supply a known trip and assert a specific base fare without concern for surge. A test of the surge calculator can assert a specific multiplier for a given time and location without concern for the underlying fare. Combining them forces every pricing test to reason about surge, and vice versa.\nThe separation of these two concerns is an application of the Single Responsibility Principle at the design level. Each component has exactly one reason to change.\n11.2 The PricingStrategy Interface and Its Default Implementation from abc import ABC, abstractmethod class PricingStrategy(ABC): \u0026#34;\u0026#34;\u0026#34; Computes the base fare for a trip, prior to the application of any surge multiplier. The base fare should be deterministic in the sense that the same trip inputs always produce the same fare output. Anything dynamic (surge, discounts, promotions, driver-specific rates) belongs in a separate layer that composes with the base fare rather than replacing it. \u0026#34;\u0026#34;\u0026#34; @abstractmethod def compute(self, trip: \u0026#34;Trip\u0026#34;) -\u0026gt; float: ... # Per-vehicle-type rate table. In production these values would # come from a configuration file, an admin panel, or a rate # service, not be hardcoded in source. Hardcoding here keeps # the LLD design self-contained. BASE_RATES = { VehicleType.MINI: {\u0026#34;base\u0026#34;: 30.0, \u0026#34;per_km\u0026#34;: 10.0, \u0026#34;per_min\u0026#34;: 1.0}, VehicleType.SEDAN: {\u0026#34;base\u0026#34;: 50.0, \u0026#34;per_km\u0026#34;: 14.0, \u0026#34;per_min\u0026#34;: 1.5}, VehicleType.SUV: {\u0026#34;base\u0026#34;: 80.0, \u0026#34;per_km\u0026#34;: 20.0, \u0026#34;per_min\u0026#34;: 2.0}, } class DistanceAndTimeBasedPricing(PricingStrategy): \u0026#34;\u0026#34;\u0026#34; Fare = base_rate + per_km_rate * distance + per_min_rate * duration. Duration is known only after the trip has completed, so this strategy is invoked at the completion step, not at the request step. If a fare estimate is required at request time (for example, to display an estimated fare to the rider before they confirm), a separate EstimatedFarePricing implementation would be used that omits the time component. \u0026#34;\u0026#34;\u0026#34; def compute(self, trip: \u0026#34;Trip\u0026#34;) -\u0026gt; float: if trip.started_at is None or trip.completed_at is None: raise ValueError( \u0026#34;Distance-and-time-based fare requires both a \u0026#34; \u0026#34;start and a completion timestamp on the trip.\u0026#34; ) rates = BASE_RATES[trip.vehicle.type] distance = trip.distance_km() duration_min = (trip.completed_at - trip.started_at).total_seconds() / 60 return ( rates[\u0026#34;base\u0026#34;] + rates[\u0026#34;per_km\u0026#34;] * distance + rates[\u0026#34;per_min\u0026#34;] * duration_min ) The strategy is deliberately focused. It computes the base fare and nothing else. It does not know about surge, does not know about discounts, does not care whether the payment succeeds. Its single responsibility is captured in its single method.\n11.3 The SurgeCalculator Surge is not treated as a full Strategy family in this design because there is really only one kind of surge, namely a multiplier that varies with time and location. However, it is factored out as a separately-injectable component so that the mechanism by which the multiplier is calculated can be replaced without disturbing the pricing strategy or the manager.\nfrom datetime import datetime class SurgeCalculator: \u0026#34;\u0026#34;\u0026#34; Returns a surge multiplier for a given time and location. The default implementation applies a simple peak-hour multiplier (mornings between 07:00 and 10:00, evenings between 17:00 and 21:00). Area-based surge is not implemented in this default, but the interface accepts a Location so that a more sophisticated implementation can be substituted without any change to callers. \u0026#34;\u0026#34;\u0026#34; def multiplier( self, at_time: datetime, at_location: \u0026#34;Location\u0026#34;, ) -\u0026gt; float: hour = at_time.hour if 7 \u0026lt;= hour \u0026lt; 10 or 17 \u0026lt;= hour \u0026lt; 21: return 1.5 return 1.0 Two design decisions embedded here are worth noticing.\nFirst, the interface takes a Location even though the default implementation ignores it. This is a deliberate provision for future extension. When area-based surge is added, no caller of multiplier needs to change; only the implementation is replaced.\nSecond, the multiplier is composed with the base fare outside both components. The RideManager calls the pricing strategy for the base fare and the surge calculator for the multiplier, then multiplies them together. Neither component knows that the other exists. This layering is what preserves the independence of the two concerns.\n11.4 Summary of the Fare Computation Layering The fare computation involves three components acting in sequence, coordinated by the RideManager. The following table summarises the responsibilities.\nComponent Responsibility Depends On PricingStrategy Compute base fare from trip distance, duration, and vehicle type Trip data only SurgeCalculator Compute multiplier from time and location Time and location only RideManager Invoke both components and multiply their outputs to produce final fare Both components as injected dependencies The manager is the only place in the system that knows both about pricing and about surge. This is the correct place for that knowledge because coordinating independent components is the manager\u0026rsquo;s job.\n12. Stage 9: Payment Methods (Strategy Application 3 with Factory) Payment methods form the third and final application of the Strategy pattern in this design. Each payment method (cash, card, UPI, wallet) has different behaviour, different failure modes, and different external dependencies, so polymorphism through a common interface is the natural fit.\n12.1 The PaymentMethod Interface and Its Concrete Implementations from abc import ABC, abstractmethod class PaymentMethod(ABC): \u0026#34;\u0026#34;\u0026#34; Abstract payment method. Real implementations would communicate with external payment gateways, handle retries and idempotency, and return richer PaymentResult objects carrying gateway-specific status and error information. For the LLD scope, the interface is kept simple: the charge method either succeeds silently (returning None) or raises an exception. Callers must handle the exceptional case explicitly. \u0026#34;\u0026#34;\u0026#34; @abstractmethod def charge(self, amount: float, from_user: \u0026#34;User\u0026#34;) -\u0026gt; None: ... class CashPayment(PaymentMethod): def charge(self, amount: float, from_user: \u0026#34;User\u0026#34;) -\u0026gt; None: # Cash is settled in person; the system merely records # that the transaction occurred. No external interaction # is required. pass class CardPayment(PaymentMethod): def __init__(self, card_last4: str): self.card_last4 = card_last4 def charge(self, amount: float, from_user: \u0026#34;User\u0026#34;) -\u0026gt; None: # In a production implementation, this method would # invoke the card gateway, handle failure and retry, # and persist the resulting transaction identifier. For # the LLD scope, the method is stubbed to succeed. pass class UPIPayment(PaymentMethod): def __init__(self, upi_id: str): self.upi_id = upi_id def charge(self, amount: float, from_user: \u0026#34;User\u0026#34;) -\u0026gt; None: # Stubbed to succeed. Production would invoke the UPI # collect request through the appropriate PSP. pass class WalletPayment(PaymentMethod): \u0026#34;\u0026#34;\u0026#34; An in-app wallet payment. Unlike the other payment methods, this one has behaviour that can be meaningfully implemented at the LLD scope because the wallet balance is entirely internal to the system. The charge method debits the balance and raises if the balance is insufficient. \u0026#34;\u0026#34;\u0026#34; def __init__(self, balance: float): self.balance = balance def charge(self, amount: float, from_user: \u0026#34;User\u0026#34;) -\u0026gt; None: if amount \u0026gt; self.balance: raise ValueError( f\u0026#34;Wallet has {self.balance:.2f}, \u0026#34; f\u0026#34;fare is {amount:.2f}\u0026#34; ) self.balance -= amount 12.2 An Optional Factory for Payment Methods If callers specify payment methods as strings (as would be the case for an HTTP API where the request body contains something like {\u0026quot;method\u0026quot;: \u0026quot;card\u0026quot;, \u0026quot;card_last4\u0026quot;: \u0026quot;4242\u0026quot;}), a factory keeps the manager from doing string-based dispatch itself.\nclass PaymentMethodFactory: \u0026#34;\u0026#34;\u0026#34; Constructs a PaymentMethod from a type identifier and its associated configuration. The factory isolates the creation logic from callers, keeping the RideManager free of if/elif chains on payment type strings. If a new payment method is added, only the factory is edited. \u0026#34;\u0026#34;\u0026#34; @staticmethod def create(kind: str, **kwargs) -\u0026gt; PaymentMethod: if kind == \u0026#34;cash\u0026#34;: return CashPayment() if kind == \u0026#34;card\u0026#34;: return CardPayment(card_last4=kwargs[\u0026#34;card_last4\u0026#34;]) if kind == \u0026#34;upi\u0026#34;: return UPIPayment(upi_id=kwargs[\u0026#34;upi_id\u0026#34;]) if kind == \u0026#34;wallet\u0026#34;: return WalletPayment(balance=kwargs[\u0026#34;balance\u0026#34;]) raise ValueError(f\u0026#34;Unknown payment kind: {kind}\u0026#34;) Whether to include the factory in an interview design depends on how the question is framed. If the interviewer explicitly asks about constructing payment methods from user input, the factory is warranted. If the interviewer treats payment as an already-constructed object handed to the manager, the factory is optional. Mentioning its existence and moving on is generally the correct choice.\n13. Stage 10: Cancellation Encoded as a Policy Object Cancellation is the concern where many otherwise-decent designs quietly go wrong. The temptation is to embed cancellation fee logic directly inside the manager with statements of the form if trip.state.name() == \u0026quot;DRIVER_ASSIGNED\u0026quot;: fee = 30. This approach hardcodes a policy into a coordinator, which is exactly the anti-pattern that the Strategy and Policy patterns exist to prevent.\nThe correct alternative is to externalise the cancellation fee schedule as a policy object.\nclass CancellationPolicy: \u0026#34;\u0026#34;\u0026#34; Computes the fee owed by a rider who cancels a trip. The fee depends on how far into the lifecycle the trip has progressed at the time of cancellation. Extracting this computation into a dedicated policy object means the fee schedule can be revised (for example, extending the free- cancellation window or introducing type-dependent fees) without touching the RideManager or the trip state classes. \u0026#34;\u0026#34;\u0026#34; def fee(self, trip: \u0026#34;Trip\u0026#34;) -\u0026gt; float: # By the time this method is invoked, the trip has # already transitioned to Cancelled. The policy therefore # cannot look at trip.state to determine which state was # active immediately before cancellation. Instead it # inspects markers on the trip data: # # - trip.started_at set means the trip had actually # started, which is the highest-fee case. # - trip.driver set means a driver was assigned but # the trip had not yet started, which is the # moderate-fee case. # - Neither set means the trip was cancelled before # any driver was assigned, which is the free case. if trip.started_at is not None: return 100.0 if trip.driver is not None: return 30.0 return 0.0 There is a subtle design fragility embedded in this implementation, and it is worth identifying explicitly rather than glossing over. Because cancel() transitions the trip to Cancelled before the fee is computed, the policy cannot read trip.state to determine which state was active immediately prior to cancellation. Instead the policy infers the pre-cancel state from other markers on the trip: the presence of started_at implies the trip had actually started, the presence of driver without started_at implies a driver had been assigned but the ride had not begun, and the absence of both implies no driver had been assigned. This inference is correct in the current design but relies on invariants that a future change could accidentally break. For example, if the design were later modified to clear driver on cancellation, the policy would return the free-cancellation fee for cancellations that should attract a moderate fee.\nA cleaner alternative is to record the pre-cancellation state explicitly on the trip at the moment of cancellation, either as a snapshot attribute (trip.state_before_cancel) or as a parameter to the policy method. This alternative eliminates the inference and its associated fragility at the cost of a small additional piece of state on the trip. For an interview response, identifying the tradeoff and articulating both options is worth as many points as solving it fully.\n14. Stage 11: The RideManager as a Facade The RideManager is the top-level coordinator of the system. Its role is to expose a small number of high-level operations (request_ride, start_trip, complete_trip, cancel_trip, rate_trip) and to hide the choreography required to implement them. This is the definition of the Facade design pattern.\n14.1 What the Manager Owns and What It Does Not The manager owns:\nThe pool of registered drivers, indexed by driver identifier for constant-time lookup. The registry of trips (active and completed), indexed by trip identifier. References to the currently-configured matching strategy, pricing strategy, surge calculator, and cancellation policy. The manager does not own:\nThe matching algorithm itself. That is inside the MatchingStrategy implementation. The fare-computation formula. That is inside the PricingStrategy implementation. The surge-computation formula. That is inside the SurgeCalculator implementation. The cancellation fee schedule. That is inside the CancellationPolicy implementation. The trip lifecycle transitions. Those are inside the TripState subclasses. The payment execution logic. That is inside the PaymentMethod implementation supplied per call. The manager is deliberately thin. It is glue, not logic. The measure of a good facade is how little logic it needs to contain of its own.\n14.2 Implementation from typing import Optional class RideManager: \u0026#34;\u0026#34;\u0026#34; The Facade for the ride-sharing system. Orchestrates end-to-end operations by composing: - MatchingStrategy for driver selection - PricingStrategy plus SurgeCalculator for fare computation - CancellationPolicy for cancellation fee determination - PaymentMethod for money movement (supplied per call) - TripState for lifecycle enforcement The manager owns none of this logic. It knows when to invoke each component, not how each component works internally. This delegation is what keeps the manager stable: a change to the pricing formula does not touch the manager, and a change to the matching policy does not touch the manager. Only a change to the choreography itself does. \u0026#34;\u0026#34;\u0026#34; def __init__( self, matching: MatchingStrategy, pricing: PricingStrategy, surge: SurgeCalculator, cancellation: CancellationPolicy, ): self._matching = matching self._pricing = pricing self._surge = surge self._cancellation = cancellation # Registered drivers, keyed by identifier for O(1) # lookup on unregister. The matching strategy is passed # the list of values. self._drivers: dict[str, Driver] = {} # All trips ever created, keyed by identifier. In a # production system, completed trips would be moved to # cold storage; here everything remains in memory. self._trips: dict[str, Trip] = {} # Driver pool management def register_driver(self, driver: Driver) -\u0026gt; None: if driver.user_id in self._drivers: raise ValueError( f\u0026#34;Driver {driver.user_id} is already registered\u0026#34; ) self._drivers[driver.user_id] = driver def unregister_driver(self, driver_id: str) -\u0026gt; None: self._drivers.pop(driver_id, None) # Trip lifecycle def request_ride( self, rider: Rider, source: Location, destination: Location, vehicle_type: VehicleType, ) -\u0026gt; Trip: \u0026#34;\u0026#34;\u0026#34; Create a trip and attempt to assign a driver. The trip is returned in either case. If the matching strategy returned a driver, the trip is in DriverAssigned state and the driver\u0026#39;s availability flag has been cleared. If matching returned None, the trip remains in Requested state; the caller may retry later or cancel the trip. This design choice (returning the trip rather than raising when no driver is available) reflects the fact that driver unavailability is an expected, recoverable condition rather than an exceptional one. \u0026#34;\u0026#34;\u0026#34; trip = Trip(rider, source, destination, vehicle_type) self._trips[trip.id] = trip driver = self._matching.select( trip, list(self._drivers.values()) ) if driver is not None: trip.assign_driver(driver) driver.is_available = False return trip def start_trip(self, trip_id: str) -\u0026gt; None: trip = self._get_trip(trip_id) trip.start() def complete_trip( self, trip_id: str, payment_method: PaymentMethod, ) -\u0026gt; float: \u0026#34;\u0026#34;\u0026#34; End the trip, compute the final fare (base multiplied by surge), transition to Completed, charge the payment method, and release the driver. Returns the final fare that was charged. The order of operations is significant: 1. Verify that the trip actually started; refuse otherwise. 2. Set completed_at temporarily so that the pricing strategy has both timestamps available. 3. Compute the base fare through the pricing strategy. 4. Compute the surge multiplier through the surge calculator. 5. Multiply to obtain the final fare. 6. Transition the state to Completed, which also sets completed_at and fare on the trip. The double-set of completed_at is harmless. 7. Invoke the payment method to charge the rider. 8. Release the driver back to the available pool and update the driver\u0026#39;s location to the drop-off point. If the payment method raises, the trip is still Completed; the caller decides how to retry payment. This mirrors real-world behaviour: the ride happened, so the trip should reflect that; the money is a separate follow-up concern. \u0026#34;\u0026#34;\u0026#34; from datetime import datetime trip = self._get_trip(trip_id) if trip.started_at is None: raise InvalidStateTransition( \u0026#34;Cannot complete a trip that never started\u0026#34; ) now = datetime.now() # Temporarily set completed_at so that pricing has both # timestamps available. trip.completed_at = now base_fare = self._pricing.compute(trip) multiplier = self._surge.multiplier(now, trip.destination) final_fare = base_fare * multiplier # Formally transition through the state machine. This # sets fare and (redundantly) completed_at. trip.complete(final_fare) # Attempt to settle payment. If this raises, the trip # remains in Completed state; retry is the caller\u0026#39;s # responsibility. payment_method.charge(final_fare, trip.rider) # Release the driver and update location to the drop-off # point so subsequent matching sees the driver where # they actually are. if trip.driver is not None: trip.driver.is_available = True trip.driver.location = trip.destination return final_fare def cancel_trip(self, trip_id: str) -\u0026gt; float: \u0026#34;\u0026#34;\u0026#34; Cancel a trip and return the cancellation fee, which may be zero. The state transition happens first (which validates that the trip is in a cancellable state); the policy then computes the fee from the trip data; the driver, if any was assigned, is finally released back to the available pool. \u0026#34;\u0026#34;\u0026#34; trip = self._get_trip(trip_id) driver_at_cancel = trip.driver trip.cancel() # State-guarded; raises if not cancellable. fee = self._cancellation.fee(trip) trip.cancellation_fee = fee if driver_at_cancel is not None: driver_at_cancel.is_available = True return fee def rate_trip( self, trip_id: str, by_rider: Optional[int] = None, by_driver: Optional[int] = None, ) -\u0026gt; None: \u0026#34;\u0026#34;\u0026#34; Record ratings for a completed trip. Only permitted after the trip has entered the Completed state. \u0026#34;\u0026#34;\u0026#34; trip = self._get_trip(trip_id) if not isinstance(trip.state, Completed): raise InvalidStateTransition( \u0026#34;Ratings can only be recorded on completed trips\u0026#34; ) if by_rider is not None and trip.driver is not None: trip.rider_rating_of_driver = by_rider trip.driver.update_rating(by_rider) if by_driver is not None: trip.driver_rating_of_rider = by_driver trip.rider.update_rating(by_driver) # Helpers def _get_trip(self, trip_id: str) -\u0026gt; Trip: if trip_id not in self._trips: raise KeyError(f\u0026#34;Unknown trip: {trip_id}\u0026#34;) return self._trips[trip_id] The entire manager runs under 150 lines and delegates every substantive decision. This is the shape of a good Facade: it coordinates but does not compute.\n15. Stage 12: Validating the Design with Mental Walkthroughs A design that has never been walked through is a design that has never been tested. Mental walkthroughs are the LLD equivalent of unit tests: they exercise specific scenarios against the design and confirm that the resulting behaviour matches what the requirements specify. Almost every non-trivial design contains at least one bug that surfaces only during walkthrough, so this stage is not optional.\nThe following six scenarios cover the happy path, several important error paths, and one scenario that exposes a genuine limitation of the design.\n15.1 Scenario A: The Happy Path Setup. A rider named Amit requests a sedan from location (12.97, 77.59) to location (12.95, 77.55). The system has two available sedans and one available mini registered.\nTrace.\nmgr.request_ride(amit, source, destination, VehicleType.SEDAN) creates a new trip and invokes NearestDriverStrategy.select. The candidate list is filtered to the two sedans (the mini is excluded by vehicle-type mismatch). The nearer sedan is selected. The trip transitions from Requested to DriverAssigned. The selected driver\u0026rsquo;s is_available flag is cleared. mgr.start_trip(trip.id) invokes trip.start(), which delegates to the current state (DriverAssigned) and transitions to Started. The started_at timestamp is set. mgr.complete_trip(trip.id, UPIPayment(\u0026quot;amit@upi\u0026quot;)) verifies that the trip has started, computes the base fare through the pricing strategy, computes the surge multiplier (assumed to be 1.0 for off-peak), multiplies to obtain the final fare, transitions the state to Completed, invokes UPIPayment.charge, and releases the driver. mgr.rate_trip(trip.id, by_rider=5, by_driver=4) records both ratings and updates the running averages on both users. Outcome. The happy path executes exactly as intended. The trip ends in Completed state with a valid fare, both users have updated ratings, and the driver is available for another trip.\n15.2 Scenario B: No Driver Available Setup. A rider requests an SUV, but no SUV is registered or available.\nTrace.\nmgr.request_ride(...) creates a trip and invokes matching. The matching strategy filters to zero candidates and returns None. The trip\u0026rsquo;s driver attribute remains None, and the trip stays in Requested state. The caller inspects trip.driver is None, recognises that matching failed, and decides on a follow-up action (retry later, notify the rider, or cancel). If the caller invokes mgr.cancel_trip(trip.id), the trip transitions from Requested to Cancelled via Requested.cancel. The CancellationPolicy observes that no driver was assigned and no timestamp was set, and returns a fee of 0.0. No driver needs to be released because none was ever assigned. Outcome. The design correctly handles the case where no driver is available. No spurious exceptions are raised, no invalid state transitions occur, and the correct (zero) fee is computed on cancellation.\n15.3 Scenario C: Cancellation After Driver Assignment but Before Start Setup. A rider requests a sedan, matching succeeds, and the trip enters DriverAssigned. The rider then cancels before the driver reaches the pickup point.\nTrace.\nmgr.request_ride(...) succeeds; the trip is in DriverAssigned and the driver is marked unavailable. mgr.cancel_trip(trip.id) invokes trip.cancel(), which delegates to DriverAssigned.cancel and transitions to Cancelled. The policy observes that a driver was assigned (trip.driver is not None) but the trip never started (trip.started_at is None), and returns a fee of 30.0. The driver is released back to the available pool. Outcome. The moderate cancellation fee is correctly applied, and the driver is correctly released.\n15.4 Scenario D: Attempting to Start a Completed Trip Setup. Through a client bug, a replay attack, or any other spurious cause, start_trip is invoked on a trip that has already reached the Completed state.\nTrace.\nmgr.start_trip(trip.id) invokes trip.start(), which delegates to self.state.start(self). The current state is Completed. The Completed class does not override start, so the base class\u0026rsquo;s default implementation raises InvalidStateTransition(\u0026quot;Cannot start a trip in state COMPLETED\u0026quot;). Outcome. The invalid transition is rejected loudly, without any explicit state check in the manager. This is the primary benefit of the State pattern: illegal transitions are impossible by construction.\n15.5 Scenario E: Completing a Trip That Never Started Setup. A client accidentally calls complete_trip immediately after request_ride, skipping the start_trip step entirely.\nTrace.\nmgr.complete_trip(trip.id, payment) performs its precondition check and observes that trip.started_at is None. It raises InvalidStateTransition before any state transition is attempted. Even if this precondition were absent, the subsequent call to trip.complete(final_fare) would fail because the current state (DriverAssigned) does not override complete, and the base class default would raise. Outcome. The invalid operation is rejected. The design has two independent lines of defence against this class of error: the manager\u0026rsquo;s precondition check and the state pattern\u0026rsquo;s default raising.\n15.6 Scenario F: Concurrent Requests Targeting the Same Driver Setup. Two request_ride calls arrive simultaneously, and the matching strategy identifies the same driver as the best match for both.\nTrace.\nIn the single-threaded design, one call executes to completion before the other begins. The first call assigns the driver and marks them unavailable. The second call\u0026rsquo;s matching invocation sees the driver as unavailable and filters them out, selecting a different driver (or returning None if no other suitable driver exists). In a hypothetical multi-threaded design, both calls could reach the matching step and both could observe the driver as available. Both would proceed to assign the same driver, resulting in a duplicate assignment that would need to be resolved by some higher-level coordination mechanism. Outcome. The single-threaded assumption from the requirements is what saves the design here. The scenario surfaces a genuine limitation that would need to be addressed if concurrency were later introduced. This limitation is identified explicitly and discussed in the follow-up section.\n15.7 Summary of Walkthroughs The design behaves correctly for all tested scenarios within the single-threaded, in-memory scope that was agreed during requirement clarification. The one genuine limitation surfaced (concurrent driver assignment) is exactly the kind of concern that the interviewer expects to be identified and deferred rather than solved inside the LLD.\nScenario Result Notes Happy path Passes All components behave as designed No driver available Passes Returns trip with driver=None; free cancellation works Cancel after assign Passes Moderate fee applied, driver released Start completed trip Passes Rejected by State pattern default Complete unstarted trip Passes Rejected by manager precondition and by State pattern Concurrent same-driver Limitation surfaced Requires concurrency support to solve 16. Stage 13: Anticipating Follow-Up Questions Interviewers use follow-up questions to probe the depth of a candidate\u0026rsquo;s design maturity. The follow-ups on ride-sharing are usually predictable, and rehearsing answers to them is worth the effort. The following are the most common, together with grounded and specific responses.\n16.1 How Would You Scale Matching to Millions of Drivers? The linear scan in NearestDriverStrategy is O(N) per request, where N is the total number of registered drivers. At scale this is untenable. The replacement is a geospatial index that partitions drivers by geographic cell, allowing matching to consider only the drivers within cells near the pickup point.\nTwo common indexing schemes are relevant. Geohash-based indexing partitions the world into a hierarchical rectangular grid, in which each geohash string identifies a cell of a specific size. Uber\u0026rsquo;s H3 library uses a hexagonal grid, which has better neighbour-uniformity properties than the geohash grid. In either case, drivers are inserted into the appropriate cell when they go online and are moved between cells as they travel.\nThe design accommodates this change through composition rather than modification. A new GeohashNearestStrategy implementation of MatchingStrategy receives a much smaller candidate list from a driver-pool component that maintains the index internally. The RideManager continues to hold a MatchingStrategy reference, and neither the manager nor the trip class needs to change. This is the payoff for having extracted matching as a Strategy from the beginning.\n16.2 How Would You Handle Concurrent Driver Assignment? Concurrent assignment can be addressed at two levels, which are commonly layered together.\nThe first level is a lock on the driver-pool access. The read-then-mark-unavailable pair is wrapped in a critical section so that it executes atomically with respect to other assignments. This is simple, correct, and easy to reason about. Its principal disadvantage is that all matching attempts contend for the same lock, which becomes a throughput bottleneck as the system scales.\nThe second level is a compare-and-swap on the driver\u0026rsquo;s availability flag. The matching strategy returns a candidate; the manager atomically attempts to flip the flag from True to False; on failure (which indicates another manager instance beat this one to the assignment), the manager re-runs matching. This gives much higher throughput at the cost of occasional retries.\nAt the largest scales (real Uber-scale, tens of thousands of assignments per second), matching is implemented as an event-driven pipeline. Driver availability events flow through a distributed queue, assignments are committed through a coordination service such as etcd or Zookeeper, and the system tolerates a small window of inconsistency in exchange for horizontal scalability. The LLD design supports all of these approaches because the MatchingStrategy interface makes no assumptions about how availability is checked.\n16.3 How Would You Support Real-Time Supply-and-Demand Surge? The SurgeCalculator interface is already designed for this extension. The default implementation applies a fixed time-of-day multiplier, but a DemandBasedSurge implementation would query a running index of active requests per area versus available drivers per area, and would return a multiplier accordingly. The signature multiplier(at_time, at_location) -\u0026gt; float is sufficient for any such implementation, so no caller changes are required to switch from the default to a demand-based version.\nThe running indices themselves are maintained by a separate component (an aggregator that observes ride requests and driver availability changes), which is architecturally similar to the driver-pool component that maintains the geospatial index for matching.\n16.4 How Would You Support Pool (Shared) Rides? Pool rides are a substantive extension because they change the fundamental assumption that a trip has one rider. Two structural approaches are worth discussing.\nThe first approach introduces a new PoolTrip entity that contains multiple TripSegment objects, one per rider. Each segment has its own source, destination, and state, but all segments share a driver, a vehicle, and a route. Pricing splits proportionally by distance. This approach models the multi-rider nature explicitly and cleanly separates the pool concern from the point-to-point trip class.\nThe second approach extends the existing Trip with a co_riders list and a per-rider fare computation. This is simpler to implement but conflates two genuinely different kinds of trip (single-rider and multi-rider), which will produce awkward conditional logic anywhere that the difference matters.\nFor an interview, the correct response is to describe both approaches, explain the tradeoff between them, and observe that the state machine becomes meaningfully more complex under either approach because a pool trip may be in different phases for different riders simultaneously (started for rider A, still en route to pickup for rider B).\n16.5 How Would You Support Scheduled (Pre-Booked) Rides? Scheduled rides add a scheduled_for: Optional[datetime] attribute to Trip and introduce a new state Scheduled that precedes Requested. A scheduler component (whose implementation is outside the LLD scope for interview purposes) fires the transition from Scheduled to Requested at the appropriate time, at which point the normal matching pipeline takes over. The remainder of the design does not change, which is a good sign about the design\u0026rsquo;s extensibility.\n16.6 How Would Ratings Actually Affect Driver Rankings? Ratings are already recorded on the User base class. The HighestRatedNearbyStrategy defined earlier is the natural consumer. For richer ranking (combining rating, acceptance rate, cancellation rate, and other quality signals), a dedicated DriverRankingStrategy interface can be introduced that returns a numerical score for a given driver. The matching strategy then invokes the ranking strategy as part of its selection logic. This becomes a fourth strategy family composed by the manager.\n16.7 How Would Notifications Work? Notifications are the natural place for the Observer pattern to enter this design. Drivers subscribe to \u0026ldquo;request-created\u0026rdquo; events in their operating area; riders subscribe to state-change events on their own trip. The Trip (or the RideManager) acts as an event source, and the app clients act as observers. An EventBus component would be introduced to decouple publishers from subscribers, and the state transition methods would emit events at appropriate points.\nAdding notifications to the design does not require changes to any existing class beyond instrumenting the state transitions with event emissions. This is another consequence of having separated concerns cleanly from the beginning.\n16.8 How Would You Handle Payment Failures? The current design has charge() either returning or raising. In a production system the flow is richer. A PaymentResult object would carry success, failure, or pending status; gateway-specific error codes; and retry hints. The trip would record the payment attempt as a first-class object. A separate retry job would periodically pick up failed payments and retry them with exponential backoff. If all retries exhaust, the rider\u0026rsquo;s account would enter a \u0026ldquo;settle before next ride\u0026rdquo; state that blocks new ride requests until the outstanding balance is cleared.\nAll of this can be layered on without changing the trip lifecycle. The trip does not care whether payment succeeded on the first attempt or the tenth; it only cares that the ride happened and the fare was recorded.\n16.9 How Would Trust and Safety Concerns Be Modelled? Trust and safety (fraudulent drivers, abusive riders, disputes, reputation) is an entire subsystem in its own right. It touches the User base class (with flags for holds, review states, and reputation scores) and the Trip class (with a disputed marker and links to review records), but is otherwise orthogonal to the core ride-lifecycle design. The composition-based approach adopted throughout this design accommodates trust-and-safety extensions cleanly: adding fraud checks does not require restructuring matching, pricing, or lifecycle management.\n17. Summary of Design Decisions The following table consolidates every substantive design decision made during the walkthrough, together with the alternatives that were considered and the criterion that led to the choice.\nDecision Chosen Approach Alternatives Considered Deciding Criterion User modelling User base with Rider and Driver subclasses Single class with role flag; two independent classes Avoids null attributes and duplication; preserves type safety Vehicle type modelling VehicleType enum Class hierarchy (Car, Bike, etc.) Types differ in metadata only, not behaviour Location representation Immutable frozen dataclass Mutable class; tuple Value semantics require equality by attributes; immutability prevents aliasing Trip lifecycle State pattern with abstract base String or enum with conditional checks; handle(event) dispatcher Localises transition rules; makes illegal transitions impossible by construction Trip state named methods Explicit named transition methods Single handle(event) method Small number of well-defined transitions favours explicit naming Matching Strategy pattern with pluggable concrete implementations Hardcoded nearest-driver algorithm in manager Interviewer signalled variability; OCP compliance Pricing Strategy pattern separate from surge Combined pricing-and-surge strategy Pricing and surge vary along different axes and change at different rates Surge Separately-injectable calculator component Embedded in pricing strategy; embedded in manager Composition preserves independent evolution Payment Strategy pattern per method Conditional dispatch on method string Each method has distinct behaviour and failure modes Payment construction Optional Factory for string-based construction Direct construction by callers Only necessary when callers use string identifiers Cancellation Externalised policy object Inline in manager Policy may change independently of state machine Coordination RideManager as thin Facade Distribute coordination across entities; god-class manager Facade hides subsystem complexity while keeping coordination discoverable Driver availability Attribute on Driver Attribute on Trip; managed by matching strategy Availability is a property of the driver in the world, not of any particular trip Trip driver snapshot Vehicle captured on assignment Read from driver at completion time Handles vehicle switches between assignment and completion No-driver-available behaviour Return trip with driver=None Raise exception Recoverable condition, not exceptional Payment failure behaviour Trip stays Completed; payment retry is caller\u0026rsquo;s problem Roll back trip to un-completed state Mirrors real system behaviour; ride happened regardless The following table maps each SOLID principle to where in the design it is upheld.\nPrinciple Where Upheld Single Responsibility Each strategy, policy, and state has exactly one reason to change Open/Closed New matching, pricing, payment, and cancellation policies can be added without modifying the manager Liskov Substitution Any MatchingStrategy, PricingStrategy, PaymentMethod, or TripState subclass is usable wherever the base is expected Interface Segregation Each strategy interface exposes only the methods relevant to its concern Dependency Inversion The manager depends on abstractions (strategy interfaces), not on concrete implementations The following table summarises the patterns used and where each earned its place.\nPattern Applied To Justification State Trip lifecycle Rich lifecycle with state-dependent operations Strategy Matching Interviewer signalled future variability Strategy Pricing Multiple pricing schemes possible in production Strategy Payment Distinct behaviour and failure modes per method Facade RideManager Coordinates many subsystems behind a small interface Factory Payment construction Isolates string-to-instance construction from callers Observer Notifications (deferred) Not built into initial design; extension point identified Policy Cancellation fees Extracted from manager for independent evolution 18. Key Takeaways The following six principles generalise from this design to almost every LLD problem a candidate will encounter.\nDecompose relentlessly, then compose deliberately. The single most important skill in low-level design is the ability to identify independent concerns and place each in its own component. The ride-sharing system has at least five such concerns (matching, pricing, surge, payment, cancellation) plus a lifecycle state machine, and treating them as truly independent is what allows each to evolve without disturbing the others.\nA good coordinator is a thin coordinator. The RideManager in this design runs under 150 lines and contains no domain logic. Every substantive decision it makes is delegated to a component it holds a reference to. When the coordinator is thin, the system is composable; when the coordinator is fat, the system becomes a monolith with class syntax.\nEvery pattern must earn its place. Patterns are tools, not decorations. The State pattern is used because the trip has a genuinely rich lifecycle; the Strategy pattern is used three times because three independent axes of variation exist; the Facade pattern is used because many components require coordination behind a small interface. Patterns that do not correspond to a real need (such as speculative repositories, builders, or command objects) are omitted.\nDiscipline in what is not modelled matters as much as discipline in what is modelled. Every unnecessary class is future maintenance debt. The design deliberately does not model apps, notifications (as a class), maps, gateways, or routes, because none of these has a genuine operational role at the LLD scope. Restraint here is a demonstration of design maturity.\nInterfaces are contracts, not suggestions. The narrow interface of MatchingStrategy.select(trip, drivers) -\u0026gt; Optional[Driver] prevents matching strategies from developing coupling to unrelated concerns. Every interface in this design is intentionally narrow, and this narrowness is what allows implementations to be substituted freely.\nFollow-up questions are opportunities, not threats. The concerns raised in follow-up questions (scale, concurrency, notifications, pool rides, scheduled rides, payment failures, trust and safety) are all accommodated by the design without structural changes. Anticipating these questions and rehearsing grounded responses is the difference between a passable design and an impressive one.\n19. Practice Questions and Solutions The following questions are designed to consolidate understanding of the design decisions made in this deep dive. Each question is accompanied by a full solution.\nQuestion 1 Suppose the requirement changes so that a rider can specify a preference for a specific driver (a favourite driver). Which component of the design should be modified, and how?\nSolution. This is a matching-policy change, so it belongs in a new MatchingStrategy implementation. A FavouriteDriverStrategy would attempt to match the specified favourite driver first if they are available and driving the requested vehicle type, and would fall back to nearest-driver behaviour otherwise. The RideManager requires no modification; only a new strategy class is added and the manager is configured with it. This is the Open/Closed Principle in action: extension without modification.\nQuestion 2 Explain why the CancellationPolicy in this design cannot inspect trip.state at the time of fee computation. What are two alternative designs that eliminate this limitation, and what is the tradeoff of each?\nSolution. The CancellationPolicy cannot inspect trip.state because by the time the policy\u0026rsquo;s fee method is invoked, the state has already been transitioned to Cancelled. The current design works around this by inferring the pre-cancel state from other markers on the trip (started_at and driver).\nThe first alternative is to capture a snapshot of the pre-cancel state on the trip at cancellation time, for example by setting trip.state_before_cancel = trip.state before performing the transition. The policy then reads this attribute directly. The tradeoff is an additional piece of state on the trip that exists solely to support the policy computation.\nThe second alternative is to invert the order of operations: compute the fee first (while the state is still valid), then transition to Cancelled. The tradeoff is that fee computation becomes tangled with state-transition logic in the manager, and any failure in the fee computation would leave the trip in an inconsistent state.\nBoth alternatives eliminate the inference brittleness at some structural cost. The right choice depends on how frequently the cancellation policy is expected to change and how much the design values immutability of the trip\u0026rsquo;s transition sequence.\nQuestion 3 The RideManager.complete_trip method invokes the pricing strategy before performing the state transition to Completed, and it sets trip.completed_at twice (once directly and once through the state transition). Why is this awkward, and what alternative design would avoid it?\nSolution. The awkwardness arises because the pricing strategy requires both started_at and completed_at to be set on the trip in order to compute the duration, but the state transition to Completed is what conventionally sets completed_at. Setting completed_at before the state transition is required to give pricing the data it needs, and the state transition then redundantly sets it again.\nThe alternative design changes the pricing strategy\u0026rsquo;s signature to take the completion time as an explicit parameter rather than reading it from the trip. The manager then computes the fare as pricing.compute(trip, now), and only the state transition writes to the trip. This removes the double-set and cleanly separates the \u0026ldquo;compute fare\u0026rdquo; concern from the \u0026ldquo;record fare on trip\u0026rdquo; concern.\nThe current design was retained because keeping the pricing strategy\u0026rsquo;s signature narrow (compute(trip)) is arguably worth the small awkwardness in the manager. Either choice is defensible.\nQuestion 4 Consider adding a new Paused state that represents a mid-trip stop (for example, when a rider asks the driver to wait outside a shop). What changes are required across the design?\nSolution. The State pattern makes this change substantially easier than it would be under a conditional-based design. The following additions are required:\nAdd a new Paused state class that permits transitions to Started (to resume) and to Cancelled. The Started.pause method is added to allow the transition into Paused, and Paused.resume allows the transition back. Add corresponding pause and resume methods to the TripState abstract base with the default raising behaviour. Add pass-through pause and resume methods on the Trip class. Add corresponding manager methods pause_trip and resume_trip. Decide how paused time is handled by the pricing strategy: does time spent in Paused count toward the per-minute charge? If not, the pricing strategy must track paused intervals. No changes are required to matching, surge, cancellation, or payment. The lifecycle change is contained within the state machine and its immediate coordinators. This localisation is precisely what the State pattern was designed to provide.\nQuestion 5 The current NearestDriverStrategy uses straight-line distance. In production, road distance would be used instead, requiring an external routing service. How should the design accommodate this without making the strategy dependent on the routing service?\nSolution. The distance_to method on Location is the current source of distance values. Replacing straight-line distance with road distance requires a different distance computation, but embedding the routing service directly in Location would give Location a network dependency, which is a significant increase in coupling for a value object.\nThe correct approach is to inject a distance function into the matching strategy at construction time. The strategy signature becomes NearestDriverStrategy(distance_fn=lambda a, b: a.distance_to(b)) by default, and a production variant substitutes distance_fn=road_distance_service.distance_between. The routing service is thus a dependency of the matching strategy, not of Location, which preserves Location as a pure value object.\nAn alternative is to introduce a DistanceProvider abstraction that the matching strategy holds a reference to, giving even more flexibility (for example, allowing multiple distance functions to be blended). Either approach preserves the separation between Location (a pure value type) and the distance computation (a potentially expensive external operation).\nQuestion 6 The Factory pattern was introduced for PaymentMethod construction. Would a Factory also be justified for constructing Trip, Rider, or Driver instances? Explain.\nSolution. A Factory is justified when the construction of an object is either complex (many parameters with defaults or interdependencies), variable (multiple concrete subtypes depending on input), or requires isolation from callers (as in the string-to-type dispatch for payment methods).\nTrip construction is currently direct: the manager creates a Trip inside request_ride. Introducing a TripFactory would add a layer of indirection without corresponding benefit, because there is only one trip type and its constructor arguments are all supplied directly by the caller. This would be pattern-for-its-own-sake, which is a form of over-engineering.\nRider construction is similarly direct: an application creates a Rider when a user signs up. No factory is needed.\nDriver construction is a marginally more interesting case because a driver requires a vehicle and a location. If drivers are constructed from database records or from API payloads, a factory could encapsulate the mapping between the input representation and the constructor arguments. Whether this rises to the level of a factory or remains a simple mapping function depends on the complexity of the input format.\nThe general principle is that Factory is a solution to a specific problem (isolating construction from callers when construction is non-trivial), and applying it in the absence of that problem introduces cost without benefit.\nQuestion 7 The design uses a single-threaded, in-memory assumption. Describe the smallest set of changes required to make the design safe for concurrent driver assignment, and explain what invariant this preserves.\nSolution. The concurrent-assignment vulnerability is in RideManager.request_ride, where the sequence of operations (invoke matching, mark driver unavailable) is not atomic. If two threads execute this sequence with overlapping timing, both may observe the same driver as available and both may attempt to assign them.\nThe smallest change is to introduce a lock (either a global lock on the manager or a per-driver lock) around the read-then-mark sequence:\nwith self._matching_lock: driver = self._matching.select(trip, list(self._drivers.values())) if driver is not None: trip.assign_driver(driver) driver.is_available = False The invariant preserved by this lock is that at most one trip observes any given driver as available and successfully assigns them. Any two threads attempting to assign the same driver will serialise, and only the first will succeed; the second will re-invoke matching after the lock is released and either select a different driver or receive None.\nA more scalable alternative uses compare-and-swap on the is_available flag itself, avoiding the global lock and allowing higher throughput at the cost of occasional retries. The invariant preserved is identical.\nEither approach requires no changes to the strategy classes, the state classes, or the trip class. The concurrency concern is localised to the manager\u0026rsquo;s coordination logic, which is where it belongs.\nQuestion 8 Explain why the Rider class in this design is retained even though it contains no additional attributes or methods beyond what User provides. Under what circumstances would you eliminate it?\nSolution. The Rider class is retained because it provides type identity even in the absence of additional state. A method signature such as def request_ride(rider: Rider, ...) establishes at development time (through type checkers such as mypy or pyright) that a Driver cannot be accidentally passed where a Rider is expected. This kind of type-based error prevention is valuable and comes at the cost of a two-line subclass.\nThe class would be eliminated only under the following circumstances. If the design were entirely dynamically typed with no type-checker involvement, the type identity benefit would be reduced (though not eliminated, because isinstance(user, Rider) would still work). If it were established that no rider-specific behaviour or attribute would ever be added and no type-based distinction was desired, the class would become truly redundant. In practice, both of these conditions are rarely met, and retaining the subclass is the safer default.\nQuestion 9 The SurgeCalculator interface takes both a time and a location as parameters, even though the default implementation ignores the location. Explain the design principle at work and give one other example of the same principle from this design.\nSolution. The principle is that an interface should be designed for the range of expected implementations, not merely for the current default. Including the location parameter in multiplier allows future area-based surge implementations to be substituted without any caller change. Omitting the location would force every caller of multiplier to change when area-based surge is added, propagating the modification across the codebase.\nThe same principle appears in the MatchingStrategy.select(trip, drivers) signature. The trip parameter includes source location, requested vehicle type, and rider identity, even though the default NearestDriverStrategy uses only source location and vehicle type. A future strategy that considers rider preferences (favourite drivers, historical driver ratings) needs only the trip parameter as it already exists.\nThis principle is sometimes called \u0026ldquo;designing for the interface, not the implementation,\u0026rdquo; and it is one of the practical manifestations of the Open/Closed Principle at the level of method signatures.\nQuestion 10 The RideManager.rate_trip method updates the running average on both Driver and Rider through their shared update_rating method on User. Is this a violation of the Liskov Substitution Principle? Explain.\nSolution. No, this is not a violation. Liskov Substitution requires that subclass instances be usable wherever superclass instances are expected, without altering the correctness of the program. In this design, both Rider and Driver inherit update_rating from User and implement it identically (through inheritance, without override). The method\u0026rsquo;s behaviour is consistent regardless of which subtype the instance is, so substitutability holds trivially.\nA hypothetical violation would arise if, for example, Driver.update_rating were overridden to reject updates below a certain threshold (thereby narrowing the accepted input), or if Rider.update_rating had side effects that Driver.update_rating lacked (thereby introducing behavioural inconsistency). In such cases, code that relied on User.update_rating\u0026rsquo;s general contract would produce different results depending on the actual runtime type, which is the essence of a Liskov violation.\nThe current design avoids this pitfall by keeping the rating behaviour on the base class and having both subclasses inherit it unchanged.\n20. Further Reading and Cross-References This deep dive is one of a series of low-level design problem walkthroughs. The recommended reading order for a learner working through the full curriculum is as follows.\n01_oop_in_python.md for the Python object-oriented foundations assumed throughout. 02_solid_principles.md for the SOLID principles referenced at every stage of this design. 03_design_patterns_creational.md, 04_design_patterns_structural.md, and 05_design_patterns_behavioral.md for the design patterns applied here (Strategy, State, Facade, Factory). 06_lld_interview_questions.md for the general approach to LLD interview problems. 06a_lld_parking_lot_deep_dive.md, 06b_lld_elevator_deep_dive.md, and 06c_lld_library_deep_dive.md for earlier problems in the series that establish techniques used here. 07_advanced_lld_topics.md for the follow-up topics (concurrency, distribution, event-driven architecture) touched on in section 16. The accompanying modular Python project (ride_sharing_system/) implements the design described in this document. Reading the code alongside the corresponding sections here is the most effective way to internalise the design and to develop the ability to reproduce it under interview conditions.\nThis content is part of Codeverra, a platform for learning coding, data science, DSA, and AI from scratch. Explore more at https://codeverra.com\n","permalink":"https://blogs.codeverra.com/blog/low_level_design/interview_questions/06f_lld_ride_sharing_deep_dive/","summary":"\u003ch1 id=\"lld-deep-dive-4-designing-a-ride-sharing-system\"\u003eLLD Deep Dive #4: Designing a Ride-Sharing System\u003c/h1\u003e\n\u003cblockquote\u003e\n\u003cp\u003e\u003cstrong\u003eWhat this document is.\u003c/strong\u003e This is a comprehensive, first-principles walkthrough of how to design a ride-sharing system (of the Uber or Ola variety) as a low-level design (LLD) interview problem. It reasons from the vague problem statement through requirement clarification, entity identification, class hierarchy decisions, behavioural modelling with design patterns, and finally a fully composed system. Every design choice is stated together with the alternatives that were considered and the criteria that decided between them, so that a learner reading this can not only reproduce the design but also defend it and adapt it.\u003c/p\u003e","tags":null,"title":""},{"content":"LLD Deep Dive #4: Designing an ATM System What this document is: A structured, step-by-step walkthrough of the ATM low-level design problem as it would be reasoned through in a technical interview. The document proceeds from first principles, weighs alternative designs at each decision point, and arrives at a solution that combines the State pattern, the Strategy pattern, the Command pattern, and a clean separation between hardware and software concerns. Definitions of every technical term are provided when the term is introduced, so the document can be read without prior familiarity with the design-pattern literature.\nPrerequisite reading: Deep dives on the Parking Lot (#1), Elevator (#2), and Library (#3). The State pattern discussed at length in the Elevator deep dive is directly reused here. The Dependency Inversion Principle, Single Responsibility Principle, and abstract-base-class mechanics from earlier documents are assumed as background.\nTable of Contents Why This Problem Matters Stage 1: Receiving the Problem Stage 2: Clarifying Requirements Stage 3: Identifying Entities Stage 4: Modeling the ATM State Machine Stage 5: Modeling Transactions Stage 6: The Cash Dispenser (Strategy and Chain of Responsibility) Stage 7: Card, Account, and the Bank Interface Stage 8: The ATM Class and Hardware Composition Stage 9: Session, Authentication, and Timeout Stage 10: Putting It All Together Stage 11: Validating with Mental Walkthroughs Stage 12: Anticipating Follow-Up Questions Summary of Patterns and Principles Used Key Takeaways Practice Questions with Solutions Final Reflection 1. Why This Problem Matters The ATM problem occupies a distinctive position in the landscape of low-level design questions. On the surface it is a state-machine problem comparable to the elevator or the vending machine, and recognising that fact quickly is expected of any competent candidate. Beneath this surface, however, three additional dimensions raise its complexity substantially and make it a richer exercise than most state-machine problems.\nThe first dimension is integration with an external system. The ATM is not the authoritative source of information about accounts, balances, or authentication. That authority resides with the bank, which is reached over a potentially unreliable network. Every design decision must accommodate the fact that the data owned by the bank may become unavailable, may return errors, or may return stale values. Candidates who treat the ATM as a self-contained system that owns its data invariably produce a design that will fail on this axis.\nThe second dimension is the presence of multiple transaction types with genuinely different execution paths. A withdrawal is not merely a deposit with a sign flip. A withdrawal touches the cash dispenser and requires a compensating action on hardware failure; a deposit takes cash from the user and requires a compensating action on bank failure; a transfer touches two accounts and requires atomicity between them; a balance inquiry is read-only. Modeling this heterogeneity cleanly requires either inheritance-based polymorphism or the Command pattern, both of which represent real design decisions rather than mechanical rewrites.\nThe third dimension is a physical constraint that shapes the software. Cash is dispensed in denominations, and the machine holds a finite inventory of each. Producing a requested amount from the available denominations is a small optimisation problem, and how it is structured has consequences for extensibility. This is where the Strategy pattern and the Chain of Responsibility pattern become relevant. Recognising which pattern applies, and why, is one of the most distinctive signals a candidate can send on this problem.\nA candidate who demonstrates command of the state machine, models transactions cleanly, chooses an appropriate dispensing strategy, and cleanly abstracts the bank interface has covered a broader surface area of the design-pattern landscape than most LLD problems require. That surface area is what makes the ATM problem worth studying carefully, even though it does not appear in interview question banks quite as frequently as the parking lot or the elevator.\n2. Stage 1: Receiving the Problem The interviewer\u0026rsquo;s opening prompt is typically a single sentence:\n\u0026ldquo;Design an ATM.\u0026rdquo;\nThe vagueness is intentional and constitutes the first evaluation criterion. The candidate is expected to notice how underspecified the problem is and to respond with structured clarification rather than assumption. The set of interpretations consistent with the prompt is large, and each interpretation carries different design consequences.\nReasonable interpretations of what \u0026ldquo;an ATM\u0026rdquo; might mean include the following. The machine may support only cash withdrawal, or it may additionally accept deposits, execute transfers, and print statements. The machine may serve customers of a single bank, or it may accept cards issued by any bank through a switching network such as NPCI or Visa. Authentication may use only a PIN, or it may additionally use biometrics, one-time passwords, or challenge questions. The bank backend may be considered inside the scope of the design or treated as an opaque external service. The hardware model may be coarse (the whole machine is one object) or fine-grained (card reader, cash dispenser, screen, keypad, printer, and deposit slot are modeled separately).\nThe scope-defining decisions listed above cannot be inferred from the prompt itself. The interviewer expects the candidate to enumerate them and to elicit answers before proceeding. Beginning to design without doing so is the most common way this problem is failed at Stage 1.\n3. Stage 2: Clarifying Requirements The following section walks through nine structured clarifying questions. For each, the reasoning behind asking is stated (what the answer commits us to), a plausible interviewer response is supplied, and the design consequence is drawn out. This structure of \u0026ldquo;reason to ask, answer received, consequence\u0026rdquo; is exactly what is expected in the interview itself.\nQ1: What transactions does the ATM support? Reason to ask. This is the largest scope-shaping question. The set of transactions determines how many concrete transaction classes are designed, which hardware components must be present, and how the state machine branches after authentication.\nThe candidate space of transactions includes cash withdrawal, cash deposit, cheque deposit, balance inquiry, mini-statement (a printed list of recent transactions), funds transfer, and PIN change.\nAssumed response. \u0026ldquo;Withdrawal, cash deposit, balance inquiry, and same-bank funds transfer.\u0026rdquo;\nConsequence. A Transaction hierarchy with exactly four concrete subclasses will be designed. Cheque deposit is out of scope, which removes the need to model a document scanner, cheque validation, or a clearing workflow. Mini-statement is out of scope, which removes the need to model a transaction history store on the bank side. These exclusions materially reduce the design surface.\nQ2: Single bank or interbank? Reason to ask. An interbank ATM must route non-home-bank requests through a switching network, which introduces additional latency, additional failure modes, and a routing layer between the ATM and the ultimate account holder. A single-bank ATM has a simpler dependency graph: it talks directly to one bank service and no other.\nAssumed response. \u0026ldquo;Assume single bank for now. Interbank routing can be treated as an extension.\u0026rdquo;\nConsequence. The bank can be modeled as a single dependency, referred to as BankService, rather than as a tree of routing decisions. When the extension question is raised later in the interview, we can describe how a RoutingService would be inserted between the ATM and the bank without materially altering the rest of the design.\nQ3: What is the authentication mechanism? Reason to ask. The authentication phase of the state machine varies significantly depending on the mechanism. PIN-only authentication involves one credential exchange; PIN plus biometric involves two; OTP-backed authentication involves a callback into a separate delivery channel. Each mechanism dictates the states and transitions between card insertion and confirmed authentication.\nAssumed response. \u0026ldquo;Card plus 4-digit PIN. Three consecutive wrong PIN attempts result in the card being physically retained by the machine.\u0026rdquo;\nConsequence. The design must include a wrong-attempt counter, a retention (capture) action on the card reader, and clear state transitions for the retention event. Card capture is a physical event in which the machine draws the card into an internal bin, from which only a bank technician can retrieve it. This is a specific, named event and requires modeling.\nQ4: What denominations does the machine dispense, and how is the amount broken up? Reason to ask. This question determines whether the dispensing subsystem is trivial or non-trivial. If the interviewer states that any amount may be requested and the machine dispenses exact change with no concern for available denominations, the dispenser degenerates into a stub. If the interviewer specifies denominations and requires the machine to compute a valid combination from current inventory, the dispenser becomes an interesting design problem in its own right and admits either the Strategy pattern or the Chain of Responsibility pattern.\nAssumed response. \u0026ldquo;Indian denominations, namely ₹100, ₹200, ₹500, and ₹2,000. The machine dispenses using the largest denominations first. Requested amounts must be multiples of ₹100. If the requested amount cannot be produced from the currently available denominations, the request is rejected.\u0026rdquo;\nConsequence. Four denominations, a deterministic largest-first algorithm, and a real possibility of failure due to inventory shortage. This is exactly the shape of a problem for which we will introduce a DispensingStrategy abstraction with a GreedyStrategy default implementation. The Chain of Responsibility variant will be introduced as an alternative for scenarios in which per-denomination policy diverges, and the trade-off will be discussed explicitly in Stage 6.\nQ5: How does the ATM know about accounts and balances? Reason to ask. This forces an explicit decision about the boundary between the ATM and the bank. If the ATM stores accounts and balances internally, the bank is inside the design and must be modeled as a component. If the bank is treated as external, the ATM depends on an abstract service interface and forwards operations to it.\nAssumed response. \u0026ldquo;The ATM does not store accounts. It calls the bank\u0026rsquo;s service for balance lookups, debits, and credits. Assume synchronous calls that may fail. On failure, the ATM must cancel the transaction and reverse any pending changes.\u0026rdquo;\nConsequence. A BankService abstraction is introduced, and an implicit rule is established: if a bank call fails, the transaction cannot proceed and any prior state changes must be reversed. This rule has consequences for the ordering of operations inside every mutating transaction, which will be addressed in Stage 5.\nQ6: How are deposited notes counted? Reason to ask. Real ATMs use a bill validator inside the deposit slot to count and authenticate notes automatically. Whether this counting logic is inside our design or abstracted behind a component boundary is a scope decision.\nAssumed response. \u0026ldquo;Model the deposit slot as a component that returns the counted, validated total. The internals of note validation are out of scope.\u0026rdquo;\nConsequence. A DepositSlot component is introduced with a collect_and_count() method that returns the total amount collected. The design trusts the returned value. The physical mechanics of counting and validation are not modeled.\nQ7: What are the concurrency assumptions? Reason to ask. ATMs are single-user machines: only one person interacts with the physical hardware at a time. However, the bank backend is shared, and the same account can be touched by another ATM or by an online transaction while our machine is mid-transaction. Concurrency at the account level is therefore a real concern, even though concurrency within our machine is not.\nAssumed response. \u0026ldquo;Single user per ATM. Assume the bank enforces account-level serialisation correctly. Our ATM must handle any resulting failure gracefully.\u0026rdquo;\nConsequence. The ATM itself does not require internal locking. The bank interface is expected to report conflicts as failures, which the ATM handles the same way it handles insufficient funds: by cleanly reporting the failure and returning to a valid state.\nQ8: Are receipts printed? Reason to ask. Receipt printing is a minor but real feature. Some transactions produce a receipt, some do not, and the user may decline. The presence or absence of a printer is a component decision.\nAssumed response. \u0026ldquo;Optional receipt at the end of each transaction. If the user opts in, print via a Printer component.\u0026rdquo;\nConsequence. A Printer component is added to the ATM. The receipt-decision branch appears at the end of each successful transaction\u0026rsquo;s flow.\nQ9: What is the session timeout policy? Reason to ask. If a user walks away mid-transaction without ejecting the card, the machine must not remain in an authenticated state indefinitely. Otherwise, the next user finds an active session belonging to someone else, which is a serious security failure.\nAssumed response. \u0026ldquo;If no user input is received for 30 seconds during an active session, eject the card and return to the idle state.\u0026rdquo;\nConsequence. Timeout logic is added to the Session object, with the timeout enforced at the entry point of each state\u0026rsquo;s action methods. Alternative timeout mechanisms (a background scheduler that fires on the timeout deadline) are noted as production improvements but not implemented here.\nRecap of the Confirmed Scope Before beginning the design, the scope is reflected back to the interviewer in a single paragraph:\n\u0026ldquo;The ATM supports four transactions: withdrawal, cash deposit, balance inquiry, and same-bank transfer. Authentication is card plus 4-digit PIN, with card capture after three wrong attempts. Cash is dispensed in ₹100, ₹200, ₹500, and ₹2,000 denominations using a largest-first strategy, and requested amounts must be multiples of ₹100. The bank is an external service accessed through a BankService interface, and bank failures must be handled cleanly. Deposits are counted by a validated slot that returns the total. One user per ATM. Optional receipt printing. A 30-second inactivity timeout ejects the card. Account-level concurrency is handled by the bank.\u0026rdquo;\nConfirming the scope explicitly serves two purposes. It gives the interviewer an opportunity to correct any misinterpretation before design effort is spent. It also establishes a shared reference point that the candidate can return to whenever a design choice needs to be justified.\n4. Stage 3: Identifying Entities With the requirements confirmed, the next step is to extract candidate nouns and filter them into a final entity list. This is a mechanical exercise that ensures no important object is overlooked and that no unimportant one is elevated to a class.\nThe full list of candidate nouns from the requirements: ATM, card, PIN, account, bank, bank service, transaction, withdrawal, deposit, balance inquiry, transfer, receipt, screen, keypad, cash dispenser, deposit slot, printer, card reader, denomination, session, and state (with its various sub-states: idle, has-card, authenticated, transaction-selected).\nThe filtering step evaluates each noun for whether it merits a distinct class in the design. The criteria are: does the noun carry data or behavior of its own, does it have a lifecycle, and does it appear in multiple places in the design such that a shared type helps.\nNoun Class? Reasoning ATM Yes Top-level coordinator that owns hardware components, the bank reference, and the current state Card Yes Has data fields (number, holder name, expiry date, associated account number) and simple behavior (is_valid()) PIN No A plain string, validated by the bank and not persisted client-side Account Yes A lightweight data holder with identifiers; the authoritative state (balance) lives with the bank BankService Yes (interface) Modeled as an abstract interface to allow multiple concrete implementations Transaction Yes (hierarchy) Different transactions have genuinely different execution paths; polymorphism is natural Receipt Yes A small immutable value object capturing details for printing Screen Yes A component with a single responsibility (rendering text) Keypad Yes A component that reads user input CashDispenser Yes A component with substantial internal logic; the primary structural focus of Stage 6 DepositSlot Yes A component that returns a counted amount Printer Yes A component that prints receipts CardReader Yes A component that reads, ejects, and captures cards Denomination Not required The denomination values are integer keys in the dispenser\u0026rsquo;s inventory dictionary Session Yes Captures per-visit transient state including authentication flag, wrong-PIN counter, and timing ATMState Yes (hierarchy) One class per state under the State pattern Why the design has so many hardware component classes The decision to introduce five distinct hardware component classes (Screen, Keypad, CardReader, CashDispenser, DepositSlot, Printer, plus the Printer\u0026rsquo;s data object Receipt) reflects two considerations.\nFirst is the Single Responsibility Principle (SRP), which states that a class should have only one reason to change. If all hardware behavior were folded into the ATM class, the class would accumulate methods for rendering text, reading input, ejecting cards, counting deposits, dispensing cash, and printing receipts. Any change to any hardware behavior would touch this single class, and the class would rapidly become the largest and most fragile component in the system. Distributing the responsibilities across dedicated components isolates changes to their proper location.\nSecond is testability. The Dependency Inversion Principle (DIP) states that high-level modules should not depend on low-level modules and that both should depend on abstractions. By isolating each hardware behavior in its own class, we make it feasible to substitute a test double for any component during unit testing. A test that verifies the withdrawal flow does not require an actual cash dispenser; it requires an object exposing the same interface against which assertions can be made.\nWhy BankService is an interface rather than a concrete class The BankService is defined as an abstract interface rather than a concrete class because the ATM should not have any awareness of whether the bank is implemented as a REST client, a database connection, or an in-memory dictionary used only for testing. What the ATM cares about is a small set of operations: authenticating a PIN, retrieving a balance, debiting an account, crediting an account, and transferring funds between accounts. That set of operations is precisely the signature of a service interface, and defining it as such is the direct application of the Dependency Inversion Principle.\nThe interface is defined using Python\u0026rsquo;s abc module. An abstract base class (ABC) is a class that cannot be instantiated directly and typically declares one or more abstract methods that must be implemented by any concrete subclass. In Python, this is achieved by inheriting from ABC and decorating methods with @abstractmethod. Attempting to instantiate a class that has unimplemented abstract methods raises a TypeError at construction time, which is how the language enforces the contract.\nThe Final Entity List The consolidated set of classes to be designed is as follows:\nATM: the top-level coordinator Card, Account, Session, Receipt: data-holding value objects Transaction (abstract), with concrete subclasses Withdrawal, Deposit, BalanceInquiry, and Transfer ATMState (abstract), with concrete subclasses IdleState, HasCardState, AuthenticatedState, and TransactionSelectedState BankService (abstract), with concrete implementation InMemoryBankService Hardware components: CardReader, Screen, Keypad, CashDispenser, DepositSlot, Printer Dispensing-algorithm classes: DispensingStrategy (abstract), GreedyStrategy (default), ChainStrategy and DenominationHandler (alternative) 5. Stage 4: Modeling the ATM State Machine The single most consequential structural decision in this problem is how the ATM\u0026rsquo;s operational modes are represented in code. A careless decision at this stage produces a design that is difficult to extend and difficult to reason about, regardless of how carefully the remainder is executed.\nWhy an ATM Is a State Machine Observing the sequence of interactions at an actual ATM reveals that the machine operates in a small number of distinct modes, and that the set of actions the user may perform depends entirely on which mode is currently active.\nAt rest, the machine displays a welcome message and awaits card insertion. The card reader is armed. No other action is meaningful in this mode: entering a PIN, requesting a balance, or ejecting a card produces no effect because there is no card to authenticate against, no account to inquire about, and no card to eject.\nAfter a card is inserted, the machine transitions into a mode in which it awaits PIN entry. In this mode, the keypad\u0026rsquo;s input is interpreted as a PIN. Inserting a second card is not meaningful because the reader already holds one; requesting a transaction is not meaningful because the user has not yet been authenticated.\nAfter correct PIN entry, the machine is authenticated and awaits transaction selection. The keypad\u0026rsquo;s input is now interpreted as menu selection rather than PIN. Entering a PIN again is not meaningful because authentication is already complete.\nAfter a transaction is selected, the machine executes the transaction. During execution, most user actions are ignored or deferred until the execution completes.\nAfter execution completes, the machine returns to the authenticated mode so the user may perform another transaction, or the user may eject the card and return the machine to the idle mode.\nEach of these operational modes is a state in the formal sense: a distinct condition of the system in which the set of permissible actions differs from other conditions, and in which each action either transitions the system to another specified state or is rejected as invalid.\nDefinition of the State Pattern The State pattern is a behavioral design pattern in which an object\u0026rsquo;s behavior changes based on its internal state. Each state is represented as a separate class that conforms to a common interface, and the enclosing object delegates operations to the current state object rather than branching on state internally. The pattern\u0026rsquo;s purpose is to keep the invariants of each state local to a single class, rather than scattering them across many methods of a larger class that must remain mutually consistent.\nTwo alternative designs will now be presented, with a comparison of their properties.\nApproach A: Flag-Based Design with Conditional Branching The naive design maintains a single string field to record the current state and branches on that field inside every action method:\nclass ATM: def __init__(self): self.state = \u0026#34;idle\u0026#34; self.card = None self.pin_attempts = 0 def insert_card(self, card): if self.state == \u0026#34;idle\u0026#34;: self.card = card self.state = \u0026#34;has_card\u0026#34; else: raise InvalidOperation(\u0026#34;Card already inserted\u0026#34;) def enter_pin(self, pin): if self.state == \u0026#34;has_card\u0026#34;: # Verify the PIN and transition to the authenticated state. ... elif self.state == \u0026#34;idle\u0026#34;: raise InvalidOperation(\u0026#34;No card inserted\u0026#34;) elif self.state == \u0026#34;authenticated\u0026#34;: raise InvalidOperation(\u0026#34;Already authenticated\u0026#34;) # Continue with additional branches for other states. Every action method contains a large conditional block that inspects self.state. The invariants defining what is legal in each state are scattered across every method. Adding a new state requires editing every method to insert the new case. Removing a state requires editing every method to remove the corresponding case. Renaming a state requires editing every method to update the string comparison.\nThis scattering of invariants is precisely the design defect that the State pattern is intended to prevent.\nApproach B: One Class per State (State Pattern) Under the State pattern, each operational mode becomes a distinct class that inherits from a common abstract base class. The abstract base declares every action the ATM can receive and provides a default implementation for each that raises InvalidOperation. Concrete state classes override only the actions that are valid in their state, leaving invalid actions to inherit the base\u0026rsquo;s rejection behavior.\nfrom abc import ABC from atm_system.exceptions import InvalidOperation class ATMState(ABC): \u0026#34;\u0026#34;\u0026#34;Abstract base class for all ATM states. Every action the ATM can receive is declared here with a default implementation that raises InvalidOperation. Concrete states override only the actions valid in their state. This design keeps the invariants of each state local to a single class. \u0026#34;\u0026#34;\u0026#34; def insert_card(self, atm, card): raise InvalidOperation(f\u0026#34;Cannot insert card in state {self.name()}\u0026#34;) def enter_pin(self, atm, pin): raise InvalidOperation(f\u0026#34;Cannot enter PIN in state {self.name()}\u0026#34;) def select_transaction(self, atm, txn): raise InvalidOperation(f\u0026#34;Cannot select transaction in state {self.name()}\u0026#34;) def eject_card(self, atm): raise InvalidOperation(f\u0026#34;Cannot eject card in state {self.name()}\u0026#34;) def name(self) -\u0026gt; str: return self.__class__.__name__ Concrete states override only the actions that are meaningful in that state:\nclass IdleState(ATMState): \u0026#34;\u0026#34;\u0026#34;The machine is at rest, waiting for a card. Only insert_card is valid.\u0026#34;\u0026#34;\u0026#34; def insert_card(self, atm, card): atm.session = Session(card=card) atm.set_state(HasCardState()) atm.screen.show(\u0026#34;Please enter your PIN\u0026#34;) class HasCardState(ATMState): \u0026#34;\u0026#34;\u0026#34;A card has been inserted; the PIN has not yet been verified.\u0026#34;\u0026#34;\u0026#34; def enter_pin(self, atm, pin): if atm.bank.authenticate(atm.session.card, pin): atm.session.authenticated = True atm.set_state(AuthenticatedState()) atm.screen.show(\u0026#34;Authenticated. Select a transaction.\u0026#34;) else: atm.session.wrong_pin_attempts += 1 if atm.session.wrong_pin_attempts \u0026gt;= 3: atm.card_reader.capture(atm.session.card) atm.set_state(IdleState()) atm.session = None atm.screen.show(\u0026#34;Card retained. Contact your bank.\u0026#34;) else: remaining = 3 - atm.session.wrong_pin_attempts atm.screen.show(f\u0026#34;Incorrect PIN. {remaining} attempts remaining.\u0026#34;) def eject_card(self, atm): atm.card_reader.eject(atm.session.card) atm.set_state(IdleState()) atm.session = None The remaining states (AuthenticatedState, TransactionSelectedState) follow the same shape and are presented in full in Stage 10 when the complete design is assembled.\nComparison of the Two Approaches The following table summarises the trade-offs between the two approaches on the dimensions that matter most for maintenance and extension.\nConcern Flag-Based State Pattern Locality of invariants Scattered across all action methods Local to each state class Cost of adding a new state Edit every action method Add one new class Cost of removing a state Edit every action method Delete one class Overriding-only-what-is-valid Requires disciplined convention Enforced by inheritance Testing a specific transition Requires walking the machine through prior states Instantiate the state and test directly Number of classes One (fat) Several (thin) Overhead on small state machines None Non-trivial ceremony The pattern\u0026rsquo;s cost is the ceremony of several small classes. Its benefit is the elimination of the scattered-invariant problem. For a state machine with three or fewer states, the ceremony arguably outweighs the benefit; for one with five or more states, each supporting several actions, the flag-based design becomes a maintenance liability. The ATM has five effective states and four action methods on each, giving twenty potential (state, action) combinations. The State pattern is the correct choice.\nThe Complete State Diagram The state diagram for the ATM shows the transitions available from each state, labeled by the triggering action:\n[IdleState] | | insert_card (valid card) v [HasCardState] / \\ eject_card enter_pin (correct) | | v v [IdleState] [AuthenticatedState] / \\ eject_card select_transaction | | v v [IdleState] [TransactionSelectedState] | | execute v [AuthenticatedState] (loop or eject_card) Two features of this diagram merit explicit attention.\nFirst, after a successful transaction, the machine returns to AuthenticatedState rather than to IdleState. This allows the user to perform multiple transactions without re-authenticating for each. Requiring re-authentication after each transaction would be technically simpler but would produce a user experience unlike any real ATM.\nSecond, the diagram does not show a separate EjectingCardState. In real hardware, ejection is a physical event that takes measurable time and during which no user input should be accepted. In this simplified design, ejection is treated as an atomic operation that completes immediately, and the machine transitions directly to IdleState. In Stage 12, we discuss how the design accommodates a proper EjectingCardState if the interviewer raises the point.\n6. Stage 5: Modeling Transactions The four transaction types (withdrawal, deposit, balance inquiry, transfer) differ from one another in their required inputs, their execution paths, and their failure modes. Modeling this heterogeneity cleanly requires a design decision comparable in significance to the state-machine decision.\nApproach A: A Single Transaction Class with a Type Flag The naive design uses a single Transaction class with optional fields and branches on a type discriminator:\nclass Transaction: def __init__(self, txn_type, amount=None, target_account=None): self.txn_type = txn_type # \u0026#34;WITHDRAWAL\u0026#34;, \u0026#34;DEPOSIT\u0026#34;, \u0026#34;INQUIRY\u0026#34;, \u0026#34;TRANSFER\u0026#34; self.amount = amount self.target_account = target_account def execute(self, bank, dispenser=None, deposit_slot=None): if self.txn_type == \u0026#34;WITHDRAWAL\u0026#34;: ... elif self.txn_type == \u0026#34;DEPOSIT\u0026#34;: ... elif self.txn_type == \u0026#34;INQUIRY\u0026#34;: ... elif self.txn_type == \u0026#34;TRANSFER\u0026#34;: ... This design has three concrete defects. The target_account field is meaningful only for transfers, so it is None for three-quarters of transactions and its presence in the constructor signature is misleading. The amount field is meaningless for balance inquiries. The execute method is a large conditional block on the type field, and every additional transaction type extends this block. These are the same categories of defect that afflict the flag-based state-machine design in the previous stage.\nApproach B: A Transaction Hierarchy Under the polymorphic approach, Transaction is an abstract base class that declares a single abstract execute method. Each concrete transaction subclass carries only the fields it requires and provides its own implementation of execute:\nfrom abc import ABC, abstractmethod class Transaction(ABC): \u0026#34;\u0026#34;\u0026#34;A transaction encapsulates one unit of work against an account. Each transaction knows how to execute itself against a BankService. The ATM does not branch on transaction type; it invokes execute() and relies on polymorphism to dispatch to the correct implementation. This design is a variant of the Command pattern. \u0026#34;\u0026#34;\u0026#34; def __init__(self, account): self.account = account self.status = \u0026#34;PENDING\u0026#34; @abstractmethod def execute(self, bank): \u0026#34;\u0026#34;\u0026#34;Execute the transaction and return a TransactionResult.\u0026#34;\u0026#34;\u0026#34; Definition of the Command Pattern The Command pattern is a behavioral design pattern that encapsulates a request as an object, allowing the request to be parameterised, queued, logged, or undone. In the classical formulation, a Command object holds a reference to a receiver (the object on which the operation acts) and exposes an execute method that performs the operation. The invoker (in our case, the ATM) calls execute without knowing which specific command is being executed.\nThe transaction hierarchy is a lightweight application of this pattern. Each transaction object holds references to the resources it needs (an account, an amount, a dispenser, a target account) and exposes an execute method. The ATM invokes execute without dispatching on type, and the concrete transaction determines what actually happens.\nThe Four Concrete Transactions The withdrawal transaction is the most complex because it touches two subsystems (the bank and the cash dispenser) and requires a compensating action if the second subsystem fails after the first has succeeded.\nclass Withdrawal(Transaction): \u0026#34;\u0026#34;\u0026#34;Withdraw cash from an account. Execution order: 1. Check that the dispenser can produce the requested amount. 2. Debit the account through the bank. 3. Physically dispense the cash. 4. If step 3 fails, credit the account back (compensating action). The ordering matters. If we debited first and then discovered that the dispenser could not produce the amount, the user\u0026#39;s account would be debited without the user receiving cash. Feasibility-check-first eliminates most of that risk; the compensating credit handles the residual risk of a hardware failure between plan and commit. \u0026#34;\u0026#34;\u0026#34; def __init__(self, account, amount, dispenser): super().__init__(account) self.amount = amount self.dispenser = dispenser def execute(self, bank): # Step 1: Feasibility. Read-only check that does not modify inventory. if not self.dispenser.can_dispense(self.amount): self.status = \u0026#34;FAILED\u0026#34; return TransactionResult.fail( \u0026#34;ATM cannot dispense the requested amount\u0026#34; ) # Step 2: Debit. The bank enforces balance, limits, and holds. if not bank.debit(self.account, self.amount): self.status = \u0026#34;FAILED\u0026#34; return TransactionResult.fail(\u0026#34;Insufficient funds\u0026#34;) # Step 3: Physically dispense. Any failure here triggers step 4. try: self.dispenser.dispense(self.amount) except DispenseError: # Step 4: Compensating action. bank.credit(self.account, self.amount) self.status = \u0026#34;FAILED\u0026#34; return TransactionResult.fail( \u0026#34;Dispensing failed; account has been credited back\u0026#34; ) self.status = \u0026#34;SUCCESS\u0026#34; return TransactionResult.ok( value=self.amount, message=f\u0026#34;Dispensed ₹{self.amount}\u0026#34;, ) The deposit transaction accepts cash from the user and credits the account, with a compensating physical return of cash if the bank rejects the deposit:\nclass Deposit(Transaction): \u0026#34;\u0026#34;\u0026#34;Deposit cash into an account. Execution order: 1. Collect and count the cash via the deposit slot. 2. Credit the account through the bank. 3. If crediting fails, physically return the cash to the user. \u0026#34;\u0026#34;\u0026#34; def __init__(self, account, deposit_slot): super().__init__(account) self.deposit_slot = deposit_slot self.amount = 0.0 def execute(self, bank): self.amount = self.deposit_slot.collect_and_count() if self.amount == 0: self.status = \u0026#34;CANCELLED\u0026#34; return TransactionResult.fail(\u0026#34;No cash deposited\u0026#34;) if not bank.credit(self.account, self.amount): self.deposit_slot.return_cash() self.status = \u0026#34;FAILED\u0026#34; return TransactionResult.fail(\u0026#34;Bank rejected the deposit\u0026#34;) self.status = \u0026#34;SUCCESS\u0026#34; return TransactionResult.ok( value=self.amount, message=f\u0026#34;Deposited ₹{self.amount:.2f}\u0026#34;, ) The balance inquiry transaction is read-only and has no compensating actions:\nclass BalanceInquiry(Transaction): \u0026#34;\u0026#34;\u0026#34;Read the current balance for the account. No side effects.\u0026#34;\u0026#34;\u0026#34; def execute(self, bank): balance = bank.get_balance(self.account) self.status = \u0026#34;SUCCESS\u0026#34; return TransactionResult.ok( value=balance, message=f\u0026#34;Balance: ₹{balance:.2f}\u0026#34;, ) The transfer transaction moves funds between two accounts and delegates atomicity to the bank:\nclass Transfer(Transaction): \u0026#34;\u0026#34;\u0026#34;Transfer funds between two accounts. Atomicity of the two-sided operation is delegated to the BankService, which is expected to either commit both sides or roll back both sides. \u0026#34;\u0026#34;\u0026#34; def __init__(self, source, target, amount): super().__init__(source) self.target = target self.amount = amount def execute(self, bank): if not bank.transfer(self.account, self.target, self.amount): self.status = \u0026#34;FAILED\u0026#34; return TransactionResult.fail( \u0026#34;Transfer failed (insufficient funds or bank error)\u0026#34; ) self.status = \u0026#34;SUCCESS\u0026#34; return TransactionResult.ok( value=self.amount, message=f\u0026#34;Transferred ₹{self.amount:.2f} to {self.target.account_number}\u0026#34;, ) The TransactionResult Value Object A uniform result type is defined so that calling code does not need to interpret tuples or booleans and does not need to know which specific fields each transaction produces:\nclass TransactionResult: \u0026#34;\u0026#34;\u0026#34;Uniform result type across all transaction executions. Having a single return type simplifies the ATM\u0026#39;s handling of results: the ATM inspects `success` to decide whether to print a receipt and inspects `message` to display to the user, without needing to know which transaction type produced the result. \u0026#34;\u0026#34;\u0026#34; def __init__(self, success, value=None, message=\u0026#34;\u0026#34;): self.success = success self.value = value # amount, balance, or None depending on transaction self.message = message # user-facing message @classmethod def ok(cls, value=None, message=\u0026#34;OK\u0026#34;): return cls(True, value=value, message=message) @classmethod def fail(cls, message): return cls(False, message=message) The Discipline of Order-of-Operations The execution order in Withdrawal.execute (feasibility check, then debit, then dispense, with a compensating credit on dispense failure) reflects a general discipline that is worth stating explicitly. Whenever code contains two or more mutating operations, the following question should be asked for each pair: if the second operation fails after the first has succeeded, is the state resulting from the first recoverable?\nIf the answer is yes, the operations may be executed in either order, though there may still be a preferred order for other reasons (efficiency, common-case fast paths, minimising work done before a failure).\nIf the answer is no, the operations must be either reordered so that the recoverable one occurs first, wrapped in a transaction that provides atomicity, or paired with an explicit compensating action that reverses the first operation when the second fails.\nThis discipline is the foundation of correct behavior in distributed systems more generally. The withdrawal transaction is a small instance of the same reasoning that produces two-phase commit, saga patterns, and outbox patterns in larger systems.\n7. Stage 6: The Cash Dispenser (Strategy and Chain of Responsibility) The cash dispenser is the most technically distinctive component of the ATM design. The stated problem is as follows: given a requested amount (say, ₹4,600) and an inventory of denominations (say, five ₹2,000 notes, ten ₹500 notes, five ₹200 notes, and eight ₹100 notes), produce the requested amount using the largest denominations first, and fail cleanly if the available inventory cannot produce the exact amount.\nTwo design patterns are candidates for structuring this component: the Strategy pattern and the Chain of Responsibility pattern. These patterns operate at different levels of abstraction, and understanding the distinction is one of the most valuable exercises the ATM problem offers. The design presented here uses Strategy as the primary abstraction and treats Chain of Responsibility as an alternative implementation strategy that becomes justified only under specific conditions.\nThe Naive Approach as a Baseline Before introducing either pattern, consider the direct implementation: a single function that iterates over denominations from largest to smallest and accumulates a plan:\ndef dispense_naive(amount, inventory): plan = {} for denom in sorted(inventory.keys(), reverse=True): available = inventory[denom] count = min(amount // denom, available) if count \u0026gt; 0: plan[denom] = count amount -= denom * count if amount == 0: break if amount != 0: raise DispenseError(\u0026#34;Cannot dispense with current inventory\u0026#34;) return plan This implementation is short, clear, and correct for the stated problem. Any design change that adds complexity beyond this baseline must be justified by a real requirement, not by pattern-application for its own sake.\nThe naive implementation has two limitations. First, the algorithm is hardcoded: only greedy largest-first behavior is supported. If a different algorithm is needed later (for example, an algorithm that preserves high-denomination notes for future large withdrawals), the function must be modified. Second, if per-denomination policy diverges (for example, if the ₹2,000 handler must reserve stock below a threshold, or if the ₹100 handler must cap contributions per transaction), the branching logic accumulates in this function and it stops being clean.\nDefinition of the Strategy Pattern The Strategy pattern is a behavioral design pattern in which a family of interchangeable algorithms is defined behind a common interface, allowing the algorithm used by a client to be selected or replaced at runtime without modifying the client. Each concrete strategy is a self-contained implementation of the algorithm. The client holds a reference to a strategy through the interface and delegates the algorithmic decision to it.\nApplied to cash dispensing, the Strategy pattern captures the observation that different algorithms may be used to break an amount into denominations. Greedy largest-first is one such algorithm. Others include:\nA balanced algorithm that preserves high-denomination notes when the requested amount is small, so that future large withdrawals remain possible. An optimal algorithm that minimises the total number of notes dispensed. For canonical currency systems such as the Indian rupee, this is equivalent to greedy; for pathological denomination sets it differs. An exact-only algorithm that refuses to dispense unless a valid plan exists that uses only whole notes. This is trivially the same as greedy for the ATM but differs for vending machines that must also give change. The Strategy pattern answers the question: which algorithm do we use to solve the whole problem? Each strategy is a complete, self-contained solution.\nDefinition of the Chain of Responsibility Pattern The Chain of Responsibility (CoR) pattern is a behavioral design pattern in which a request is passed along a sequence of handler objects. Each handler either handles the request in whole or in part and forwards the remainder, or forwards the request unchanged. The chain terminates when either a handler fully handles the request or the request reaches the end of the chain without being fully handled.\nApplied to cash dispensing, the CoR pattern makes each denomination a distinct handler that contributes what it can from its own inventory and forwards the remaining amount to the next handler in the chain. Handlers are linked in order from largest denomination to smallest.\nThe CoR pattern answers the question: how do we structure the execution across multiple handlers, each with its own logic? It shines when handlers have genuinely different behavior, not merely different parameters.\nThe Two Patterns Compared The distinction between the two patterns is subtle but consequential. The following table summarises when each pattern earns its place in the design.\nAspect Strategy Chain of Responsibility Question answered Which algorithm for the whole problem? How to distribute execution across handlers? Number of \u0026ldquo;solvers\u0026rdquo; per request One (the selected strategy) Multiple (one per handler in the chain) Concrete objects differ in Algorithmic approach Per-handler logic and configuration Justified when Multiple interchangeable algorithms exist Per-handler policy genuinely diverges Not justified when Only one algorithm makes sense All handlers do the same thing with different parameters Client sees The strategy interface The head of the chain (or a facade over it) For a uniform greedy algorithm across all denominations, the Strategy pattern with a single GreedyStrategy implementation is sufficient. The Chain of Responsibility becomes justified only when per-denomination policy diverges. Introducing CoR without such divergence produces a chain in which every handler is essentially identical modulo a parameter (the denomination value), which is a loop with additional ceremony.\nThe honest presentation of the design is therefore to lead with Strategy and treat CoR as a refinement available for when per-denomination logic becomes non-uniform. Both are implemented below so that the trade-off can be studied directly.\nThe DispensingStrategy Interface The abstract strategy is declared with a single method that computes a dispensing plan without mutating any state:\nfrom abc import ABC, abstractmethod from typing import Optional class DispensingStrategy(ABC): \u0026#34;\u0026#34;\u0026#34;Abstract algorithm for planning a cash dispense. Strategies are stateless. They take a target amount and a snapshot of the current inventory and return a plan (or None if the amount cannot be produced). The dispenser itself owns the inventory and is responsible for committing the plan; the strategy never mutates inventory. Separating \u0026#34;plan\u0026#34; (which the strategy computes) from \u0026#34;commit\u0026#34; (which the dispenser performs) ensures that a failed plan never corrupts the inventory state. \u0026#34;\u0026#34;\u0026#34; @abstractmethod def plan(self, amount: int, inventory: dict) -\u0026gt; Optional[dict]: \u0026#34;\u0026#34;\u0026#34;Compute a dispensing plan. Args: amount: The amount to dispense. inventory: Current stock as a mapping of denomination to count. Returns: A dict mapping denomination to count to dispense, or None if the amount cannot be produced from the available inventory. \u0026#34;\u0026#34;\u0026#34; The GreedyStrategy Implementation The default strategy implements the greedy largest-first algorithm described in the requirements:\nclass GreedyStrategy(DispensingStrategy): \u0026#34;\u0026#34;\u0026#34;Largest-denomination-first greedy allocation. For each denomination from largest to smallest, take as many notes as fit into the remaining amount, capped by the available inventory. If the remaining amount reaches zero, a valid plan has been constructed; otherwise the amount cannot be produced from current stock. This algorithm is optimal (minimum note count) when denominations form a canonical system, which is true for standard Indian and most Western currencies. It is not optimal for pathological denomination sets such as {1, 3, 4} for a target of 6, where greedy gives 4+1+1 (three notes) and the optimum is 3+3 (two notes). This limitation is not a concern for real currency. \u0026#34;\u0026#34;\u0026#34; def plan(self, amount, inventory): plan = {} remaining = amount for denom in sorted(inventory.keys(), reverse=True): available = inventory[denom] count = min(remaining // denom, available) if count \u0026gt; 0: plan[denom] = count remaining -= count * denom if remaining == 0: break return plan if remaining == 0 else None The Chain of Responsibility Implementation For scenarios in which per-denomination policy diverges, the CoR variant provides a structural alternative. A DenominationHandler class represents one denomination and its associated logic; a ChainStrategy builds a chain of handlers from an inventory and delegates planning to the chain.\nclass DenominationHandler: \u0026#34;\u0026#34;\u0026#34;One handler in the chain, responsible for one denomination. Handlers form a linked chain from largest denomination to smallest. Each handler determines how much of the request it can contribute from its own stock, records its contribution in the plan, and delegates the remainder to the next handler. A subclass may override contribute() to inject per-denomination policy: reserving stock below a threshold, capping the contribution per transaction, or applying priority rules that depend on context. This extension point is the reason the pattern exists. \u0026#34;\u0026#34;\u0026#34; def __init__(self, denomination, count): self.denomination = denomination self.count = count self.next = None def set_next(self, handler): # Returns the passed handler so that setup can be chained fluently: # a.set_next(b).set_next(c).set_next(d) self.next = handler return handler def contribute(self, amount, plan): \u0026#34;\u0026#34;\u0026#34;Contribute what this denomination can. Return the remainder.\u0026#34;\u0026#34;\u0026#34; needed = amount // self.denomination used = min(needed, self.count) if used \u0026gt; 0: plan[self.denomination] = used amount -= used * self.denomination if amount \u0026gt; 0 and self.next is not None: return self.next.contribute(amount, plan) return amount class ChainStrategy(DispensingStrategy): \u0026#34;\u0026#34;\u0026#34;DispensingStrategy that plans by walking a chain of DenominationHandlers. Produces the same result as GreedyStrategy for the default uniform-policy case. The chain becomes valuable only when per-denomination policy diverges, at which point a subclass of DenominationHandler for the relevant denomination is the natural home for the divergent logic. \u0026#34;\u0026#34;\u0026#34; def plan(self, amount, inventory): head = None prev = None for denom in sorted(inventory.keys(), reverse=True): handler = DenominationHandler(denom, inventory[denom]) if head is None: head = handler if prev is not None: prev.set_next(handler) prev = handler if head is None: return {} if amount == 0 else None plan = {} remaining = head.contribute(amount, plan) return plan if remaining == 0 else None The CashDispenser Facade The dispenser is a facade over the inventory and the strategy. It exposes a small, clear interface (can_dispense, dispense, restock) and internally coordinates the strategy\u0026rsquo;s planning with the atomic commit to inventory.\nFacade pattern definition: The Facade pattern is a structural design pattern that provides a simplified, unified interface to a set of interfaces in a subsystem. The facade defines a higher-level interface that makes the subsystem easier to use, without preventing clients from accessing the subsystem directly when needed.\nclass CashDispenser: \u0026#34;\u0026#34;\u0026#34;Manages cash inventory and coordinates dispensing. The dispenser is the single source of truth for what notes exist in the physical machine. It uses a DispensingStrategy to decide how to dispense a given amount, but the strategy does not modify inventory; the dispenser does that, atomically, after the strategy returns a valid plan. \u0026#34;\u0026#34;\u0026#34; def __init__(self, inventory, strategy=None): # Copy the inventory so external mutations of the passed-in # dictionary do not corrupt the dispenser\u0026#39;s state. self.inventory = dict(inventory) self.strategy = strategy or GreedyStrategy() def can_dispense(self, amount): \u0026#34;\u0026#34;\u0026#34;Non-mutating feasibility check. Returns True if the requested amount can be produced from current inventory using the current strategy. Safe to call before committing to a debit. \u0026#34;\u0026#34;\u0026#34; return self.strategy.plan(amount, self.inventory) is not None def dispense(self, amount): \u0026#34;\u0026#34;\u0026#34;Produce a physical dispense of the given amount. Two-phase: first compute the plan, and only if the plan is feasible do we commit the inventory changes. If planning fails, we raise before touching inventory, ensuring that no partial dispense or inconsistent state can occur. \u0026#34;\u0026#34;\u0026#34; plan = self.strategy.plan(amount, self.inventory) if plan is None: raise DispenseError( f\u0026#34;Cannot dispense ₹{amount} with available inventory\u0026#34; ) for denom, count in plan.items(): self.inventory[denom] -= count return plan def restock(self, denom, count): \u0026#34;\u0026#34;\u0026#34;Refill inventory of a given denomination.\u0026#34;\u0026#34;\u0026#34; self.inventory[denom] = self.inventory.get(denom, 0) + count def total_cash(self): return sum(denom * count for denom, count in self.inventory.items()) A Worked Example To make the algorithm concrete, consider a request for ₹4,600 against an inventory of 2,000 × 3, 500 × 10, 200 × 5, and 100 × 8.\nThe greedy algorithm proceeds as follows. The ₹2,000 denomination contributes two notes for ₹4,000; the remaining amount is ₹600. The ₹500 denomination contributes one note for ₹500; the remaining amount is ₹100. The ₹200 denomination contributes zero notes because ₹200 does not fit into ₹100; the remaining amount is unchanged at ₹100. The ₹100 denomination contributes one note for ₹100; the remaining amount is ₹0.\nThe resulting plan is {2000: 2, 500: 1, 100: 1}, which totals ₹4,600 as required. Both the GreedyStrategy and the ChainStrategy produce this plan; they differ only in code organisation, not in output.\nAn Example of Failure Consider the same request for ₹4,600 against a degenerate inventory containing only ten ₹200 notes (with the ₹2,000, ₹500, and ₹100 slots empty).\nThe greedy algorithm proceeds as follows. The ₹2,000 handler has zero notes; contribution is zero. The ₹500 handler has zero notes; contribution is zero. The ₹200 handler has ten notes and could contribute up to twenty-three notes (₹4,600 divided by ₹200), so it is limited to ten notes for a contribution of ₹2,000; the remaining amount is ₹2,600. The ₹100 handler has zero notes; contribution is zero. The remaining amount is ₹2,600, which is non-zero.\nplan() returns None, and can_dispense returns False. The bank is not called, no debit occurs, and no inventory is modified. The transaction is rejected cleanly with a message advising the user to try a different amount.\nInterview Framing of the Strategy vs Chain Trade-off The recommended sequence for presenting this component in an interview is:\nIntroduce the DispensingStrategy abstraction and implement GreedyStrategy as the default. This handles the stated requirement completely and correctly. If the interviewer asks about alternative algorithms, introduce BalancedStrategy, OptimalStrategy, or ExactChangeStrategy as sketches. Each is a self-contained algorithm and demonstrates why Strategy is the right level of abstraction for the algorithmic question. If the interviewer asks about per-denomination policies such as reserving high-denomination notes or capping contributions of small-denomination notes, introduce the ChainStrategy and DenominationHandler classes, explaining that subclasses of DenominationHandler are the natural home for divergent per-denomination logic. The reverse order (leading with Chain of Responsibility) is a common mistake and reflects a misunderstanding of when the pattern earns its place. A pattern must be earned by the problem it solves; if all handlers do the same thing modulo a parameter, the pattern has not been earned.\n8. Stage 7: Card, Account, and the Bank Interface The ATM does not own accounts, balances, or authentication credentials. These belong to the bank. This section models the client-side view of each and defines the interface through which the ATM communicates with the bank.\nThe Card Class A card carries the identifiers needed to look up the associated account with the bank, along with the metadata needed to validate the card physically. It does not carry a balance because the balance belongs to the account and is authoritative only at the bank.\nfrom datetime import datetime class Card: \u0026#34;\u0026#34;\u0026#34;A debit card as presented to the ATM. The card carries the account identifier that the bank uses to look up the associated account, along with expiry metadata that the ATM can validate locally before making any bank call. The card does not carry a balance, since the balance is owned by the account and is authoritative only at the bank. \u0026#34;\u0026#34;\u0026#34; def __init__(self, number, holder_name, expiry, account_number): self.number = number self.holder_name = holder_name self.expiry = expiry self.account_number = account_number def is_valid(self): \u0026#34;\u0026#34;\u0026#34;Return True if the card has not passed its expiry date.\u0026#34;\u0026#34;\u0026#34; return datetime.now() \u0026lt; self.expiry The is_valid method is invoked by the IdleState.insert_card handler as a preliminary check before any bank interaction. Rejecting an expired card locally saves an unnecessary round trip to the bank and returns the card to the user immediately.\nThe Account Class The account class holds only the identifiers required to address the account in bank calls. It is deliberately anemic: it does not hold a balance, does not hold behavior, and does not hold cached state that could become stale.\nclass Account: \u0026#34;\u0026#34;\u0026#34;A bank account as seen from the ATM\u0026#39;s perspective. Deliberately anemic. The bank owns the authoritative state (balance, limits, holds, ledger entries). This class carries only the identifiers the ATM needs to address the account in bank service calls. If the design were of the bank backend itself, this class would grow considerably. From the ATM\u0026#39;s viewpoint, growth would be a mistake: every additional field would be a candidate for staleness. \u0026#34;\u0026#34;\u0026#34; def __init__(self, account_number, holder_name): self.account_number = account_number self.holder_name = holder_name The term anemic domain model refers to a design in which entity classes carry data with little or no behavior, with the behavior implemented in separate service classes. It is often cited as an antipattern in domain-driven design because it can lead to a proliferation of service classes and a loss of the encapsulation benefits of object orientation. In this specific case, however, the anemic model is appropriate because the account\u0026rsquo;s behavior genuinely belongs to the bank, which is external to our design. Attempting to give the account behavior on the ATM side would require duplicating logic that lives authoritatively at the bank, creating a source of potential inconsistency.\nThe BankService Abstract Interface The BankService is defined as an abstract base class that declares every operation the ATM performs against the bank. Concrete implementations may include a REST client for production, a database-backed implementation for direct integration, and an in-memory implementation for testing and demonstration.\nfrom abc import ABC, abstractmethod from typing import Optional class BankService(ABC): \u0026#34;\u0026#34;\u0026#34;Abstract interface between the ATM and the bank. The ATM depends on this abstraction rather than on any concrete implementation. This is the Dependency Inversion Principle: the high-level ATM code depends on an abstraction (BankService), not on a low-level concrete class (InMemoryBankService or RestBankClient). Two categories of failure are distinguished: - Business-level failures (insufficient funds, invalid PIN, closed account) are signalled by return values. False indicates a verifiable business rule violation; None indicates an absent lookup. - Infrastructure failures (network timeout, malformed response, bank unavailable) are signalled by raising BankError. These are caught at the transaction boundary inside the state machine. \u0026#34;\u0026#34;\u0026#34; @abstractmethod def authenticate(self, card: Card, pin: str) -\u0026gt; bool: \u0026#34;\u0026#34;\u0026#34;Verify the PIN for a given card. Return True if valid.\u0026#34;\u0026#34;\u0026#34; @abstractmethod def get_balance(self, account: Account) -\u0026gt; float: \u0026#34;\u0026#34;\u0026#34;Return the current balance for an account.\u0026#34;\u0026#34;\u0026#34; @abstractmethod def debit(self, account: Account, amount: float) -\u0026gt; bool: \u0026#34;\u0026#34;\u0026#34;Debit amount from account. Return False on insufficient funds.\u0026#34;\u0026#34;\u0026#34; @abstractmethod def credit(self, account: Account, amount: float) -\u0026gt; bool: \u0026#34;\u0026#34;\u0026#34;Credit amount to account. Return False on failure.\u0026#34;\u0026#34;\u0026#34; @abstractmethod def transfer(self, source: Account, target: Account, amount: float) -\u0026gt; bool: \u0026#34;\u0026#34;\u0026#34;Transfer amount from source to target. Return False on failure. Implementations must ensure atomicity: either both accounts change or neither does. \u0026#34;\u0026#34;\u0026#34; @abstractmethod def get_account(self, account_number: str) -\u0026gt; Optional[Account]: \u0026#34;\u0026#34;\u0026#34;Look up an account by account number. Return None if not found.\u0026#34;\u0026#34;\u0026#34; The Concrete In-Memory Implementation The concrete implementation used for demonstration stores all state in dictionaries. It is deliberately minimal: no daily limits, no overdraft protection, no ledger entries, no audit trail, no concurrency handling. Its purpose is to make the ATM runnable end-to-end, not to model a production bank.\nclass InMemoryBankService(BankService): \u0026#34;\u0026#34;\u0026#34;A dict-backed BankService for demonstrations and tests. In production, this class is replaced by a REST client that talks to the bank\u0026#39;s core-banking system, or by a direct database client. The ATM code does not change; it depends on BankService, which both implementations satisfy. \u0026#34;\u0026#34;\u0026#34; def __init__(self): self.accounts = {} # account_number -\u0026gt; Account self.balances = {} # account_number -\u0026gt; balance self.pins = {} # card_number -\u0026gt; pin (plaintext for demo only) def register_account(self, account, balance, card, pin): \u0026#34;\u0026#34;\u0026#34;Setup utility (not part of the BankService contract). Registers an account with a starting balance and associates a card with a PIN. In production, the bank would already contain these records; this method exists purely to seed data for demos. \u0026#34;\u0026#34;\u0026#34; self.accounts[account.account_number] = account self.balances[account.account_number] = balance self.pins[card.number] = pin def authenticate(self, card, pin): return self.pins.get(card.number) == pin def get_balance(self, account): return self.balances.get(account.account_number, 0.0) def debit(self, account, amount): current = self.balances.get(account.account_number, 0.0) if current \u0026lt; amount: return False self.balances[account.account_number] = current - amount return True def credit(self, account, amount): self.balances[account.account_number] = ( self.balances.get(account.account_number, 0.0) + amount ) return True def transfer(self, source, target, amount): # Two-phase with compensation. In a production database, this # would be a single transactional operation covered by a database # transaction. Here the compensation is manual. if not self.debit(source, amount): return False if not self.credit(target, amount): self.credit(source, amount) # compensating action return False return True def get_account(self, account_number): return self.accounts.get(account_number) The plaintext PIN storage in this class is a simplification for demonstration only. In a production system, PINs are never stored in plaintext. They are stored as salted cryptographic hashes, and the comparison happens on the bank side using constant-time comparison to avoid timing-attack leakage. The mechanism for transmitting the PIN from the card to the bank uses encryption keys that never leave the hardware security module.\n9. Stage 8: The ATM Class and Hardware Composition The ATM class is the top-level coordinator. Its role is to hold references to the hardware components, the bank service, the cash dispenser, and the current state, and to delegate every user-initiated action to the current state. It contains almost no branching logic of its own; all branching is handled by polymorphism through the state objects.\nThe Hardware Component Classes Each hardware component is kept small and focused on a single responsibility. The classes are grouped in one file (hardware/components.py) because each is small, but each remains a distinct class with its own methods.\nclass Screen: \u0026#34;\u0026#34;\u0026#34;Renders text to the user. In production this drives an LCD or touchscreen through a hardware driver. In this simulation, output is directed to standard output. \u0026#34;\u0026#34;\u0026#34; def show(self, message): print(f\u0026#34;[SCREEN] {message}\u0026#34;) class Keypad: \u0026#34;\u0026#34;\u0026#34;Reads user input. In production this reads from a physical keypad or touchscreen input. In this simulation, input is read from standard input. In the demo driver used later, this component is bypassed because the demo drives the ATM programmatically. \u0026#34;\u0026#34;\u0026#34; def read(self, prompt=\u0026#34;\u0026#34;): return input(prompt) class CardReader: \u0026#34;\u0026#34;\u0026#34;Handles card insertion, ejection, and physical retention (capture). Capture refers to the physical retention of a card after too many failed PIN attempts. In real hardware the card is drawn into an internal bin from which only a technician can retrieve it. In this simulation, capture is represented by printing a message. \u0026#34;\u0026#34;\u0026#34; def read(self, card): # In real hardware, this reads the chip or magnetic stripe and # validates cryptographic material. Here the card is returned # unchanged. return card def eject(self, card): print(f\u0026#34;[CARD READER] Ejecting card {card.number}\u0026#34;) def capture(self, card): print(f\u0026#34;[CARD READER] Card {card.number} retained. Contact your bank.\u0026#34;) class DepositSlot: \u0026#34;\u0026#34;\u0026#34;Accepts, counts, and validates deposited cash. In production, a bill validator inside the slot counts and authenticates notes automatically. In this simulation, the count is obtained from a prompt. \u0026#34;\u0026#34;\u0026#34; def __init__(self): self._pending_amount = 0.0 def collect_and_count(self): raw = input(\u0026#34;[DEPOSIT SLOT] Enter deposit amount (simulated): \u0026#34;) try: amount = float(raw) except ValueError: amount = 0.0 self._pending_amount = amount return amount def return_cash(self): \u0026#34;\u0026#34;\u0026#34;Physically return the cash to the user (used if the bank rejects the deposit after counting).\u0026#34;\u0026#34;\u0026#34; print(f\u0026#34;[DEPOSIT SLOT] Returning ₹{self._pending_amount} to user\u0026#34;) self._pending_amount = 0.0 class Printer: \u0026#34;\u0026#34;\u0026#34;Prints receipts to the user.\u0026#34;\u0026#34;\u0026#34; def print(self, receipt): print(\u0026#34;=\u0026#34; * 40) print(receipt.render()) print(\u0026#34;=\u0026#34; * 40) class Receipt: \u0026#34;\u0026#34;\u0026#34;Immutable record of a completed transaction.\u0026#34;\u0026#34;\u0026#34; def __init__(self, txn_type, amount, status, timestamp): self.txn_type = txn_type self.amount = amount self.status = status self.timestamp = timestamp def render(self): lines = [ f\u0026#34;Transaction: {self.txn_type}\u0026#34;, f\u0026#34;Status: {self.status}\u0026#34;, ] if self.amount is not None: lines.append(f\u0026#34;Amount: ₹{self.amount:.2f}\u0026#34;) lines.append(f\u0026#34;Time: {self.timestamp.isoformat()}\u0026#34;) return \u0026#34;\\n\u0026#34;.join(lines) The ATM Class The ATM class is thin by design. Each user-initiated method is a single line that delegates to the current state. The class contains no if statements based on state, no isinstance checks based on transaction type, and no direct execution of business logic. This thinness is diagnostic: when a top-level coordinator class stays thin, it usually means responsibilities have been located correctly.\nclass ATM: \u0026#34;\u0026#34;\u0026#34;Top-level coordinator for the ATM. Owns the hardware components, the external bank service, the cash dispenser, and the current state. Every user-facing method delegates to the current state, which is responsible for validating the action and performing any transition. The absence of branching logic in this class is intentional and diagnostic. If branching accumulates here, it usually indicates that logic has been placed in the wrong location and should be moved to the appropriate state class. \u0026#34;\u0026#34;\u0026#34; def __init__(self, bank, dispenser, screen=None, keypad=None, card_reader=None, deposit_slot=None, printer=None): # Hardware components. Constructor accepts injections so that # tests can substitute mocks; defaults are provided for # production use. self.screen = screen or Screen() self.keypad = keypad or Keypad() self.card_reader = card_reader or CardReader() self.deposit_slot = deposit_slot or DepositSlot() self.printer = printer or Printer() self.dispenser = dispenser # External dependency (abstract interface). self.bank = bank # Runtime state. self._state = IdleState() self.session = None self.screen.show(\u0026#34;Welcome. Please insert your card.\u0026#34;) def set_state(self, state): \u0026#34;\u0026#34;\u0026#34;Called by state objects during transitions to advance the machine.\u0026#34;\u0026#34;\u0026#34; self._state = state def state_name(self): return self._state.name() # Public user-facing actions. All delegate to the current state. def insert_card(self, card): self._state.insert_card(self, card) def enter_pin(self, pin): self._state.enter_pin(self, pin) def select_transaction(self, txn): self._state.select_transaction(self, txn) def eject_card(self): self._state.eject_card(self) The Composition Root The main.py module (shown in Stage 10) is the composition root of the application. The composition root is the location in an application where the object graph is constructed and dependencies are wired together, typically at application startup. Every concrete implementation is chosen here: which BankService to use, which DispensingStrategy to use, what initial inventory to load, which hardware components to instantiate.\nThe importance of concentrating construction in the composition root is that every class below this point can be written against abstractions and remain ignorant of concrete implementations. Testing, extension, and replacement all become simpler because the seams for substitution are all located in one place.\n10. Stage 9: Session, Authentication, and Timeout The Session object carries transient state that belongs to a single user\u0026rsquo;s visit to the ATM. It is created when a card is inserted and destroyed when the card is ejected or captured. Session state includes the card reference, the authentication flag, the wrong-PIN counter, and timestamps used for inactivity timeout.\nThe Session Class from datetime import datetime class Session: \u0026#34;\u0026#34;\u0026#34;Transient state of one user\u0026#39;s visit to the ATM. Created when a card is inserted; destroyed when the card is ejected or captured. Tracks authentication status, wrong-PIN attempts, and timing information used for the inactivity timeout. The session is the natural home for any state that spans the entire interaction but does not survive it. Balance snapshots, authorised transaction limits, and per-visit audit information all belong here. \u0026#34;\u0026#34;\u0026#34; TIMEOUT_SECONDS = 30 def __init__(self, card): self.card = card self.authenticated = False self.wrong_pin_attempts = 0 self.started_at = datetime.now() self.last_activity = datetime.now() def touch(self): \u0026#34;\u0026#34;\u0026#34;Update the last-activity timestamp. Called on every user action.\u0026#34;\u0026#34;\u0026#34; self.last_activity = datetime.now() def is_timed_out(self): \u0026#34;\u0026#34;\u0026#34;Return True if no activity has occurred for TIMEOUT_SECONDS.\u0026#34;\u0026#34;\u0026#34; elapsed = (datetime.now() - self.last_activity).total_seconds() return elapsed \u0026gt; self.TIMEOUT_SECONDS Where Timeout Enforcement Belongs Two designs for enforcing the timeout are possible, and the choice between them depends on the deployment environment.\nDesign A: Enforcement on user action. The timeout is checked at the entry point of each state\u0026rsquo;s action methods. Before performing any real work, the state consults session.is_timed_out() and, if the session has expired, ejects the card and transitions to the idle state. This design is simple and requires no additional infrastructure, but it enforces the timeout only when the user attempts an action. A session that has expired remains logically alive until the user does something, at which point the expiration is discovered and enforced.\nDesign B: Enforcement by a background timer. A scheduler thread runs continuously and, at each configured interval, checks whether the current session has expired. If so, the scheduler triggers the ejection procedure regardless of user activity. This design is more accurate but requires a threading or async model, and it introduces concerns about safely mutating state from a thread other than the one processing user actions.\nFor the interview design, Design A is sufficient and is the one implemented. Design B would be identified in the follow-up discussion as the production improvement that would be introduced when the design moves into production.\nIntegration of Timeout Enforcement into the State Methods The AuthenticatedState.select_transaction method illustrates the enforcement:\nclass AuthenticatedState(ATMState): def select_transaction(self, atm, txn): if atm.session.is_timed_out(): self._handle_timeout(atm) return atm.session.touch() # Proceed with the transaction flow. from atm_system.states.transaction_selected import TransactionSelectedState new_state = TransactionSelectedState(txn) atm.set_state(new_state) atm.screen.show(f\u0026#34;Processing {txn.__class__.__name__}...\u0026#34;) new_state.execute(atm) def _handle_timeout(self, atm): atm.card_reader.eject(atm.session.card) from atm_system.states.idle import IdleState atm.set_state(IdleState()) atm.session = None atm.screen.show(\u0026#34;Session timed out. Card returned.\u0026#34;) The timeout check appears at the top of every state method that mutates session state. In a production refactor, this check could be extracted into a decorator to eliminate the repetition, but for a design walkthrough the inline version is clearer.\n11. Stage 10: Putting It All Together This section assembles the complete design. The state classes are shown in full (including those deferred from Stage 4), the exception hierarchy is defined, and an end-to-end demonstration runs a scripted user journey. Every class defined in the preceding stages participates in this final assembly.\nThe Exception Hierarchy Three exception types capture the failure modes distinguished in the design:\nclass InvalidOperation(Exception): \u0026#34;\u0026#34;\u0026#34;Raised when an action is attempted in a state that does not permit it. Example: attempting to enter a PIN before a card has been inserted. This exception is the mechanism by which the State pattern enforces its invariants. \u0026#34;\u0026#34;\u0026#34; class DispenseError(Exception): \u0026#34;\u0026#34;\u0026#34;Raised when the cash dispenser cannot fulfil a request. Typically caused by insufficient inventory of the right denominations to produce the requested amount. Distinct from an authorisation failure, which is a bank concern. \u0026#34;\u0026#34;\u0026#34; class BankError(Exception): \u0026#34;\u0026#34;\u0026#34;Raised on bank service infrastructure failures. Distinct from business-level failures such as insufficient funds, which are signalled by return values from BankService methods. BankError covers network timeouts, malformed responses, and other conditions where the outcome of a bank operation cannot be determined. \u0026#34;\u0026#34;\u0026#34; The Complete State Classes The abstract base class and all four concrete states, presented in the order they appear during a normal user journey:\nfrom abc import ABC from datetime import datetime class ATMState(ABC): \u0026#34;\u0026#34;\u0026#34;Abstract base class for all ATM states.\u0026#34;\u0026#34;\u0026#34; def insert_card(self, atm, card): raise InvalidOperation(f\u0026#34;Cannot insert card in state {self.name()}\u0026#34;) def enter_pin(self, atm, pin): raise InvalidOperation(f\u0026#34;Cannot enter PIN in state {self.name()}\u0026#34;) def select_transaction(self, atm, txn): raise InvalidOperation(f\u0026#34;Cannot select transaction in state {self.name()}\u0026#34;) def eject_card(self, atm): raise InvalidOperation(f\u0026#34;Cannot eject card in state {self.name()}\u0026#34;) def name(self): return self.__class__.__name__ class IdleState(ATMState): \u0026#34;\u0026#34;\u0026#34;The machine is at rest, waiting for a card.\u0026#34;\u0026#34;\u0026#34; def insert_card(self, atm, card): if not card.is_valid(): atm.card_reader.eject(card) atm.screen.show(\u0026#34;Card expired. Please use a valid card.\u0026#34;) return atm.session = Session(card=card) atm.set_state(HasCardState()) atm.screen.show(\u0026#34;Please enter your PIN\u0026#34;) class HasCardState(ATMState): \u0026#34;\u0026#34;\u0026#34;A card has been inserted; the PIN has not yet been verified.\u0026#34;\u0026#34;\u0026#34; MAX_PIN_ATTEMPTS = 3 def enter_pin(self, atm, pin): atm.session.touch() if atm.bank.authenticate(atm.session.card, pin): atm.session.authenticated = True atm.set_state(AuthenticatedState()) atm.screen.show(\u0026#34;Authenticated. Select a transaction.\u0026#34;) return atm.session.wrong_pin_attempts += 1 if atm.session.wrong_pin_attempts \u0026gt;= self.MAX_PIN_ATTEMPTS: atm.card_reader.capture(atm.session.card) atm.set_state(IdleState()) atm.session = None atm.screen.show(\u0026#34;Card retained. Contact your bank.\u0026#34;) else: remaining = self.MAX_PIN_ATTEMPTS - atm.session.wrong_pin_attempts atm.screen.show(f\u0026#34;Incorrect PIN. {remaining} attempts remaining.\u0026#34;) def eject_card(self, atm): atm.card_reader.eject(atm.session.card) atm.set_state(IdleState()) atm.session = None atm.screen.show(\u0026#34;Card returned. Welcome.\u0026#34;) class AuthenticatedState(ATMState): \u0026#34;\u0026#34;\u0026#34;The user is authenticated and may select transactions or eject the card.\u0026#34;\u0026#34;\u0026#34; def select_transaction(self, atm, txn): if atm.session.is_timed_out(): self._handle_timeout(atm) return atm.session.touch() new_state = TransactionSelectedState(txn) atm.set_state(new_state) atm.screen.show(f\u0026#34;Processing {txn.__class__.__name__}...\u0026#34;) new_state.execute(atm) def eject_card(self, atm): atm.card_reader.eject(atm.session.card) atm.set_state(IdleState()) atm.session = None atm.screen.show(\u0026#34;Thank you. Please take your card.\u0026#34;) def _handle_timeout(self, atm): atm.card_reader.eject(atm.session.card) atm.set_state(IdleState()) atm.session = None atm.screen.show(\u0026#34;Session timed out. Card returned.\u0026#34;) class TransactionSelectedState(ATMState): \u0026#34;\u0026#34;\u0026#34;A transaction has been selected and is being executed. Held only briefly during execution. After completion, the machine returns to AuthenticatedState so the user may perform another transaction or eject the card. \u0026#34;\u0026#34;\u0026#34; def __init__(self, transaction): self.transaction = transaction def execute(self, atm): # Wrap execution in try/except so that a bank exception does not # leave the ATM stuck in this state indefinitely. try: result = self.transaction.execute(atm.bank) except Exception as exc: atm.screen.show(f\u0026#34;System error: {exc}. Please try again.\u0026#34;) self._return_to_authenticated(atm) return self._display_result(atm, result) if self.transaction.status == \u0026#34;SUCCESS\u0026#34;: self._print_receipt(atm, result) self._return_to_authenticated(atm) def _display_result(self, atm, result): prefix = \u0026#34;Success\u0026#34; if result.success else \u0026#34;Failed\u0026#34; atm.screen.show(f\u0026#34;{prefix}: {result.message}\u0026#34;) def _print_receipt(self, atm, result): amount = result.value if isinstance(result.value, (int, float)) else None receipt = Receipt( txn_type=self.transaction.__class__.__name__, amount=amount, status=self.transaction.status, timestamp=datetime.now(), ) atm.printer.print(receipt) def _return_to_authenticated(self, atm): atm.set_state(AuthenticatedState()) atm.screen.show(\u0026#34;Another transaction? Or eject card.\u0026#34;) End-to-End Demonstration The following demonstration wires together every class defined in the preceding stages and runs a scripted user journey through the ATM. The demonstration exercises card insertion, PIN entry, balance inquiry, withdrawal, transfer, and card ejection.\ndef demo(): # Set up the bank with two customers. bank = InMemoryBankService() alice_account = Account(\u0026#34;ACC-1001\u0026#34;, \u0026#34;Alice\u0026#34;) alice_card = Card( number=\u0026#34;4111-1111-1111-1111\u0026#34;, holder_name=\u0026#34;Alice\u0026#34;, expiry=datetime(2030, 12, 31), account_number=\u0026#34;ACC-1001\u0026#34;, ) bank.register_account(alice_account, balance=10_000, card=alice_card, pin=\u0026#34;4321\u0026#34;) bob_account = Account(\u0026#34;ACC-1002\u0026#34;, \u0026#34;Bob\u0026#34;) bob_card = Card( number=\u0026#34;4222-2222-2222-2222\u0026#34;, holder_name=\u0026#34;Bob\u0026#34;, expiry=datetime(2030, 12, 31), account_number=\u0026#34;ACC-1002\u0026#34;, ) bank.register_account(bob_account, balance=500, card=bob_card, pin=\u0026#34;0000\u0026#34;) # Set up the ATM with a stocked dispenser. dispenser = CashDispenser({ 2000: 5, # five ₹2,000 notes 500: 10, # ten ₹500 notes 200: 5, # five ₹200 notes 100: 8, # eight ₹100 notes }) atm = ATM(bank=bank, dispenser=dispenser) # Simulated user journey. print(\u0026#34;\\n--- Alice inserts her card ---\u0026#34;) atm.insert_card(alice_card) print(\u0026#34;\\n--- Alice enters her PIN ---\u0026#34;) atm.enter_pin(\u0026#34;4321\u0026#34;) print(\u0026#34;\\n--- Alice checks her balance ---\u0026#34;) atm.select_transaction(BalanceInquiry(alice_account)) print(\u0026#34;\\n--- Alice withdraws ₹4,600 ---\u0026#34;) atm.select_transaction(Withdrawal(alice_account, 4600, dispenser)) print(\u0026#34;\\n--- Alice transfers ₹1,000 to Bob ---\u0026#34;) atm.select_transaction(Transfer(alice_account, bob_account, 1000)) print(\u0026#34;\\n--- Alice ejects the card ---\u0026#34;) atm.eject_card() if __name__ == \u0026#34;__main__\u0026#34;: demo() Running this program produces a sequence of screen messages that trace the state transitions and transaction results. The final balances after the demonstration are as follows: Alice has ₹4,400 (₹10,000 minus ₹4,600 withdrawn minus ₹1,000 transferred), and Bob has ₹1,500 (₹500 initial plus ₹1,000 received). The dispenser inventory has been reduced by two ₹2,000 notes, one ₹500 note, and one ₹100 note (totaling ₹4,600).\n12. Stage 11: Validating with Mental Walkthroughs Design walkthroughs are the stage at which quiet bugs surface. The purpose of a walkthrough is not to demonstrate the design to the interviewer; it is to expose defects that are invisible when the code is read as static text. Each walkthrough traces a specific scenario through the design, noting the state transitions, the calls made to external components, and the resulting state of the system. Five walkthroughs are presented, covering the primary happy path, two failure modes visible in the requirements, one edge case that reveals subtle correctness, and one defect that only walkthrough analysis would reveal.\nFlow 1: Happy-Path Withdrawal Scenario. Alice inserts her card, enters her PIN correctly, and withdraws ₹4,600. Her account has ₹10,000. The dispenser holds sufficient inventory of every denomination.\nTrace. Card insertion triggers IdleState.insert_card. The card\u0026rsquo;s expiry is validated locally and passes. A Session object is created; the state advances to HasCardState. PIN entry triggers HasCardState.enter_pin. The bank\u0026rsquo;s authenticate call returns True. The session\u0026rsquo;s authenticated flag is set; the state advances to AuthenticatedState. Transaction selection triggers AuthenticatedState.select_transaction. A TransactionSelectedState is constructed and execute is called on it. Withdrawal.execute runs the three-step sequence: dispenser.can_dispense(4600) returns True, bank.debit succeeds (balance becomes ₹5,400), and dispenser.dispense(4600) succeeds with the plan {2000: 2, 500: 1, 100: 1}. The receipt is printed. The state returns to AuthenticatedState. Ejection triggers AuthenticatedState.eject_card. The card is ejected, the session is cleared, and the state advances to IdleState.\nOutcome. All transitions are valid. Alice\u0026rsquo;s balance is consistent. The dispenser inventory is decremented once, and the physical dispense occurred once. No exceptions were raised.\nFlow 2: Insufficient Funds Scenario. Alice authenticates successfully, then requests a withdrawal of ₹15,000 against her balance of ₹10,000.\nTrace. Withdrawal.execute runs step one: dispenser.can_dispense(15000) returns True because the dispenser holds sufficient cash. Step two: bank.debit(alice_account, 15000) returns False because the account does not have sufficient funds. The transaction status becomes FAILED and a failure result is returned. The dispenser is not called for step three, so no inventory is modified. The state returns to AuthenticatedState, from which Alice may retry with a smaller amount.\nOutcome. The order of operations prevented an inventory modification for a transaction that could not complete. If the ordering had placed dispensing before the debit check, cash would have been physically dispensed against a debit that then failed, producing a serious integrity defect.\nFlow 3: Dispenser Cannot Produce the Amount Scenario. Alice has ₹50,000 in her account. The ATM\u0026rsquo;s inventory has been depleted such that only ten ₹200 notes remain (the ₹2,000, ₹500, and ₹100 slots are empty). Alice requests ₹4,600.\nTrace. Withdrawal.execute runs step one: dispenser.can_dispense(4600) invokes GreedyStrategy.plan, which iterates over denominations. The ₹2,000 handler contributes zero (empty inventory). The ₹500 handler contributes zero (empty inventory). The ₹200 handler could theoretically contribute twenty-three notes but is limited to ten by available inventory, contributing ₹2,000; the remaining amount is ₹2,600. The ₹100 handler contributes zero (empty inventory). The remaining amount is non-zero. plan returns None; can_dispense returns False. The transaction status becomes FAILED. The bank is not called; no debit occurs; no inventory is modified.\nOutcome. The infeasibility of the request was detected before any state was mutated. The user is informed and may try a different amount (for example, ₹2,000, which the dispenser can produce).\nFlow 4: Three Wrong PIN Attempts Scenario. Alice inserts her card but enters an incorrect PIN three times in a row.\nTrace. First entry: HasCardState.enter_pin calls bank.authenticate, which returns False. The session\u0026rsquo;s wrong_pin_attempts counter is incremented to 1, and a screen message reports two remaining attempts. Second entry: same behavior; counter is 2, one remaining. Third entry: bank returns False, counter becomes 3, and the branch that handles the maximum-attempts condition activates. card_reader.capture is invoked, which physically retains the card. The state is reset to IdleState. The session is cleared. The screen displays a message directing Alice to contact her bank.\nOutcome. The card capture is triggered exactly at the threshold. The session is fully cleared, preventing any subsequent action from operating against stale session state. The machine is ready for the next user.\nFlow 5: A Defect Revealed Only by Walkthrough Scenario. Alice authenticates and initiates a withdrawal. During the transaction\u0026rsquo;s execution, the bank\u0026rsquo;s debit method raises BankError (network timeout) rather than returning False.\nTrace. Consider what happens if the TransactionSelectedState.execute method is written without exception handling. Withdrawal.execute raises BankError, which propagates out of execute. The state was set to TransactionSelectedState before execute was called and is never reset. The screen was updated to \u0026ldquo;Processing Withdrawal\u0026hellip;\u0026rdquo; and is never updated with a result. The user is stuck in TransactionSelectedState, which has no defined action methods, so every subsequent user action raises InvalidOperation. The card is not ejected, and no message informs the user of the problem.\nDefect and fix. This is a real defect not visible from reading the code as static text. The fix is to wrap the transaction\u0026rsquo;s execution in a try/except at the state level, catching broad exceptions, displaying a user-friendly message, and returning the state to AuthenticatedState:\ndef execute(self, atm): try: result = self.transaction.execute(atm.bank) except Exception as exc: atm.screen.show(f\u0026#34;System error: {exc}. Please try again.\u0026#34;) self._return_to_authenticated(atm) return # ... continue with normal result handling Outcome. After the fix, a bank exception produces a user-visible error message and a return to AuthenticatedState, from which the user may retry or eject the card. The system does not become stuck. This defect illustrates why walkthrough analysis is not optional: no amount of pattern-application substitutes for tracing specific failure paths through the actual code.\n13. Stage 12: Anticipating Follow-Up Questions Interviewers almost always follow the primary design with a set of extension questions that probe how well the design accommodates change. A strong candidate has concise, confident answers to each. The following eight questions cover the extensions most commonly raised for the ATM problem, with the reasoning for each answer.\nQ1: How would the design support interbank transactions? A RoutingService is introduced between the ATM and the concrete bank. The ATM does not know which bank issued a given card; it delegates the routing decision to the router, which inspects the card\u0026rsquo;s identifier prefix (in real systems, the Bank Identification Number or BIN), determines the issuing bank, and forwards the transaction to that bank\u0026rsquo;s service. Failure modes multiply because the routing layer itself may be unavailable, timeouts may occur at any hop, and the issuing bank may reject transactions for reasons the routing layer does not know. The ATM code changes only at the single point where it currently calls self.bank; that call becomes self.router.route(...). The routing layer is itself a service that satisfies the same BankService interface, so the interaction with the ATM remains unchanged.\nQ2: How does the design handle concurrent access to the same account from multiple ATMs? Concurrency at the account level is not the ATM\u0026rsquo;s responsibility. The bank owns the account state and must serialise concurrent access using database-level mechanisms (row-level locks, optimistic concurrency with version numbers, or serialisable transactions). The ATM\u0026rsquo;s responsibility is to react cleanly to whichever failure mode the bank reports. The compensating-action discipline demonstrated in Withdrawal.execute already handles this: if a concurrent transaction consumes the balance between the ATM\u0026rsquo;s feasibility check and its debit call, the debit call returns False, and the transaction is failed cleanly. If the bank supports optimistic concurrency and returns a conflict indication, the ATM treats this exactly like insufficient funds: fail the transaction, restore consistency, and inform the user.\nQ3: How would the design accommodate physical dispensing time and jam handling? In the current design, dispenser.dispense is synchronous and atomic. In real hardware, dispensing a large amount takes several seconds during which the machine must accept no user input, and the physical mechanism can jam mid-way. To accommodate this, a DispensingState is inserted between TransactionSelectedState and the return to AuthenticatedState. The machine remains in DispensingState while cash physically emerges. All user actions in this state raise InvalidOperation because no user input is meaningful. A callback from the physical dispenser signals either completion or jam. On completion, the state advances to AuthenticatedState. On jam, the transaction is captured as disputed: the debit stands (money has left the account), but the physical dispense is incomplete. The user is instructed to contact the bank for reconciliation, and the incident is logged for the bank\u0026rsquo;s dispute-resolution process. This scenario is one of the most common sources of real-world ATM complaints, and naming it explicitly in the interview demonstrates awareness of production concerns.\nQ4: How does the design handle receipt-printing failure? Receipt printing is not part of the transaction\u0026rsquo;s atomicity. The transaction has completed at the bank and the dispense (if any) has completed at the hardware before the receipt is printed. If the printer fails (out of paper, mechanical fault), the transaction is still complete. The correct response is to display a screen message (\u0026ldquo;Receipt could not be printed; your transaction was successful\u0026rdquo;) and to log the printer failure for maintenance. Under no circumstance should a printer failure trigger a reversal of the transaction.\nQ5: How would the design accommodate biometric authentication? The cleanest extension introduces an AuthenticationStrategy interface with concrete implementations for PIN, biometric, and one-time password. The HasCardState holds a reference to the current authentication strategy and delegates verification to it:\nclass AuthenticationStrategy(ABC): @abstractmethod def verify(self, card, credential) -\u0026gt; bool: ... class PINStrategy(AuthenticationStrategy): def __init__(self, bank): self.bank = bank def verify(self, card, credential): return self.bank.authenticate(card, credential) class BiometricStrategy(AuthenticationStrategy): def __init__(self, biometric_service): self.biometric_service = biometric_service def verify(self, card, credential): return self.biometric_service.match(card, credential) This is the Strategy pattern again, applied at the authentication layer. Multiple strategies can be composed by requiring more than one to succeed (multi-factor authentication) or by allowing any one to succeed (either PIN or biometric). The state machine is unaffected: the state that awaits authentication delegates to a strategy, and strategies are swapped at composition-root time.\nQ6: How would the design support cardless withdrawal? Cardless withdrawal is initiated through a channel other than card insertion, typically a mobile banking application that generates a reference code. A new entry point is added to the ATM: initiate_cardless_withdrawal(reference_code). This bypasses the IdleState.insert_card transition and enters a new state, HasReferenceState, which verifies the reference code with the bank. Upon verification, the state advances to AuthenticatedState, from which the rest of the flow (transaction selection, dispensing, ejection) reuses the existing states unchanged. The State pattern accommodates this cleanly: new entry paths become new starting states, without requiring modification of the states downstream.\nQ7: How would the design be tested? Each layer is tested independently. State tests construct a state, invoke an action with a mock ATM, and assert the resulting transition and the mock calls. Transaction tests construct a transaction with a mock bank service and a mock dispenser, invoke execute, and assert the sequence of calls and the resulting TransactionResult. Dispenser tests stock the dispenser with a specific inventory, request an amount, and assert the resulting plan and the resulting inventory. Bank tests exercise the InMemoryBankService implementation directly, and the BankService interface is mocked using unittest.mock.Mock for tests that focus on the ATM\u0026rsquo;s handling of specific bank behaviors. End-to-end tests wire the whole system together and run scripted user journeys such as the demonstration in Stage 10. The isolation produced by the design makes each of these categories small, independent, and fast.\nQ8: How does the design handle the ATM being offline? Two policies are possible. The deny-all policy refuses all transactions and displays \u0026ldquo;ATM is offline\u0026rdquo; to any user who attempts to insert a card. This is the safest policy and is the one used by most real ATMs. The cached-read policy allows read-only transactions (balance inquiry) using cached data from the last successful bank contact and refuses all mutating transactions. This policy is difficult to implement correctly because the cached balance may be stale, and it is rarely worth the operational risk. The current design accommodates either policy without modification: the BankService implementation reports its own availability by raising BankError on operations, and the transaction-level try/except (Flow 5 in Stage 11) handles the resulting exception cleanly.\nQ9: How would an audit trail be added? A TransactionLog is introduced that receives every completed or failed transaction. Log entries are immutable and append-only, satisfying regulatory audit requirements. Two implementation locations are possible. The first places the logging call inside the TransactionSelectedState.execute method, immediately after the transaction returns: atm.transaction_log.record(txn, result). The second places the logging call inside each transaction\u0026rsquo;s execute method. The first location is cleaner because it keeps transactions focused on their own execution and does not require every new transaction subclass to remember to log itself. The state acts as the crosscutting concern\u0026rsquo;s integration point.\nQ10: How would low-inventory alerting be added? The Observer pattern is applied to the dispenser. Observer pattern definition: the Observer pattern is a behavioral design pattern in which an object (the subject) maintains a list of dependents (observers) and notifies them of state changes. Applied here, the dispenser is the subject; observers include a maintenance system that dispatches refills and an operations dashboard that displays inventory levels. When any denomination\u0026rsquo;s count drops below a configured threshold, the dispenser notifies its observers with the affected denomination and the new count. The chain-of-handlers logic is unaffected; only the observer wiring is added. This composition of patterns (Strategy for algorithm selection, Observer for cross-cutting notification, State for the machine\u0026rsquo;s mode) is characteristic of well-designed systems in which multiple patterns collaborate, each doing one job.\n14. Summary of Patterns and Principles Used The ATM design integrates several design patterns and principles. The following table summarises where each is applied and what problem it solves in this context.\nPattern or Principle Where Applied Problem Solved State pattern ATMState hierarchy with IdleState, HasCardState, AuthenticatedState, TransactionSelectedState Localises the invariants of each operational mode; allows new states to be added without modifying existing code Command pattern (light) Transaction hierarchy with Withdrawal, Deposit, BalanceInquiry, Transfer Encapsulates a request as an object; enables polymorphic execution without type-based branching Strategy pattern DispensingStrategy with GreedyStrategy (and alternatives such as BalancedStrategy, OptimalStrategy); also applied to AuthenticationStrategy in the biometric extension Allows algorithms to be selected or replaced at runtime without modifying the client Chain of Responsibility pattern DenominationHandler chain inside ChainStrategy Distributes execution across handlers with distinct per-denomination logic; justified only when policy diverges Facade pattern CashDispenser over DispensingStrategy and inventory Presents a simplified interface (can_dispense, dispense) to a subsystem with multiple internal parts Observer pattern Dispenser inventory alerts (Q10 extension) Notifies interested parties of state changes without coupling the source to the recipients Dependency Inversion Principle BankService interface with concrete implementations (InMemoryBankService, RestBankClient) High-level modules (ATM) depend on abstractions rather than concrete implementations Single Responsibility Principle Separate classes for each hardware component and each state Each class has one reason to change; changes remain localised Open/Closed Principle Adding a new state, transaction, dispensing strategy, or denomination handler requires no modification of existing classes Software entities are open for extension but closed for modification Liskov Substitution Principle Any concrete BankService, DispensingStrategy, or ATMState may be substituted for its abstract type without altering correctness Subtypes preserve the contracts of their base types Composition Root main.py (the demo driver) Concentrates dependency wiring in one location; keeps every other class free of concrete-implementation knowledge Compensating Action Withdrawal.execute reversing the debit on dispense failure; Deposit.execute returning cash on credit failure; InMemoryBankService.transfer reversing the debit on credit failure Restores consistency when a multi-step operation fails partway 15. Key Takeaways The following six lessons distill the transferable insights from the ATM problem. Each generalises beyond the ATM to a broad category of low-level design problems.\nLesson 1: The State pattern is the correct choice for any problem with named operational modes. The ATM has clearly separated modes (idle, has-card, authenticated, transaction-selected), and each mode admits a different set of actions. This is the diagnostic signature of a State-pattern problem. Recognising the signature quickly and reaching for the pattern produces a design that is more maintainable and more extensible than any flag-based alternative. The same signature appears in the elevator, the vending machine, order processing systems, document workflows, network connections, and payment gateways.\nLesson 2: The Strategy pattern is the right level of abstraction for algorithmic variability, while the Chain of Responsibility pattern is the right structure for divergent per-handler logic. The two patterns operate at different levels and answer different questions. Confusing them is a common design error. When the problem admits multiple interchangeable algorithms, use Strategy. When the problem requires distinct logic per handler in an ordered sequence, use Chain of Responsibility. When both apply, the correct structure is Strategy at the outer level with Chain of Responsibility as one strategy implementation.\nLesson 3: Order of operations matters for integrity, and the discipline of compensating actions is foundational. Withdrawal.execute checks dispenser feasibility first, then debits, then dispenses, and reverses the debit on dispense failure. This ordering is not incidental. It is what prevents the user\u0026rsquo;s money from disappearing on a hardware failure. Whenever code contains two or more mutating operations, the following question must be asked: if the second operation fails after the first has succeeded, is the state from the first recoverable? If not, either reorder the operations, wrap them in a transaction, or add explicit compensation. This discipline is the miniature of distributed-transaction management and applies far beyond ATMs.\nLesson 4: External systems must be modeled as interfaces, not as concrete classes. The ATM does not own accounts or balances; it calls into BankService, defined as an abstract interface. This is the Dependency Inversion Principle expressed concretely: the high-level ATM depends on an abstraction rather than a concrete InMemoryBankService or RestBankClient. Any implementation plugs in without modification of the ATM. Every LLD problem worth solving has at least one such boundary. Locating it and modeling it as an interface simplifies the rest of the design substantially.\nLesson 5: Design walkthroughs surface defects that reading code does not. The Flow 5 defect (a bank exception leaving the ATM stuck in TransactionSelectedState) was found by tracing a specific execution path, not by reading the code as static text. Every design should be validated by tracing at least one happy path and one failure path. If the designer cannot describe what the system does when the network fails or when hardware jams, the design is incomplete regardless of how many patterns it applies.\nLesson 6: A thin coordinator is a diagnostic sign of good design. The ATM class contains one-line methods with no branching logic. It knows about states, transactions, hardware, and the bank, but it does not decide anything. The states decide. When a top-level class stays thin, responsibilities have usually been located correctly. When it grows fat, something is misplaced. The god-class antipattern is exactly the failure of this discipline. Whenever a top-level class accumulates logic, the question to ask is: where else could this logic live? The answer is almost always: with the object that owns the data the logic operates on.\n16. Practice Questions with Solutions The following practice questions test the design and its extensions. Solutions are provided immediately after each question to support self-paced study.\nQuestion 1 The Withdrawal.execute method performs three operations in a specific order: feasibility check, debit, dispense. Explain what could go wrong if the order were rearranged to dispense first, then debit, and describe the resulting integrity problem in detail.\nSolution. If dispensing occurs before the debit, the physical cash emerges from the machine before the account has been debited. If the debit then fails (for example, because the account has insufficient funds, or because the bank is unreachable, or because a concurrent transaction has consumed the balance), the user has already received cash for which the account has not been charged. There is no way to recover the physical cash from the user\u0026rsquo;s hand. The bank is out of the amount, and there is no legitimate way to reconcile the difference from a system-integrity standpoint. This produces a financial loss for the bank and a compliance failure because bank records do not accurately reflect what happened. The correct ordering (feasibility check first, then debit, then dispense) ensures that the debit succeeds before any irreversible physical action occurs, and the compensating credit handles the residual case in which dispensing fails despite passing the feasibility check.\nQuestion 2 The HasCardState.enter_pin method contains a hardcoded constant MAX_PIN_ATTEMPTS = 3. Describe two ways to make this configurable, discuss the trade-offs of each approach, and identify which approach is more consistent with the design principles used elsewhere in the ATM.\nSolution. The first approach makes the constant a class attribute that can be overridden by subclassing HasCardState. This is a lightweight change but couples configuration to inheritance, which is generally reserved for behavioral variation. The second approach passes the value into the HasCardState constructor as a parameter, with the ATM providing the value from its configuration. This decouples configuration from inheritance and allows the value to be sourced from a configuration file, environment variable, or centralised policy service without modifying any state class.\nThe second approach is more consistent with the design principles used elsewhere. The ATM already uses constructor injection for its major dependencies (bank service, dispenser, hardware components), and applying the same discipline to policy values keeps the injection pattern uniform. A production refinement would introduce a Policy or SecurityPolicy object holding all such thresholds (maximum PIN attempts, session timeout duration, maximum withdrawal per transaction), which is then injected into the ATM and passed down to state constructors as needed. This centralises policy configuration and simplifies both testing and operational tuning.\nQuestion 3 Design a BalancedStrategy implementation of DispensingStrategy with the following requirement: for requested amounts below ₹5,000, the strategy preserves ₹2,000 notes by treating them as unavailable (contributing zero regardless of inventory). For requested amounts of ₹5,000 or greater, the strategy uses standard greedy behavior. Provide the Python implementation and explain in which file it belongs.\nSolution. The implementation belongs in a new file dispenser/balanced_strategy.py because it is a new strategy that should not modify any existing file. Adding a new file rather than modifying existing ones satisfies the Open/Closed Principle directly.\nfrom typing import Optional from atm_system.dispenser.strategy import DispensingStrategy class BalancedStrategy(DispensingStrategy): \u0026#34;\u0026#34;\u0026#34;Preserves high-denomination notes for large withdrawals. For amounts below ₹5,000, the ₹2,000 denomination is treated as unavailable, forcing the algorithm to use smaller denominations. For amounts of ₹5,000 or greater, standard greedy behavior applies. The threshold is configurable via the constructor. \u0026#34;\u0026#34;\u0026#34; def __init__(self, preserve_threshold=5000, preserved_denomination=2000): self.preserve_threshold = preserve_threshold self.preserved_denomination = preserved_denomination def plan(self, amount, inventory): # Construct a view of inventory that respects the preservation policy. effective_inventory = dict(inventory) if amount \u0026lt; self.preserve_threshold: effective_inventory[self.preserved_denomination] = 0 plan = {} remaining = amount for denom in sorted(effective_inventory.keys(), reverse=True): available = effective_inventory[denom] count = min(remaining // denom, available) if count \u0026gt; 0: plan[denom] = count remaining -= count * denom if remaining == 0: break return plan if remaining == 0 else None Wiring this into the ATM happens at the composition root. In main.py, the dispenser is constructed with the new strategy:\ndispenser = CashDispenser(inventory={...}, strategy=BalancedStrategy()) No other file requires modification. The CashDispenser, Withdrawal, ATM, and all state classes remain unchanged.\nQuestion 4 The current design executes transactions synchronously inside TransactionSelectedState.execute. Suppose the interviewer asks how the design would accommodate transactions that take significant time (for example, a physical dispense of thousands of notes taking tens of seconds). Describe the modifications required, identify which existing classes are affected, and specify what a new DispensingState would look like.\nSolution. The current design assumes that transaction execution is instantaneous. Accommodating physically time-consuming operations requires introducing a state in which the ATM sits while the operation completes and during which no user input is accepted.\nA new state DispensingState is added to the state machine. It is entered from TransactionSelectedState immediately before the physical dispense begins, and it is exited when the dispenser signals completion (or jam). All user-facing actions in DispensingState inherit the base ATMState default and raise InvalidOperation, correctly rejecting input during the physical operation.\nclass DispensingState(ATMState): \u0026#34;\u0026#34;\u0026#34;Physical dispensing is in progress. No user input is accepted.\u0026#34;\u0026#34;\u0026#34; def __init__(self, transaction, on_complete, on_jam): self.transaction = transaction self.on_complete = on_complete self.on_jam = on_jam def notify_complete(self, atm): \u0026#34;\u0026#34;\u0026#34;Called by the physical dispenser when dispensing succeeds.\u0026#34;\u0026#34;\u0026#34; self.on_complete(atm, self.transaction) def notify_jam(self, atm): \u0026#34;\u0026#34;\u0026#34;Called by the physical dispenser when a jam is detected.\u0026#34;\u0026#34;\u0026#34; self.on_jam(atm, self.transaction) The dispenser interface is extended to expose a callback-based dispense method: dispense_async(amount, on_complete, on_jam) rather than the synchronous dispense(amount). Withdrawal.execute initiates dispensing and transitions the state to DispensingState; it does not wait for completion.\nThe callbacks translate the physical outcome into state transitions: on_complete prints the receipt and returns to AuthenticatedState; on_jam marks the transaction as disputed, logs the incident, and returns to AuthenticatedState with a message directing the user to contact the bank.\nThe classes affected are Withdrawal, CashDispenser, DispensingStrategy (which gains an async variant), and the state hierarchy (which gains DispensingState). The ATM class itself is unchanged because the delegation to states remains uniform.\nQuestion 5 Explain why the Account class in this design does not carry a balance field, and describe under what design change the decision would be reconsidered.\nSolution. The Account class does not carry a balance because the balance is owned authoritatively by the bank, not by the ATM. Any balance value held by the ATM would be a cached snapshot at some point in time, and the value would become stale the moment any transaction occurs. Since transactions can occur concurrently from other ATMs, from online banking, or from card payments elsewhere, a cached balance cannot be treated as reliable. Displaying a stale balance to the user is a serious usability defect, and executing a transaction based on a stale balance is a serious correctness defect. The safe policy is to fetch the balance from the bank at the moment it is needed and to treat every subsequent operation as requiring bank confirmation.\nThe decision would be reconsidered under two design changes. First, if the ATM were extended to operate in an offline mode using cached data, a balance snapshot would need to be stored. The design accommodates this by treating the snapshot as belonging to the Session rather than to the Account itself, preserving the invariant that Account is a stable identifier and not a mutable state holder. Second, if the design scope were extended to include the bank backend itself, Account would take on behavior and state because it would then be the authoritative holder rather than a client-side reference. In that scope, methods such as debit, credit, and get_balance would move onto the account class, and the anemic-model criticism would apply. Within the ATM\u0026rsquo;s scope, the anemic model is the correct choice.\n17. Final Reflection The ATM problem consolidates several distinct low-level design techniques into a single exercise. The State pattern organises the machine\u0026rsquo;s operational modes. The Command pattern (in its lightweight form) organises the transaction hierarchy. The Strategy pattern organises the choice of dispensing algorithm. The Chain of Responsibility pattern becomes available for the specific extension in which per-denomination policy diverges. The Facade pattern presents the dispenser as a single object to the rest of the system. The Observer pattern is available for cross-cutting concerns such as inventory alerting. The Dependency Inversion Principle governs the boundary between the ATM and the bank. The compensating-action discipline governs the ordering of operations within each transaction that touches more than one subsystem.\nA candidate who executes each of these decisions correctly demonstrates competence across a broader surface area of the design-pattern landscape than most LLD problems require. The problem is therefore worth studying carefully even though it does not appear on interview lists as frequently as the parking lot or the elevator.\nBeyond the specific patterns, the ATM problem teaches four transferable habits. The first is the discipline of clarifying requirements before designing; the nine-question walkthrough in Stage 2 is the model for how any LLD problem should be opened. The second is the discipline of validating designs by mental walkthrough; the five flows in Stage 11 illustrate how quiet defects surface only when specific execution paths are traced. The third is the discipline of choosing patterns based on the problem\u0026rsquo;s structure rather than on their availability; the Strategy-versus-Chain-of-Responsibility discussion in Stage 6 illustrates how the same problem admits different structural choices and how the choice must be justified. The fourth is the discipline of keeping coordinators thin and locating decision logic with the object that owns the relevant data; the ATM class\u0026rsquo;s one-line methods demonstrate this principle concretely.\nTaken together with the parking lot (which teaches relationships and pluggable policy), the elevator (which teaches state machines and scheduling), and the library (which teaches policy-driven domain modeling), the ATM completes the four canonical shapes of LLD interview problems. Most other LLD questions decompose into some combination of these four shapes, and mastery of them provides the reasoning framework for addressing any specific problem encountered in an interview.\nThis content is part of Codeverra, a platform for learning coding, data science, DSA, and AI from scratch. Explore more at https://codeverra.com\n","permalink":"https://blogs.codeverra.com/blog/low_level_design/interview_questions/06g_lld_atm_deep_dive/","summary":"\u003ch1 id=\"lld-deep-dive-4-designing-an-atm-system\"\u003eLLD Deep Dive #4: Designing an ATM System\u003c/h1\u003e\n\u003cblockquote\u003e\n\u003cp\u003e\u003cstrong\u003eWhat this document is:\u003c/strong\u003e A structured, step-by-step walkthrough of the ATM low-level design problem as it would be reasoned through in a technical interview. The document proceeds from first principles, weighs alternative designs at each decision point, and arrives at a solution that combines the State pattern, the Strategy pattern, the Command pattern, and a clean separation between hardware and software concerns. Definitions of every technical term are provided when the term is introduced, so the document can be read without prior familiarity with the design-pattern literature.\u003c/p\u003e","tags":null,"title":""},{"content":"LLD Deep Dive #4: Designing a Movie Ticket Booking System (BookMyShow-style) Purpose of this document. A staged walkthrough of how to reason through a Movie Ticket Booking Low-Level Design (LLD) problem in an interview setting. Each design decision is presented alongside the alternatives that were considered and the rationale for the final choice. The resulting design is modular, extensible, and treats concurrency as a first-class concern rather than a post-hoc extension.\nWhat makes this problem distinct. Previous deep dives in this series (Parking Lot, Elevator, Library, ATM) treated concurrency as an addendum that could be layered on later. Movie Ticket Booking does not permit that treatment. From the moment two users attempt to reserve the same seat, the design either handles contention correctly or it generates duplicate bookings. The concurrency techniques introduced here (per-resource locking, the hold-then-confirm pattern, idempotency keys, ordered lock acquisition) generalise directly to hotel reservations, flight bookings, e-commerce checkout, and every other shared-resource allocation system.\nRecommended reading approach. Progress deliberately through each stage. At the start of each section, pause and consider the design decision you would make before reading the analysis that follows.\nTable of Contents Why This Problem Matters Stage 1: Receiving the Problem Stage 2: Clarifying Requirements Stage 3: Identifying Entities Stage 4: The Domain Hierarchy Stage 5: Modelling Seats and the Seat Lifecycle Stage 6: The Concurrency Problem in Depth 7.1 The Naive Design and Its Race Condition 7.2 Lock Granularity: Global, Per-Show, Per-Seat 7.3 Pessimistic vs Optimistic Locking 7.4 The Hold-Then-Confirm Pattern 7.5 Deadlock and Ordered Locking 7.6 Idempotency 7.7 Sequence Diagram: Two Users Contending for the Same Seat Stage 7: Booking as a State Machine Stage 8: Pricing as a Strategy Stage 9: Payment as a Strategy Stage 10: Notifications via Observer Stage 11: The Complete Class Diagram Stage 12: Validating the Design with a Mental Walkthrough Stage 13: Anticipating Follow-Up Questions Summary and Key Takeaways Practice Questions with Solutions 1. Why This Problem Matters Each LLD problem covered in this series emphasises a distinct modelling skill. Parking Lot addressed entity relationships. Elevator addressed state machines. Library addressed policy-driven design. ATM addressed transactional flows. Movie Ticket Booking addresses a fundamentally different concern: the correctness of the model under concurrent load.\nThe relevance becomes clear from the traffic pattern of a popular movie release. When a major film opens on BookMyShow, the platform receives thousands of booking requests per second directed at the same show, the same screen, and often the same specific seats. A design that lacks a coherent answer to the question of what happens when two threads select the same seat simultaneously will sell that seat twice, produce two confirmation records, and generate two aggrieved customers at the box office.\nSenior interviewers favour this problem precisely for this reason. The problem exposes three distinct layers of design competence, each of which must be handled explicitly.\nLayer Concern Techniques Modelling Domain hierarchy and entity relationships Composition, aggregation, distinguishing physical from per-instance state Patterns Pricing rules, booking lifecycle, event dispatch Strategy, State-as-enum, Observer Concurrency Correctness under simultaneous access Locks, lock granularity, hold-then-confirm, deadlock avoidance, idempotency Weak candidates address layer 1 only. Competent candidates address layers 1 and 2. Strong candidates address all three and articulate the tradeoffs at each level. This document develops the design across all three layers.\n2. Stage 1: Receiving the Problem The interviewer opens with the following prompt:\n\u0026ldquo;Design a movie ticket booking system, like BookMyShow.\u0026rdquo;\nThe phrasing is deliberately underspecified. The reference to BookMyShow signals that the interviewer expects a production-grade design rather than a simplified toy model. Beyond that hint, the scope remains open, and the candidate must determine which aspects of the system to model through clarifying questions.\nThe unresolved scope areas include whether the system spans multiple cities and multiple theaters per city, how many screens each theater contains, how many shows run per screen per day, whether seat types (regular, premium, recliner) require distinguishing, whether pricing varies by show or time slot, whether promotional codes are supported, whether seat selection is interactive or automatic, and whether the system must handle concurrent bookings, payment failures, and cancellations. Each of these areas materially changes the design. The next stage resolves them systematically.\n3. Stage 2: Clarifying Requirements Requirement clarification proceeds by asking questions whose answers change design decisions. Each question below is presented with the reason it matters and the assumed answer we will proceed with.\nQ1. What is the scope of the system? Reason for asking. BookMyShow is a nationwide platform. The design changes significantly depending on whether the full hierarchy (city, theater, screen, show, seat) is in scope or only a subset.\nAssumed answer. \u0026ldquo;Full hierarchy. Users pick a city, then a movie, then a theater and showtime, then seats.\u0026rdquo;\nThis fixes the containment structure as City to Theater to Screen to Show to Seat, and establishes Movie as a top-level entity that plays across many theaters.\nQ2. Are multiple concurrent users permitted on the same show? Reason for asking. The answer determines whether concurrency is central to the design or a footnote.\nAssumed answer. \u0026ldquo;Yes. Assume high traffic. Multiple users may attempt to book the same seat concurrently. The system must guarantee no double-booking.\u0026rdquo;\nThis grants explicit permission, and creates the obligation, to allocate significant design time to concurrency.\nQ3. Are there seat types and variable pricing? Reason for asking. Variable pricing rules typically justify the Strategy pattern, which is a design pattern where the algorithm for a computation (in this case, price calculation) is encapsulated in an interchangeable object.\nAssumed answer. \u0026ldquo;Yes. Regular, premium, and recliner seat types with distinct base prices. Weekend shows apply a 20% surcharge. Student, senior, and coupon-based discounts are supported.\u0026rdquo;\nPricing is confirmed as non-trivial and variable. Strategy pattern is the appropriate response.\nQ4. What happens if a user begins the booking flow but does not complete payment? Reason for asking. This question determines whether the system uses a hold mechanism, in which selected seats are temporarily reserved during the payment window, or leaves seats available to all users until payment completes.\nWithout holds, users who reach the payment screen frequently discover their selected seats have been taken by a faster buyer, producing poor user experience. With holds, an unresponsive user can block seats for others until the hold expires.\nAssumed answer. \u0026ldquo;Seats are held for 5 minutes upon selection. If payment does not complete within that window, the hold expires and the seats return to the available pool.\u0026rdquo;\nThis confirms the need for a hold-then-confirm flow, which is developed in detail in Stage 6.\nQ5. What payment methods are supported? Reason for asking. Payment integration is a large surface area and must be scoped.\nAssumed answer. \u0026ldquo;Multiple payment methods: card, UPI, wallet. The payment gateway is external; assume a charge() API that returns success or failure.\u0026rdquo;\nThis is another instance of the Strategy pattern, applied to payment methods rather than pricing.\nQ6. Are cancellations permitted? Reason for asking. Cancellations expand the Booking lifecycle beyond a simple confirmed state.\nAssumed answer. \u0026ldquo;Yes. Users may cancel up to 2 hours before showtime with a refund. Cancellations closer to showtime are not permitted.\u0026rdquo;\nThe Booking entity is now clearly a state machine with the states PENDING, CONFIRMED, CANCELLED, and FAILED.\nQ7. Are notifications required? Reason for asking. Notification concerns (email, SMS) are cross-cutting and typically handled via the Observer pattern, which decouples the object that generates events (the publisher, or subject) from the objects that respond to them (the observers, or subscribers).\nAssumed answer. \u0026ldquo;Yes. Notify on confirmed, cancelled, and payment-failed events. Assume email and SMS providers are pluggable.\u0026rdquo;\nObserver pattern is confirmed, with concrete implementations behind a common interface.\nQ8. What is the scope regarding persistence, scale, and distribution? Reason for asking. These concerns typically lie outside LLD scope but should be acknowledged.\nAssumed answer. \u0026ldquo;In-memory persistence is acceptable. Single process. Multi-threaded (must handle concurrent bookings correctly). Distribution across servers is out of scope but should be mentioned as an extension.\u0026rdquo;\nThe design is now scoped for a single-process, multi-threaded, in-memory implementation, with distribution treated as a follow-up topic.\nRequirements Recap At the end of the clarification stage, restate the requirements to the interviewer as follows:\n\u0026ldquo;To confirm the scope: a multi-city, multi-theater system in which users browse movies, select a show, interactively pick seats, hold those seats for 5 minutes, complete payment through a pluggable gateway, and receive notifications on confirmation or cancellation. Pricing varies by seat type and time. Cancellations are permitted up to 2 hours before showtime. The implementation is in-memory and multi-threaded, and must correctly handle concurrent seat selection. Have I missed anything?\u0026rdquo;\nDelivering this recap signals structured thinking and gives the interviewer an opportunity to correct any misunderstanding before implementation work begins.\n4. Stage 3: Identifying Entities The next step is to extract candidate entities from the requirements and evaluate which deserve to become classes.\nConfirmed Entities Entity Role City A grouping of theaters. Theater A physical venue with an address. Contains screens. Screen A physical auditorium inside a theater. Contains seats. Seat A physical seat in a screen with a fixed row, number, and type. Movie A film with metadata: title, duration, language, genre. Show A specific screening of a movie in a screen at a start time. User The account that performs a booking. Booking A completed transaction linking a user to seats on a show. Payment The transactional record of money changing hands. Supporting Components Component Role PricingStrategy Computes the total price for a set of seats on a show for a given user. PaymentMethod Abstract interface implemented by Card, UPI, Wallet. SeatLockManager The concurrency-critical component that manages seat holds during the payment window. BookingService The facade that orchestrates the complete booking flow. BookingEventPublisher The Observer-pattern subject that dispatches booking events to subscribers. Entities Rejected or Merged Ticket as a separate class was considered but merged into Booking. A booking already contains the show, the seats, and the user, so a separate Ticket entity would carry no additional information. Auditorium and Session are alternative names for Screen and Show respectively; the shorter forms are used.\nThe Critical Distinction: Seat vs ShowSeat The most important modelling decision at this stage is the separation of Seat from ShowSeat.\nA Seat (for example, seat A5 in Screen 2 of a specific theater) is a physical entity. Its properties (row, number, type) are fixed for the lifetime of the screen. Its existence does not depend on any specific show.\nAvailability, however, is not a property of the physical seat. Availability is a property of the seat with respect to a specific show. Seat A5 may be booked for the 7:00 PM show while being available for the 10:00 PM show on the same day. The physical seat is unchanged; only its per-show availability differs.\nThis separation requires two distinct classes:\nSeat: The physical entity. Attached to a Screen. Immutable once the screen is built. ShowSeat: The per-show availability record for a specific Seat. Contains the status (AVAILABLE, LOCKED, BOOKED), the computed price, and (when LOCKED) the holding user and expiry timestamp. This is structurally analogous to the Book versus BookItem distinction in the Library deep dive. The abstract entity (title, physical seat) is separated from the per-instance state (loan record, per-show availability).\nFailing to model this distinction leads to designs in which Seat objects are reset between shows, which becomes untenable once per-show pricing is introduced.\n5. Stage 4: The Domain Hierarchy With entities identified, the next step is to establish the containment and reference relationships between them.\nThe relationships are as follows. A City contains many Theater objects. A Theater contains many Screen objects. A Screen contains many Seat objects (the physical seats installed in the auditorium, which are fixed once built) and hosts many Show objects over time. A Show is defined by a Movie, a Screen, and a start time, and contains one ShowSeat per physical seat in the screen. A Movie exists independently of any specific Show and can be played by many shows across many screens across many theaters.\nThe class diagram for the structural portion of the design is shown below. Strategy components and the lock manager are added in later stages.\nclassDiagram class City { +str id +str name +list~Theater~ theaters } class Theater { +str id +str name +str address +list~Screen~ screens } class Screen { +str id +str name +list~Seat~ seats +list~Show~ shows } class Seat { +str id +str row +int number +SeatType type } class Movie { +str id +str title +int duration_minutes +str language +str genre } class Show { +str id +Movie movie +Screen screen +datetime start_time +datetime end_time +dict~seat_id, ShowSeat~ show_seats } class ShowSeat { +Seat seat +Show show +SeatStatus status +float price } City \u0026#34;1\u0026#34; *-- \u0026#34;many\u0026#34; Theater Theater \u0026#34;1\u0026#34; *-- \u0026#34;many\u0026#34; Screen Screen \u0026#34;1\u0026#34; *-- \u0026#34;many\u0026#34; Seat Screen \u0026#34;1\u0026#34; o-- \u0026#34;many\u0026#34; Show Show \u0026#34;1\u0026#34; o-- \u0026#34;1\u0026#34; Movie Show \u0026#34;1\u0026#34; o-- \u0026#34;1\u0026#34; Screen Show \u0026#34;1\u0026#34; *-- \u0026#34;many\u0026#34; ShowSeat ShowSeat \u0026#34;many\u0026#34; o-- \u0026#34;1\u0026#34; Seat Definitions relevant to this diagram.\nComposition (denoted *-- in Mermaid class diagrams): a whole-part relationship in which the part cannot exist independently of the whole. A Seat is composed into a Screen because removing the screen makes the seat meaningless. A ShowSeat is composed into a Show for the same reason. Aggregation (denoted o--): a whole-part relationship in which the part exists independently and can be associated with other wholes. A Show references a Movie by aggregation because the movie exists independently and plays in many shows. Distinguishing composition from aggregation on the diagram communicates ownership semantics to the reader. This distinction signals that the candidate understands lifecycle dependencies rather than treating all relationships as generic associations.\n6. Stage 5: Modelling Seats and the Seat Lifecycle The ShowSeat.status field encodes the availability of a specific seat for a specific show. The candidate must determine the correct set of states.\nAn initial impulse is to define two states: AVAILABLE and BOOKED. This is insufficient because seats must be reserved during the payment window without being permanently booked. The correct set of states is three.\nState Meaning AVAILABLE The seat is free and can be selected by any user. LOCKED The seat is held by a specific user with a time-to-live (TTL) expiry. Other users cannot select it, but the seat is not yet permanently booked. BOOKED Payment has succeeded and the seat is sold. Definition: TTL (Time To Live). A duration after which a resource, hold, or record automatically expires. Widely used in caching, session management, and hold mechanisms of this kind.\nThe permitted state transitions are shown below.\nstateDiagram-v2 [*] --\u0026gt; AVAILABLE : Show created AVAILABLE --\u0026gt; LOCKED : User selects seat (hold_seat) LOCKED --\u0026gt; AVAILABLE : Hold expires OR user cancels LOCKED --\u0026gt; BOOKED : Payment succeeds (confirm_booking) BOOKED --\u0026gt; AVAILABLE : Refund processed (cancel_booking) BOOKED --\u0026gt; [*] : Show ends The transition LOCKED to AVAILABLE has two triggering causes: automatic expiry after the TTL elapses, or explicit release when the user cancels or payment fails. Both causes must lead to identical cleanup logic. The transition BOOKED to AVAILABLE corresponds to a cancellation and returns the seat to the available pool.\nShould the State Pattern Be Applied Here? The State pattern encapsulates state-dependent behaviour in dedicated classes, one per state, and delegates method calls from the context object to the current state object. It is appropriate when each state exhibits genuinely different behaviour for the same method invocation.\nShowSeat does not meet this criterion. The three states differ only in permitted transitions, not in the behaviour they expose. Every state supports the same read operations (get status, get price), and the transitions can be implemented as a simple lookup table plus validation. Introducing three classes for three enum values would add ceremony without benefit.\nThe correct choice for ShowSeat is an enum-backed field with an explicit transition table. The State pattern is reconsidered for the Booking entity in Stage 7, where the tradeoff analysis produces the same conclusion for the same reason. Explicitly articulating the decision, namely that the State pattern was considered and rejected because the states lack behavioural differences, is a signal of design maturity that interviewers reward.\n7. Stage 6: The Concurrency Problem in Depth This stage is the analytical core of the design. The concurrency techniques introduced here are what distinguish a correct movie booking system from an incorrect one.\n7.1 The Naive Design and Its Race Condition The correct starting point is the naive design, since analysing its failure mode motivates every subsequent decision.\nclass NaiveBookingService: def book(self, user_id: str, show_id: str, seat_ids: list[str]) -\u0026gt; Booking: show = self.get_show(show_id) # Step 1: check every seat is available for seat_id in seat_ids: if show.show_seats[seat_id].status != SeatStatus.AVAILABLE: raise SeatUnavailableError(seat_id) # Step 2: mark them all as booked for seat_id in seat_ids: show.show_seats[seat_id].status = SeatStatus.BOOKED return Booking(user_id, show_id, seat_ids, ...) The code appears correct on inspection. It reads seats, verifies availability, and updates state. The defect is invisible until two threads execute it concurrently.\nConsider two users, Alice and Bob, both attempting to book seat A5 at the same moment. The following interleaving produces incorrect behaviour.\nTime Alice\u0026#39;s thread Bob\u0026#39;s thread ---- ------------------------------ ------------------------------ t=1 Read show_seats[A5].status -\u0026gt; AVAILABLE t=2 Read show_seats[A5].status -\u0026gt; AVAILABLE (Alice has not written yet) t=3 Set show_seats[A5].status = BOOKED t=4 Set show_seats[A5].status = BOOKED t=5 Create Booking for Alice t=6 Create Booking for Bob Result: Seat A5 is BOOKED and BOTH bookings exist. The same seat has been sold to two customers. Definition: Race condition. A defect whose presence depends on the specific temporal ordering of operations across two or more threads, when that ordering is not guaranteed by the system. Race conditions are characterised by non-deterministic reproduction: identical inputs produce different outputs on different runs.\nDefinition: Check-then-act pattern. A code structure in which a program reads state to determine an action and then performs that action based on the read. When the check and the act are not executed as a single atomic operation, another thread can modify state between the two steps, invalidating the assumption on which the act was based. The example above is a check-then-act race condition on show_seats[A5].status.\nThe naive design is broken by construction. Single-threaded tests will pass. Load tests that happen to serialise requests will pass. Only concurrent production traffic reveals the defect, at which point double-booked customers appear at the venue.\n7.2 Lock Granularity: Global, Per-Show, Per-Seat The correction requires a lock, which is a synchronisation primitive that permits at most one thread to hold it at a time. The choice of what the lock protects, referred to as lock granularity, is one of the most consequential design decisions in any concurrent system. Locks that are too coarse reduce throughput under contention. Locks that are too fine invite deadlocks and complex acquisition logic.\nThree granularities are worth analysing.\nOption A: One global lock for the entire system.\nclass BookingService: def __init__(self): self._lock = threading.Lock() def book(self, ...): with self._lock: # entire booking flow Correctness. Satisfactory. Only one booking proceeds at any moment, so no race is possible.\nPerformance. Unacceptable at scale. Every booking in the entire system serialises against every other booking. A production platform would experience throughput collapse under peak load.\nOption B: One lock per show.\nclass Show: def __init__(self, ...): self._lock = threading.RLock() Correctness. Satisfactory. A booking within a show is atomic with respect to all other bookings within the same show. Since a single booking always involves seats within a single show, multi-seat bookings are naturally covered.\nPerformance. Suitable for most real systems. A single show typically contains a few hundred seats, so contention on the show lock is bounded. Different shows scale independently. This is the granularity used in most production implementations.\nOption C: One lock per seat.\nclass ShowSeat: def __init__(self, ...): self._lock = threading.Lock() Correctness. Requires additional discipline for multi-seat bookings. Acquiring multiple locks without a defined order creates deadlock risk (see Section 7.5).\nPerformance. Maximal parallelism, at the cost of implementation complexity. Justified only when profiling shows that per-show lock contention is the bottleneck.\nThe chosen design uses Option B (per-show locks). This choice provides correct concurrent behaviour with straightforward implementation and adequate performance for a well-partitioned system (many concurrent shows across many screens across many theaters). Option C is available as a future optimisation.\nDefinition: RLock (re-entrant lock). A lock variant that permits the same thread to acquire the lock multiple times without deadlocking against itself. Required when a method that holds the lock calls another method that also attempts to acquire the same lock. The Show.lock is an RLock because SeatLockManager methods sometimes call other locked methods within the same thread.\n7.3 Pessimistic vs Optimistic Locking Two philosophical approaches exist for controlling concurrent access to shared data.\nPessimistic locking is the strategy discussed so far. The design assumes conflict will occur and blocks other threads from accessing the resource until the current thread finishes. Threads acquire the lock, perform work, and release the lock.\ndef book_seat(seat_id): with show._lock: # block until exclusive access is granted if seat.status == AVAILABLE: seat.status = BOOKED return \u0026#34;success\u0026#34; return \u0026#34;already booked\u0026#34; Advantages. Simple to reason about. Guarantees atomicity of the critical section. Correctness is easy to verify by inspection.\nDisadvantages. Blocking reduces throughput under contention. If the lock-holding thread performs a slow operation (for example, a payment gateway call that takes several seconds), all waiting threads block for that entire duration.\nOptimistic locking takes the opposite approach. The design assumes conflict is rare and does not hold a lock during the operation. Instead, it verifies at commit time that the underlying data has not changed since it was read. If verification fails, the operation retries.\nThe standard implementation uses a version number on the shared data. Every read captures the current version. Every write requests the update conditional on the version remaining unchanged.\ndef book_seat_optimistic(seat_id): while True: seat = read_seat(seat_id) # capture current version if seat.status != AVAILABLE: return \u0026#34;already booked\u0026#34; # Attempt to update only if the version is unchanged. # In a database context this maps to: # UPDATE seat SET status=\u0026#39;BOOKED\u0026#39;, version = version + 1 # WHERE id = ? AND version = ? success = compare_and_swap( seat_id, expected_version=seat.version, new_status=BOOKED, ) if success: return \u0026#34;success\u0026#34; # Otherwise, another thread won the race; retry. Advantages. No blocking. Threads make progress in parallel. High throughput when conflicts are rare.\nDisadvantages. Requires retry logic. Susceptible to livelock under sustained high contention, in which threads repeatedly retry without any thread completing. Wasted computation when conflicts do occur.\nThe following table summarises when each approach is appropriate.\nApproach Best suited to Pessimistic Short critical sections, moderate to high contention, correctness paramount. Default choice for in-process LLD designs. Optimistic Long-running operations that should not hold locks, read-heavy workloads with rare writes. Common at the database layer. This design applies pessimistic locking on the basis that the critical section is short (a check-and-mark on seat status), contention on popular shows would produce significant retry overhead under optimistic locking, and pessimistic locks are simpler to explain and verify. An interviewer question about optimistic locking should be answered with the tradeoff table above.\n7.4 The Hold-Then-Confirm Pattern Section 3 established that seats are held for 5 minutes during the payment window. The rationale requires elaboration.\nBooking involves a payment step whose duration is measured in seconds to minutes: the user enters card details, receives an OTP, completes 3-D Secure verification, and awaits gateway confirmation. During this window, the seat cannot remain in either extreme state.\nLeaving the seat AVAILABLE allows another user to select it, causing the original user\u0026rsquo;s payment to fail against a now-unavailable seat. Marking the seat BOOKED immediately means that user abandonment leaves the seat permanently unavailable until manual cleanup. The solution is a third state, LOCKED, and a two-phase flow.\nPhase 1: hold_seats(user, seat_ids). Atomically mark all requested seats as LOCKED with a 5-minute expiry timestamp and associate the hold with the requesting user. This phase is fast, since the critical section holds the lock for microseconds.\nPhase 2a: confirm_booking(hold_id, payment_result). If payment succeeds, atomically flip the locked seats to BOOKED and create the Booking record. This phase is also fast for the same reason.\nPhase 2b: release_hold(hold_id). If the user cancels or payment fails, atomically flip the locked seats back to AVAILABLE.\nPhase 2c: expire_holds(). A background process periodically identifies holds whose TTL has elapsed and releases them, returning the seats to the available pool. A supplementary lazy-check on seat access ensures that stale holds cannot block new selections even if the sweeper is delayed.\nThe complete flow is depicted below.\nsequenceDiagram participant User participant BookingService participant SeatLockManager participant Show participant PaymentGateway participant NotificationService User-\u0026gt;\u0026gt;BookingService: select_seats(show, [A5, A6]) BookingService-\u0026gt;\u0026gt;SeatLockManager: hold_seats(user, show, [A5, A6]) SeatLockManager-\u0026gt;\u0026gt;Show: acquire show lock SeatLockManager-\u0026gt;\u0026gt;Show: check all seats AVAILABLE SeatLockManager-\u0026gt;\u0026gt;Show: mark all as LOCKED (with TTL) SeatLockManager-\u0026gt;\u0026gt;Show: release show lock SeatLockManager--\u0026gt;\u0026gt;BookingService: hold_id (valid 5 min) BookingService--\u0026gt;\u0026gt;User: hold_id + total_price User-\u0026gt;\u0026gt;BookingService: pay(hold_id, card_details) BookingService-\u0026gt;\u0026gt;SeatLockManager: verify hold still valid BookingService-\u0026gt;\u0026gt;PaymentGateway: charge(amount) PaymentGateway--\u0026gt;\u0026gt;BookingService: success BookingService-\u0026gt;\u0026gt;SeatLockManager: confirm_booking(hold_id) SeatLockManager-\u0026gt;\u0026gt;Show: acquire show lock SeatLockManager-\u0026gt;\u0026gt;Show: mark all LOCKED to BOOKED SeatLockManager-\u0026gt;\u0026gt;Show: release show lock BookingService-\u0026gt;\u0026gt;NotificationService: notify(BookingConfirmed) BookingService--\u0026gt;\u0026gt;User: booking confirmed The pattern\u0026rsquo;s correctness under load follows from two properties. First, both lock-holding operations (hold_seats and confirm_booking) are trivially fast, so lock contention is negligible. Second, the slow operation (payment) occurs outside any lock, so an arbitrary number of users can be in the payment step simultaneously without blocking one another. The lock is held for essentially zero time, slow external operations proceed freely, and no seat can be double-sold.\nThis pattern is used in essentially every production ticket-booking, hotel-reservation, and flight-booking system. It appears in the software architecture literature under several names, including reserve-and-commit and tentative allocation.\n7.5 Deadlock and Ordered Locking The chosen per-show lock granularity avoids deadlock for the common case of booking multiple seats within a single show. If the design used per-seat locks instead, deadlock becomes a genuine risk that must be actively managed.\nDefinition: Deadlock. A state in which two or more threads are permanently blocked, each holding a resource that another thread is waiting for. Deadlock requires the simultaneous presence of four conditions (the Coffman conditions): mutual exclusion (resources held exclusively), hold-and-wait (a thread holds resources while requesting others), no preemption (resources cannot be forcibly taken), and circular wait (a cycle of threads each waiting on the next).\nConsider the per-seat lock design. Alice requests seats A5 and A6. Bob requests seats A6 and A5.\nAlice\u0026#39;s thread Bob\u0026#39;s thread -------------- -------------- acquire lock(A5) acquire lock(A6) try to acquire lock(A6) try to acquire lock(A5) BLOCKED: Bob holds it BLOCKED: Alice holds it [neither thread proceeds] Both threads wait indefinitely. Neither releases its held lock, and the process must be terminated externally.\nThe standard prevention technique is ordered locking: establish a total order on all locks (for example, alphabetical by seat identifier) and require every thread to acquire locks strictly in that order. Under ordered locking, both Alice and Bob attempt to acquire A5 before A6, so the second thread blocks at A5, and the circular-wait condition cannot form.\ndef hold_seats_ordered(user, seat_ids): # Sort seat IDs to enforce consistent lock acquisition order. sorted_seats = sorted(seat_ids) acquired_locks = [] try: for seat_id in sorted_seats: lock = get_seat_lock(seat_id) lock.acquire() acquired_locks.append(lock) # perform check-and-mark on all seats finally: for lock in reversed(acquired_locks): lock.release() Ordered locking eliminates the circular-wait condition, which breaks the deadlock possibility.\nBecause the current design uses per-show locking, deadlock is not a concern for single-show bookings. If a future requirement introduced cross-show bookings (for example, booking the same seat number for two consecutive shows in one transaction), the same ordered-locking discipline would apply at the show-lock level.\n7.6 Idempotency Concurrency defects are not limited to conflicts between different users. The same user\u0026rsquo;s request can arrive multiple times through several common mechanisms.\nThe user clicks the \u0026ldquo;Pay\u0026rdquo; button twice because the first click appeared unresponsive. The mobile client experiences a network interruption and retries a request the server already received. The payment gateway\u0026rsquo;s webhook delivery mechanism retries a callback that was already processed. Without protection, the duplicate request creates a second booking against the same held seats. The second attempt either fails (correctly, because the seats are already booked by attempt one) or succeeds against a different set of seats (which is incorrect and results in double-charging).\nDefinition: Idempotency. A property of an operation such that executing it multiple times has the same effect as executing it once. An idempotent operation is safe to retry.\nThe standard implementation uses an idempotency key, which is a unique client-supplied identifier for the operation. The server records the key on first receipt and, if the same key arrives again, returns the original response without re-executing the operation.\nclass BookingService: def __init__(self): self._idempotency_cache: dict[str, Booking] = {} self._idempotency_lock = threading.Lock() def confirm_booking(self, hold_id, payment_details, idempotency_key: str) -\u0026gt; Booking: # Check for duplicate request first. with self._idempotency_lock: if idempotency_key in self._idempotency_cache: return self._idempotency_cache[idempotency_key] # Perform the booking operation. booking = self._do_booking(hold_id, payment_details) # Record the result under the key. with self._idempotency_lock: self._idempotency_cache[idempotency_key] = booking return booking All reputable payment gateways (Stripe, Razorpay, Braintree, Adyen) support idempotency keys, precisely because network retries are ubiquitous in payment flows. Any LLD design involving payment should include this mechanism as a core feature rather than an addition.\n7.7 Sequence Diagram: Two Users Contending for the Same Seat The following diagram illustrates the resolution of a concurrent seat selection under the hold-then-confirm design with per-show locking.\nsequenceDiagram participant Alice participant Bob participant SLM as SeatLockManager participant Show par Alice-\u0026gt;\u0026gt;SLM: hold_seats(A5) and Bob-\u0026gt;\u0026gt;SLM: hold_seats(A5) end Note over SLM,Show: Both requests reach the SeatLockManager\u0026lt;br/\u0026gt;and contend for the show lock. SLM-\u0026gt;\u0026gt;Show: try acquire lock (Alice wins) Show--\u0026gt;\u0026gt;SLM: lock granted to Alice SLM-\u0026gt;\u0026gt;Show: A5.status == AVAILABLE? YES SLM-\u0026gt;\u0026gt;Show: A5.status = LOCKED (by Alice, TTL 5min) SLM-\u0026gt;\u0026gt;Show: release lock SLM--\u0026gt;\u0026gt;Alice: hold_id_1 SLM-\u0026gt;\u0026gt;Show: try acquire lock (Bob\u0026#39;s turn) Show--\u0026gt;\u0026gt;SLM: lock granted to Bob SLM-\u0026gt;\u0026gt;Show: A5.status == AVAILABLE? NO (LOCKED by Alice) SLM-\u0026gt;\u0026gt;Show: release lock SLM--\u0026gt;\u0026gt;Bob: SeatUnavailableError Note over Alice,Bob: Alice proceeds to payment.\u0026lt;br/\u0026gt;Bob receives an immediate error and may select a different seat. The contention is resolved deterministically by the lock. Whichever thread the operating system schedules first acquires the resource; the other receives an immediate, clean error. No double-booking is possible.\nThe subsequent success flow, in which Alice completes payment, follows the sequence below.\nsequenceDiagram participant Alice participant BS as BookingService participant SLM as SeatLockManager participant PG as PaymentGateway Alice-\u0026gt;\u0026gt;BS: confirm(hold_id_1, card_details, idempotency_key) BS-\u0026gt;\u0026gt;SLM: is hold still valid? SLM--\u0026gt;\u0026gt;BS: yes, seat A5 locked by Alice BS-\u0026gt;\u0026gt;PG: charge($15.00, idempotency_key) PG--\u0026gt;\u0026gt;BS: success (txn_id) BS-\u0026gt;\u0026gt;SLM: promote_to_booked(hold_id_1) SLM-\u0026gt;\u0026gt;SLM: acquire show lock, flip A5 LOCKED to BOOKED, release SLM--\u0026gt;\u0026gt;BS: ok BS--\u0026gt;\u0026gt;Alice: Booking(id=B1, seat=A5, txn=txn_id) The failure and abandonment cases are illustrated below.\nsequenceDiagram participant Alice participant BS as BookingService participant SLM as SeatLockManager participant PG as PaymentGateway alt Payment fails Alice-\u0026gt;\u0026gt;BS: confirm(hold_id_1, card_details) BS-\u0026gt;\u0026gt;PG: charge($15.00) PG--\u0026gt;\u0026gt;BS: FAILED (card declined) BS-\u0026gt;\u0026gt;SLM: release_hold(hold_id_1) SLM-\u0026gt;\u0026gt;SLM: A5 LOCKED to AVAILABLE BS--\u0026gt;\u0026gt;Alice: PaymentFailedError else User abandons Note over SLM: 5 minutes pass... SLM-\u0026gt;\u0026gt;SLM: background sweep finds expired hold SLM-\u0026gt;\u0026gt;SLM: A5 LOCKED to AVAILABLE Note over SLM: The client eventually times out. end Regardless of the outcome (successful contention, payment failure, or user abandonment), the seat state converges to a valid configuration without manual intervention.\n8. Stage 7: Booking as a State Machine The Booking entity has a lifecycle that must be modelled explicitly. The permitted states and transitions are shown below.\nstateDiagram-v2 [*] --\u0026gt; PENDING : Booking created (payment in progress) PENDING --\u0026gt; CONFIRMED : Payment succeeds PENDING --\u0026gt; FAILED : Payment fails CONFIRMED --\u0026gt; CANCELLED : User cancels (\u0026gt;=2h before show) FAILED --\u0026gt; [*] CANCELLED --\u0026gt; [*] CONFIRMED --\u0026gt; [*] : Show ends The transition rules are as follows. A PENDING booking may transition to CONFIRMED or FAILED. A CONFIRMED booking may transition to CANCELLED only when the show is at least 2 hours away. The FAILED and CANCELLED states are terminal and permit no further transitions.\nAs discussed in Stage 5, the State pattern is not applied here. The transitions differ across states, but the behaviours do not. The implementation uses an enum-backed status field with transition validation in the setter. Attempting an illegal transition raises an InvalidBookingTransition exception, providing safety without the ceremony of dedicated state classes.\n9. Stage 8: Pricing as a Strategy Pricing depends on multiple orthogonal factors:\nSeat type (regular, premium, recliner) affects the base price. Time (weekend versus weekday, matinee versus evening) applies multipliers. User attributes (student, senior) unlock discounts. Promotional codes apply flat or percentage-based discounts. Show popularity may drive surge pricing. Encoding these rules inside a monolithic Booking.calculate_price() method with conditional branches produces unmaintainable code. Any new rule requires modifying the existing method, violating the Open/Closed Principle.\nThe Strategy pattern provides the appropriate structure. Each pricing rule becomes a class implementing a common PricingStrategy interface. Composite rules use the Decorator pattern, in which one strategy wraps another and adjusts its result.\nclassDiagram class PricingStrategy { \u0026lt;\u0026lt;interface\u0026gt;\u0026gt; +calculate(show, seats, user) float } class StandardPricing { +calculate(show, seats, user) float } class WeekendPricing { +float weekend_multiplier +calculate(show, seats, user) float } class StudentDiscountPricing { +PricingStrategy base +float discount_pct +calculate(show, seats, user) float } PricingStrategy \u0026lt;|.. StandardPricing PricingStrategy \u0026lt;|.. WeekendPricing PricingStrategy \u0026lt;|.. StudentDiscountPricing StudentDiscountPricing o-- PricingStrategy : wraps The composition StudentDiscountPricing(WeekendPricing(StandardPricing())) produces weekend-adjusted base pricing with a student discount applied on top. This composition satisfies the Open/Closed Principle: adding a new pricing rule requires only a new class, with no changes to existing pricing code.\n10. Stage 9: Payment as a Strategy The payment subsystem uses the Strategy pattern for the same reasons as pricing. The PaymentMethod abstract base defines the charge interface; CardPayment, UPIPayment, and WalletPayment provide concrete implementations. Each concrete class encapsulates the specifics of its gateway or backend.\nclass PaymentMethod(ABC): @abstractmethod def charge(self, amount: float, idempotency_key: str) -\u0026gt; PaymentResult: ... class CardPayment(PaymentMethod): def __init__(self, card_number, cvv, expiry): ... def charge(self, amount, idempotency_key): # invoke the payment gateway return PaymentResult(status=SUCCESS, txn_id=\u0026#34;...\u0026#34;) The BookingService accepts a PaymentMethod argument and calls its charge method. It has no knowledge of the concrete payment mechanism, which is the Dependency Inversion Principle applied at the boundary between business logic and infrastructure.\nAdding a new payment method (UPI Autopay, Apple Pay, or any other rail) requires implementing the interface. No changes to BookingService or any other component are needed.\n11. Stage 10: Notifications via Observer Booking events (confirmation, cancellation, failure) must trigger notifications through multiple channels (email, SMS, audit log). Coupling BookingService directly to EmailNotifier and SMSNotifier would violate the Single Responsibility Principle by mixing business logic with notification infrastructure.\nThe Observer pattern provides the correct decoupling. BookingService holds a BookingEventPublisher and calls its notify method when significant events occur. Interested components register as observers.\nclassDiagram class BookingEventPublisher { -list~BookingObserver~ observers +subscribe(observer) +unsubscribe(observer) +notify(event) } class BookingObserver { \u0026lt;\u0026lt;interface\u0026gt;\u0026gt; +on_booking_event(event) } class EmailNotifier { +on_booking_event(event) } class SMSNotifier { +on_booking_event(event) } class AuditLogger { +on_booking_event(event) } BookingEventPublisher o-- BookingObserver BookingObserver \u0026lt;|.. EmailNotifier BookingObserver \u0026lt;|.. SMSNotifier BookingObserver \u0026lt;|.. AuditLogger Adding a new notification channel (WhatsApp, push notification, Slack integration for enterprise customers) requires implementing BookingObserver and registering the instance with the publisher. BookingService remains unchanged.\n12. Stage 11: The Complete Class Diagram Combining the structural entities from Stage 4 with the strategies, lock manager, and publisher from Stages 6 through 10 produces the complete class diagram below.\nclassDiagram class BookingService { -SeatLockManager lock_manager -PricingStrategy pricing -BookingEventPublisher publisher -dict~str, Booking~ bookings +hold_seats(user, show, seat_ids) Hold +confirm_booking(hold_id, payment, idempotency_key) Booking +release_hold(hold_id) +cancel_booking(booking_id) } class SeatLockManager { -dict~str, Hold~ holds +hold_seats(user, show, seat_ids, ttl) Hold +is_hold_valid(hold_id) bool +promote_to_booked(hold_id) +release_hold(hold_id) +expire_stale_holds() } class Hold { +str id +User user +Show show +list~ShowSeat~ seats +datetime expires_at +bool is_expired() bool } class Booking { +str id +User user +Show show +list~ShowSeat~ seats +float total_amount +BookingStatus status +PaymentResult payment +confirm(payment) +fail(reason) +cancel() } class Show { +str id +Movie movie +Screen screen +datetime start_time +dict~str, ShowSeat~ show_seats +RLock lock } class ShowSeat { +Seat seat +Show show +SeatStatus status +float price +User locked_by +datetime locked_until } class PricingStrategy { \u0026lt;\u0026lt;interface\u0026gt;\u0026gt; +calculate(show, seats, user) float } class PaymentMethod { \u0026lt;\u0026lt;interface\u0026gt;\u0026gt; +charge(amount, idempotency_key) PaymentResult } class BookingEventPublisher { -list~BookingObserver~ observers +subscribe(o) +notify(event) } BookingService o-- SeatLockManager BookingService o-- PricingStrategy BookingService o-- BookingEventPublisher BookingService --\u0026gt; Booking : creates SeatLockManager --\u0026gt; Hold : creates SeatLockManager --\u0026gt; Show : locks Hold o-- ShowSeat Booking o-- ShowSeat Booking o-- User Show *-- ShowSeat The diagram makes the architectural roles explicit.\nComponent Role BookingService The Facade. The only class that clients interact with. SeatLockManager The concurrency boundary. All operations that modify seat availability route through it. Show Owner of the per-show lock. Enforces the chosen lock granularity. PricingStrategy, PaymentMethod Injected strategies. Swappable and independently testable. BookingEventPublisher Decouples business logic from notification infrastructure. 13. Stage 12: Validating the Design with a Mental Walkthrough Before declaring the design complete, trace a concrete scenario end-to-end to verify that the components interact correctly.\nScenario. Alice books two premium seats (A5, A6) for the 7:00 PM show of a film at Screen 1, paying by credit card. Simultaneously, Bob attempts to book A5 for the same show.\nThe sequence of events is as follows.\nAlice\u0026rsquo;s request arrives. BookingService.hold_seats(alice, show_id, [\u0026quot;A5\u0026quot;, \u0026quot;A6\u0026quot;]). SeatLockManager acquires the lock on show. Availability check: A5 is AVAILABLE, A6 is AVAILABLE. Both seats are marked LOCKED with locked_by=alice and locked_until=now + 5min. A Hold(id=\u0026quot;h1\u0026quot;, user=alice, seats=[A5, A6], expires_at=...) is created. The show lock is released. The hold_id and total price are returned to Alice\u0026rsquo;s frontend. Bob\u0026rsquo;s request arrives. hold_seats(bob, show_id, [\u0026quot;A5\u0026quot;]). SeatLockManager waits briefly for the show lock (Alice\u0026rsquo;s operation has completed, so contention is negligible). The lock is acquired. The availability check finds A5 is LOCKED (by Alice) and fails immediately. The show lock is released. SeatUnavailableError is raised for Bob. Bob\u0026rsquo;s UI immediately displays \u0026ldquo;seat unavailable\u0026rdquo; and prompts him to select a different seat. Alice enters card details. The frontend calls confirm_booking(h1, card, idempotency_key=\u0026quot;a-uuid-1\u0026quot;). The idempotency cache is checked; no prior request exists with a-uuid-1, so processing proceeds. Hold h1 is verified as still valid (not expired, user matches). card.charge(30.00, \u0026quot;a-uuid-1\u0026quot;) is invoked. The gateway returns success with txn_id=\u0026quot;tx-999\u0026quot;. SeatLockManager.promote_to_booked(h1) acquires the show lock, flips both seats from LOCKED to BOOKED, and releases the lock. A Booking(id=\u0026quot;b1\u0026quot;, user=alice, seats=[A5, A6], status=CONFIRMED, payment=tx-999) is created. The booking is stored and cached under the idempotency key. publisher.notify(BookingConfirmed(booking)) fires; EmailNotifier and SMSNotifier dispatch their messages. The booking is returned to Alice\u0026rsquo;s UI. Two hours later, Alice decides to cancel. cancel_booking(\u0026quot;b1\u0026quot;) is invoked. The booking is fetched. The system verifies the show is at least 2 hours away. The booking status is set to CANCELLED. Seats flip from BOOKED to AVAILABLE under the show lock. The publisher notifies BookingCancelled, and the refund workflow proceeds. Failure Modes Identified During the Walkthrough The walkthrough surfaces edge cases that the design must address, whether now or in follow-up discussion.\nIssue 1: Payment succeeds but promote-to-booked crashes before the Booking record is created. The user has been charged, but no booking record exists. Mitigation strategies include auto-refund on booking-record failure, or writing a PENDING_PAYMENT booking record before initiating the charge so that a record always exists to reconcile against. This is a variant of the classic distributed transaction problem.\nIssue 2: An abandoned hold blocks other users for the full TTL duration. For high-demand shows, 5 minutes of blocked seats represents significant lost bookings. Mitigations include dynamic TTL adjustment based on show demand and active prompts to confirm continued user presence.\nIssue 3: A stampede of users clicks the same seat simultaneously. With 1000 concurrent clicks on the same seat, 999 users receive SeatUnavailableError. This is architecturally correct but produces poor user experience. Mitigation belongs at the UI layer (refresh the seat map so unavailable seats are not shown as available) and at the API layer (rate limiting, virtual waiting rooms).\nNone of these issues represents a design flaw. All are operational concerns that would be addressed in follow-up discussion.\n14. Stage 13: Anticipating Follow-Up Questions The interviewer typically probes the boundaries of the design with follow-up questions. Prepared responses to the most common questions are provided below.\nHow would you handle 100,000 shows if per-show locks became a memory issue? Move the locks into a lock manager that lazily creates locks by show identifier and evicts inactive ones. Alternatively, use a striped lock design (a fixed pool of N locks, with shows hashed to a lock). Striping trades some concurrency for bounded memory usage.\nHow would the design scale across multiple servers? In-process locks do not synchronise across servers. Three common alternatives exist.\nApproach Mechanism Tradeoff Database row locks SELECT ... FOR UPDATE on ShowSeat rows during holds Simple; adds DB load and increases lock contention there. Distributed lock service Redis SETNX with TTL, or Zookeeper, etcd Scalable; introduces an external dependency and its failure modes. Partitioned message queue Route all requests for a given show to a single consumer (partition by show_id) Highest scalability; changes the system to asynchronous processing. What happens if the payment gateway is slow? The hold-then-confirm design specifically addresses this. The show lock is held for microseconds, not for the duration of payment. A slow gateway affects only the user awaiting that specific charge; other users can browse seats and initiate bookings without any impact.\nHow would you test this system? Testing spans multiple levels of granularity.\nTest level Approach Unit tests Test SeatLockManager in isolation with a mocked Show. Verify hold, release, expire, and promote in single-threaded scenarios. Concurrency tests Spawn many threads racing for the same seat; assert exactly one succeeds. Repeat many times to detect race conditions statistically. Idempotency tests Call confirm_booking twice with the same key; assert the same booking is returned and the payment charge is invoked only once. Integration tests Complete end-to-end flow with a fake payment gateway. Can users be notified when a seat opens up after cancellation? Add a per-show Waitlist. On cancellation, pop the next waiting user and notify them via the Observer system. If they accept, place a hold on their behalf with a limited window. The mechanism reuses the existing hold infrastructure without new patterns.\nWhat about group bookings (10 seats for a birthday party)? The existing flow accommodates this directly. Extending the hold TTL for larger group bookings is prudent, since coordination takes longer. If seat adjacency is required, add a SeatAllocationStrategy for locating N adjacent available seats, again using the Strategy pattern.\nWhat if a show is cancelled by the theater? Add a Show.status field. Setting Show.status = CANCELLED causes hold_seats to reject new attempts. All existing bookings for the cancelled show transition to CANCELLED status, with automatic refunds published through the existing Observer channels.\nHow is scalping or bot-based bulk booking prevented? This concern belongs at the API layer rather than in LLD (rate limiting per user, CAPTCHA challenges, anti-fraud machine learning). It should be acknowledged as a known concern, but it is not typically addressed in the low-level design itself.\n15. Summary and Key Takeaways Lesson 1: Concurrency is a Design Concern, Not a Post-Hoc Fix Every prior LLD problem in this series treated concurrency as an extension to be added in production hardening. Movie Ticket Booking requires the opposite treatment. The presence of two arrows pointing to the same seat on the whiteboard establishes concurrency as central to the design. Interviewers distinguish between candidates who acknowledge this and candidates who do not.\nLesson 2: The Hold-Then-Confirm Pattern is Universal The pattern developed in Section 7.4 is used by essentially every reservation system in production: ticket sales, hotel bookings, flight reservations, restaurant tables, event registration. The specifics vary by domain, but the structure is invariant: atomically claim a resource with a short-lived hold, perform slow external work outside any lock, atomically promote the hold to a permanent claim or release it, and clean up expired holds in the background. Recognising this pattern is a transferable skill.\nLesson 3: Lock Granularity Requires Deliberate Choice Coarse locks are simple and correct but limit throughput. Fine locks maximise throughput but require careful discipline to avoid deadlock. The principle for choosing granularity is to lock at the level of the atomic unit of contention. For movie bookings, that unit is the show, because a booking involves N seats within a single show atomically. Locking at a finer granularity would require ordered acquisition; locking at a coarser granularity would collapse throughput.\nLesson 4: Idempotency is Not Optional Any system involving money and network calls requires idempotency mechanisms from the outset. Retrofitting idempotency is expensive because it changes API signatures throughout the system. Design idempotency keys into APIs before they become necessary in production.\nLesson 5: Not Every State-Bearing Entity Requires the State Pattern State-machine thinking was applied to both ShowSeat and Booking, but both were implemented as enum-backed fields with transition tables rather than full State-pattern class hierarchies. The State pattern applies when method invocations produce genuinely different behaviours in different states. When only transitions differ (as here), an enum with validation is cleaner and more direct. Recognising when a pattern is unnecessary is a signal of design maturity.\nLesson 6: Concentrate Concurrency Logic in a Single Component The SeatLockManager is a dedicated component with a single responsibility: orchestrate seat holds atomically. All concurrency logic resides there. Every other component (BookingService, PricingStrategy, Booking) is single-threaded logic that relies on the lock manager to serialise access appropriately. Distributing concurrency logic across multiple components makes the design impossible to reason about. Concentration keeps the concurrent parts small, testable, and reviewable while the remaining components stay simple.\nConsolidated Pattern Summary Pattern Where Applied Purpose Facade BookingService Provides a single, simplified interface over the subsystem. Strategy PricingStrategy, PaymentMethod Interchangeable algorithms for pricing and payment. Decorator Composed pricing strategies Layered composition of pricing rules. Observer BookingEventPublisher and subscribers Decouples event generation from event handling. Dependency Injection Constructor arguments of BookingService Enables testing with substitute implementations. State-as-enum ShowSeat.status, Booking.status Explicit lifecycle with transition validation, without the ceremony of dedicated state classes. Consolidated Concurrency Summary Technique Purpose Per-show RLock Serialises access to seats within a single show while allowing independent shows to proceed in parallel. Hold-then-confirm Separates the fast lock-holding operations from the slow payment operation. Lazy expiry on access Ensures users are never blocked by stale holds, even if the background sweeper is delayed. Active expiry sweep Reclaims memory occupied by expired hold records. Idempotency keys Ensures duplicate requests do not create duplicate bookings or double charges. Ordered lock acquisition (if per-seat locks used) Prevents deadlock by breaking the circular-wait condition. 16. Practice Questions with Solutions Question 1 The current design uses a per-show RLock. Suppose a specific popular show experiences 500 concurrent booking attempts per second, and profiling reveals that lock contention on that show is limiting throughput. Describe two independent optimisations that could reduce contention without abandoning correctness.\nSolution Optimisation 1: Move to per-seat locks with ordered acquisition. Contention shifts from the show-level lock to per-seat locks. Only threads targeting the same seat contend; threads targeting different seats within the same show proceed in parallel. The ordered-acquisition discipline (acquire seat locks in sorted order of seat identifier) prevents deadlock during multi-seat bookings.\nOptimisation 2: Read-write partitioning. Split the show\u0026rsquo;s operations into read (browsing available seats) and write (holding, confirming, cancelling). Use a ReadWriteLock that permits multiple concurrent readers but requires exclusive access for writers. Since browsing is far more frequent than booking, this significantly increases effective throughput. Note that the write path itself is unchanged in its correctness properties.\nQuestion 2 Explain why the following code contains a race condition even though every mutation is guarded by a lock. Then propose a fix.\nclass BookingCounter: def __init__(self): self.count = 0 self._lock = threading.Lock() def get_count(self) -\u0026gt; int: with self._lock: return self.count def increment(self): with self._lock: self.count += 1 # Client code if counter.get_count() \u0026lt; MAX_BOOKINGS: counter.increment() Solution The individual methods get_count and increment are each atomic. The race occurs in the client code, which performs a check-then-act sequence across two separate lock acquisitions. Between the return of get_count and the entry to increment, another thread may increment the counter, causing the current thread to increment past MAX_BOOKINGS.\nFix. Introduce a compound increment_if_below(max_value) method that performs the check and the increment inside a single lock acquisition.\ndef increment_if_below(self, max_value: int) -\u0026gt; bool: with self._lock: if self.count \u0026lt; max_value: self.count += 1 return True return False The general principle is that the lock must protect the invariant, not merely individual reads and writes. When the invariant spans multiple operations, all of those operations must occur inside a single critical section.\nQuestion 3 The design uses an in-memory _idempotency_cache in BookingService to prevent duplicate bookings. Identify three limitations of this implementation and propose corrections for a production system.\nSolution Limitation 1: Unbounded memory growth. Every idempotency key is retained indefinitely. In a production system, this dictionary grows without bound.\nCorrection. Apply a TTL to cache entries (for example, 24 hours) and evict expired entries periodically, or use a bounded LRU cache.\nLimitation 2: Loss on process restart. The cache is in-memory. A process restart loses all recorded keys, and duplicate requests submitted after the restart are treated as new requests.\nCorrection. Persist the idempotency cache in durable storage (a database table, or a distributed cache such as Redis with persistence enabled).\nLimitation 3: No cross-server coordination. In a multi-server deployment, each server maintains its own cache. A request that reaches server A and is retried against server B is not detected as a duplicate.\nCorrection. Use a shared cache accessible to all servers, or route requests to a specific server based on a hash of the idempotency key.\nQuestion 4 The SeatLockManager expires holds through two mechanisms: a lazy check inside hold_seats and an active expire_stale_holds sweeper. Explain why both mechanisms are needed rather than relying on either one alone.\nSolution If only the active sweeper were used: the interval between sweeps creates a window during which expired holds still appear valid. A user attempting to select a seat during this window would be incorrectly told the seat is unavailable, even though the actual owner has already abandoned the hold. Users would experience apparent failures for reasons they cannot diagnose.\nIf only lazy expiry were used: stale hold records accumulate indefinitely in the _holds dictionary. Memory usage grows without bound, since only active seat-access operations trigger cleanup. Holds for seats that are never re-attempted remain forever.\nThe combination provides both correctness (lazy expiry ensures no user is blocked by a stale hold) and resource hygiene (active sweeping bounds memory usage). Each mechanism compensates for the weakness of the other.\nQuestion 5 The design currently permits cancellation up to 2 hours before showtime. Suppose the business requires a tiered refund policy: full refund up to 24 hours before, 50% refund between 24 and 4 hours before, no refund within 4 hours. How would you extend the design to accommodate this while minimising changes to existing components?\nSolution Introduce a RefundPolicy interface following the Strategy pattern, and inject it into BookingService.\nclass RefundPolicy(ABC): @abstractmethod def calculate_refund(self, booking: Booking, cancellation_time: datetime) -\u0026gt; float: ... class TieredRefundPolicy(RefundPolicy): def calculate_refund(self, booking, cancellation_time): hours_to_show = (booking.show.start_time - cancellation_time).total_seconds() / 3600 if hours_to_show \u0026gt;= 24: return booking.total_amount if hours_to_show \u0026gt;= 4: return booking.total_amount * 0.5 return 0.0 BookingService.cancel_booking invokes self._refund_policy.calculate_refund(...) to determine the refund amount, which is then included in the BookingCancelled event for downstream processing. The existing 2-hour cancellation cutoff logic is replaced by a check that the refund policy returns a non-zero amount, or the cutoff is removed entirely if partial cancellations are permitted at any time.\nThe changes are localised to BookingService.cancel_booking and the BookingCancelled event payload. All other components (lock manager, pricing, notifications) remain unchanged.\nThis content is part of Codeverra, a platform for learning coding, data science, DSA, and AI from scratch. Explore more at https://codeverra.com\n","permalink":"https://blogs.codeverra.com/blog/low_level_design/interview_questions/06h_lld_movie_booking_deep_dive/","summary":"\u003ch1 id=\"lld-deep-dive-4-designing-a-movie-ticket-booking-system-bookmyshow-style\"\u003eLLD Deep Dive #4: Designing a Movie Ticket Booking System (BookMyShow-style)\u003c/h1\u003e\n\u003cblockquote\u003e\n\u003cp\u003e\u003cstrong\u003ePurpose of this document.\u003c/strong\u003e A staged walkthrough of how to reason through a Movie Ticket Booking Low-Level Design (LLD) problem in an interview setting. Each design decision is presented alongside the alternatives that were considered and the rationale for the final choice. The resulting design is modular, extensible, and treats concurrency as a first-class concern rather than a post-hoc extension.\u003c/p\u003e","tags":null,"title":""},{"content":"LLD Deep Dive #4 — Designing a Text Editor What this document is: A genuine, step-by-step walkthrough of how to think through a Text Editor LLD problem in an interview. Not a finished design dropped on you — but the reasoning, the false starts, the alternatives weighed, and the justifications behind each design decision.\nTime investment: Read this slowly. If you\u0026rsquo;re doing this for interview prep, treat each stage as its own thinking exercise — pause and ask yourself \u0026ldquo;what would I do here?\u0026rdquo; before reading on.\nTable of Contents Why This Problem Matters Stage 1 — Receiving the Problem Stage 2 — Clarifying Requirements Stage 3 — Identifying Entities Stage 4 — The Text Buffer: Choosing a Data Structure Stage 5 — Cursor and Selection Stage 6 — Insert and Delete: The Core Operations Stage 7 — Undo/Redo: The Command Pattern Stage 8 — Clipboard: Cut, Copy, Paste Stage 9 — Find and Replace Stage 10 — Formatting: Bold, Italic, Rich Text Stage 11 — File I/O and Dirty State Stage 12 — View Notifications: The Observer Pattern Stage 13 — Putting It All Together Stage 14 — Validating with a Mental Walkthrough Stage 15 — Anticipating Follow-Up Questions Final Reflection — What This Problem Teaches Practice Questions 1. Why This Problem Matters The three previous deep dives centered on relationships (Parking Lot), state machines (Elevator), and policy (Library). The Text Editor is a different beast: it\u0026rsquo;s an operations-on-a-data-structure problem.\nWhat makes it interesting:\nData structure choice is design. The entire performance profile of your editor lives or dies on how you represent the text. Every design pattern is bolted on top of that one central decision. It\u0026rsquo;s the canonical Command pattern problem. Undo/redo isn\u0026rsquo;t just a feature — it\u0026rsquo;s the feature that reveals whether a candidate understands command objects. Interviewers love seeing this done well. Observer shows up naturally. The editor\u0026rsquo;s state changes; the view must re-render. This coupling problem practically screams for Observer. Rich text introduces a subtle modeling problem. How do you represent \u0026ldquo;characters 5 through 10 are bold\u0026rdquo;? The answer isn\u0026rsquo;t obvious, and there are multiple defensible approaches. It rewards restraint. You could design for collaborative editing, multi-cursor, syntax highlighting, autosave, macros — and lose the interview. Knowing what to leave out is half the skill. If Parking Lot is \u0026ldquo;can you model a domain,\u0026rdquo; Elevator is \u0026ldquo;can you handle state transitions,\u0026rdquo; and Library is \u0026ldquo;can you enforce policy,\u0026rdquo; then Text Editor is \u0026ldquo;can you pick the right data structure and pattern for a mutation-heavy system.\u0026rdquo;\nLet\u0026rsquo;s think through it.\n2. Stage 1 — Receiving the Problem The interviewer says:\n\u0026ldquo;Design a text editor.\u0026rdquo;\nThat\u0026rsquo;s the full prompt. Nothing more.\nSame trap as always: the vagueness is intentional. Let me list what\u0026rsquo;s undefined:\nConsole-based (like nano) or GUI-based (like VS Code)? Plain text (Notepad) or rich text (Word)? Single document or multiple tabs? What operations? Just type/delete? Undo? Find? Formatting? File loading and saving? Concurrency? Collaboration? Scale — 1KB documents or 100MB source files? If we design for the maximum interpretation (VS Code with collaboration and syntax highlighting), we\u0026rsquo;ll burn the entire interview on scope. If we design for the minimum (echo \u0026quot;hello\u0026quot; \u0026gt; file.txt), the interviewer will conclude we lack depth.\nNeither works. So we pause and ask.\nThe mental posture here: \u0026ldquo;They\u0026rsquo;re not testing whether I know the answer; they\u0026rsquo;re testing whether I can define the question.\u0026rdquo;\n3. Stage 2 — Clarifying Requirements Time to structure our clarifications. We do this in three buckets:\nFunctional requirements — what does it do? Non-functional requirements — what qualities does it have? Out of scope — what we\u0026rsquo;re not doing. Let\u0026rsquo;s walk through the questions we\u0026rsquo;d actually ask, and why each matters.\nQuestion 1: Plain text or rich text (with formatting)? Why ask: This is the biggest fork in the design. Rich text needs a way to store formatting attributes (bold, italic, color, font). Plain text is just characters. Choosing rich text triples the complexity of the buffer.\nSuppose the interviewer says: \u0026ldquo;Let\u0026rsquo;s support basic formatting — bold and italic — but keep it simple.\u0026rdquo;\nNow we know: rich text, but limited. We won\u0026rsquo;t need multi-font layout engines, but we need some way to associate style with ranges of characters.\nQuestion 2: Is undo/redo required? Why ask: Undo/redo is the single biggest architectural constraint on a text editor. If it\u0026rsquo;s required, every operation must be captured as a reversible command. That decision propagates everywhere.\nSuppose: \u0026ldquo;Yes — unlimited undo/redo.\u0026rdquo;\nWe\u0026rsquo;re committing to the Command pattern. Note this — it\u0026rsquo;s the spine of the design.\nQuestion 3: Cursor, selection, and clipboard operations? Why ask: These are the core interactive features. Without them, we\u0026rsquo;re just writing an append-only log.\nSuppose: \u0026ldquo;Yes to all — single cursor, single-range selection, cut/copy/paste.\u0026rdquo;\nWe\u0026rsquo;ll model a Cursor (a position) and a Selection (a range). Clipboard is a small standalone thing.\nQuestion 4: Find and replace? Why ask: Search complicates the design in interesting ways — how do we search efficiently, are matches case-sensitive, do we support regex? But it\u0026rsquo;s a bounded feature.\nSuppose: \u0026ldquo;Basic find and replace all — no regex.\u0026rdquo;\nSimple enough. We\u0026rsquo;ll add a small SearchService.\nQuestion 5: File load/save? Why ask: Persistence layer. Whether we support it affects our design (e.g., we need a \u0026ldquo;dirty\u0026rdquo; flag) but its actual implementation is usually trivial.\nSuppose: \u0026ldquo;Yes — load from a file path, save to a file path. Track whether the document has unsaved changes.\u0026rdquo;\nNote the \u0026ldquo;dirty\u0026rdquo; state requirement.\nQuestion 6: Multiple documents / tabs? Why ask: A multi-document editor is essentially N single-document editors + a coordinator. It\u0026rsquo;s easy to add if the single-document design is clean.\nSuppose: \u0026ldquo;Single document is fine for now — but design so multi-document is a natural extension.\u0026rdquo;\nGood — no immediate work needed, but we\u0026rsquo;ll keep our Editor non-singleton and non-global.\nQuestion 7: GUI or CLI? Why ask: Because they may confuse an interview about UI plumbing with an LLD interview.\nSuppose: \u0026ldquo;Ignore the UI. Assume something calls into your editor. We want the core model, not the view.\u0026rdquo;\nPerfect — no UI to worry about. But we still model a View abstraction, so a real UI can attach as an observer.\nQuestion 8: Scale? Why ask: A 1MB text file behaves totally differently from a 100MB log file. Data structure choice depends on this.\nSuppose: \u0026ldquo;Assume documents up to a few megabytes — a large source file, not an entire book. Insertions and deletions should be efficient.\u0026rdquo;\nThat eliminates naive whole-buffer copies. We need a proper text buffer, not just a Python string.\nQuestion 9: Concurrency? Collaboration? Why ask: Collaborative editing (Google Docs style) is dramatically different from local editing. It needs CRDTs or operational transforms. We want to bound this out.\nSuppose: \u0026ldquo;Single user. No collaboration. Single-threaded.\u0026rdquo;\nBig relief. We\u0026rsquo;ll mention collaboration as an extension only.\nRecap to the Interviewer Now we play back:\n\u0026ldquo;To confirm: a single-user text editor with rich text (bold/italic), full undo/redo, single-cursor with selection, clipboard (cut/copy/paste), find and replace all (no regex), file load/save with dirty tracking, single document (extensible to multi), no UI (headless model), documents of a few megabytes, single-threaded. Correct?\u0026rdquo;\nIf the interviewer nods, we\u0026rsquo;re aligned. If not, they adjust and we adjust. Either way — we don\u0026rsquo;t write a class until this handshake is done.\n4. Stage 3 — Identifying Entities Nouns from the requirements:\ntext / document character cursor selection command / operation undo / redo (history) clipboard editor view search file formatting / style user Now we filter using the three-question test:\nDoes it have identity? Does it have behavior or state? Is it part of the domain or generic infrastructure? Noun Identity? Behavior/State? Verdict document / text buffer Yes Yes (store text, expose operations) Class character No Just a value Reject (primitive) cursor Yes Position, movement Class (small) selection Yes Range Class (small) command Yes (many) execute/undo Class hierarchy history Yes Two stacks Class clipboard Yes Holds content Class (small) editor Yes Orchestrates everything Class view External-ish Reacts to state changes Interface (Observer) search No identity Algorithm Strategy file External I/O Service or method formatting / style Yes (spans) Associates ranges with attributes Class user External actor We don\u0026rsquo;t model them Not a class Rejected:\ncharacter — it\u0026rsquo;s just str[i]. Making a Character class would be Java-ism, not design. user — no login, no accounts, no per-user state. External actor. Our skeleton:\nTextBuffer, Cursor, Selection, Command (+ subclasses), History, Clipboard, Editor, View (interface), FormattingSpans, SearchService Plus one Strategy: SearchStrategy (if we want swappable search algorithms).\nNow we design each piece — and we do it in an order that matters. The text buffer comes first, because everything else depends on it.\n5. Stage 4 — The Text Buffer: Choosing a Data Structure This is the pivotal decision. Let me lay out the alternatives before picking.\nThe Naive Approach: A Python String class TextBuffer: def __init__(self): self.text = \u0026#34;\u0026#34; def insert(self, pos, s): self.text = self.text[:pos] + s + self.text[pos:] def delete(self, start, end): self.text = self.text[:start] + self.text[end:] How it works: Concatenate slices to insert; concatenate slices to delete.\nAnalogy: It\u0026rsquo;s like editing an essay by rewriting the entire essay every time you add a comma.\nCost: Every insert or delete is O(n) — you copy the whole buffer. For a 5MB document, every keystroke rewrites 5MB. Absolutely fine for tiny files. Unusable for a real editor.\nReject. But note it — this is the baseline against which better approaches justify themselves.\nApproach A: List of Characters class TextBuffer: def __init__(self): self.chars = [] # list of str characters Better? In some ways — Python lists have amortized O(1) append. But list.insert(pos, ch) is still O(n) because everything after pos shifts. Same problem, slightly better constants.\nReject for the same reason.\nApproach B: List of Lines class TextBuffer: def __init__(self): self.lines = [\u0026#34;\u0026#34;] # a list of strings, one per line Insert: Find the line, splice within it. O(line_length + line_lookup). Delete: Similar.\nPros: Natural for line-oriented tasks — go to line N, render line N, etc. Editors that surface line numbers (basically all of them) benefit.\nCons: Insertions within a very long line are still O(line_length). If someone pastes a 1MB block onto one line, we\u0026rsquo;re back to slow. Multi-line paste requires splicing lines.\nReal-world usage: Historically common. Simple and effective for well-formatted code files (source lines are usually short).\nNot bad. But we can do better on worst case.\nApproach C: Gap Buffer Analogy: Imagine a whiteboard where you keep writing left to right, but sometimes you insert in the middle. Instead of rewriting everything to the right when you insert, you keep a gap — an empty region you can fill. When the cursor moves, you shift text across the gap to keep the gap centered where you\u0026rsquo;re editing.\nBefore: [H][e][l][l][o]____________[W][o][r][l][d] ^ gap starts ^ gap ends ^ cursor position Type \u0026#39;X\u0026#39; → [H][e][l][l][o][X]__________[W][o][r][l][d] ^ gap shrinks by one Complexity:\nInsert at cursor: O(1) amortized (just write into the gap; grow the gap when full). Delete at cursor: O(1) (expand the gap). Move cursor: O(distance moved) — you shift characters across the gap. Random-access lookup: O(1) via index arithmetic. Editors that use gap buffers: Emacs, historically vim. It\u0026rsquo;s beautifully suited to editing where the cursor stays roughly in one region.\nCons: When the cursor jumps far, we pay O(distance). Real users mostly edit near where they just typed, so this is rare — but not free.\nApproach D: Piece Table Analogy: Think of a book editor working with the original manuscript and a stack of correction slips. The final \u0026ldquo;text\u0026rdquo; is a sequence of pointers: \u0026ldquo;read from original page 3, then insert this correction slip, then continue from page 4.\u0026rdquo; You never modify the original — you just build a list of pieces.\noriginal: \u0026#34;the quick brown fox\u0026#34; add: \u0026#34;very \u0026#34; pieces: [ (original, 0, 10), (add, 0, 5), (original, 10, 19) ] = \u0026#34;the quick very brown fox\u0026#34; Complexity:\nInsert: O(1) to add a piece (plus splitting existing piece — a small local op). Delete: Similar; may need to trim or drop pieces. Lookup at index: O(pieces) unless you index the pieces (piece tree makes it O(log n)). Real-world usage: VS Code uses a piece tree. Microsoft Word famously used a piece table. It\u0026rsquo;s ideal for undo-heavy systems because old buffers are immutable — you can reconstruct any prior state cheaply.\nCons: More complex to implement. Reads (extracting substring) require walking pieces.\nApproach E: Rope Analogy: A binary tree of string chunks. Concatenation is O(log n) — you just make a new root. Splitting is O(log n). Great for very large documents.\nReal-world usage: Some IDEs and specialty tools. Overkill for most.\nThe Decision The interviewer said \u0026ldquo;up to a few megabytes.\u0026rdquo; That rules out the naive approaches, and it doesn\u0026rsquo;t require the exotic ones. Any of B / C / D would work.\nFor teaching purposes and interview clarity, I\u0026rsquo;ll go with a gap buffer. Reasons:\nConceptually clean — one array, one gap, three pointers (start, gap_start, gap_end). The performance argument is easy to state: \u0026ldquo;insertions near the cursor are O(1); we pay only when the cursor jumps.\u0026rdquo; It\u0026rsquo;s a well-known editor data structure — showing you know it signals depth without going full academic. Simpler code than a piece table, but a real improvement over \u0026ldquo;just use a string.\u0026rdquo; Piece table would be a defensible answer too. What matters isn\u0026rsquo;t which one — it\u0026rsquo;s that you can articulate why one wins over another for this workload.\nIn interview I\u0026rsquo;d say:\n\u0026ldquo;For a few-MB document with typical editing patterns (cursor edits near a working region), a gap buffer gives O(1) insertion and deletion at the cursor with simple code. A piece table would also work well and has the nice property of immutability for undo, but adds implementation complexity. A plain Python string is O(n) per keystroke — unacceptable for MB-sized files.\u0026rdquo;\nThe Code class GapBuffer: \u0026#34;\u0026#34;\u0026#34; A gap buffer text storage. Internally, we hold a single character array. Somewhere inside it lives a \u0026#34;gap\u0026#34; — an unused region we treat as free space. The cursor is always positioned at the start of the gap. - Characters before the gap are the text to the LEFT of the cursor. - Characters after the gap are the text to the RIGHT of the cursor. - The gap itself is invisible to the outside world. Why this design? - Inserting a character is just: write into the gap, shrink the gap by one. O(1) amortized. - Deleting a character (backspace) is just: grow the gap by one. O(1). No shifting. - Moving the cursor by K positions requires shifting K characters across the gap. O(K), but K is usually small in practice. Alternatives considered and rejected: - Naive Python string: O(n) per keystroke; unusable for MB files. - Piece table: better for very large docs and cheap undo history, but more complex code. Overkill here. - Rope: only useful for enormous documents. \u0026#34;\u0026#34;\u0026#34; INITIAL_GAP = 64 # size of gap when the buffer is created def __init__(self, initial_text: str = \u0026#34;\u0026#34;): # We store a Python list because Python strings are immutable. # A list of single-char strings gives us mutable random access. n = len(initial_text) # Layout: [initial_text chars ...][gap of INITIAL_GAP slots] # gap_start is where the gap begins; gap_end is one past its last slot. # Initially, cursor is at end of initial_text. self._buf = list(initial_text) + [\u0026#34;\u0026#34;] * self.INITIAL_GAP self._gap_start = n self._gap_end = n + self.INITIAL_GAP # ---- helpers ---- @property def size(self) -\u0026gt; int: \u0026#34;\u0026#34;\u0026#34;Number of actual characters (excluding the gap).\u0026#34;\u0026#34;\u0026#34; return len(self._buf) - (self._gap_end - self._gap_start) def _gap_size(self) -\u0026gt; int: return self._gap_end - self._gap_start def _grow_gap(self, min_extra: int) -\u0026gt; None: \u0026#34;\u0026#34;\u0026#34; Ensure at least `min_extra` free slots in the gap. Doubles the gap size (a common growth strategy — amortized O(1)). \u0026#34;\u0026#34;\u0026#34; needed = max(min_extra, self._gap_size() * 2 or self.INITIAL_GAP) # Insert `needed` empty slots at gap_end. self._buf[self._gap_end:self._gap_end] = [\u0026#34;\u0026#34;] * needed self._gap_end += needed def _move_gap_to(self, pos: int) -\u0026gt; None: \u0026#34;\u0026#34;\u0026#34; Move the gap so it starts at `pos`. This is how the cursor moves. Shifts characters across the gap — cost O(|pos - current_gap_start|). \u0026#34;\u0026#34;\u0026#34; if pos \u0026lt; 0 or pos \u0026gt; self.size: raise IndexError(f\u0026#34;position {pos} out of range [0, {self.size}]\u0026#34;) if pos \u0026lt; self._gap_start: # Moving gap left: shift chars from left-of-gap to right-of-gap. shift = self._gap_start - pos self._buf[self._gap_end - shift:self._gap_end] = \\ self._buf[pos:self._gap_start] self._gap_start -= shift self._gap_end -= shift elif pos \u0026gt; self._gap_start: # Moving gap right: shift chars from right-of-gap to left-of-gap. shift = pos - self._gap_start self._buf[self._gap_start:self._gap_start + shift] = \\ self._buf[self._gap_end:self._gap_end + shift] self._gap_start += shift self._gap_end += shift # If pos == gap_start, nothing to do. # ---- public API ---- def insert(self, pos: int, text: str) -\u0026gt; None: \u0026#34;\u0026#34;\u0026#34;Insert `text` at position `pos`.\u0026#34;\u0026#34;\u0026#34; self._move_gap_to(pos) if len(text) \u0026gt; self._gap_size(): self._grow_gap(len(text)) for ch in text: self._buf[self._gap_start] = ch self._gap_start += 1 def delete(self, start: int, end: int) -\u0026gt; str: \u0026#34;\u0026#34;\u0026#34; Delete characters in [start, end). Return the deleted text (needed by undo commands to know what to restore). \u0026#34;\u0026#34;\u0026#34; if not (0 \u0026lt;= start \u0026lt;= end \u0026lt;= self.size): raise IndexError(f\u0026#34;invalid delete range [{start}, {end})\u0026#34;) # Extract the doomed text before we lose access to it. deleted = self.get_text(start, end) # Position the gap at `start`, then expand it to swallow `end - start`. self._move_gap_to(start) self._gap_end += (end - start) return deleted def get_text(self, start: int = 0, end: int | None = None) -\u0026gt; str: \u0026#34;\u0026#34;\u0026#34;Return the text in [start, end).\u0026#34;\u0026#34;\u0026#34; if end is None: end = self.size if not (0 \u0026lt;= start \u0026lt;= end \u0026lt;= self.size): raise IndexError(f\u0026#34;invalid range [{start}, {end})\u0026#34;) # Convert logical positions to physical buffer positions, skipping the gap. def phys(p: int) -\u0026gt; int: return p if p \u0026lt; self._gap_start else p + (self._gap_end - self._gap_start) result = [] for p in range(start, end): result.append(self._buf[phys(p)]) return \u0026#34;\u0026#34;.join(result) def __len__(self): return self.size def __str__(self): return self.get_text() Notice a few pedagogical touches:\ndelete returns the deleted text. Not because \u0026ldquo;return values are good practice\u0026rdquo; — but because undo needs it. If you delete \u0026ldquo;World\u0026rdquo; and later undo, you must know it was \u0026ldquo;World\u0026rdquo; you deleted. Baking this into the API from day one prevents bolt-on rework later. Insertion errors, deletion errors — we raise, not return False. Same reasoning as the parking-lot spot: an out-of-range delete is a caller bug. We didn\u0026rsquo;t optimize get_text for large substrings. The per-character loop is O(n) but has clarity. A production version would slice-copy in chunks. That\u0026rsquo;s the buffer. Everything from here builds on it.\n6. Stage 5 — Cursor and Selection The cursor is a position — an integer offset into the buffer. The selection is a range: start and end offsets.\nDecision Point 1: Store cursor on buffer or separately? Option A: TextBuffer.cursor — the buffer tracks its own cursor. Option B: Cursor is a separate object, held by the editor, that references the buffer.\nI lean B because:\nA buffer with no cursor is more reusable (e.g., could be used for a text-processing pipeline where \u0026ldquo;cursor\u0026rdquo; makes no sense). Multi-cursor is an easier future extension when cursor isn\u0026rsquo;t baked in. Selection is a range, not a point — it can\u0026rsquo;t sensibly live \u0026ldquo;on\u0026rdquo; the buffer either. The gap buffer does internally know where the gap is (which happens to be the cursor position in most operations). But that\u0026rsquo;s an implementation detail. The public API takes explicit positions.\nDecision Point 2: What happens to the cursor when text is inserted before it? Two options:\nCursor is a \u0026ldquo;sticky\u0026rdquo; position. Insert 5 chars before the cursor → cursor stays where it is logically (i.e., the same character is still to its right), so its offset increases by 5. Cursor is a \u0026ldquo;raw\u0026rdquo; offset. Insert 5 chars before the cursor → cursor stays at the same offset, so it\u0026rsquo;s now 5 characters earlier in the visible text. Practically every editor uses the sticky model. Users expect that if they type at position 10, the cursor advances to position 11, not stays at 10.\nDecision: sticky. This means every insert/delete operation must also adjust the cursor.\nThe Code class Cursor: \u0026#34;\u0026#34;\u0026#34; Represents a single cursor position in the document. A cursor is just an integer offset — the position between two characters (or before the first / after the last). It\u0026#39;s \u0026#34;sticky\u0026#34;: when text is inserted before the cursor, the cursor moves forward. Why not just use an int on the Editor? - Wrapping it in a class lets us attach behavior (e.g., `move_to`) and validate. - Makes future multi-cursor support natural (Editor could hold a list of Cursor objects). \u0026#34;\u0026#34;\u0026#34; def __init__(self, position: int = 0): self._pos = position @property def position(self) -\u0026gt; int: return self._pos def move_to(self, pos: int, max_pos: int) -\u0026gt; None: \u0026#34;\u0026#34;\u0026#34; Move cursor to `pos`, clamped to [0, max_pos]. `max_pos` is the current buffer size — the caller knows this. \u0026#34;\u0026#34;\u0026#34; self._pos = max(0, min(pos, max_pos)) def shift(self, delta: int, max_pos: int) -\u0026gt; None: \u0026#34;\u0026#34;\u0026#34;Move cursor by `delta` (positive or negative), clamped.\u0026#34;\u0026#34;\u0026#34; self.move_to(self._pos + delta, max_pos) def __repr__(self): return f\u0026#34;Cursor(pos={self._pos})\u0026#34; class Selection: \u0026#34;\u0026#34;\u0026#34; A range [start, end) of characters currently selected. An empty selection (start == end) means \u0026#34;no selection\u0026#34; — many editors treat this as identical to just having a cursor at that position. Invariant: start \u0026lt;= end. The `anchor` (the fixed end when the user is drag-selecting) is not modeled here; we simplify to just the range. \u0026#34;\u0026#34;\u0026#34; def __init__(self, start: int = 0, end: int = 0): if start \u0026gt; end: start, end = end, start self.start = start self.end = end @property def is_empty(self) -\u0026gt; bool: return self.start == self.end @property def length(self) -\u0026gt; int: return self.end - self.start def clear(self) -\u0026gt; None: \u0026#34;\u0026#34;\u0026#34;Reset to empty (used after operations consume the selection).\u0026#34;\u0026#34;\u0026#34; self.end = self.start def __repr__(self): return f\u0026#34;Selection({self.start}, {self.end})\u0026#34; Simple, focused. The cursor and selection do not know about the buffer — the editor coordinates them. This will pay off when we bring in commands.\n7. Stage 6 — Insert and Delete: The Core Operations Now we start assembling the Editor. Its first two responsibilities: insert text at the cursor, and delete either a character (backspace) or the current selection.\nDecision Point 1: Should Editor call buffer directly, or go through a Command? Option A: Editor.insert(text) → directly calls buffer.insert(...). Option B: Editor.insert(text) → creates an InsertCommand, calls execute().\nSince we need undo/redo, B is inevitable. Every mutation must become a Command so it can be pushed onto the undo stack.\nWe\u0026rsquo;ll design commands next — but for now, our Editor\u0026rsquo;s public API should look like the user\u0026rsquo;s API (insert, delete, move_cursor), and internally it wraps each in a Command.\nDecision Point 2: What if a selection is active when the user types? Universal editor behavior: typing replaces the selection. So insert(\u0026quot;x\u0026quot;) when the selection covers [5, 10) should delete [5, 10) first, then insert \u0026quot;x\u0026quot; at position 5. That\u0026rsquo;s two operations — but from the user\u0026rsquo;s undo perspective, it\u0026rsquo;s one.\nThis foreshadows the need for composite commands — a single command that groups multiple sub-commands, executed and undone atomically.\nWe\u0026rsquo;re getting ahead of ourselves. Let\u0026rsquo;s design commands.\n8. Stage 7 — Undo/Redo: The Command Pattern This is the design lynchpin of the entire editor. Get this right and everything falls into place.\nWhat is the Command pattern? Analogy: Imagine a restaurant. When you order, the waiter doesn\u0026rsquo;t yell \u0026ldquo;make a pizza!\u0026rdquo; at the chef — they write a ticket and pin it to the wheel. The ticket is a self-contained object: it knows what was ordered, when, for whom. It can be re-fired if the food burns, canceled, or filed for the night\u0026rsquo;s records.\nA Command object is the ticket. It encapsulates a request as a first-class object: it knows what to do, how to undo, and what data it needs to do both.\nStructure Each command implements:\nexecute() — perform the action. undo() — reverse it. The Editor keeps two stacks:\nundo_stack — commands that have been executed. redo_stack — commands that have been undone (available to redo). Rules:\nRunning a new command → push to undo stack, clear the redo stack (because the redo history is now invalid — the user branched). undo() → pop from undo stack, call undo(), push to redo stack. redo() → pop from redo stack, call execute(), push to undo stack. Why can\u0026rsquo;t we just store snapshots (Memento) for undo? Because for a 5MB file, storing a snapshot after every keystroke would eat gigabytes. Commands store only the delta — \u0026ldquo;you inserted \u0026lsquo;x\u0026rsquo; at position 42\u0026rdquo; — which is tiny. Snapshots are used for coarser-grained undo (e.g., \u0026ldquo;revert to save point\u0026rdquo;); commands are used for fine-grained undo.\nThe Command Interface from abc import ABC, abstractmethod class Command(ABC): \u0026#34;\u0026#34;\u0026#34; A reversible operation on the editor. Each Command knows: - How to execute itself (state change). - How to undo itself (revert that change). We pass the editor into execute/undo rather than storing it as a field for two reasons: 1. Commands become reusable across editors (unusual but conceptually clean). 2. It\u0026#39;s clearer at call sites where the mutation happens. Some frameworks also add a `redo()` method distinct from `execute()`. We don\u0026#39;t — for our operations, redo IS execute. If a command needed different logic for redo (rare), it could override. \u0026#34;\u0026#34;\u0026#34; @abstractmethod def execute(self, editor: \u0026#34;Editor\u0026#34;) -\u0026gt; None: ... @abstractmethod def undo(self, editor: \u0026#34;Editor\u0026#34;) -\u0026gt; None: ... Concrete Commands class InsertCommand(Command): \u0026#34;\u0026#34;\u0026#34; Insert `text` at `position`. Undo: delete [position, position + len(text)). We record `position` at execute time. The cursor is NOT captured inside the command — the editor manages the cursor as a consequence of the mutation. If we captured cursor here, undo would need to restore both text and cursor state, coupling the two concerns. \u0026#34;\u0026#34;\u0026#34; def __init__(self, position: int, text: str): self.position = position self.text = text def execute(self, editor: \u0026#34;Editor\u0026#34;) -\u0026gt; None: editor._buffer.insert(self.position, self.text) # Move cursor to just after the inserted text. editor._cursor.move_to(self.position + len(self.text), len(editor._buffer)) def undo(self, editor: \u0026#34;Editor\u0026#34;) -\u0026gt; None: editor._buffer.delete(self.position, self.position + len(self.text)) editor._cursor.move_to(self.position, len(editor._buffer)) class DeleteCommand(Command): \u0026#34;\u0026#34;\u0026#34; Delete text in [start, end). Undo: reinsert the deleted text at `start`. We capture the deleted text during execute so undo has something to reinsert. Design note: We don\u0026#39;t take `text` in the constructor because at construction time the caller doesn\u0026#39;t know the exact content — the buffer does. So we store `deleted_text` as internal state set during execute(). This makes the command STATEFUL (execute must be called before undo can work), which is fine because commands always execute at least once before being undoable. \u0026#34;\u0026#34;\u0026#34; def __init__(self, start: int, end: int): self.start = start self.end = end self._deleted_text: str = \u0026#34;\u0026#34; def execute(self, editor: \u0026#34;Editor\u0026#34;) -\u0026gt; None: self._deleted_text = editor._buffer.delete(self.start, self.end) editor._cursor.move_to(self.start, len(editor._buffer)) def undo(self, editor: \u0026#34;Editor\u0026#34;) -\u0026gt; None: editor._buffer.insert(self.start, self._deleted_text) editor._cursor.move_to(self.end, len(editor._buffer)) class CompositeCommand(Command): \u0026#34;\u0026#34;\u0026#34; A command composed of multiple sub-commands, treated as one unit. Used when: - User types with a selection active → delete selection + insert (one undo). - Paste over selection → delete + insert. - Replace-all → many delete+insert pairs, one undo. Undo runs sub-commands in REVERSE order — this is the general rule for undoing composed operations. This is the Composite pattern applied to Commands. \u0026#34;\u0026#34;\u0026#34; def __init__(self, commands: list[Command]): self.commands = commands def execute(self, editor: \u0026#34;Editor\u0026#34;) -\u0026gt; None: for cmd in self.commands: cmd.execute(editor) def undo(self, editor: \u0026#34;Editor\u0026#34;) -\u0026gt; None: for cmd in reversed(self.commands): cmd.undo(editor) The History class History: \u0026#34;\u0026#34;\u0026#34; Two-stack undo/redo manager. Invariants: - `undo_stack` is the sequence of commands that have been executed (and not undone) since some baseline. Top of stack is most recent. - `redo_stack` is the sequence of commands that have been undone (and not re-executed). When a NEW command is executed, redo_stack is cleared. This is the standard \u0026#34;branching invalidates future history\u0026#34; behavior — the user has diverged from their previous undo path. \u0026#34;\u0026#34;\u0026#34; def __init__(self, max_size: int | None = None): # max_size lets us bound memory. None = unlimited. self._undo_stack: list[Command] = [] self._redo_stack: list[Command] = [] self._max_size = max_size def record(self, cmd: Command) -\u0026gt; None: \u0026#34;\u0026#34;\u0026#34;Called after a new command has been executed.\u0026#34;\u0026#34;\u0026#34; self._undo_stack.append(cmd) # New action invalidates the redo path. self._redo_stack.clear() # Bound the stack size, dropping oldest. if self._max_size is not None and len(self._undo_stack) \u0026gt; self._max_size: self._undo_stack.pop(0) def can_undo(self) -\u0026gt; bool: return bool(self._undo_stack) def can_redo(self) -\u0026gt; bool: return bool(self._redo_stack) def pop_undo(self) -\u0026gt; Command: return self._undo_stack.pop() def push_redo(self, cmd: Command) -\u0026gt; None: self._redo_stack.append(cmd) def pop_redo(self) -\u0026gt; Command: return self._redo_stack.pop() def push_undo(self, cmd: Command) -\u0026gt; None: self._undo_stack.append(cmd) The History doesn\u0026rsquo;t do the executing itself — the Editor does. History just tracks. This split matters because it keeps history dumb: it doesn\u0026rsquo;t need to know what a command does.\n9. Stage 8 — Clipboard: Cut, Copy, Paste The clipboard is tiny. It holds text. Three operations use it:\nCopy: read selected text, store in clipboard. Buffer unchanged. Not undoable. Cut: read selected text, store in clipboard, delete selection. Undoable. Paste: insert clipboard content at cursor (replacing selection if active). Undoable. Decision Point: Clipboard as separate class or inline in Editor? Very small chunk of state — arguable. But we separate it because:\nIt might be system-wide (a real OS clipboard is shared across apps). Abstraction makes future swap-in trivial. It has behavior (get, set, has_content) — enough to earn a class. The Code class Clipboard: \u0026#34;\u0026#34;\u0026#34; Holds the current clipboard content (a plain string). In a real system this might wrap the OS clipboard via a platform API; here it\u0026#39;s an in-memory store. Keeping it as an interface (with a single default implementation) means we can plug in an OSClipboard later without changing the Editor. We don\u0026#39;t currently store rich-text formatting on paste. That\u0026#39;s a conscious simplification — a real editor\u0026#39;s clipboard is a bag of typed payloads (plain text, rich text, image, ...). Extension point. \u0026#34;\u0026#34;\u0026#34; def __init__(self): self._content: str = \u0026#34;\u0026#34; def set(self, text: str) -\u0026gt; None: self._content = text def get(self) -\u0026gt; str: return self._content def has_content(self) -\u0026gt; bool: return bool(self._content) The Cut/Paste operations use commands we\u0026rsquo;ve already built. We don\u0026rsquo;t need new command classes — cut is DeleteCommand + a clipboard write, paste is InsertCommand, cut-over-selection is CompositeCommand([DeleteCommand, ...]). The clipboard write itself is not undoable (universal editor convention — undo doesn\u0026rsquo;t restore the clipboard).\nWe\u0026rsquo;ll wire it into the Editor shortly.\n10. Stage 9 — Find and Replace Requirements recap: basic find, replace all. No regex. Find returns a list of positions. Replace all replaces every occurrence and must be one undo step (that\u0026rsquo;s the user expectation).\nDecision Point 1: Where does search live? On the buffer — buffer.find(pattern)? On the editor — editor.find(pattern)? A separate SearchService? The buffer is a low-level storage. Search is a text-processing algorithm. They should be separate. Going with a SearchService.\nDecision Point 2: Strategy pattern for search algorithm? For plain literal search on small docs, str.find in a loop is fine. But interviewers appreciate seeing an interface that admits multiple algorithms:\nLiteral (case-sensitive) Case-insensitive Regex (extension) Whole word (extension) That\u0026rsquo;s a natural Strategy pattern.\nEven if we implement only one now, sketching the interface is cheap and shows extensibility.\nThe Code import re class SearchStrategy(ABC): \u0026#34;\u0026#34;\u0026#34; Given a text and a query, return all match positions. A position is (start, end). We return end so that replace-all knows how much to delete (queries can be different lengths from replacements, and case-insensitive search may match different-cased text). Why return positions instead of doing the whole replace here? - Separation of concerns: search algorithm knows text, not how to mutate. - The Editor turns positions into commands, wrapping them for undo. \u0026#34;\u0026#34;\u0026#34; @abstractmethod def find_all(self, text: str, query: str) -\u0026gt; list[tuple[int, int]]: ... class LiteralSearch(SearchStrategy): def find_all(self, text, query): if not query: return [] positions = [] i = 0 while True: idx = text.find(query, i) if idx == -1: break positions.append((idx, idx + len(query))) i = idx + len(query) # skip past this match (non-overlapping) return positions class CaseInsensitiveSearch(SearchStrategy): def find_all(self, text, query): if not query: return [] # Use regex with IGNORECASE to correctly get match lengths. # (We can\u0026#39;t just lowercase both — locale-specific edge cases with # Unicode ß → SS mess up index arithmetic.) return [(m.start(), m.end()) for m in re.finditer(re.escape(query), text, re.IGNORECASE)] class SearchService: \u0026#34;\u0026#34;\u0026#34; A small orchestrator. Holds a strategy; can be swapped at runtime. \u0026#34;\u0026#34;\u0026#34; def __init__(self, strategy: SearchStrategy | None = None): self._strategy = strategy or LiteralSearch() def set_strategy(self, strategy: SearchStrategy) -\u0026gt; None: self._strategy = strategy def find(self, text: str, query: str) -\u0026gt; list[tuple[int, int]]: return self._strategy.find_all(text, query) Replace-All as a Single Undo Step The user hits \u0026ldquo;Replace All\u0026rdquo; once. If they undo, they expect every replacement to be undone in one step — not one-by-one.\nSolution: build a CompositeCommand containing all the individual delete+insert pairs, execute it once, and push a single entry onto the undo stack.\nSubtle detail: replacing changes indices. If we replace \u0026quot;foo\u0026quot; (3 chars) with \u0026quot;bar-baz\u0026quot; (7 chars), every subsequent match\u0026rsquo;s index shifts by +4. To avoid this bookkeeping, we go right-to-left: process matches from the end of the document backward. Positions to the left are unaffected by changes to the right.\nThis will be a method on the Editor. We\u0026rsquo;ll show it in the \u0026ldquo;putting it all together\u0026rdquo; stage.\n11. Stage 10 — Formatting: Bold, Italic, Rich Text Now we enter the trickier part. How do you represent \u0026ldquo;characters 5 through 10 are bold and characters 8 through 15 are italic\u0026rdquo;?\nApproach A: Character-Level Attributes Every character carries its own attributes:\n[ (\u0026#34;H\u0026#34;, set()), (\u0026#34;e\u0026#34;, set()), (\u0026#34;l\u0026#34;, {\u0026#34;bold\u0026#34;}), (\u0026#34;l\u0026#34;, {\u0026#34;bold\u0026#34;}), (\u0026#34;o\u0026#34;, {\u0026#34;bold\u0026#34;}), ] Pros: Simple to query — what\u0026rsquo;s the style of character 5? Look at character 5. Cons: Bloats memory 5–10x. Every attribute change touches every character in the range. Hard to serialize efficiently.\nApproach B: Attribute Spans (Range-Based) Store attributes as ranges:\nspans = [ Span(start=2, end=5, attrs={\u0026#34;bold\u0026#34;}), Span(start=4, end=8, attrs={\u0026#34;italic\u0026#34;}), ] Pros: Compact. Natural for range operations (\u0026ldquo;make [5, 10) bold\u0026rdquo;). Cons: Complex to maintain — when you insert or delete text in the buffer, existing spans must be shifted / split / merged. When you apply overlapping formatting, spans must be split.\nReal-world editors (Word, VS Code) essentially use span-based models with careful bookkeeping.\nDecision Approach B. For the interview, we\u0026rsquo;ll implement the interface cleanly and describe the maintenance rules; a fully bulletproof implementation would be its own hour.\nThe Model from dataclasses import dataclass, field @dataclass class FormatSpan: \u0026#34;\u0026#34;\u0026#34; A range [start, end) with a set of formatting attributes. Attributes are stored as a set of string labels (\u0026#34;bold\u0026#34;, \u0026#34;italic\u0026#34;). A future extension could carry attribute *values* (e.g., color=\u0026#34;#FF0000\u0026#34;). \u0026#34;\u0026#34;\u0026#34; start: int end: int attrs: set[str] = field(default_factory=set) def contains(self, pos: int) -\u0026gt; bool: return self.start \u0026lt;= pos \u0026lt; self.end class FormattingSpans: \u0026#34;\u0026#34;\u0026#34; Manages the collection of format spans over the document. Invariants (we won\u0026#39;t enforce these strictly here, but they matter): - Spans may overlap (bold and italic can coexist on the same range). - Two spans with identical attrs may be merged for compactness — but we don\u0026#39;t do this for pedagogical simplicity. When the buffer changes: - insert at pos: shift all spans with start \u0026gt;= pos by +len(text); for a span that STRADDLES pos, extend its end by +len(text) (the inserted text inherits the formatting of the surrounding span). - delete [s, e): remove or trim spans overlapping the deletion. This module is the source of ALL span bookkeeping. The Editor calls into it whenever text changes. \u0026#34;\u0026#34;\u0026#34; def __init__(self): self._spans: list[FormatSpan] = [] def apply(self, start: int, end: int, attr: str) -\u0026gt; None: \u0026#34;\u0026#34;\u0026#34;Add an attribute to the range [start, end).\u0026#34;\u0026#34;\u0026#34; self._spans.append(FormatSpan(start, end, {attr})) def remove(self, start: int, end: int, attr: str) -\u0026gt; None: \u0026#34;\u0026#34;\u0026#34; Remove the attribute from any span overlapping [start, end). This may require splitting spans — a span that partially overlaps must be split into overlapping and non-overlapping parts. We keep the implementation compact by rebuilding spans: for each existing span, produce 0-3 replacement spans as needed. \u0026#34;\u0026#34;\u0026#34; new_spans: list[FormatSpan] = [] for span in self._spans: if attr not in span.attrs or span.end \u0026lt;= start or span.start \u0026gt;= end: # No overlap or attribute not present here — keep unchanged. new_spans.append(span) continue # There is overlap and the attribute is present. # Emit up to three pieces: pre-overlap, overlap-without-attr, post-overlap. if span.start \u0026lt; start: new_spans.append(FormatSpan(span.start, start, set(span.attrs))) # Overlap piece: keep other attrs; drop the removed one. overlap_attrs = set(span.attrs) - {attr} if overlap_attrs: new_spans.append(FormatSpan(max(span.start, start), min(span.end, end), overlap_attrs)) if span.end \u0026gt; end: new_spans.append(FormatSpan(end, span.end, set(span.attrs))) self._spans = new_spans def attrs_at(self, pos: int) -\u0026gt; set[str]: \u0026#34;\u0026#34;\u0026#34;All attributes active at `pos`.\u0026#34;\u0026#34;\u0026#34; result: set[str] = set() for span in self._spans: if span.contains(pos): result |= span.attrs return result def on_insert(self, pos: int, length: int) -\u0026gt; None: \u0026#34;\u0026#34;\u0026#34; Called after `length` chars are inserted at `pos`. Spans starting at or after `pos` shift right by `length`. Spans straddling `pos` grow their end by `length`. \u0026#34;\u0026#34;\u0026#34; for span in self._spans: if span.start \u0026gt;= pos: span.start += length span.end += length elif span.end \u0026gt; pos: # span straddles the insertion point span.end += length def on_delete(self, start: int, end: int) -\u0026gt; None: \u0026#34;\u0026#34;\u0026#34; Called after chars in [start, end) are deleted. Adjust spans: shift, trim, or drop. \u0026#34;\u0026#34;\u0026#34; length = end - start surviving: list[FormatSpan] = [] for span in self._spans: if span.end \u0026lt;= start: # Wholly before deletion — unchanged. surviving.append(span) elif span.start \u0026gt;= end: # Wholly after deletion — shift left. span.start -= length span.end -= length surviving.append(span) else: # Overlaps the deletion; clip. new_start = min(span.start, start) new_end = max(span.end - length, new_start) if new_end \u0026gt; new_start: surviving.append(FormatSpan(new_start, new_end, span.attrs)) # else: span was entirely consumed by the deletion — drop it. self._spans = surviving Formatting as Commands Applying bold to a range is a mutation → it must be undoable → it\u0026rsquo;s a Command.\nclass ApplyFormatCommand(Command): \u0026#34;\u0026#34;\u0026#34; Apply an attribute (e.g. \u0026#39;bold\u0026#39;) to the range [start, end). Undo removes that attribute from the range. We could snapshot the entire spans structure before/after, but incremental undo (add on execute, remove on undo) is much cheaper. \u0026#34;\u0026#34;\u0026#34; def __init__(self, start: int, end: int, attr: str): self.start = start self.end = end self.attr = attr def execute(self, editor): editor._formatting.apply(self.start, self.end, self.attr) def undo(self, editor): editor._formatting.remove(self.start, self.end, self.attr) There\u0026rsquo;s a subtle question: what if the range was already bold before we applied bold? Undo would remove bold, which is wrong (it was bold to start with). A production system captures a small delta (\u0026ldquo;which sub-ranges actually changed?\u0026rdquo;) on execute. For interview scope, we call it out and move on:\n\u0026ldquo;For simplicity, I\u0026rsquo;m treating ApplyFormat as a delta: execute adds the attribute; undo removes it. This assumes the range wasn\u0026rsquo;t already fully-attributed. A production version would record only the sub-ranges that actually changed.\u0026rdquo;\nThat\u0026rsquo;s the tradeoff spoken aloud. Interview gold.\n12. Stage 11 — File I/O and Dirty State Two operations, one flag.\nLoad: open a file, read text, replace the buffer\u0026rsquo;s content. Save: write the buffer\u0026rsquo;s text to a file. Dirty flag: true if the buffer has been modified since the last save. Decision Point: How does Load interact with undo history? Two reasonable choices:\nClear the history on load. Loading a new file makes the prior document\u0026rsquo;s history nonsensical. Preserve the history. You could \u0026ldquo;undo the load\u0026rdquo; and get the old document back. We\u0026rsquo;ll clear the history. That\u0026rsquo;s Notepad/VS Code behavior and matches user intuition.\nThe Code class FileService: \u0026#34;\u0026#34;\u0026#34; Thin wrapper around file I/O. Extracted as a class so we can: - Mock it in tests. - Swap in a cloud storage backend later. - Handle encoding issues in one place. \u0026#34;\u0026#34;\u0026#34; def load(self, path: str) -\u0026gt; str: with open(path, \u0026#34;r\u0026#34;, encoding=\u0026#34;utf-8\u0026#34;) as f: return f.read() def save(self, path: str, text: str) -\u0026gt; None: with open(path, \u0026#34;w\u0026#34;, encoding=\u0026#34;utf-8\u0026#34;) as f: f.write(text) The dirty flag lives on the Editor — we\u0026rsquo;ll wire it in the assembly stage.\n13. Stage 12 — View Notifications: The Observer Pattern The Editor is a model. There\u0026rsquo;s a view (whatever renders text — a terminal, a GUI, a browser). The view must update when the model changes. But the model must not know about specific views — that would tightly couple model to presentation and prevent us from ever swapping the UI.\nSolution: Observer The Editor exposes a subscribe/publish mechanism. Views (or anyone else) subscribe. When state changes, the Editor emits events. Subscribers react.\nEvent Types We could send one generic \u0026ldquo;state changed\u0026rdquo; event — but that forces every observer to figure out what changed. Better to be specific:\ntext_changed(start, end_before, end_after) — the region [start, end_before) was replaced with content ending at end_after. cursor_moved(new_position). selection_changed(selection). dirty_changed(is_dirty). format_changed(start, end). The Code from typing import Callable # We use a very simple observer mechanism: named events → list of callbacks. # For interviews, this is clearer than a full Observer class hierarchy. class EventBus: \u0026#34;\u0026#34;\u0026#34; A tiny pub/sub bus. The Editor uses it to notify observers of granular state changes without knowing who they are. Why not a formal Observer / Subject hierarchy? - For this size of system, callable subscribers are lighter and idiomatic Python. - Multiple event types are hard to model with a single-method Observer interface (would end up dispatching on event type string). - Callable subscribers let a single view subscribe to only the events it cares about (a syntax highlighter only wants text_changed, a status bar wants dirty_changed). \u0026#34;\u0026#34;\u0026#34; def __init__(self): self._subs: dict[str, list[Callable]] = {} def subscribe(self, event: str, callback: Callable) -\u0026gt; None: self._subs.setdefault(event, []).append(callback) def unsubscribe(self, event: str, callback: Callable) -\u0026gt; None: if event in self._subs and callback in self._subs[event]: self._subs[event].remove(callback) def emit(self, event: str, *args, **kwargs) -\u0026gt; None: for cb in self._subs.get(event, []): cb(*args, **kwargs) The Editor holds an EventBus and emits events after each mutation. Views subscribe. No class dependency in either direction.\n14. Stage 13 — Putting It All Together Now we assemble the Editor. This is the orchestrator — the class users interact with.\nDesign Point: How thin should the Editor be? Ideally, the Editor is a thin coordinator: it takes user input, builds Commands, executes them via History, and emits events. All actual work lives in Buffer, Formatting, Clipboard, etc.\nIf the Editor starts doing things directly (mutating spans in-place, walking the buffer), we\u0026rsquo;ve broken the layered design.\nThe Code class Editor: \u0026#34;\u0026#34;\u0026#34; The main coordinator. Users interact only with this class. Composition (dependencies, all injected or default-constructed): - _buffer: TextBuffer (gap buffer under the hood) - _cursor: Cursor - _selection: Selection - _history: History (undo/redo) - _clipboard: Clipboard - _formatting: FormattingSpans - _search: SearchService - _files: FileService - _events: EventBus The Editor NEVER touches _buffer\u0026#39;s internals directly outside of trivial reads. All mutations go through Command objects. This is the invariant that guarantees undo/redo correctness. \u0026#34;\u0026#34;\u0026#34; def __init__(self, buffer: GapBuffer | None = None, files: FileService | None = None): self._buffer = buffer or GapBuffer() self._cursor = Cursor(0) self._selection = Selection(0, 0) self._history = History() self._clipboard = Clipboard() self._formatting = FormattingSpans() self._search = SearchService() self._files = files or FileService() self._events = EventBus() self._dirty = False self._current_file: str | None = None # ---- state accessors ---- @property def text(self) -\u0026gt; str: return str(self._buffer) @property def cursor_position(self) -\u0026gt; int: return self._cursor.position @property def selection(self) -\u0026gt; Selection: return self._selection @property def is_dirty(self) -\u0026gt; bool: return self._dirty def subscribe(self, event: str, callback: Callable) -\u0026gt; None: self._events.subscribe(event, callback) # ---- helper: run a command and record it ---- def _run(self, command: Command) -\u0026gt; None: \u0026#34;\u0026#34;\u0026#34; Execute a command, record it for undo, mark dirty, emit events. This is THE single point through which every mutation flows. Any operation that skips this path breaks undo/redo. That\u0026#39;s why we keep this method small and central. \u0026#34;\u0026#34;\u0026#34; command.execute(self) self._history.record(command) self._set_dirty(True) self._events.emit(\u0026#34;text_changed\u0026#34;) self._events.emit(\u0026#34;cursor_moved\u0026#34;, self._cursor.position) def _set_dirty(self, value: bool) -\u0026gt; None: if self._dirty != value: self._dirty = value self._events.emit(\u0026#34;dirty_changed\u0026#34;, value) # ---- cursor and selection ---- def move_cursor(self, position: int) -\u0026gt; None: self._cursor.move_to(position, len(self._buffer)) # Moving the cursor collapses the selection (standard behavior). self._selection.start = self._selection.end = self._cursor.position self._events.emit(\u0026#34;cursor_moved\u0026#34;, self._cursor.position) self._events.emit(\u0026#34;selection_changed\u0026#34;, self._selection) def select(self, start: int, end: int) -\u0026gt; None: self._selection = Selection(start, end) self._cursor.move_to(end, len(self._buffer)) self._events.emit(\u0026#34;selection_changed\u0026#34;, self._selection) self._events.emit(\u0026#34;cursor_moved\u0026#34;, self._cursor.position) # ---- editing ---- def insert(self, text: str) -\u0026gt; None: \u0026#34;\u0026#34;\u0026#34; Insert text at the cursor. If a selection is active, the selection is replaced with the inserted text (as one undo step). \u0026#34;\u0026#34;\u0026#34; if not self._selection.is_empty: # Composite: delete the selection, then insert. delete = DeleteCommand(self._selection.start, self._selection.end) insert = InsertCommand(self._selection.start, text) self._selection.clear() self._run(CompositeCommand([delete, insert])) else: self._run(InsertCommand(self._cursor.position, text)) def backspace(self) -\u0026gt; None: \u0026#34;\u0026#34;\u0026#34;Delete the char before the cursor (or the selection, if any).\u0026#34;\u0026#34;\u0026#34; if not self._selection.is_empty: self._delete_selection() return if self._cursor.position == 0: return # nothing to delete start = self._cursor.position - 1 end = self._cursor.position self._run(DeleteCommand(start, end)) def _delete_selection(self) -\u0026gt; None: if self._selection.is_empty: return start, end = self._selection.start, self._selection.end self._selection.clear() self._run(DeleteCommand(start, end)) # ---- clipboard ---- def copy(self) -\u0026gt; None: \u0026#34;\u0026#34;\u0026#34;Copy selection to clipboard. Not undoable.\u0026#34;\u0026#34;\u0026#34; if self._selection.is_empty: return text = self._buffer.get_text(self._selection.start, self._selection.end) self._clipboard.set(text) def cut(self) -\u0026gt; None: \u0026#34;\u0026#34;\u0026#34;Copy + delete. Undoable as a single step.\u0026#34;\u0026#34;\u0026#34; if self._selection.is_empty: return text = self._buffer.get_text(self._selection.start, self._selection.end) self._clipboard.set(text) self._delete_selection() def paste(self) -\u0026gt; None: \u0026#34;\u0026#34;\u0026#34;Insert clipboard content at cursor (replacing selection if any).\u0026#34;\u0026#34;\u0026#34; content = self._clipboard.get() if not content: return self.insert(content) # reuses insert\u0026#39;s selection-handling logic # ---- undo / redo ---- def undo(self) -\u0026gt; None: if not self._history.can_undo(): return cmd = self._history.pop_undo() cmd.undo(self) self._history.push_redo(cmd) # Note: dirty flag is intentionally left true — the standard # convention is \u0026#34;any change since save is dirty, and undo is # still a change since save.\u0026#34; A stricter version would track # a \u0026#34;clean checkpoint\u0026#34; position in the history and compare # current position to it. self._events.emit(\u0026#34;text_changed\u0026#34;) self._events.emit(\u0026#34;cursor_moved\u0026#34;, self._cursor.position) def redo(self) -\u0026gt; None: if not self._history.can_redo(): return cmd = self._history.pop_redo() cmd.execute(self) self._history.push_undo(cmd) self._events.emit(\u0026#34;text_changed\u0026#34;) self._events.emit(\u0026#34;cursor_moved\u0026#34;, self._cursor.position) # ---- find / replace ---- def find_all(self, query: str) -\u0026gt; list[tuple[int, int]]: return self._search.find(self.text, query) def replace_all(self, query: str, replacement: str) -\u0026gt; int: \u0026#34;\u0026#34;\u0026#34; Replace every occurrence of `query` with `replacement`. Undoable as ONE step. Returns the number of replacements. \u0026#34;\u0026#34;\u0026#34; matches = self._search.find(self.text, query) if not matches: return 0 # Process right-to-left so earlier indices remain valid as we go. subcommands: list[Command] = [] for start, end in reversed(matches): subcommands.append(DeleteCommand(start, end)) subcommands.append(InsertCommand(start, replacement)) self._run(CompositeCommand(subcommands)) return len(matches) # ---- formatting ---- def apply_format(self, attr: str) -\u0026gt; None: \u0026#34;\u0026#34;\u0026#34;Apply an attribute to the current selection.\u0026#34;\u0026#34;\u0026#34; if self._selection.is_empty: return self._run(ApplyFormatCommand(self._selection.start, self._selection.end, attr)) self._events.emit(\u0026#34;format_changed\u0026#34;, self._selection.start, self._selection.end) def attrs_at_cursor(self) -\u0026gt; set[str]: return self._formatting.attrs_at(self._cursor.position) # ---- file I/O ---- def load(self, path: str) -\u0026gt; None: \u0026#34;\u0026#34;\u0026#34; Load a file. Clears undo history and dirty flag. This is NOT an undoable operation. \u0026#34;\u0026#34;\u0026#34; text = self._files.load(path) self._buffer = GapBuffer(text) self._cursor = Cursor(0) self._selection = Selection(0, 0) self._history = History() # reset self._formatting = FormattingSpans() self._current_file = path self._set_dirty(False) self._events.emit(\u0026#34;text_changed\u0026#34;) def save(self, path: str | None = None) -\u0026gt; None: \u0026#34;\u0026#34;\u0026#34;Save to `path`, or to the current file if none given.\u0026#34;\u0026#34;\u0026#34; target = path or self._current_file if not target: raise ValueError(\u0026#34;no file path known — provide one\u0026#34;) self._files.save(target, self.text) self._current_file = target self._set_dirty(False) Two things to notice:\nThe Editor is thin. Every mutation is either delegating to _run (which wraps it in commands and history) or a small non-mutating helper. The _run method is the single choke point through which state changes flow. If we ever need to add logging, autosave triggers, remote sync — we add it here, once. That\u0026rsquo;s the design.\n15. Stage 14 — Validating with a Mental Walkthrough Let\u0026rsquo;s simulate a user session. We do this to catch bugs the code doesn\u0026rsquo;t reveal — sequences that \u0026ldquo;should work\u0026rdquo; but don\u0026rsquo;t.\n1. User creates editor. text = \u0026#34;\u0026#34;, cursor = 0, dirty = false. 2. User types \u0026#34;Hello\u0026#34;. → insert(\u0026#34;Hello\u0026#34;) called. selection empty, so InsertCommand(0, \u0026#34;Hello\u0026#34;). → buffer becomes \u0026#34;Hello\u0026#34;, cursor = 5, dirty = true. → undo_stack: [InsertCommand(0, \u0026#34;Hello\u0026#34;)] 3. User types \u0026#34; World\u0026#34;. → InsertCommand(5, \u0026#34; World\u0026#34;). → buffer = \u0026#34;Hello World\u0026#34;, cursor = 11. → undo_stack: [Insert(\u0026#34;Hello\u0026#34;), Insert(\u0026#34; World\u0026#34;)] 4. User hits undo. → pop Insert(\u0026#34; World\u0026#34;), call undo → delete [5, 11) → buffer = \u0026#34;Hello\u0026#34;. → cursor moves to 5. → redo_stack: [Insert(\u0026#34; World\u0026#34;)] 5. User types \u0026#34;!\u0026#34; — this should INVALIDATE the redo. → InsertCommand(5, \u0026#34;!\u0026#34;). → buffer = \u0026#34;Hello!\u0026#34;, cursor = 6. → undo_stack: [Insert(\u0026#34;Hello\u0026#34;), Insert(\u0026#34;!\u0026#34;)] → redo_stack: [] ✓ correctly cleared 6. User selects [0, 5), types \u0026#34;Hi\u0026#34;. → composite: [Delete(0, 5), Insert(0, \u0026#34;Hi\u0026#34;)]. → execute: buffer becomes \u0026#34;Hi!\u0026#34;, cursor = 2. → undo: as one step, restore \u0026#34;Hello!\u0026#34; 7. User undo → back to \u0026#34;Hello!\u0026#34;, selection cleared, cursor = 6. (Actually: undo of composite restores the deleted text and removes the inserted; cursor ends at wherever the LAST sub-command\u0026#39;s undo places it. For our composite, the undo iterates in reverse: undo(Insert) puts cursor at 0, then undo(Delete) puts cursor at 5. That\u0026#39;s the \u0026#34;after undoing the deletion\u0026#34; position. Is that what we want? Marginal — different editors do it differently. Note it.) 8. User selects [0, 5), copies. clipboard = \u0026#34;Hello\u0026#34;. buffer unchanged. 9. User moves cursor to 6, pastes. → InsertCommand(6, \u0026#34;Hello\u0026#34;) → buffer = \u0026#34;Hello!Hello\u0026#34;. 10. User saves → dirty = false. 11. User types \u0026#34;x\u0026#34; → dirty = true. Save-prompt behavior triggers on close, etc. The walkthrough surfaces one soft spot: cursor position after composite undo is a bit ill-defined. We noted it. In an interview, saying:\n\u0026ldquo;Composite undo leaves the cursor at the position implied by the last sub-command\u0026rsquo;s undo. That\u0026rsquo;s semantically fuzzy — a real editor might explicitly record \u0026lsquo;undo cursor\u0026rsquo; and \u0026lsquo;redo cursor\u0026rsquo; in each command. I\u0026rsquo;d add that if pressed.\u0026rdquo;\n\u0026hellip;is exactly the level of scrutiny an interviewer wants.\nA Second Walkthrough: Replace-All 1. Buffer = \u0026#34;foo bar foo baz foo\u0026#34;. 2. User: replace_all(\u0026#34;foo\u0026#34;, \u0026#34;xyz\u0026#34;). 3. Search finds [(0,3), (8,11), (16,19)]. 4. We reverse to [(16,19), (8,11), (0,3)]. 5. Composite command: Delete(16,19), Insert(16, \u0026#34;xyz\u0026#34;), Delete(8, 11), Insert(8, \u0026#34;xyz\u0026#34;), Delete(0, 3), Insert(0, \u0026#34;xyz\u0026#34;) 6. Executed: buffer becomes \u0026#34;xyz bar xyz baz xyz\u0026#34;. One undo step. 7. Undo → walks composite in reverse: Insert(0,\u0026#34;xyz\u0026#34;).undo (removes it), Delete(0,3).undo (restores \u0026#34;foo\u0026#34;), then next pair, etc. Result: buffer back to \u0026#34;foo bar foo baz foo\u0026#34;. Confirmed one undo restores all. ✓ Correct. Good.\n16. Stage 15 — Anticipating Follow-Up Questions Interviewers always stress-test the design. Here\u0026rsquo;s what to expect and how to answer.\nQ: How would you support very large files (say, 100MB)? The gap buffer holds everything in memory. For 100MB, that\u0026rsquo;s fine on modern machines (100MB is nothing), but the O(distance) cursor move becomes noticeable.\nAnswers:\nSwitch to a piece table — cursor moves are O(1); random access is O(log n) with a piece tree. VS Code does this. Or split into a rope — good for very large concatenations and edits. For truly enormous files (multi-GB logs), consider memory-mapped files with a windowed view — you don\u0026rsquo;t load everything, just what\u0026rsquo;s rendered. Q: How would you support collaborative editing? This is a huge topic. The short answer:\nEvery operation becomes an operation in the OT (Operational Transform) or CRDT sense. Concurrent operations must be transformed against each other so their final effect converges on all clients. Local undo becomes tricky — you can\u0026rsquo;t simply reverse if someone else has since made changes on top of yours. Real systems: Google Docs uses OT; many modern collab editors use CRDTs. For interview scope: \u0026ldquo;Beyond LLD; would need OT or CRDTs, and touches distributed systems territory.\u0026rdquo;\nQ: How would you support multi-cursor? Change the Editor to hold list[Cursor] and list[Selection]. Every insert operation applies at each cursor position — must go right to left to keep indices valid. Every command records positions for all cursors.\nNot that hard structurally — the code just needs to iterate cursors. The design already admits this because Cursor is a class and not just int on Editor.\nQ: What about undo grouping? Typing \u0026ldquo;hello\u0026rdquo; is 5 InsertCommands — but users expect one undo to remove the whole word. Right. The fix: coalesce consecutive character insertions when they\u0026rsquo;re contiguous and quick (say, no cursor movement between them, within a time threshold).\nImplementation: on record, if the incoming command is an InsertCommand, look at the top of the undo stack. If the top is also an InsertCommand and its end equals the new one\u0026rsquo;s start (and it happened recently), merge them. Effectively, replace the top with a single command whose text is the concatenation.\nMention this. Don\u0026rsquo;t implement in a 45-min interview.\nQ: How would you add syntax highlighting? Syntax highlighting is a separate concern layered on top:\nA highlighter subscribes to text_changed. On each change, it re-tokenizes the affected region (not the whole file — an incremental parser like tree-sitter does this). Each token\u0026rsquo;s range is stored as a formatting-span-like structure (a different set from user-applied formatting). The view queries both sets when rendering. Nice: syntax highlighting is just another observer of the model. Doesn\u0026rsquo;t require touching the Editor at all.\nQ: Auto-save? An observer on dirty_changed(True) starts a debounced timer. When it fires, call editor.save(). No editor-side changes needed.\nNotice the pattern: most extensions are new observers, not new methods on the Editor. That\u0026rsquo;s a healthy design.\nQ: How would you test this? Buffer tests: insert/delete correctness, edge cases (empty buffer, end-of-buffer). Command tests: execute-then-undo returns to original state (property-based testing shines here). Editor integration tests: sequences of user operations, verify final state. Regression tests: for every bug found, add a test that would have caught it. Q: What about Unicode? We use Python strings, which are Unicode-aware, but indexes are code-point-based, not grapheme-based. \u0026ldquo;👨‍👩‍👧\u0026rdquo; is one visible glyph but multiple code points. A real editor uses a grapheme-cluster-aware cursor. Extension point.\nQ: Undo across sessions? Persistent undo? Serialize the command stack alongside the document (VS Code and Emacs both do this). Each command must be serializable — since our commands hold only primitives (positions, text), they already are.\nQ: What if the buffer runs out of memory? Real editors handle \u0026ldquo;out of memory\u0026rdquo; via file-backed swap or an error prompt. Beyond LLD scope; mention only.\n17. Final Reflection — What This Problem Teaches Lesson 1: The Data Structure Choice IS the Design We spent an entire stage on the buffer choice, and rightly so. Every operation\u0026rsquo;s complexity flows from it. If you\u0026rsquo;d picked a naive Python string, your editor works fine for tiny toy inputs and collapses at any real scale. The design of the surrounding classes is easy compared to picking the right container underneath.\nIn interview: never skip the data structure justification. Even a two-line \u0026ldquo;considered X, Y, Z — chose Y because\u0026hellip;\u0026rdquo; is worth ten lines of code.\nLesson 2: The Command Pattern Is the Spine Everything mutating in this system goes through a Command. Not because \u0026ldquo;commands are cool,\u0026rdquo; but because the requirement of undo/redo forces it. Once you accept that, everything downstream falls into place: replace-all as a composite, cut as delete+clipboard-write, formatting as its own command class.\nThe Command pattern is one of the highest-leverage patterns you can know for LLD. Text editors, transaction systems, task queues, GUI frameworks, macros — anywhere you need to represent an action as data.\nLesson 3: Observer Decouples Model from View We designed the editor so a UI could attach without the editor knowing. Same for auto-save, syntax highlighting, status bars, dirty indicators — all just observers. This is what makes the design extensible. Add a feature by adding a subscriber; don\u0026rsquo;t modify the model.\nLesson 4: YAGNI, Continued Look at what we deliberately didn\u0026rsquo;t build:\nMulti-cursor. Regex search. Collaborative editing. Syntax highlighting. Auto-save. Undo grouping. Session-persistent undo. Unicode grapheme-aware cursor. Each of these is a real feature. Each was scoped out or mentioned as an extension. The design would collapse under its own weight if we\u0026rsquo;d built all of them. The discipline of \u0026ldquo;not now\u0026rdquo; is exactly what senior engineers exercise.\nLesson 5: Every Layer Has One Job GapBuffer — store text and support efficient edits. Doesn\u0026rsquo;t know about cursors. Cursor — a position. Doesn\u0026rsquo;t know about the buffer. Command — a reversible operation. Doesn\u0026rsquo;t know about the history. History — track stacks. Doesn\u0026rsquo;t know what commands do. Editor — orchestrate. Doesn\u0026rsquo;t do any actual mutation itself. EventBus — pub/sub. Doesn\u0026rsquo;t know about the domain. Each of those \u0026ldquo;doesn\u0026rsquo;t know\u0026rdquo; is a deliberate boundary. Those boundaries are where extensibility lives. Break them for convenience and you lose the design\u0026rsquo;s structure.\nLesson 6: The Composite Pattern Emerges Naturally We didn\u0026rsquo;t set out to use the Composite pattern. But when \u0026ldquo;typing with a selection\u0026rdquo; or \u0026ldquo;replace all\u0026rdquo; needed to be one undo step, CompositeCommand fell out of the requirements. The pattern didn\u0026rsquo;t drive the design; the design produced the pattern. That\u0026rsquo;s the right direction.\nLesson 7: Talk Out Loud We flagged tradeoffs everywhere: buffer choice, cursor storage, replace-all ordering, undo cursor semantics, span invariants, unicode. Every acknowledgment of a tradeoff is worth more than pretending you got it perfect. Interviewers reward candidates who see the messiness of the real problem; they penalize candidates who paper over it.\n18. Practice Questions Work through these before reading the solutions. They\u0026rsquo;re designed to test whether you\u0026rsquo;ve internalized the reasoning, not just remembered the code.\nQuestion 1 (Warm-up) Suppose we removed the requirement for undo/redo. Which classes could we simplify or eliminate? Which decisions would we revisit?\nSolution Eliminate: Command hierarchy, CompositeCommand, History. Simplify: The Editor._run choke point collapses — mutations become direct calls to the buffer. insert calls buffer.insert; backspace calls buffer.delete; etc. Revisit: The choice of gap buffer stands (it\u0026rsquo;s about performance, not undo). But we\u0026rsquo;d no longer need DeleteCommand to return the deleted text — the reason we captured it was for undo restoration. Impact on replace_all: No longer needs to be a composite. Can just loop and mutate directly. The Command pattern imposes real complexity. Without undo, we don\u0026rsquo;t pay it. This is a good reminder: patterns aren\u0026rsquo;t free — apply them when a requirement demands them.\nQuestion 2 (Design tradeoff) We chose the gap buffer. Suppose the interviewer says: \u0026ldquo;Actually, undo history must survive across editor restarts, and I want to be able to inspect the state of the document at any historical point.\u0026rdquo; Does this change your buffer choice? Why?\nSolution Yes. This is a strong hint toward a piece table.\nWhy: a piece table is naturally immutable at the piece level. When you edit, you don\u0026rsquo;t overwrite existing content — you add new pieces to the \u0026ldquo;add buffer\u0026rdquo; and rearrange the piece list. This means:\nHistorical snapshots are free. The old piece list, if retained, still refers to unchanged buffer contents. Reconstruct any prior state by remembering the piece list at that point. Serialization of history is easier. Commands hold positions and small text fragments — plus you can serialize the underlying add-buffer once. Gap buffer would also work, but you\u0026rsquo;d need to snapshot the whole buffer at checkpoints (memory-heavy) or replay commands from a persisted stack against a persisted starting document (correct but slow to load).\nThe design pivot: swap GapBuffer for PieceTable. The Editor and its interfaces are unchanged — that\u0026rsquo;s the payoff of layering.\nQuestion 3 (Extend the design) Add multi-cursor support. Sketch the changes to Editor, Cursor, and InsertCommand. What edge cases arise?\nSolution Editor changes:\nself._cursors: list[Cursor] = [Cursor(0)] # replaces single _cursor self._selections: list[Selection] = [Selection(0, 0)] # parallel list Every operation that referenced self._cursor now iterates the list. move_cursor(index, pos) targets a specific cursor.\nInsertCommand changes:\nA \u0026ldquo;user inserts \u0026lsquo;x\u0026rsquo; with 3 cursors\u0026rdquo; is now one user action but three insertions. The insertion order matters — do them right-to-left so earlier positions don\u0026rsquo;t shift.\nclass MultiInsertCommand(Command): def __init__(self, positions: list[int], text: str): # Sort in reverse so leftward positions aren\u0026#39;t invalidated. self.positions = sorted(positions, reverse=True) self.text = text def execute(self, editor): for pos in self.positions: editor._buffer.insert(pos, self.text) # Cursors: each shifts by len(text) if it was at or after an inserted position. ... def undo(self, editor): # Delete inserted texts, left-to-right (deleting reverse-order would also work). for pos in reversed(self.positions): editor._buffer.delete(pos, pos + len(self.text)) Edge cases:\nOverlapping cursors. Two cursors at the same position should be deduplicated or treated as one — otherwise a single character gets typed twice at that position. Cursor within another cursor\u0026rsquo;s selection. Semantics get ambiguous — most editors resolve by collapsing overlapping selections into one. Undo of a multi-op is one step. Already handled by the single command wrapping the multi. The key insight: multi-cursor doesn\u0026rsquo;t change the architecture; it just plumbs a list where there was one. The Cursor abstraction we designed makes this natural.\nQuestion 4 (Bug hunting) There\u0026rsquo;s a subtle bug in this snippet from our replace_all:\nmatches = self._search.find(self.text, query) # ... build composite command using matches ... self._run(CompositeCommand(subcommands)) Under what conditions would this misbehave? How would you fix it?\nSolution Bug: If replacement contains query as a substring, our command sequence is still fine (we process right-to-left so already-replaced text isn\u0026rsquo;t scanned). But if we later re-execute (redo), matches are recomputed\u0026hellip; except we don\u0026rsquo;t recompute — the command captured the exact positions and edits at execute time. That\u0026rsquo;s actually safe.\nThe real subtle bug: matches is computed once, then we build the composite. If some caller mutates self.text between the find call and the _run call (unlikely here, but possible in multi-threaded scenarios), the positions become stale.\nFix for the general case:\nSnapshot the text before finding, and validate no changes have occurred (in a single-threaded model, this is trivially true). Or make the whole replace-all atomic under a lock in a multi-threaded editor. A more common subtle bug: empty query. If query == \u0026quot;\u0026quot;, str.find returns 0 forever. Our LiteralSearch.find_all guards against this with if not query: return []. Verify that guard exists.\nAnother subtle bug: what if query == replacement? We\u0026rsquo;d do a no-op wrapped in a composite of delete-plus-insert per match — technically correct but wasteful, and dirty flag gets set for no visible change. Optimization: return early if query == replacement.\nInterviewer-pleaser: mentioning \u0026ldquo;empty query causes infinite loop\u0026rdquo; is a classic gotcha to know.\nQuestion 5 (Design a new feature) Design a \u0026ldquo;Go to Line\u0026rdquo; feature. go_to_line(n) moves the cursor to the start of the nth line (1-indexed). What class does this method live on? What data structure would you use to make this O(log n) rather than O(document size)?\nSolution Where does it live? On the Editor, as a small public method. It uses the buffer\u0026rsquo;s contents but doesn\u0026rsquo;t mutate them — no command needed.\nNaive implementation: scan the text, count newlines, return the position after the (n-1)th newline. O(document size). Fine for MB documents; sluggish for 100MB logs.\nFaster: maintain a line index — a data structure that maps line numbers to byte offsets.\nOptions:\nPrecomputed array of newline positions. Fast lookup (line_starts[n] is O(1)). But every insert/delete requires updating this array, and inserts near the front are O(newlines-after-position). Sorted tree of newline positions (e.g., a SortedList or Fenwick tree). Insertions O(log n). Line-to-position lookup O(1) after finding. Piece table with per-piece line count. VS Code\u0026rsquo;s approach. Very fast for both edits and lookups. Interaction with commands: The line index is a derived cache. It should be maintained by the same code that mutates the buffer — cleanest to update it inside TextBuffer.insert and TextBuffer.delete. That means either:\nThe buffer knows about lines (couples two concerns). The buffer exposes hooks/observers for edits, and the line index subscribes. I\u0026rsquo;d choose the observer approach: the buffer emits low-level edit events; a LineIndex subscribes and stays in sync. That preserves the buffer\u0026rsquo;s single responsibility.\nNotice: this is another observer! Same pattern as the view. Once you set up event emission, everything derived from the model becomes a subscriber. This is the design shape scaling as more features arrive.\nQuestion 6 (Deep tradeoff) We used a set of string labels for format attributes ({\u0026quot;bold\u0026quot;, \u0026quot;italic\u0026quot;}). What are the limitations of this? How would you extend it to support:\nColors (e.g., font color = #FF0000) Multiple simultaneous colors on the same range (probably impossible — but how would you decide?) Named styles (heading1, code) that carry compound formatting? Solution Current limitation: attributes are just strings — they can\u0026rsquo;t carry values. \u0026quot;bold\u0026quot; is on or off; there\u0026rsquo;s no data to associate.\nExtension 1: Colors.\nChange attrs from set[str] to dict[str, Any]. Now attrs = {\u0026quot;bold\u0026quot;: True, \u0026quot;color\u0026quot;: \u0026quot;#FF0000\u0026quot;} works. Note the semantic shift: a \u0026ldquo;bold\u0026rdquo; attribute is boolean-valued; a \u0026ldquo;color\u0026rdquo; attribute has a value. You need type-safety at some level (e.g., a FormatAttribute schema).\nExtension 2: Simultaneous colors.\nCan you be both red and blue at the same position? No — a character has one color at a time. So color is a single-valued attribute; setting a new color on a range overwrites the old. That\u0026rsquo;s different from bold, which is idempotent.\nThis means the model has to know each attribute\u0026rsquo;s kind:\nToggle attributes (bold, italic, underline) — set-like semantics; multiple ranges can add or remove. Single-valued attributes (color, font, font size) — the most recent (or highest z-order) wins. You\u0026rsquo;d introduce something like:\nclass AttributeSpec: def __init__(self, name: str, kind: str): # kind in {\u0026#34;toggle\u0026#34;, \u0026#34;single_value\u0026#34;} ... When applying, the FormattingSpans logic checks the spec and behaves accordingly.\nExtension 3: Named styles.\nA named style like heading1 is really a bundle of attributes: {font_size: 24, bold: True, color: \u0026quot;#000000\u0026quot;}. You could:\nExpand at apply time: applying heading1 inserts multiple spans (one per constituent). Loses the \u0026ldquo;this is a heading1\u0026rdquo; semantic when reading back. Store as a reference: the span carries style = \u0026quot;heading1\u0026quot;, and a StyleRegistry maps names to concrete attributes at render time. This preserves the semantic, allows themes (\u0026ldquo;change what heading1 looks like globally\u0026rdquo;), and is closer to how Word / Google Docs work. Approach 2 is more sophisticated but far more powerful. It also introduces new questions: what if you override just one attribute of a named style? (Answer: store both — the style name and a delta.) You\u0026rsquo;ve now stumbled into a small but real inheritance model for formatting.\nThe lesson: what looked like a small extension (add color) actually requires reworking the attribute type system. This is common. Push back gently in interview: \u0026ldquo;Are we treating attributes uniformly, or do some like color have single-value semantics? That changes the model.\u0026rdquo;\nQuestion 7 (End-to-end) Sketch the operations needed to implement \u0026ldquo;undo the last N operations as one step\u0026rdquo; — a batch undo, e.g., \u0026ldquo;undo the last 5 keystrokes together.\u0026rdquo; What changes in Editor and History?\nSolution Semantics: the user wants to say \u0026ldquo;undo N times, treating it as one redo step.\u0026rdquo;\nIn History: we don\u0026rsquo;t lose the individual commands — we just batch them.\nApproach 1: Batch on the fly.\ndef undo_batch(self, editor, n: int) -\u0026gt; None: batched: list[Command] = [] for _ in range(n): if not self._history.can_undo(): break cmd = self._history.pop_undo() cmd.undo(editor) batched.append(cmd) if batched: # Wrap them (in original execute order) into a composite for redo. composite = CompositeCommand(list(reversed(batched))) self._history.push_redo(composite) Note the reversal: we undid in reverse order; the composite should redo in original order.\nApproach 2: Explicit transaction markers.\nAdd Editor.begin_transaction() and Editor.end_transaction(). Inside, all commands are collected into a pending composite. On end, one composite is pushed to the undo stack. This is more powerful — the user gets to decide grouping ahead of time.\nWhich is better? Depends on the use case. Batch-undo of the last N is more like a UI convenience. Transactions are more like a programmatic contract. Real editors often use auto-coalescing (Q on typing \u0026ldquo;hello\u0026rdquo; as one undo) rather than either explicit approach — because users don\u0026rsquo;t want to think about grouping.\nThe takeaway: the design supports these extensions because commands are first-class objects that can be reordered, batched, and rewrapped. Try doing this with a \u0026ldquo;mutate directly\u0026rdquo; editor — you\u0026rsquo;d be re-architecting.\nKey Takeaways Data structure first. In text editors, the buffer implementation determines the whole system\u0026rsquo;s performance ceiling. Justify your choice out loud. Command pattern is unavoidable when undo/redo is required. Every mutation → a Command. Every user action → an execute call. Every reversal → an undo call. Composite commands emerge from multi-step user actions. Typing over a selection = delete + insert = one composite. Replace-all = many pairs = one composite. This lets one undo step reverse multi-step operations. Observer decouples model from view — and from every other derived cache. Line index, syntax highlighter, autosave, dirty indicator — all subscribers. Layer the design. Buffer knows text; cursor knows position; command knows how to mutate; history knows stacks; editor orchestrates. Break these boundaries and extensions become invasive. YAGNI, aggressively. Multi-cursor, collaboration, syntax highlighting, undo grouping — all extensions, none in v1. Cost of adding later ≈ cost of adding now, if the design is layered right. Talk about tradeoffs. Gap buffer vs piece table, attribute spans vs per-char attributes, undo cursor semantics, replace-all direction — every one is a spoken tradeoff, not a silent choice. What\u0026rsquo;s Next You\u0026rsquo;ve now internalized four LLD deep dives — Parking Lot (relationships), Elevator (state machines), Library (policy), Text Editor (operations-on-data). Notice again what stays the same: the discipline of clarifying, the entity filtering, the alternative-weighing, the mental walkthrough, the anticipation of follow-ups.\nThe framework transfers. Every future LLD problem — Vending Machine, Splitwise, Ride-Hailing, Chess, Snake and Ladder, ATM — is the same set of thinking moves applied to a new domain.\nThe specific designs are notes; the framework is the melody.\nThis content is part of Codeverra — a platform for learning coding, data science, DSA, and AI from scratch. Explore more: https://codeverra.com\n","permalink":"https://blogs.codeverra.com/blog/low_level_design/interview_questions/06i_lld_text_editor_deep_dive/","summary":"\u003ch1 id=\"lld-deep-dive-4--designing-a-text-editor\"\u003eLLD Deep Dive #4 — Designing a Text Editor\u003c/h1\u003e\n\u003cblockquote\u003e\n\u003cp\u003e\u003cstrong\u003eWhat this document is:\u003c/strong\u003e A genuine, step-by-step walkthrough of how to \u003cem\u003ethink through\u003c/em\u003e a Text Editor LLD problem in an interview. Not a finished design dropped on you — but the reasoning, the false starts, the alternatives weighed, and the justifications behind each design decision.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eTime investment:\u003c/strong\u003e Read this slowly. If you\u0026rsquo;re doing this for interview prep, treat each stage as its own thinking exercise — pause and ask yourself \u0026ldquo;what would I do here?\u0026rdquo; before reading on.\u003c/p\u003e","tags":null,"title":""},{"content":"LLD Deep Dive #5: Designing Splitwise (Expense Sharing) What this document is. This is a complete, step by step walkthrough of how to design an expense sharing system of the kind popularised by the Splitwise application. It reasons from first principles, states the alternatives at every decision point, and justifies the option that is chosen. The design developed here is implemented as a small, runnable, modular Python project that accompanies this document, so that every concept discussed in prose can be examined and executed as real code.\nHow to use this document alongside the code. Each design stage names the exact module in the accompanying project that implements the idea under discussion. The project is laid out under splitwise_lld/, and the module paths quoted in this document (for example splitwise/core/balance_sheet.py) refer to files inside that project. Reading the prose and the corresponding module together is the most effective way to absorb the material.\nPrerequisites. This is the fifth problem in the low level design series. It assumes familiarity with the design patterns covered earlier in the course, in particular the Strategy pattern, the Decorator pattern, the Observer pattern, and the Facade pattern, together with the SOLID principles. Where a pattern is applied, this document names it and connects it back to its earlier appearances so that the recurring structure becomes visible.\nTable of Contents Learning Objectives Why This Problem Matters Stage 1: Receiving the Problem Stage 2: Clarifying Requirements Stage 3: Identifying the Entities Stage 4: The Central Decision, How to Represent Debt Stage 5: The Shape of the Project Stage 6: Representing Money in Minor Units Stage 7: Modeling Users and Groups Stage 8: Split Types as a Strategy Stage 9: The Expense and the Balance Update Rule Stage 10: The BalanceSheet as the Source of Truth Stage 11: Debt Simplification Stage 12: The ExpenseManager as a Facade Stage 13: Validating the Design with a Walkthrough Stage 14: Anticipating Follow Up Questions Stage 15: Running the Project in PyCharm Summary Tables Key Takeaways Practice Questions and Solutions 1. Learning Objectives By the end of this document a reader should be able to do the following. First, explain why the choice of data model is the most consequential decision in a data centred design problem, and defend the net balance model against the more obvious alternative of storing one record per transaction. Second, apply the Strategy pattern to a family of related algorithms and state the correctness contract that binds them. Third, describe and implement a greedy algorithm for reducing a set of mutual debts to a small number of settlement payments, and state honestly where that algorithm sits relative to the theoretical optimum. Fourth, assemble these parts behind a Facade so that the coordinating class remains thin. Fifth, run and extend the accompanying modular project.\nA note on terminology before proceeding. Low level design, abbreviated LLD, refers to the activity of specifying the classes, their responsibilities, their relationships, and their interactions for a single system or component, as distinct from high level design, which concerns the arrangement of services, databases, and network boundaries across a whole distributed system. The present problem is an LLD problem, so the emphasis throughout is on classes, responsibilities, and the interactions between objects held in the memory of a single process.\n2. Why This Problem Matters Splitwise is the canonical problem for the skill of modeling money that flows between people. It is valuable in a design curriculum because it exercises two distinct competencies at the same time, and because it contains a tempting wrong path that teaches an important lesson when it is recognised and rejected.\nThe first competency is domain modeling, which is the activity of choosing the classes and the data structures that represent the concepts of the problem. The problem requires a representation of the fact that one person owes another a certain amount, and this representation must support the operations that the application performs most frequently. A representation that makes the common operation cheap is a good model, and a representation that makes the common operation expensive is a poor model even if it appears more natural at first sight.\nThe second competency is algorithm design. The feature that distinguishes a serious expense sharing system from a simple ledger is the ability to reduce a tangle of mutual debts to the smallest practical number of payments. This is a genuine computational problem with a well understood structure, and reasoning about it demonstrates that a candidate can move beyond arranging classes into designing the procedure that operates on them.\nThe problem also teaches restraint in the selection of patterns. Earlier problems in this series were built around behavioural patterns such as State. It would be a mistake to assume that every design must be organised around such a pattern. Splitwise has no state machine at its core. Its centre is a data model and an algorithm. Recognising that a problem does not call for a behavioural pattern, and resisting the urge to introduce one, is itself a mark of design maturity.\n3. Stage 1: Receiving the Problem The interviewer states the problem in a single sentence.\n\u0026ldquo;Design Splitwise.\u0026rdquo;\nA reader who has used the application will have strong intuitions, and those intuitions are a hazard rather than an asset at this moment. The application appears simple, because its visible behaviour is only the recording of who paid for what. Its true difficulty is concealed behind that surface, in the questions of how debt is represented and how a group of people can settle their mutual obligations with as few payments as possible.\nThe correct opening move is therefore not to begin declaring classes. It is to establish what the phrase \u0026ldquo;settling up\u0026rdquo; means in this system and how debt is to be represented, because that single decision determines the shape of everything that follows. The clarification stage that follows is the vehicle for establishing this.\n4. Stage 2: Clarifying Requirements A clarification round is a structured activity rather than a scatter of questions. The questions are organised into functional requirements, which describe what the system does, non functional requirements, which describe the qualities the system must possess, and an explicit statement of what lies outside the scope of the design. Each question below is presented together with the reason it influences the design, because asking a question without understanding why it matters is of little value.\n4.1 Which kinds of split must the system support? This is the largest single driver of the design. A system that only ever divides a cost equally requires one algorithm. A system that also divides by exact amounts, by percentage, and by weighted shares requires several algorithms that perform the same task differently. The presence of several interchangeable algorithms for one task is the precise condition under which the Strategy pattern applies.\nAssume the interviewer answers that the system must support equal division, division by exact amounts, and division by percentage, and that division by weighted shares may be required later. The mention of a future requirement is significant, because it instructs the designer to build the mechanism for selecting a split algorithm in a way that admits a new algorithm without modification to the existing ones.\n4.2 Does an expense have a single payer or several? An expense in which one person settles the entire bill is considerably simpler to model than an expense whose cost is advanced by two or more people. The data model must record which case it is dealing with.\nAssume the interviewer answers that a single payer per expense is sufficient for the first version, and that multiple payers are a desirable later addition. The design will therefore record one payer per expense and will note the extension explicitly.\n4.3 Are groups required, or only pairwise expenses? A group is a named collection of users, such as the participants in a trip or the members of a household. Groups change how expenses are organised and make it possible to settle the balances of one group in isolation.\nAssume the interviewer answers that both are required. Users may share one off expenses with one another, and they may also form groups. The design will therefore include a Group entity that scopes a set of expenses without owning any balance information.\n4.4 Must the system simplify debts? This is the question that separates a serious system from a simple ledger. To simplify debts means to take a web of mutual obligations and produce the minimum practical set of payments that settles everyone to a zero balance. This is a real algorithmic feature and it is the most likely subject of a deep follow up.\nAssume the interviewer answers that the system must be able to display the minimum number of transactions required to settle a group. This requirement must be flagged prominently, because it exerts a strong influence on the choice of debt representation made in the next stage.\n4.5 Is more than one currency required? Support for multiple currencies introduces conversion, fluctuating exchange rates, and additional rounding concerns. It is frequently excluded from a first version.\nAssume the interviewer answers that a single currency is sufficient for now. The design will nevertheless store all amounts as integers in the smallest unit of the currency, for reasons of arithmetic correctness that are explained in Stage 6.\n4.6 What are the non functional requirements? These are the qualities of the system rather than its behaviours. The relevant ones here are persistence, meaning whether data must survive beyond the life of the process, scale, meaning how many users and expenses the system must accommodate, and concurrency, meaning whether several operations proceed at the same time.\nAssume the interviewer answers that an in memory implementation is acceptable, that the system may be single threaded, and that the priority is a clean object model to which storage can be added later. The design will therefore hold all state in memory and will treat persistence and concurrency as extensions discussed at the end.\n4.7 Restating the scope After the clarification round the designer restates the agreed scope so that the interviewer can confirm or correct it before any class is written.\n\u0026ldquo;To confirm the scope: users record expenses that are paid by a single payer and divided equally, by exact amounts, or by percentage, with division by shares as a likely later addition, optionally within groups. The system tracks the net balance between users and can produce the minimum practical set of payments that settles a group. A single currency is used, and all amounts are stored as integers in the smallest currency unit to avoid rounding error. The implementation is in memory and single threaded, and persistence, concurrency, and multiple currencies are treated as extensions.\u0026rdquo;\nWith the scope agreed, the design proceeds to the decision on which everything else depends.\n5. Stage 3: Identifying the Entities The candidate classes of a design are drawn from the nouns of the requirements. Not every noun deserves a class. A noun becomes a class only if it possesses its own identity, meaning that the system must refer to a specific instance of it over time, and its own behaviour or state, meaning that it does more than hold a single value. Nouns that are merely values become fields or value objects, and nouns that describe actors outside the system are not modelled at all.\nThe following table records this analysis for the nouns that arise in the Splitwise requirements.\nNoun Has identity Has state or behaviour Verdict User Yes Yes, participates in expenses Entity class Group Yes Yes, holds members and expenses Entity class Expense Yes Yes, records payer, total, and shares Entity class Split No Holds one user and one amount Value object Split type such as equal or percentage No An algorithm Strategy, not an entity Balance or debt Not applicable The central data of the system Resolved in Stage 4 Settlement or payment Yes Records a transfer Modelled as a reverse debt and a result type Currency amount No A single value Integer field in minor units Expense manager Yes Coordinates the subsystem Facade class Two verdicts deserve comment. The split type is not an entity, because it has no identity and consists only of an algorithm. It is therefore modelled as a Strategy. The balance is the one cell of the table that cannot be resolved by inspection, because the choice of representation is itself the principal design decision. It is given its own stage.\nThe skeleton that emerges from this analysis is as follows. The entity classes are User, Group, and Expense, supported by the Split value object. The splitting algorithms form a Strategy hierarchy. The balance representation and the settlement algorithm form the core of the system, and a coordinating class ties everything together. Each of these appears as a module in the accompanying project, which is described in Stage 5.\n6. Stage 4: The Central Decision, How to Represent Debt Every subsequent decision depends on how debt is represented, so this stage examines the options with the care the decision deserves. Three representations are considered.\n6.1 Approach A: One debt record per expense Under this approach the system stores, for every expense, an individual record for each obligation that the expense creates, such as a record stating that Alice owes Bob two hundred as a result of a particular dinner. All such records are retained in a list.\nThe advantage of this approach is that it preserves a complete and literal history. It is possible to trace every obligation back to the expense that produced it.\nThe disadvantages are decisive. The most frequent question the application must answer is how much one user owes another in total, and under this approach that question requires a scan and summation of every record between the two users, which costs time proportional to the length of the entire history. Furthermore, obligations never cancel. If Alice owes Bob two hundred and Bob later owes Alice two hundred, the approach stores two records and never recognises that they offset one another. The stored data grows without bound and never simplifies, and settling up requires locating and neutralising records scattered throughout the history.\n6.2 Approach B: A pairwise net balance map Under this approach the system maintains, for every ordered pair of users, a single number representing the net amount one owes the other. The entry balance[A][B] holds the amount that A owes B, and a negative value indicates that B in fact owes A. Every expense updates the relevant entries.\nThe advantages are substantial. The question of how much one user owes another is answered in constant time by a single lookup. Obligations cancel automatically, because an offsetting expense simply moves the stored number toward zero. The stored data is compact, requiring at most one number per pair of users that has ever transacted.\nThe one apparent disadvantage, that the per expense history is not visible in the balance itself, is not a genuine loss, because the list of expenses is retained separately as an immutable history for auditing. The balance map and the expense history serve different purposes and coexist without conflict. A second consideration is that the representation must be kept consistent, so that the entry for A owing B always equals the negation of the entry for B owing A. This consistency requirement is addressed in Stage 10 by confining all mutation to a single method.\n6.3 Approach C: A single net position per user Under this approach each user carries one number representing the total amount they have overpaid or underpaid across all expenses. A positive number means the user is owed money overall, and a negative number means the user owes money overall.\nThe advantage is that this is the smallest possible representation, requiring one number per user, and it is precisely the input required by the debt simplification algorithm.\nThe disadvantage is that it is too lossy to serve as the source of truth, because it cannot answer the question of how much one specific user owes another. It records only the aggregate position of each user.\n6.4 The decision The design adopts Approach B, the pairwise net balance map, as the source of truth, and derives Approach C, the net position per user, on demand for the purpose of debt simplification.\nThe reasoning is as follows. Approach A is the intuitive path and it is the trap, because it makes the most common query expensive and never allows debts to cancel. Approach B answers the common query in constant time and cancels offsetting debts automatically, which is the defining behaviour of an expense sharing application. Approach C is ideal as the input to the simplification algorithm but too lossy to be the source of truth, so rather than storing it separately the design computes it from Approach B whenever it is needed. The result combines the strengths of both: the pairwise map holds the precise truth, and the net position per user is a projection derived from that truth when the algorithm requires it.\nThis choice is an instance of a principle that recurs throughout good design, namely derive rather than duplicate. When one quantity can be computed from another, storing both invites the two copies to drift out of agreement. Computing the net position from the pairwise map on demand guarantees that the two views can never disagree, because there is only one stored copy of the information.\nThe pairwise map is implemented in splitwise/core/balance_sheet.py and the derivation of net positions is the net_position method of that same module. The full analysis of that module appears in Stage 10.\n7. Stage 5: The Shape of the Project Before implementing the classes it is worth establishing the structure of the project, because a modular arrangement makes the responsibilities visible and keeps each concept in a place where it can be studied and tested in isolation. The accompanying project is organised into four layers, each of which is a Python package.\nThe domain layer, under splitwise/domain/, contains the passive data types and the money helpers. These are the User, Group, Split, and Expense types together with the money conversion and formatting functions. A passive data type holds information and enforces its own small invariants but does not orchestrate the behaviour of other objects.\nThe strategy layer, under splitwise/strategies/, contains the interchangeable splitting algorithms. It holds the abstract SplitStrategy interface and the concrete implementations for equal, exact, percentage, and shares based division.\nThe core layer, under splitwise/core/, contains the two most important pieces of the design, namely the BalanceSheet, which is the source of truth for debt, and the DebtSimplifier, which is the settlement algorithm.\nThe service layer, under splitwise/services/, contains the ExpenseManager, which is the facade that coordinates the other layers and presents a small interface to the outside world.\nThis separation into layers is a direct application of the Single Responsibility Principle at the level of packages rather than classes. A change to the formatting of money touches only the domain layer, a new splitting algorithm touches only the strategy layer, and a change to the settlement algorithm touches only the core layer. The dependencies flow in one direction, from the service layer inward to the core, strategy, and domain layers, so that the passive data types at the centre depend on nothing above them.\nThe following table maps each design concept to the module that implements it.\nConcept Module Money convention and formatting splitwise/domain/money.py User identity splitwise/domain/user.py One participant\u0026rsquo;s share of one expense splitwise/domain/split.py Immutable record of a spending event splitwise/domain/expense.py Named collection of users splitwise/domain/group.py Interchangeable splitting algorithms splitwise/strategies/split_strategy.py Source of truth for debt splitwise/core/balance_sheet.py Settlement algorithm splitwise/core/debt_simplifier.py Coordinating facade splitwise/services/expense_manager.py Demonstration main.py 8. Stage 6: Representing Money in Minor Units Money is stored throughout the system as an integer count of the smallest indivisible unit of the currency, which is referred to as the minor unit. For the Indian rupee the minor unit is the paisa, of which there are one hundred to a rupee, and for the United States dollar the minor unit is the cent. This convention is established in splitwise/domain/money.py.\nThe justification is a matter of arithmetic correctness. A floating point number cannot represent most decimal fractions exactly, because it stores values in binary. The consequence is that an expression such as the sum of one tenth and two tenths does not equal three tenths in floating point arithmetic, but differs from it by a minute error. Financial software that stores money as a floating point number of rupees accumulates these errors across many operations and eventually reports balances that are incorrect by a fraction of a unit. Storing money as an integer count of minor units removes this class of error completely, because integer arithmetic in Python is exact and is not subject to a fixed width overflow.\nThe module provides two helpers. The rupees function converts a human readable rupee amount into integer paise, so that demonstration code and tests may be written in familiar units while the system stores paise internally. The format_paise function renders an integer paise amount as a readable rupee string, handling the sign explicitly so that negative balances display cleanly and always showing two digits for the paise portion so that amounts align when printed in a column.\nThe decision to store money as integers is small in code but significant in what it signals about the maturity of the design, and it is worth stating aloud when presenting the solution.\n9. Stage 7: Modeling Users and Groups The user and group classes are deliberately thin. They are implemented in splitwise/domain/user.py and splitwise/domain/group.py.\nA User is an entity, which means it has a stable identity that persists across many expenses and balances. The identity is a universally unique identifier assigned at construction, so that two user records are regarded as the same user precisely when they share this identifier, regardless of any later change to name or email. The User class overrides the hashing and equality operations to be based on this identifier, because user objects are used as dictionary keys inside the balance model, and a dictionary key must have a stable hash that does not depend on mutable fields.\nimport uuid class User: \u0026#34;\u0026#34;\u0026#34;A person who can pay for and participate in expenses.\u0026#34;\u0026#34;\u0026#34; def __init__(self, name: str, email: str): self.user_id: str = str(uuid.uuid4()) self.name: str = name self.email: str = email def __hash__(self) -\u0026gt; int: return hash(self.user_id) def __eq__(self, other: object) -\u0026gt; bool: return isinstance(other, User) and self.user_id == other.user_id A User deliberately holds no balance information. A balance is a relationship between two users rather than a property of a single user, and placing it inside the user class would scatter the source of truth for debt across many objects and invite inconsistency. This is an application of the Single Responsibility Principle: the user class is responsible for identity and profile data and for nothing else. The balance is held instead in the BalanceSheet described in Stage 10.\nA Group is a named collection of users that provides a scope within which expenses are organised. It records which users belong together and, by identifier, which expenses were charged to the group. Like the user, it holds no balance information, because the balance remains the exclusive responsibility of the BalanceSheet. Keeping the source of truth for debt in exactly one place is the same discipline applied twice, once to the user and once to the group.\n10. Stage 8: Split Types as a Strategy The division of an expense among its participants is a task that admits several algorithms, namely equal division, division by exact amounts, division by percentage, and division by weighted shares. A single well defined task that has several interchangeable implementations is the defining condition of the Strategy pattern, which is the pattern that encapsulates each algorithm in its own class behind a common interface so that the algorithm can be selected and exchanged without altering the code that uses it.\nThis is the fourth appearance of the Strategy pattern in this series. It was used for pricing in the parking lot problem, for scheduling and target selection in the elevator problem, and for change making in the vending machine problem. The recurring signal is the presence of several interchangeable algorithms for one task, together with the expectation that further algorithms may be added. When the interviewer stated in Stage 2 that division by shares might be required later, that statement was in effect an instruction to use the Strategy pattern, because only an open ended mechanism for selecting an algorithm accommodates a future addition without modification to the existing algorithms.\n10.1 The result type The output of a splitting algorithm is a list of shares, one per participant. Each share is represented by the Split value object defined in splitwise/domain/split.py. A value object is an object that has no identity of its own and is defined entirely by its contents. The class is declared frozen, meaning immutable, so that a computed share cannot be altered after it has been produced, which removes an entire category of defects in which a share is accidentally modified after the expense has been recorded.\nfrom dataclasses import dataclass from splitwise.domain.user import User @dataclass(frozen=True) class Split: user: User amount: int 10.2 The interface and its correctness contract The abstract interface, defined in splitwise/strategies/split_strategy.py, declares a single method that divides a total among a list of participants and returns one Split per participant.\nfrom abc import ABC, abstractmethod from typing import List from splitwise.domain.split import Split from splitwise.domain.user import User class SplitStrategy(ABC): \u0026#34;\u0026#34;\u0026#34;Abstract interface for an algorithm that divides an expense.\u0026#34;\u0026#34;\u0026#34; @abstractmethod def calculate(self, total_amount: int, participants: List[User]) -\u0026gt; List[Split]: raise NotImplementedError Every implementation is bound by a correctness contract that is as important as the interface itself. The amounts of the returned shares must sum exactly to the total amount of the expense. Money must never be created or lost through rounding. Where integer division leaves a remainder, the implementation is responsible for distributing that remainder deterministically, so that the sum remains exact. This contract is the reason the money is stored in minor units, because a remainder can then be assigned as a whole number of minor units rather than lost to a floating point approximation.\n10.3 Equal division The equal strategy divides the total into equal shares. Because an integer total rarely divides evenly among the participants, the remainder is distributed one minor unit at a time to the first participants in the list. This satisfies the contract while distributing the unavoidable rounding difference in a deterministic manner. A total of three hundred and one paise divided among three participants therefore yields shares of one hundred and one, one hundred, and one hundred.\nclass EqualSplit(SplitStrategy): def calculate(self, total_amount, participants): count = len(participants) if count == 0: raise ValueError(\u0026#34;Cannot split an expense among zero participants\u0026#34;) base_share = total_amount // count remainder = total_amount - base_share * count splits = [] for index, user in enumerate(participants): extra = 1 if index \u0026lt; remainder else 0 splits.append(Split(user, base_share + extra)) return splits 10.4 Division by exact amounts The exact strategy assigns an explicitly specified amount to each participant. The amounts are supplied to the strategy through its constructor, which is why the strategy object encapsulates both the algorithm and its configuration. The strategy validates that the supplied amounts sum to the expense total, because a set of exact amounts that does not reconcile with the total is an error on the part of the caller that must be rejected rather than silently absorbed.\nclass ExactSplit(SplitStrategy): def __init__(self, amounts): self._amounts = dict(amounts) def calculate(self, total_amount, participants): supplied_total = sum(self._amounts.get(u, 0) for u in participants) if supplied_total != total_amount: raise ValueError( f\u0026#34;Exact amounts sum to {supplied_total} \u0026#34; f\u0026#34;but the expense total is {total_amount}\u0026#34;) return [Split(u, self._amounts[u]) for u in participants] 10.5 Division by percentage and by shares The percentage strategy divides the total according to a percentage assigned to each participant, requiring that the percentages sum to one hundred. The shares strategy divides the total in proportion to a set of integer weights, which expresses a ratio directly. In both cases the percentages or shares are converted to integer minor units by rounding, and the final participant receives the exact residual so that the shares still sum to the total. This preservation of the total despite intermediate rounding is the mechanism by which the correctness contract is honoured. The full implementations of both strategies appear in splitwise/strategies/split_strategy.py.\n10.6 Why the strategy object carries its configuration An alternative design would pass the per participant inputs, such as the exact amounts or the percentages, into the calculate method rather than into the constructor. The chosen design places them in the constructor so that the calculate method has a uniform signature across every strategy, taking only the total and the participants. This uniformity is what allows the ExpenseManager to hold a strategy of any concrete type behind the abstract interface and invoke it without knowing which concrete algorithm it holds. The strategy object thereby encapsulates both the algorithm and the parameters that algorithm requires, which is the more complete expression of the Strategy pattern.\n11. Stage 9: The Expense and the Balance Update Rule An Expense is an immutable record of a single spending event, implemented in splitwise/domain/expense.py. It captures the payer, the total amount, and the list of per participant shares produced by a splitting strategy. It is an entity because the system retains a history of expenses for auditing and for display.\nThe expense is deliberately passive. It does not update balances and it contains no splitting logic. The division of the total into shares is performed by a strategy before the expense is constructed, and the recording of the resulting debts is performed by the ExpenseManager. Keeping the expense passive means it can be stored, listed, and replayed without triggering side effects, which in turn makes it possible to rebuild the balance model from the expense history should that ever be required.\nThe rule by which an expense updates the balances is the conceptual centre of the system, and it is stated precisely as follows. When one participant, the payer, settles the entire bill, every other participant owes the payer the amount of their own share. The payer\u0026rsquo;s own share requires no entry, because the payer has already paid for it. This rule is applied in the ExpenseManager and not in the expense, so that the expense remains a passive record and the mutation of the balance model is confined to a single place.\n12. Stage 10: The BalanceSheet as the Source of Truth The BalanceSheet, implemented in splitwise/core/balance_sheet.py, is the concrete form of Approach B from Stage 4. It is the single source of truth for who owes whom, and it stores a pairwise net balance for every ordered pair of users that has transacted.\nThe entry balance[debtor][creditor] holds the net amount, in minor units, that the debtor currently owes the creditor. A positive value means the debtor owes the creditor, and a negative value means the creditor owes the debtor. The class maintains an antisymmetry invariant, which is the property that for every pair of users A and B the entry for A owing B equals the negation of the entry for B owing A. This invariant is what allows offsetting debts to cancel automatically, because an obligation in one direction and an equal obligation in the other direction sum to zero in both entries.\nfrom collections import defaultdict from typing import Dict from splitwise.domain.user import User class BalanceSheet: def __init__(self): self._balances: Dict[User, Dict[User, int]] = defaultdict( lambda: defaultdict(int)) def record_debt(self, debtor: User, creditor: User, amount: int) -\u0026gt; None: if debtor == creditor or amount == 0: return self._balances[debtor][creditor] += amount self._balances[creditor][debtor] -= amount def amount_owed(self, debtor: User, creditor: User) -\u0026gt; int: return self._balances.get(debtor, {}).get(creditor, 0) def net_position(self, user: User) -\u0026gt; int: total = 0 for other in list(self._balances.keys()): if other == user: continue total += self._balances[other].get(user, 0) return total The invariant is preserved by a single design decision, namely that the only method which mutates the balances, record_debt, updates both directions together. Because every mutation passes through this one method, and that method always adjusts both entries, no caller can ever leave the two entries inconsistent. This is a general and valuable technique: an invariant that is enforced in exactly one place cannot be violated from anywhere else.\nThe net_position method is the derivation of Approach C from the pairwise map. It computes a user\u0026rsquo;s single net position by summing, over every other user, the amount that the other user owes this user. Because of the antisymmetry invariant this single sum accounts for amounts flowing in both directions, so a positive result means the user is owed money overall and a negative result means the user owes money overall. The method reads the entries with a defensive lookup so that querying a pair that has never transacted does not create spurious entries in the map.\n13. Stage 11: Debt Simplification Debt simplification is the algorithmic centrepiece of the system, implemented in splitwise/core/debt_simplifier.py. The task is to take the net position of every user and produce a small set of payments that settles everyone to a zero balance.\n13.1 The insight that reduces the problem Once the balances have been reduced to net positions, the original detail of who owed whom is no longer relevant to the task of settlement. The only facts that matter are how much each user owes or is owed. The users with a negative net position are debtors who must pay, the users with a positive net position are creditors who must be paid, and the total owed equals the total that is owed, because the balance model conserves money. The problem is therefore to move money from the debtors to the creditors using as few transfers as possible.\n13.2 The greedy algorithm The algorithm repeatedly matches the user who owes the most with the user who is owed the most, and settles the smaller of the two magnitudes between them. Each such transfer reduces at least one of the two users to a zero balance, so the number of participants who still carry a balance strictly decreases at every step and the procedure terminates.\nThe implementation uses two heaps. A heap, also called a priority queue, is a data structure that allows the largest or smallest element of a collection to be retrieved efficiently. Python provides a min heap, which yields the smallest element first, so the magnitudes are negated in order to obtain the behaviour of a max heap, which yields the largest element first. One heap holds the creditors ordered by the amount they are owed, and the other holds the debtors ordered by the magnitude of what they owe.\nimport heapq import itertools from dataclasses import dataclass from typing import Dict, List from splitwise.domain.user import User @dataclass(frozen=True) class Settlement: debtor: User creditor: User amount: int class DebtSimplifier: def simplify(self, net_positions: Dict[User, int]) -\u0026gt; List[Settlement]: counter = itertools.count() creditors, debtors = [], [] for user, amount in net_positions.items(): if amount \u0026gt; 0: heapq.heappush(creditors, (-amount, next(counter), user)) elif amount \u0026lt; 0: heapq.heappush(debtors, (amount, next(counter), user)) settlements = [] while creditors and debtors: owed_negated, _, creditor = heapq.heappop(creditors) owe_signed, _, debtor = heapq.heappop(debtors) owed, owe = -owed_negated, -owe_signed transfer = min(owed, owe) settlements.append(Settlement(debtor, creditor, transfer)) if owed - transfer \u0026gt; 0: heapq.heappush(creditors, (-(owed - transfer), next(counter), creditor)) if owe - transfer \u0026gt; 0: heapq.heappush(debtors, (-(owe - transfer), next(counter), debtor)) return settlements A detail of the implementation deserves explanation. Each heap entry contains a monotonic counter value between the magnitude and the user. This counter is a tie breaker. Without it, two entries of equal magnitude would cause the heap to compare the user objects, and because user objects are not ordered this comparison would raise an error. The counter, being unique and ordered, ensures that the comparison never reaches the user object.\n13.3 Optimality and complexity An honest account of the algorithm requires stating where it sits relative to the theoretical optimum. Producing the provably minimum number of transfers is NP hard in the general case, which means that no algorithm is known that solves every instance optimally in time that grows only polynomially with the number of users, and the problem is believed to admit none. The difficulty arises because the problem is related to the partition problem, in which one seeks to divide a set of numbers into groups of equal sum. The greedy matching used here is therefore a heuristic rather than an exact optimiser. It is the standard near optimal heuristic, it runs quickly, and it produces very few transfers in practice, which is why real expense sharing applications use it. Stating this boundary plainly, rather than claiming that the greedy result is provably minimal, is the mark of an engineer who understands the solution rather than one who has memorised it.\nThe complexity of the algorithm is as follows. Building the two heaps costs time proportional to the number of users multiplied by the logarithm of that number. Each transfer removes at least one participant, so there are at most as many transfers as there are users, and each heap operation costs time proportional to the logarithm of the number of users. The overall cost is therefore proportional to the number of users multiplied by the logarithm of that number.\n13.4 Why greedy matching outperforms arbitrary pairing Matching the largest debtor with the largest creditor at each step maximises the chance of reducing at least one participant to zero at every transfer, which minimises the total number of transfers. Pairing participants in an arbitrary order can leave small residual amounts that require additional transfers to clear. The greedy rule consistently eliminates a participant at each step and avoids creating new small residuals, which keeps the number of transfers low.\n14. Stage 12: The ExpenseManager as a Facade The ExpenseManager, implemented in splitwise/services/expense_manager.py, is a Facade, which is the structural pattern that presents a small and stable interface over a subsystem of several cooperating classes. Client code adds an expense, records a settlement, queries a balance, or requests a simplified settlement plan, and never needs to interact with the splitting strategies, the balance sheet, or the debt simplifier directly.\nThe manager holds no algorithmic logic of its own. It does not compute splits, because that is the responsibility of a strategy. It does not maintain the antisymmetry invariant, because that is the responsibility of the balance sheet. It does not compute settlement plans, because that is the responsibility of the debt simplifier. Keeping the coordinator thin is what prevents it from growing into a single class that accumulates every responsibility in the system, a failure that was identified in the first problem of this series and that recurs whenever a coordinating class is allowed to absorb the work of its collaborators.\nclass ExpenseManager: def __init__(self): self.balance_sheet = BalanceSheet() self.expenses = [] self._simplifier = DebtSimplifier() def add_expense(self, description, total_amount, paid_by, participants, strategy): splits = strategy.calculate(total_amount, participants) computed_total = sum(split.amount for split in splits) if computed_total != total_amount: raise ValueError( f\u0026#34;Splits sum to {computed_total} rather than {total_amount}\u0026#34;) expense = Expense(description, total_amount, paid_by, splits) self.expenses.append(expense) for split in splits: if split.user != paid_by: self.balance_sheet.record_debt( debtor=split.user, creditor=paid_by, amount=split.amount) return expense def settle_up(self, payer, payee, amount): self.balance_sheet.record_debt(debtor=payee, creditor=payer, amount=amount) def simplify_debts(self): return self._simplifier.simplify(self.balance_sheet.all_net_positions()) Two aspects of this class merit attention. First, the splitting strategy is passed into add_expense by the caller rather than chosen inside the method. This is an application of the Dependency Inversion Principle, under which a high level component depends on an abstraction rather than on a concrete implementation. The manager depends on the abstract SplitStrategy interface and is therefore indifferent to which concrete algorithm it receives, which allows a new algorithm to be introduced without any change to the manager.\nSecond, the settle_up method reveals a useful unification. A real world payment from one person to another is modelled as a debt in the opposite direction, because when the payer hands money to the payee the amount the payer owes the payee is reduced. This reuse of the existing debt machinery means that settling up requires no separate code path. Recognising that a payment and an obligation are the same operation viewed from opposite directions is exactly the kind of simplification that a clean data model makes available.\nThe add_expense method also validates that the shares returned by the strategy sum to the total before recording any debt. Although every supplied strategy is expected to honour the correctness contract, verifying it at the boundary protects the balance model from a faulty strategy, and a faulty balance model would be far harder to diagnose than a rejected expense.\n15. Stage 13: Validating the Design with a Walkthrough A design that has not been simulated is a design that is not yet trusted. The following walkthrough traces the classic three friends trip, which is also the scenario implemented in main.py and asserted in the test suite. The amounts are written in rupees for readability and are stored internally as paise.\nThe participants are Alice, Bob, and Carol. The first expense is a dinner of three hundred paid by Alice and divided equally, which makes Bob and Carol each owe Alice one hundred, while Alice\u0026rsquo;s own share of one hundred requires no entry. The second expense is a cab fare of three hundred paid by Bob and divided equally, which makes Alice and Carol each owe Bob one hundred. At this point the obligation between Alice and Bob illustrates the central advantage of the chosen model. Alice now owes Bob one hundred from the cab, and Bob owed Alice one hundred from the dinner, so the two entries cancel and the balance between Alice and Bob returns to zero automatically, without any special handling.\nAfter these two expenses Carol owes one hundred to each of Alice and Bob. If a third expense is added, namely tickets of six hundred paid by Carol and divided in the proportion fifty, twenty five, and twenty five percent, then Alice owes Carol three hundred and Bob owes Carol one hundred and fifty, while Carol\u0026rsquo;s own share requires no entry. The resulting net positions are that Alice owes two hundred, Bob owes fifty, and Carol is owed two hundred and fifty, and these three figures sum to zero as required by the conservation of money.\nThe simplification of these positions proceeds as follows. Carol is the sole creditor, owed two hundred and fifty. The largest debtor is Alice, owing two hundred, who pays Carol two hundred and is thereby reduced to zero. The remaining debtor is Bob, owing fifty, who pays Carol fifty, and both are reduced to zero. The settlement plan is therefore two payments, Alice to Carol of two hundred and Bob to Carol of fifty, which is the minimum for this configuration. This is exactly the output produced by main.py, and it is asserted by the tests in tests/test_expense_manager.py.\nThe walkthrough confirms the property that most distinguishes the chosen model, namely that the obligation between Alice and Bob cancelled itself the moment the offsetting expense was recorded. Under the rejected Approach A the system would carry several stale records and would be obliged to discover the cancellation during simplification. The chosen model made the difficult behaviour automatic, and this is the point worth emphasising when presenting the design.\nThe walkthrough also surfaces a matter that is a scope note rather than a defect. The equal division assigns any leftover minor units to the first participants in the list. For a total that divides evenly this is invisible, but for a total that does not divide evenly the first participants pay one minor unit more than the others. The distribution is deterministic and the sum is exact, but a refinement worth mentioning is to rotate which participant absorbs the remainder across successive expenses, so that the burden of the leftover unit is shared over time.\n16. Stage 14: Anticipating Follow Up Questions 16.1 How would the system support several payers on one expense? The single payer field would be replaced by a mapping from payer to the amount that payer advanced, with the amounts summing to the total. The balance update rule generalises so that each participant owes each payer a proportional share. Neither the balance sheet nor the strategies change, because only the bookkeeping loop inside the manager is affected. The isolation of the balance update inside a single method is what confines the change to one place.\n16.2 How would a new split type be added? A new split type is introduced as a new class that implements the SplitStrategy interface, such as the shares based strategy already present in the project. No existing strategy and no other component requires modification, which is the property described by the Open Closed Principle, under which software should be open to extension but closed to modification. This is the benefit for which the Strategy seam was created in Stage 8.\n16.3 How would the system support several currencies? Money would become a value object carrying both an amount in minor units and a currency, and balances would be tracked separately per currency, because a user may owe an amount in one currency and be owed an amount in another. Cross currency settlement would require an exchange rate service, which would be injected as a dependency in accordance with the Dependency Inversion Principle. This is a substantial extension and would be scoped as such rather than built into the first version.\n16.4 How would users be notified when they are added to an expense? Notification is the province of the Observer pattern, under which a subject maintains a list of observers and informs them when an event occurs. The manager or the group would act as the subject, and notifiers for email and for push messages would act as observers, each informed when an expense is added. This is the same pattern that drove the display boards in the parking lot problem. It is not built in the first version, in accordance with the principle of not adding capability before it is required, but the point at which it would attach is clear.\n16.5 How would the system persist data? Two approaches are available. The first is to persist only the immutable expense history and to rebuild the balance sheet by replaying that history when the system starts, which is safe because the balances are derivable from the history. The second is to persist the balances directly through a repository abstraction for speed, with periodic reconciliation against a replay. In both cases the repository is introduced as an abstraction on which the manager depends, again in accordance with the Dependency Inversion Principle. That the balances can be replayed from history is a direct dividend of having kept the expense an immutable record.\n16.6 Is the settlement plan guaranteed to be minimal? It is not, and this must be stated plainly. The exact minimisation of the number of transfers is NP hard, so the greedy heuristic is used instead. It is near optimal, it runs quickly, and it is what production applications use. If exact minimality were required for small groups, an exact solver based on examining subsets could be added for groups below a modest size, with the greedy heuristic used beyond that threshold.\n16.7 How would the design be tested? The design is highly testable, which is a consequence of its separation of concerns. The strategies are pure functions of their inputs and are tested by asserting that their output sums to the total and divides correctly. The balance sheet is tested by asserting the antisymmetry invariant and the automatic cancellation of offsetting debts. The simplifier is tested by supplying crafted net positions and asserting that the settlements conserve money, clear every balance, and are few in number. The manager is tested end to end with the three friends trip. The accompanying project contains all of these tests under the tests directory, and they may be run with a single command.\n17. Stage 15: Running the Project in PyCharm The accompanying project is arranged so that it can be demonstrated directly in the PyCharm integrated development environment. The steps are as follows.\nFirst, open the project by choosing Open from the PyCharm welcome screen and selecting the splitwise_lld folder. PyCharm indexes the project and recognises the package structure. If prompted, mark the project root as a sources root so that the imports of the form from splitwise.domain.user import User resolve correctly.\nSecond, run the demonstration by right clicking the file main.py in the project view and choosing the run action. The output shows the three expenses, the resulting net positions, and the simplified settlement plan in which Carol is paid by both Alice and Bob. This provides a concrete artefact to project during a class.\nThird, run the test suite by right clicking the tests folder and choosing the option to run the tests within it. PyCharm displays the results in its test runner, and all tests should pass. Running the tests live demonstrates that the design behaves as the walkthrough claimed, and it allows a lesson to show the effect of deliberately breaking a piece of code and observing which test fails.\nA recommended order for presenting the code in a class follows the direction of the dependencies, from the innermost data types outward to the coordinator. Begin with the money module to establish the minor unit convention, proceed to the user, split, expense, and group types, then to the splitting strategies, then to the balance sheet, which contains the central modeling decision, then to the debt simplifier, then to the expense manager, and finally to the demonstration in main.py, which shows the whole system in operation.\n18. Summary Tables The following table summarises the classes of the design, their responsibilities, and the design idea each one embodies.\nClass or module Responsibility Design idea embodied User Identity and profile of a person Entity with identity based equality Group A named scope for a set of expenses Entity that holds no balance data Split One participant\u0026rsquo;s share of one expense Immutable value object Expense Immutable record of a spending event Passive entity supporting replay SplitStrategy and subclasses Interchangeable division algorithms Strategy pattern BalanceSheet Source of truth for pairwise debt Net balance model, single mutation point DebtSimplifier Minimal set of settlement payments Greedy heuristic over a heap ExpenseManager Coordination of the subsystem Facade with dependency injection The following table summarises the design decisions, the alternatives that were considered, and the reason for the choice.\nDecision Alternatives considered Reason for the choice Store money as integer minor units Floating point rupees Integer arithmetic is exact and avoids rounding error Represent debt as a pairwise net map One record per expense; single net position per user Constant time queries and automatic cancellation, with net positions derived on demand Encapsulate split algorithms as strategies A single method with conditional branches New algorithms added without modifying existing ones Confine balance mutation to one method Allow callers to update entries directly The antisymmetry invariant cannot then be violated Simplify debts with a greedy heuristic An exact optimiser Exact minimisation is NP hard, and greedy is near optimal and fast Coordinate through a thin facade A single class holding all logic Prevents the coordinator from becoming a class that absorbs every responsibility 19. Key Takeaways The data model is the design. The single decision to represent debt as a pairwise net map, with net positions derived on demand, made every subsequent operation efficient and made offsetting debts cancel automatically. The rejected alternative of storing one record per expense would have made the most common query expensive and would never have allowed debts to cancel. In a data centred problem the greatest share of the reasoning budget should be spent on the choice of model.\nDerive rather than duplicate. The net position of a user is computed from the pairwise map rather than stored, and the balances can be replayed from the immutable expense history. Each derivation removes an opportunity for two copies of the same information to disagree.\nMoney requires integer minor units and explicit handling of rounding. Amounts are stored as integers, and every splitting strategy distributes any rounding remainder deterministically so that the shares sum exactly to the total. In a financial system rounding is a first class concern rather than an afterthought.\nNot every problem calls for a behavioural pattern. This design contains no state machine, and introducing one would have been a mistake. Its centre is a data model and an algorithm. Recognising when not to apply a pattern is part of the skill of applying patterns well.\nInvariants are enforced most reliably in a single place. The antisymmetry of the balance sheet is guaranteed because the only method that mutates the balances updates both directions together. An invariant confined to one method cannot be violated from elsewhere.\nHonesty about complexity is a professional virtue. The design states plainly that exact debt minimisation is NP hard and that the greedy algorithm is a near optimal heuristic. Naming the boundary of a solution distinguishes an engineer who understands the solution from one who has memorised it.\n20. Practice Questions and Solutions The following questions are intended to exercise the reasoning developed in this document rather than the recall of code. Each should be attempted before its solution is read.\nQuestion 1 A colleague proposes to store a list of records of the form debt from one user to another with an associated expense identifier, one record for each participant of each expense, and never to delete any record. Identify the principal problem with this proposal and describe the representation that resolves it.\nSolution The proposal is Approach A from Stage 4, and it has two connected problems. The most frequent query, namely the net balance between two users, requires a scan and summation of the entire history for those two users, which costs time proportional to the length of the history rather than constant time. In addition, offsetting obligations never cancel, so the stored data grows without bound and the system must rediscover cancellations during every simplification. The resolution is the pairwise net balance map of Approach B, in which a single entry per ordered pair is updated in place, queries are answered in constant time, and offsetting debts cancel automatically. The per expense records may still be retained, but as an immutable history for auditing and replay rather than as the source of truth for balances.\nQuestion 2 Explain why the record_debt method of the balance sheet updates both the entry for the debtor owing the creditor and the entry for the creditor owing the debtor within a single method, rather than allowing callers to update one entry at a time. State the category of defect this design prevents.\nSolution Updating both entries within one method guarantees the antisymmetry invariant, which is the property that the entry for A owing B always equals the negation of the entry for B owing A. If callers were permitted to update one entry at a time, any caller that omitted the mirroring update would silently corrupt the balances, and the corruption would surface only later as an incorrect settlement plan, which is difficult to trace to its origin. By making the single mutation path adjust both entries together, the invariant becomes impossible to violate from outside the class. The prevented defect is the class of silent data corruption arising from a partially applied update.\nQuestion 3 The team wishes to support, at the time an expense is created, a choice among equal, exact, percentage, and shares based division, and additionally to apply an optional service charge that is added to the total and divided in the same manner as the underlying expense. Identify the patterns that apply and describe where each attaches without requiring a new class for every combination.\nSolution The choice among division methods is the Strategy pattern, expressed by injecting a concrete SplitStrategy when the expense is created, and the addition of a division method such as shares is a new strategy class that requires no modification to the existing strategies, in accordance with the Open Closed Principle. The optional service charge is naturally expressed as a Decorator over a splitting strategy, that is, a strategy that wraps another strategy, inflates the total by the service charge, and then delegates the division of the inflated total to the wrapped strategy. This composition avoids writing a separate class for every combination of division method and service charge, because the service charge decorator can wrap any strategy. This is the same relationship between the Strategy pattern and the Decorator pattern that arose in the layered pricing of the parking lot problem.\nQuestion 4 In the greedy simplifier, explain why the largest debtor is matched with the largest creditor at each step rather than pairing participants in an arbitrary order, and state honestly what guarantee the algorithm does and does not provide.\nSolution Matching the largest debtor with the largest creditor maximises the probability that at least one of the two participants is reduced to a zero balance at each transfer, which minimises the total number of transfers. Pairing participants in an arbitrary order can leave small residual amounts that require additional transfers to clear. The algorithm provides the guarantee that every balance is settled and that money is conserved, and it produces a small number of transfers in practice. It does not provide the guarantee that the number of transfers is the theoretical minimum, because exact minimisation is NP hard. The correct characterisation is therefore that the algorithm is a near optimal heuristic rather than an exact optimiser.\nQuestion 5 Describe how the design would change to allow an expense to be advanced by more than one payer, and identify which components remain unchanged.\nSolution The single payer field of the expense would be replaced by a mapping from each payer to the amount that payer advanced, with the amounts summing to the total. The balance update rule would generalise so that each participant owes each payer a share proportional to that payer\u0026rsquo;s contribution. The splitting strategies remain unchanged, because they continue to divide the total among the participants without regard to who paid. The balance sheet remains unchanged, because it continues to record debts between pairs of users. Only the bookkeeping loop inside the add_expense method of the manager changes, because the recording of debts now iterates over several payers. The confinement of the balance update to a single method is what limits the change to one location.\nThis content is part of Codeverra, a platform for learning coding, data science, DSA, and AI from scratch. Explore more at https://codeverra.com\n","permalink":"https://blogs.codeverra.com/blog/low_level_design/interview_questions/06j_lld_splitwise_deep_dive/","summary":"\u003ch1 id=\"lld-deep-dive-5-designing-splitwise-expense-sharing\"\u003eLLD Deep Dive #5: Designing Splitwise (Expense Sharing)\u003c/h1\u003e\n\u003cblockquote\u003e\n\u003cp\u003e\u003cstrong\u003eWhat this document is.\u003c/strong\u003e This is a complete, step by step walkthrough of how to design an expense sharing system of the kind popularised by the Splitwise application. It reasons from first principles, states the alternatives at every decision point, and justifies the option that is chosen. The design developed here is implemented as a small, runnable, modular Python project that accompanies this document, so that every concept discussed in prose can be examined and executed as real code.\u003c/p\u003e","tags":null,"title":""},{"content":"\u003c!DOCTYPE html\u003e Python: Where to Start and Where It Can Take You | Codeverra \u003c! - HEADER - \u003e // codeverra Python Fundamentals Series \u003c! - HERO - \u003e Lecture 01 / Python Fundamentals Python: Where to Start\nand Where It Can Take You A complete introduction to why Python matters, how to set it up, the basics of its syntax, and an honest roadmap of everything that lies ahead for you. Series Python Fundamentals Level Absolute Beginner Read time 20 min \u003c! - TABLE OF CONTENTS - \u003e In this article\nWhy Python is Popular Python vs Other Languages Setting Up Python and Jupyter Python Syntax and Rules Basic Data Types Your Learning Roadmap \u003c! - MAIN CONTENT - \u003e \u003c! - SECTION 1: WHY PYTHON - \u003e 01 / Why Python Why Python is the Language to Learn Right Now Every few years, a programming language rises to the top of developer surveys and job listings in a way that is hard to ignore. Python has been doing this consistently since 2015, and in 2024 it sits at number one on the TIOBE index, the Stack Overflow developer survey, and nearly every \"most in-demand languages\" ranking published by hiring platforms. Python is not just a language. It is the lingua franca of modern computing - from a five-line script to systems running at Google, ISRO, and Netflix. Here is what actually drives that popularity:\nIt reads like English Python was designed with one philosophy at its core: code should be easy to read. Guido van Rossum, who created Python in 1991, believed that code is read far more often than it is written. The result is a language where you spend less time fighting syntax and more time solving actual problems. # Calculate GST for a list of prices - this almost reads like a sentence prices = [299, 499, 1299, 2499] with_gst = [round(price * 1.18, 2) for price in prices] print(with_gst) # [352.82, 588.82, 1532.82, 2948.82] Python One language, every domain Most languages have a primary domain. Java is enterprise backend. JavaScript is the web. C++ is systems programming. Python refuses to be boxed in. The same language powers Zomato's recommendation engine, ISRO's satellite telemetry processing, a data analyst's quarterly sales report, and the fine-tuning of large language models. You learn it once and it opens multiple doors. The ecosystem is unmatched Python has over 400,000 packages on PyPI. Libraries like NumPy, Pandas, Scikit-learn, TensorFlow, FastAPI, and Django represent hundreds of thousands of engineering hours distilled into tools you can import in one line. Whatever problem you are trying to solve, someone has almost certainly built a Python library for it. The job market is real As of 2024, Python is required or preferred in the majority of data science, machine learning, backend engineering, and automation job postings in India and globally. Companies like Swiggy, Meesho, Razorpay, PhonePe, and nearly every product startup in Bangalore list Python as a core skill. The demand is not abstract - it is in active job listings right now. Note for beginners You do not need to understand data science or machine learning to start with Python. The fundamentals you learn in the first few weeks apply equally whether you end up in web development, analytics, or AI. Start here. Specialise later. \u003c! - SECTION 2: PYTHON VS OTHERS - \u003e 02 / Language Comparison Python vs Other Popular Languages Understanding where Python fits relative to other languages helps you make an informed decision about what to learn and why. This is not about declaring one language \"better\" - every language was built to solve specific problems. It is about understanding the trade-offs. Language Primary Use Learning Curve Speed vs Python Python Data, AI/ML, Backend, Automation Low Moderate The baseline JavaScript Web frontend, Node.js backend Low Moderate Dominates the browser; Python dominates data Java Enterprise backend, Android Medium Fast More verbose; Python is faster to write C / C++ Systems, embedded, game engines High Very Fast Python is 10-100x slower but 10x faster to write R Statistical computing, research Medium Moderate Python has surpassed R for most data science work SQL Database queries Low Fast Not a general language; Python and SQL are used together The honest trade-off: Speed vs Productivity Python is interpreted, dynamically typed, and runs slower than compiled languages like C++ or Java. For building a video game engine or an operating system kernel, Python is the wrong choice. But for data analysis, web APIs, machine learning pipelines, and automation scripts - which is where most software jobs actually are - Python's development speed far outweighs its execution speed disadvantage. And when raw performance is needed, Python code calls into C libraries like NumPy under the hood anyway. The Python + SQL combination If you are heading toward data analytics or data engineering, Python and SQL together are far more powerful than either alone. SQL handles what lives in databases. Python handles everything else. Both are covered in the Codeverra curriculum. \u003c! - SECTION 3: SETUP - \u003e 03 / Setup Setting Up Python and Jupyter Notebook Getting Python running on your machine takes about ten minutes. There are two things to install: Python itself, and Jupyter Notebook - the interactive environment we will use for all exercises. Step 1: Install Python Download Python Go to python.org/downloads and download the latest stable version (3.12 or newer). On Windows, make sure to check \"Add Python to PATH\" during installation. On macOS and Linux, Python may already be installed - run python3 - version in your terminal to check.\nVerify the installation Open a terminal (Command Prompt on Windows, Terminal on Mac/Linux) and run:\npython - version # Expected: Python 3.12.x (or newer) python -c \"print('Python is working!')\" # Expected: Python is working! Shell Step 2: Install Jupyter via pip pip is Python's package manager. It is installed automatically with Python. Use it to install Jupyter: pip install notebook # This installs Jupyter Notebook and all its dependencies Shell Step 3: Launch Jupyter Notebook jupyter notebook # Opens a browser tab at http://localhost:8888 # Navigate to a folder and click \"New \u003e Python 3\" to create your first notebook Shell Alternative: Anaconda Distribution If you prefer an all-in-one installer, Anaconda bundles Python, Jupyter, and 250+ data science libraries. Download it at anaconda.com. Recommended if you plan to focus on data science or machine learning. Why Jupyter Notebook? Jupyter lets you write and run Python code in small, isolated cells. You see the output immediately below each cell. This makes it ideal for learning because you can experiment, make mistakes, and observe results without running an entire file each time. It is also the standard tool used by data analysts and data scientists worldwide. Your First Cell Once Jupyter is open, create a new notebook and type this in the first cell, then press Shift + Enter to run it:\n# Your first Python program name = \"Aarav\" city = \"Bangalore\" print(f\"Hello! My name is {name} and I am from {city}.\") # Output: Hello! My name is Aarav and I am from Bangalore. Python If you see the output, your setup is complete.\n\u003c! - SECTION 4: SYNTAX - \u003e 04 / Syntax and Rules Python Syntax and the Rules You Need to Know Every language has grammar rules. Python's rules are minimal but strict in certain areas. Understanding them from day one saves you hours of debugging confusing errors later. Indentation is structure Python uses indentation (4 spaces, not tabs) to define code blocks. There are no curly braces. Get this wrong and your code either fails or does something unintended.\nCase sensitivity name, Name, and NAME are three different variables. Python is case-sensitive everywhere.\nNo semicolons needed Each statement goes on its own line. Semicolons are valid but never required and rarely used in Python code.\nComments with # Anything after a # on a line is a comment. Python ignores it. Use comments liberally to explain your thinking.\nVariable naming rules Start with a letter or underscore. No spaces. No special characters. Use lowercase with underscores: student_name, not StudentName.\nDynamic typing You do not declare a variable's type. Python figures it out from the value you assign. x = 5 makes x an integer. x = \"hello\" makes x a string.\nIndentation in Practice This is the most important rule to internalize early. Here is what correct and incorrect indentation looks like: # CORRECT - consistent 4-space indentation score = 85 if score \u003e= 60: print(\"Passed\") # 4 spaces in if score \u003e= 85: print(\"Distinction\") # 8 spaces in (nested block) else: print(\"Failed\") # 4 spaces in # WRONG - mixing indentation levels causes IndentationError if score \u003e= 60: print(\"Passed\") # 2 spaces - inconsistent! print(\"Continuing...\") # 6 spaces - IndentationError Python Variables and Assignment # Assigning values student_name = \"Priya Sharma\" age = 21 gpa = 8.7 is_enrolled = True # Multiple assignment in one line x, y, z = 10, 20, 30 # Reassigning changes the type - this is valid in Python value = 42 # integer value = \"hello\" # now a string - Python allows this # Check the type of any variable print(type(age)) # \u0026lt;class 'int'\u0026gt; print(type(gpa)) # \u0026lt;class 'float'\u0026gt; Python Printing Output # Basic print print(\"Namaste, World!\") # Print multiple values print(\"Name:\", student_name, \"| Age:\", age) # f-string formatting - the modern, preferred way print(f\"{student_name} is {age} years old with a GPA of {gpa}.\") # Format numbers inside f-strings price = 1234567.89 print(f\"Price: Rs.{price:,.2f}\") # Rs.1,234,567.89 Python \u003c! - SECTION 5: DATA TYPES - \u003e 05 / Basic Data Types Basic Data Types in Python A data type tells Python what kind of value a variable holds and what operations are allowed on it. Python has a handful of built-in types that cover the vast majority of what you will work with in the beginning. Here is an overview before we go into each one: int Integer age = 21 Whole numbers, positive or negative, no decimal point.\nfloat Floating Point price = 299.99 Numbers with a decimal component. Used for measurements and money.\nstr String city = \"Delhi\" Text data. Any sequence of characters inside quotes.\nbool Boolean active = True Only two values: True or False. The foundation of all conditions.\nNoneType None result = None Represents the intentional absence of a value. Not zero, not empty - nothing.\ncomplex Complex z = 3 + 4j Complex numbers with real and imaginary parts. Used in scientific computing.\nIntegers and Floats # Integers - no limit on size in Python population = 1_428_627_663 # underscores allowed for readability odi_centuries = 50 # Floats gst_rate = 0.18 ipl_fee = 1_294_000_000.00 # Arithmetic operators print(10 + 3) # 13 addition print(10 - 3) # 7 subtraction print(10 * 3) # 30 multiplication print(10 / 3) # 3.333... always returns float print(10 // 3) # 3 floor division (integer result) print(10 % 3) # 1 modulo (remainder) print(10 ** 3) # 1000 exponentiation Python Strings # Creating strings name = \"Rohan Mehta\" city = 'Hyderabad' # single or double quotes, both work address = \"\"\" 123, MG Road, Bangalore - 560001 \"\"\" # triple quotes for multiline strings # Common string operations print(len(name)) # 11 - length of string print(name.upper()) # ROHAN MEHTA print(name.lower()) # rohan mehta print(name.replace(\"Mehta\", \"Sharma\")) # Rohan Sharma print(name.split(\" \")) # ['Rohan', 'Mehta'] # Indexing and slicing print(name[0]) # R - first character print(name[-1]) # a - last character print(name[0:5]) # Rohan - slice Python Booleans and Comparison # Boolean values is_registered = True has_paid = False # Comparison operators return booleans print(10 \u003e 5) # True print(10 == 10) # True - note: == not = print(10 != 5) # True print(10 \u003e= 10) # True print(10 \u003c 5) # False # Logical operators print(True and False) # False print(True or False) # True print(not True) # False # Real use case age = 22 has_degree = True eligible = age \u003e= 21 and has_degree print(eligible) # True Python Type Conversion # Convert between types explicitly age_str = \"25\" age_int = int(age_str) # string to int price_int = 299 price_float = float(price_int) # int to float: 299.0 score_float = 87.9 score_int = int(score_float) # float to int: 87 (truncates, does NOT round) number = 42 number_str = str(number) # int to string: \"42\" # Common pattern: reading user input (always a string, must be converted) age = int(input(\"Enter your age: \")) Python \u003c! - ROADMAP SECTION - full width dark - \u003e 06 / Your Roadmap The Complete Python Learning Roadmap This is the map we will follow together at Codeverra. Every lesson, practice sheet, and masterclass published on this site corresponds to a node on this roadmap. Bookmark this page. Click a phase tab to explore what you will learn and what resources are available for that phase. \u003c! - Phase tabs - \u003e Phase 1: Foundations Phase 2: Core Python Phase 3: Data Tools Phase 4: Specialisation \u003c! - PHASE 1 - \u003e 01 Python Foundations The building blocks that everything else depends on. Master these before moving forward.\nWhy Python and setting up your environment This Article Basic data types: int, float, str, bool, None This Article Variables, assignment, and naming conventions This Article Operators: arithmetic, comparison, logical Lecture 01 Input and output: print(), input(), f-strings Lecture 01 Type conversion: int(), float(), str() Lecture 01 Control flow: if, elif, else Lecture 02 Loops: for, while, break, continue, range() Lecture 03 Available on Codeverra Python Loops Masterclass - covers for, while, enumerate, zip, break, continue, and comprehensions Loops Practice Workbook - 20 questions with full solutions (Easy to Hard) \u003c! - PHASE 2 - \u003e 02 Core Python The essential Python toolkit that professional developers use every day.\nCollections: List, Tuple, Dictionary, Set Practice Available List comprehensions and dict comprehensions Practice Available Functions: def, parameters, return values Practice Available *args, **kwargs, default parameters Practice Available Scope, closures, and higher order functions Practice Available Lambda, map(), filter(), reduce() Practice Available Decorators: basic, with arguments, chaining Practice Available Exception handling: try, except, finally, raise Coming Soon File handling: reading, writing, with statement Coming Soon Modules and packages: import, pip, venv Coming Soon Available on Codeverra Python Collections Practice Sheet - List, Tuple, Dict, Set with 40+ questions and solutions Python Functions Masterclass - all concepts from basics through decorators and generators Functions Practice Workbook - 20 questions (Easy to Hard) with full solutions \u003c! - PHASE 3 - \u003e 03 Data Tools and Object-Oriented Python The libraries and patterns that connect foundational Python to real-world work.\nObject-Oriented Programming: classes and objects Coming Soon Inheritance, encapsulation, polymorphism Coming Soon NumPy: arrays, operations, linear algebra Masterclass Available Pandas: DataFrames, cleaning, GroupBy, merge Masterclass Available Matplotlib and Seaborn: data visualisation Coming Soon SQL with Python: sqlite3, SQLAlchemy, psycopg2 Separate SQL Series Working with APIs: requests, JSON, REST Coming Soon Itertools, functools, and collections modules Coming Soon Available on Codeverra NumPy Masterclass - 17 sections covering all major topics with Indian-context examples and 15 practice questions with solutions Pandas Masterclass - 22 sections from DataFrame basics to MultiIndex, GroupBy, and performance tips with 13 practice questions and solutions SQL with Python course - Modules 1-3 available (Lessons through real Kaggle datasets) \u003c! - PHASE 4 - \u003e 04 Specialisations After Phase 3, you have enough to choose a direction. Each path builds on the same core.\nAnalytics Data Analyst Turn raw data into decisions. Heavy use of SQL, Pandas, Excel, and visualisation tools. Often the fastest path to a first data job.\nPandas SQL Matplotlib Seaborn Power BI / Tableau Statistics Science Data Scientist Build predictive models and extract deeper insight from data. Requires stronger statistics and machine learning foundations.\nNumPy Pandas Scikit-learn Statistics Feature Engineering Experiment Design ML / AI ML Engineer Build, train, and deploy machine learning models at scale. Bridges data science and software engineering.\nPyTorch / TensorFlow MLflow Docker FastAPI Cloud (AWS/GCP) MLOps AI / LLM AI / LLM Engineer Build applications powered by large language models. One of the fastest-growing roles in tech as of 2024.\nLangChain OpenAI API RAG Vector DBs Prompt Engineering Fine-tuning Backend Backend Engineer Build the APIs and server-side logic that power web and mobile applications. Python's frameworks are production-grade and widely used.\nFastAPI Django PostgreSQL Redis Docker REST / GraphQL Engineering Data Engineer Build the pipelines that move and transform data at scale. Combines Python, SQL, and cloud infrastructure.\nApache Spark Airflow dbt Kafka BigQuery / Redshift ETL Pipelines Note on specialisation timing Do not specialise too early. Phases 1 and 2 are shared across all paths. Rushing into Scikit-learn before understanding Python fundamentals is one of the most common mistakes beginners make. A solid Phase 1 and Phase 2 takes 3 to 5 months of consistent practice for most beginners. Phase 3 onwards, the direction you choose will determine which electives and projects to focus on. \u003c! - CLOSING SECTION - \u003e Where to Go From Here You now have Python installed, an understanding of its syntax, a working knowledge of its basic data types, and a clear map of where this journey leads. That is more than most beginners have before writing their first line of code. The next lecture covers control flow - if/elif/else conditions and how to make your programs make decisions. After that, loops. Then collections. Each concept builds on the last. The roadmap above shows exactly where each piece fits. The single best thing you can do right now is open Jupyter, type out the code examples in this article by hand (not copy-paste), and run them. Understanding comes from doing, not reading. Every practice sheet, masterclass, and project on Codeverra is designed to fit into one of the four phases on the roadmap. When you complete a resource, come back to this page and see which box it fills. That progress tracking will keep you oriented as the curriculum grows. \u003c! - FOOTER - \u003e codeverra.com \u0026nbsp;\u0026middot;\u0026nbsp; Python Fundamentals Series \u0026nbsp;\u0026middot;\u0026nbsp; Lecture 01\ncodeverra.com \u0026nbsp;\u0026middot;\u0026nbsp; learn.codeverra.com ","permalink":"https://blogs.codeverra.com/blog/python-beginners-introduction/","summary":"\u003c!DOCTYPE html\u003e\n\u003chtml lang=\"en\"\u003e\n\u003chead\u003e\n\u003cmeta charset=\"UTF-8\"\u003e\n\u003cmeta name=\"viewport\" content=\"width=device-width, initial-scale=1.0\"\u003e\n\u003ctitle\u003ePython: Where to Start and Where It Can Take You | Codeverra\u003c/title\u003e\n\u003clink rel=\"preconnect\" href=\"https://fonts.googleapis.com\"\u003e\n\u003clink href=\"https://fonts.googleapis.com/css2?family=Playfair+Display:ital,wght@0,400;0,600;0,700;1,400\u0026family=Source+Sans+3:wght@300;400;500;600\u0026family=JetBrains+Mono:wght@400;500\u0026display=swap\" rel=\"stylesheet\"\u003e\n\u003cstyle\u003e\n :root {\n - ink: #1a1a1a;\n - ink-light: #444;\n - ink-faint: #888;\n - paper: #f7f4ef;\n - paper-dark: #ede9e2;\n - rule: #d8d2c8;\n - accent: #c0392b;\n - accent-2: #2c6e8a;\n - accent-3: #27704a;\n - code-bg: #1e1e2e;\n - code-text: #cdd6f4;\n - code-green: #a6e3a1;\n - code-blue: #89b4fa;\n - code-pink: #f38ba8;\n - code-yellow:#f9e2af;\n - serif: 'Playfair Display', Georgia, serif;\n - sans: 'Source Sans 3', system-ui, sans-serif;\n - mono: 'JetBrains Mono', 'Fira Code', monospace;\n }\n\n *, *::before, *::after { box-sizing: border-box; margin: 0; padding: 0; }\n\n html { scroll-behavior: smooth; }\n\n body {\n background: var( - paper);\n color: var( - ink);\n font-family: var( - sans);\n font-size: 17px;\n line-height: 1.75;\n -webkit-font-smoothing: antialiased;\n }\n\n /* ---- HEADER ---- */\n .site-header {\n background: var( - ink);\n padding: 14px 40px;\n display: flex;\n align-items: center;\n justify-content: space-between;\n position: sticky;\n top: 0;\n z-index: 100;\n border-bottom: 2px solid var( - accent);\n }\n .site-logo {\n font-family: var( - mono);\n font-size: 15px;\n color: #fff;\n letter-spacing: 0.08em;\n text-decoration: none;\n }\n .site-logo span { color: var( - accent); }\n .header-tag {\n font-family: var( - sans);\n font-size: 12px;\n font-weight: 500;\n color: #888;\n letter-spacing: 0.1em;\n text-transform: uppercase;\n }\n\n /* ---- HERO ---- */\n .hero {\n background: var( - ink);\n color: #fff;\n padding: 80px 40px 70px;\n position: relative;\n overflow: hidden;\n }\n .hero::before {\n content: '';\n position: absolute;\n top: 0; left: 0; right: 0; bottom: 0;\n background:\n repeating-linear-gradient(\n 90deg,\n transparent,\n transparent 79px,\n rgba(255,255,255,0.03) 79px,\n rgba(255,255,255,0.03) 80px\n );\n pointer-events: none;\n }\n .hero-inner {\n max-width: 820px;\n margin: 0 auto;\n position: relative;\n }\n .hero-label {\n font-family: var( - mono);\n font-size: 12px;\n color: var( - accent);\n letter-spacing: 0.15em;\n text-transform: uppercase;\n margin-bottom: 24px;\n display: block;\n }\n .hero h1 {\n font-family: var( - serif);\n font-size: clamp(36px, 5vw, 58px);\n font-weight: 700;\n line-height: 1.15;\n margin-bottom: 28px;\n letter-spacing: -0.01em;\n }\n .hero h1 em {\n font-style: italic;\n color: #c8bfae;\n }\n .hero-sub {\n font-size: 18px;\n color: #b0a898;\n font-weight: 300;\n max-width: 600px;\n line-height: 1.6;\n margin-bottom: 40px;\n }\n .hero-meta {\n display: flex;\n gap: 32px;\n font-size: 13px;\n color: #666;\n font-family: var( - mono);\n flex-wrap: wrap;\n }\n .hero-meta span { color: #888; }\n .hero-meta strong { color: #aaa; }\n\n /* ---- TABLE OF CONTENTS ---- */\n .toc-bar {\n background: var( - paper-dark);\n border-bottom: 1px solid var( - rule);\n padding: 0;\n }\n .toc-inner {\n max-width: 820px;\n margin: 0 auto;\n padding: 20px 40px;\n }\n .toc-title {\n font-family: var( - mono);\n font-size: 11px;\n letter-spacing: 0.12em;\n text-transform: uppercase;\n color: var( - ink-faint);\n margin-bottom: 12px;\n }\n .toc-list {\n display: flex;\n flex-wrap: wrap;\n gap: 8px 0;\n list-style: none;\n counter-reset: toc;\n }\n .toc-list li {\n counter-increment: toc;\n width: 50%;\n font-size: 14px;\n color: var( - ink-light);\n padding: 2px 0;\n }\n .toc-list li::before {\n content: counter(toc, decimal-leading-zero) \". \";\n font-family: var( - mono);\n font-size: 11px;\n color: var( - accent);\n margin-right: 6px;\n }\n .toc-list a {\n color: inherit;\n text-decoration: none;\n border-bottom: 1px solid transparent;\n transition: border-color 0.2s, color 0.2s;\n }\n .toc-list a:hover { color: var( - ink); border-bottom-color: var( - accent); }\n\n /* ---- MAIN CONTENT ---- */\n .main {\n max-width: 820px;\n margin: 0 auto;\n padding: 60px 40px 80px;\n }\n\n /* ---- SECTIONS ---- */\n .section { margin-bottom: 72px; }\n\n .section-label {\n font-family: var( - mono);\n font-size: 11px;\n color: var( - accent);\n letter-spacing: 0.15em;\n text-transform: uppercase;\n display: block;\n margin-bottom: 12px;\n }\n\n h2 {\n font-family: var( - serif);\n font-size: 32px;\n font-weight: 700;\n line-height: 1.2;\n color: var( - ink);\n margin-bottom: 20px;\n padding-bottom: 14px;\n border-bottom: 2px solid var( - rule);\n }\n\n h3 {\n font-family: var( - serif);\n font-size: 22px;\n font-weight: 600;\n margin-top: 40px;\n margin-bottom: 14px;\n color: var( - ink);\n }\n\n h4 {\n font-family: var( - sans);\n font-size: 14px;\n font-weight: 600;\n letter-spacing: 0.08em;\n text-transform: uppercase;\n color: var( - ink-faint);\n margin-top: 28px;\n margin-bottom: 10px;\n }\n\n p { margin-bottom: 18px; color: var( - ink-light); }\n p strong { color: var( - ink); font-weight: 600; }\n\n /* ---- PULLQUOTE ---- */\n .pullquote {\n border-left: 4px solid var( - accent);\n margin: 36px 0;\n padding: 20px 28px;\n background: var( - paper-dark);\n font-family: var( - serif);\n font-size: 20px;\n font-style: italic;\n color: var( - ink);\n line-height: 1.5;\n }\n\n /* ---- CALLOUT BOX ---- */\n .callout {\n border: 1px solid var( - rule);\n border-left: 4px solid var( - accent-2);\n background: #f0f6fa;\n padding: 20px 24px;\n margin: 28px 0;\n border-radius: 0 4px 4px 0;\n }\n .callout.green { border-left-color: var( - accent-3); background: #f0faf4; }\n .callout-title {\n font-family: var( - mono);\n font-size: 11px;\n letter-spacing: 0.12em;\n text-transform: uppercase;\n color: var( - accent-2);\n margin-bottom: 8px;\n display: block;\n }\n .callout.green .callout-title { color: var( - accent-3); }\n .callout p { margin: 0; font-size: 15px; }\n\n /* ---- CODE BLOCKS ---- */\n pre {\n background: var( - code-bg);\n border-radius: 6px;\n padding: 24px 28px;\n overflow-x: auto;\n margin: 24px 0;\n position: relative;\n border: 1px solid #313244;\n }\n pre .lang-tag {\n position: absolute;\n top: 10px;\n right: 14px;\n font-family: var( - mono);\n font-size: 10px;\n color: #45475a;\n letter-spacing: 0.1em;\n text-transform: uppercase;\n }\n code {\n font-family: var( - mono);\n font-size: 14px;\n line-height: 1.7;\n color: var( - code-text);\n }\n p code, li code {\n background: var( - paper-dark);\n color: var( - accent);\n padding: 1px 6px;\n border-radius: 3px;\n font-size: 13.5px;\n border: 1px solid var( - rule);\n }\n .kw { color: var( - code-pink); }\n .fn { color: var( - code-blue); }\n .st { color: var( - code-green); }\n .cm { color: #6c7086; font-style: italic; }\n .nm { color: var( - code-yellow); }\n .op { color: #cba6f7; }\n\n /* ---- COMPARISON TABLE ---- */\n .compare-table {\n width: 100%;\n border-collapse: collapse;\n margin: 28px 0;\n font-size: 15px;\n }\n .compare-table thead tr {\n background: var( - ink);\n color: #fff;\n }\n .compare-table th {\n font-family: var( - sans);\n font-weight: 600;\n font-size: 13px;\n letter-spacing: 0.05em;\n padding: 12px 16px;\n text-align: left;\n }\n .compare-table td {\n padding: 11px 16px;\n border-bottom: 1px solid var( - rule);\n color: var( - ink-light);\n vertical-align: top;\n }\n .compare-table tbody tr:nth-child(even) { background: var( - paper-dark); }\n .compare-table tbody tr:hover { background: #e8e3da; }\n .badge {\n display: inline-block;\n font-family: var( - mono);\n font-size: 11px;\n padding: 2px 8px;\n border-radius: 2px;\n font-weight: 500;\n }\n .badge-green { background: #e6f4ea; color: #27704a; }\n .badge-yellow { background: #fef9e7; color: #9a7d0a; }\n .badge-red { background: #fdecea; color: #c0392b; }\n\n /* ---- SYNTAX RULES LIST ---- */\n .rules-grid {\n display: grid;\n grid-template-columns: 1fr 1fr;\n gap: 16px;\n margin: 24px 0;\n }\n .rule-card {\n background: var( - paper-dark);\n border: 1px solid var( - rule);\n border-top: 3px solid var( - accent-2);\n padding: 18px 20px;\n border-radius: 0 0 4px 4px;\n }\n .rule-card h5 {\n font-family: var( - sans);\n font-size: 13px;\n font-weight: 600;\n text-transform: uppercase;\n letter-spacing: 0.06em;\n color: var( - ink);\n margin-bottom: 8px;\n }\n .rule-card p { font-size: 14px; margin: 0; }\n\n /* ---- DATATYPE CARDS ---- */\n .dtype-grid {\n display: grid;\n grid-template-columns: repeat(3, 1fr);\n gap: 16px;\n margin: 28px 0;\n }\n .dtype-card {\n background: var( - paper-dark);\n border: 1px solid var( - rule);\n padding: 20px;\n border-radius: 4px;\n transition: box-shadow 0.2s, transform 0.2s;\n }\n .dtype-card:hover { box-shadow: 0 4px 16px rgba(0,0,0,0.08); transform: translateY(-2px); }\n .dtype-name {\n font-family: var( - mono);\n font-size: 13px;\n font-weight: 500;\n color: var( - accent);\n display: block;\n margin-bottom: 4px;\n }\n .dtype-label {\n font-size: 16px;\n font-weight: 600;\n color: var( - ink);\n display: block;\n margin-bottom: 8px;\n }\n .dtype-eg {\n font-family: var( - mono);\n font-size: 12px;\n color: var( - accent-2);\n background: #e8f0f5;\n padding: 4px 8px;\n border-radius: 3px;\n display: inline-block;\n margin-bottom: 10px;\n }\n .dtype-card p { font-size: 13px; margin: 0; color: var( - ink-faint); }\n\n /* ---- INSTALL STEPS ---- */\n .steps { list-style: none; counter-reset: steps; margin: 24px 0; }\n .steps li {\n counter-increment: steps;\n display: flex;\n gap: 20px;\n margin-bottom: 24px;\n align-items: flex-start;\n }\n .steps li::before {\n content: counter(steps);\n flex-shrink: 0;\n width: 36px;\n height: 36px;\n background: var( - ink);\n color: #fff;\n border-radius: 50%;\n display: flex;\n align-items: center;\n justify-content: center;\n font-family: var( - mono);\n font-size: 13px;\n font-weight: 500;\n margin-top: 2px;\n }\n .steps li div { flex: 1; }\n .steps li strong { display: block; font-size: 16px; margin-bottom: 4px; color: var( - ink); }\n .steps li p { font-size: 15px; margin: 0; }\n\n /* ---- DIVIDER ---- */\n .divider {\n border: none;\n border-top: 1px solid var( - rule);\n margin: 56px 0;\n }\n\n /* ===================== ROADMAP ===================== */\n .roadmap-section {\n background: var( - ink);\n color: #fff;\n margin: 0 -40px;\n padding: 72px 40px;\n }\n .roadmap-section h2 {\n color: #fff;\n border-bottom-color: #333;\n font-size: 36px;\n }\n .roadmap-section p { color: #a0998e; }\n\n /* Phase selector */\n .phase-tabs {\n display: flex;\n gap: 0;\n margin: 40px 0 0;\n border-bottom: 1px solid #333;\n flex-wrap: wrap;\n }\n .phase-tab {\n font-family: var( - mono);\n font-size: 12px;\n letter-spacing: 0.08em;\n text-transform: uppercase;\n padding: 12px 22px;\n border: none;\n background: none;\n color: #555;\n cursor: pointer;\n border-bottom: 3px solid transparent;\n margin-bottom: -1px;\n transition: color 0.2s, border-color 0.2s;\n }\n .phase-tab.active { color: #fff; border-bottom-color: var( - accent); }\n .phase-tab:hover:not(.active) { color: #aaa; }\n\n /* Phase content */\n .phase-panel { display: none; padding-top: 36px; }\n .phase-panel.active { display: block; }\n\n .phase-header {\n display: flex;\n align-items: flex-start;\n gap: 24px;\n margin-bottom: 32px;\n }\n .phase-num {\n font-family: var( - serif);\n font-size: 72px;\n font-weight: 700;\n color: #2a2a2a;\n line-height: 1;\n flex-shrink: 0;\n }\n .phase-info h3 {\n font-family: var( - serif);\n font-size: 28px;\n font-weight: 600;\n color: #fff;\n margin: 0 0 6px;\n }\n .phase-info p { font-size: 15px; color: #888; margin: 0; }\n\n .topics-grid {\n display: grid;\n grid-template-columns: repeat(2, 1fr);\n gap: 12px;\n margin-bottom: 32px;\n }\n .topic-item {\n display: flex;\n align-items: center;\n gap: 12px;\n background: #1e1e1e;\n border: 1px solid #2a2a2a;\n border-radius: 4px;\n padding: 14px 16px;\n transition: border-color 0.2s, background 0.2s;\n cursor: default;\n }\n .topic-item:hover { background: #222; border-color: #444; }\n .topic-dot {\n width: 8px;\n height: 8px;\n border-radius: 50%;\n flex-shrink: 0;\n background: var( - accent);\n }\n .topic-dot.blue { background: var( - accent-2); }\n .topic-dot.green { background: var( - accent-3); }\n .topic-dot.yellow { background: #d4a017; }\n .topic-dot.purple { background: #8b5cf6; }\n\n .topic-text {\n font-size: 14px;\n color: #ccc;\n flex: 1;\n }\n .topic-badge {\n font-family: var( - mono);\n font-size: 10px;\n padding: 2px 7px;\n border-radius: 2px;\n background: #2a2a2a;\n color: #666;\n white-space: nowrap;\n }\n .topic-badge.core { background: #2a1a1a; color: var( - accent); }\n .topic-badge.practice { background: #1a2a1a; color: var( - accent-3); }\n .topic-badge.advanced { background: #1a1a2a; color: var( - accent-2); }\n\n /* Resource links within phase */\n .phase-resources {\n background: #161616;\n border: 1px solid #2a2a2a;\n border-left: 3px solid var( - accent);\n padding: 20px 24px;\n border-radius: 0 4px 4px 0;\n }\n .phase-resources h5 {\n font-family: var( - mono);\n font-size: 11px;\n letter-spacing: 0.12em;\n text-transform: uppercase;\n color: var( - accent);\n margin-bottom: 12px;\n }\n .resource-links { list-style: none; }\n .resource-links li {\n font-size: 14px;\n color: #888;\n padding: 5px 0;\n border-bottom: 1px solid #222;\n }\n .resource-links li:last-child { border-bottom: none; }\n .resource-links a { color: #6ba8c4; text-decoration: none; }\n .resource-links a:hover { color: #89b4fa; text-decoration: underline; }\n\n /* ---- SPECIALISATIONS ---- */\n .spec-grid {\n display: grid;\n grid-template-columns: repeat(2, 1fr);\n gap: 16px;\n margin: 32px 0;\n }\n .spec-card {\n background: #161616;\n border: 1px solid #2a2a2a;\n border-radius: 4px;\n padding: 24px;\n transition: border-color 0.2s;\n }\n .spec-card:hover { border-color: #444; }\n .spec-tag {\n font-family: var( - mono);\n font-size: 10px;\n letter-spacing: 0.12em;\n text-transform: uppercase;\n padding: 3px 9px;\n border-radius: 2px;\n display: inline-block;\n margin-bottom: 14px;\n }\n .spec-card h4 {\n font-family: var( - sans);\n font-size: 17px;\n font-weight: 600;\n text-transform: none;\n letter-spacing: 0;\n color: #e0d8cc;\n margin-bottom: 10px;\n margin-top: 0;\n }\n .spec-card p { font-size: 13.5px; color: #777; margin-bottom: 14px; }\n .spec-skills {\n display: flex;\n flex-wrap: wrap;\n gap: 6px;\n }\n .skill-chip {\n font-family: var( - mono);\n font-size: 11px;\n padding: 3px 8px;\n background: #222;\n border: 1px solid #333;\n border-radius: 2px;\n color: #999;\n }\n\n /* FOOTER */\n .site-footer {\n background: #111;\n color: #555;\n text-align: center;\n padding: 40px;\n font-size: 13px;\n font-family: var( - mono);\n border-top: 1px solid #222;\n }\n .site-footer a { color: #777; text-decoration: none; }\n .site-footer a:hover { color: #aaa; }\n\n /* ---- RESPONSIVE ---- */\n @media (max-width: 680px) {\n .hero { padding: 50px 24px 40px; }\n .main { padding: 40px 24px 60px; }\n .toc-inner { padding: 16px 24px; }\n .toc-list li { width: 100%; }\n .rules-grid { grid-template-columns: 1fr; }\n .dtype-grid { grid-template-columns: 1fr 1fr; }\n .topics-grid { grid-template-columns: 1fr; }\n .spec-grid { grid-template-columns: 1fr; }\n .roadmap-section { margin: 0 -24px; padding: 48px 24px; }\n .site-header { padding: 12px 24px; }\n h2 { font-size: 26px; }\n }\n\n /* ANIMATIONS */\n @keyframes fadeUp {\n from { opacity: 0; transform: translateY(20px); }\n to { opacity: 1; transform: translateY(0); }\n }\n .hero-inner { animation: fadeUp 0.6s ease both; }\n .section { animation: fadeUp 0.5s ease both; }\n\u003c/style\u003e\n\u003c/head\u003e\n\u003cbody\u003e\n\n\u003c! - HEADER - \u003e\n\u003cheader class=\"site-header\"\u003e\n \u003ca class=\"site-logo\" href=\"https://codeverra.com\"\u003e\u003cspan\u003e//\u003c/span\u003e codeverra\u003c/a\u003e\n \u003cspan class=\"header-tag\"\u003ePython Fundamentals Series\u003c/span\u003e\n\u003c/header\u003e\n\n\u003c! - HERO - \u003e\n\u003cdiv class=\"hero\"\u003e\n \u003cdiv class=\"hero-inner\"\u003e\n \u003cspan class=\"hero-label\"\u003eLecture 01 / Python Fundamentals\u003c/span\u003e\n \u003ch1\u003ePython: Where to Start\u003cbr\u003eand \u003cem\u003eWhere It Can Take You\u003c/em\u003e\u003c/h1\u003e\n \u003cp class=\"hero-sub\"\u003e\n A complete introduction to why Python matters, how to set it up, the basics of its syntax,\n and an honest roadmap of everything that lies ahead for you.\n \u003c/p\u003e","tags":null,"title":""}]