Advanced Language Features & Internals

1. What are Python's decorators under the hood? How does @functools.wraps work?

• Answer:
  Decorators are essentially syntactic sugar for function transformation. @decorator is equivalent to func = decorator(func).

  • Under the hood: A decorator is a callable that takes a function and returns a new function (or any callable), replacing the original.

  • @functools.wraps: When you decorate a function, you lose its original metadata (__name__, __doc__, __module__). wraps is a decorator itself that copies this metadata from the original function to the wrapper. It's implemented by updating the __dict__ of the wrapper function.

  python

  from functools import wraps
  def my_decorator(func):
      @wraps(func)  # Preserves func's metadata
      def wrapper(*args, **kwargs):
          return func(*args, **kwargs)
      return wrapper

   

2. Explain the difference between @staticmethod and @classmethod. When would you use each?

• Answer:

  • @classmethod: Takes the class (cls) as the first implicit argument. It can access and modify class state. Used for alternative constructors or operations that require the class but not a specific instance.

  • @staticmethod: Does not take any implicit first argument. It behaves like a plain function but belongs to the class's namespace. Used for utility functions that have a logical connection to the class but don't need class or instance data.

  python

  class MyClass:
      @classmethod
      def from_string(cls, string):
          # cls is MyClass, can create an instance
          return cls(...)  # Acts as an alternative constructor
  
      @staticmethod
      def is_valid(data):
          # No access to cls or self, just a utility
          return isinstance(data, int)

   

3. What are Coroutines and how do they differ from Generators?

• Answer:

  • Generators are primarily used for producing iteration (data production) using yield. They are a subset of coroutines.

  • Coroutines are more general and used for consuming data and supporting more complex concurrency patterns. They can both yield and receive values using yield (or await in asyncio). While generators are pull-based (data is consumed), coroutines are push-based (data can be sent to them).

  • Modern Python: The asyncio library uses async/await syntax to build coroutines that are more readable and less error-prone than the older generator-based coroutines.

4. How does Python's garbage collection handle reference cycles?

• Answer:
  Reference counting alone cannot collect reference cycles (e.g., object A refers to B, B refers to A). Python employs a generational garbage collector (GC) as a secondary mechanism.

  • Generations: Objects are grouped into three generations (0, 1, 2). New objects go into generation 0.

  • Cycle Detection: The GC periodically runs, starting with the youngest generation. It traces object references to identify unreachable cycles, even if their reference count isn't zero.

  • Collection: If an object survives a GC collection, it's moved to the next, older generation. This is efficient because most objects die young.

5. What is the collections.abc module used for? Give examples.

• Answer:
  This module provides abstract base classes (ABCs) to define and test if a class provides a specific interface (like a sequence, mapping, or iterator). It helps in implementing custom container types correctly.

  • Use Cases:

    • Inheritance: Create a custom list by inheriting from collections.abc.MutableSequence. You only need to implement __getitem__, __setitem__, __delitem__, __len__, and insert, and you get append, pop, etc., for free.

    • Type Checking: isinstance(my_obj, collections.abc.Iterable) is more reliable than checking for __iter__ directly.

  python

  from collections.abc import MutableSequence
  class MyList(MutableSequence):
      def __init__(self): self._data = []
      def __getitem__(self, index): return self._data[index]
      def __setitem__(self, index, value): self._data[index] = value
      def __delitem__(self, index): del self._data[index]
      def __len__(self): return len(self._data)
      def insert(self, index, value): self._data.insert(index, value)
  # MyList automatically has .append(), .pop(), etc.

   

Concurrency & Performance Deep Dive

6. What are the key differences between asyncio.create_task(), asyncio.gather(), and asyncio.as_completed()?

• Answer:

  • asyncio.create_task(coro): Schedules a coroutine to run "in the background" and returns a Task object. It starts the task but doesn't wait for it to complete. Used for fire-and-forget concurrency.

  • asyncio.gather(*coros): Runs multiple coroutines concurrently and waits for all to finish. It returns a list of results in the same order as the input. You can handle exceptions collectively.

  • asyncio.as_completed(coros): Returns an iterator that yields coroutines/futures as they complete. This allows you to process results in the order they finish, not the order they were submitted.

7. How does the selectors module work, and what problem does it solve?

• Answer:
  The selectors module provides high-level I/O multiplexing. It allows a single thread to monitor multiple file descriptors (sockets, files) for I/O readiness.

  • Problem Solved: Building efficient, single-threaded concurrent servers without the overhead of threading. It's the foundation for asyncio's event loop.

  • How it works: You register file objects with a selector, specifying events you're interested in (e.g., EVENT_READ, EVENT_WRITE). The select() method blocks until one or more file objects are ready, then returns a list of ready objects.

8. Explain the concept of "Lazy Evaluation" in Python. Where is it used?

• Answer:
  Lazy evaluation defers the computation of a value until it is actually needed. This saves memory and can improve performance.

  • Examples:

    • Generators: (x for x in range(10)) doesn't compute all values upfront.

    • range(): In Python 3, range is a lazy sequence; it doesn't generate all numbers in memory.

    • itertools functions: count, cycle, islice are all lazy.

    • Django Querysets: They are lazy; the database query isn't executed until the data is iterated over.

9. What is the purpose of the __call__ method? Give a practical use case.

• Answer:
  The __call__ method allows an instance of a class to be called as a function.

  • Use Case 1: Functors (Function Objects): Creating stateful functions where the behavior depends on internal state.

  python

  class Adder:
      def __init__(self, n):
          self.n = n
      def __call__(self, x):
          return self.n + x
  
  add5 = Adder(5)
  print(add5(10))  # Output: 15 (the instance is callable!)

  • Use Case 2: Decorator Classes: Creating decorators that maintain state across calls.

  python

  class CountCalls:
      def __init__(self, func):
          self.func = func
          self.count = 0
      def __call__(self, *args, **kwargs):
          self.count += 1
          return self.func(*args, **kwargs)
  
  @CountCalls
  def say_hello():
      print("Hello!")

   

Testing & Debugging Advanced

10. What are fixtures in pytest? How do they help in testing?

• Answer:
  Fixtures are functions that provide a fixed, reliable baseline for tests. They run before (and after) the test functions that use them.

  • Help they provide:

    • Setup/Teardown: Manage test dependencies (database connections, temporary files, API clients).

    • Dependency Injection: Pass reusable data or objects to tests without repetition.

    • Scope: Control how often a fixture is executed (e.g., function, class, module, session).

  python

  import pytest
  @pytest.fixture
  def database_connection():
      conn = create_connection()  # Setup
      yield conn                  # Provide value to test
      conn.close()                # Teardown
  
  def test_query(database_connection):  # Fixture is injected
      result = database_connection.execute("...")
      assert result is not None

   

11. How do you mock an object in Python tests? Explain the unittest.mock module.

• Answer:
  unittest.mock allows you to replace parts of your system under test with mock objects and make assertions about how they were used.

  • Mock: A flexible object that can be used to replace any object. You can set its return value, side effects, and check if it was called.

  • patch: A decorator/context manager used to replace a target with a Mock object for the duration of a test. It's the most common way to inject mocks.

  python

  from unittest.mock import patch, MagicMock
  @patch('my_module.requests.get')  # Patch where the object is used
  def test_api_call(mock_get):
      mock_get.return_value.status_code = 200  # Configure the mock
      result = my_function_that_uses_requests()
      mock_get.assert_called_once_with('https://api.example.com')
      assert result == 200

   

12. What is "property-based testing" and which libraries support it in Python?

• Answer:
  Property-based testing verifies that certain properties or invariants hold for a wide range of automatically generated input data, rather than just for specific examples.

  • How it works: The library (like Hypothesis) generates many random inputs that match a specified strategy. The test is run against all these inputs, trying to find a minimal failing case.

  • Libraries: Hypothesis is the main library. It helps find edge cases you might not have considered.

  python

  from hypothesis import given, strategies as st
  @given(st.integers(), st.integers())
  def test_commutative_property(a, b):
      assert my_add(a, b) == my_add(b, a)  # Should hold for all integers

   

System Design & Architecture

13. How would you design a caching system in Python? What are the trade-offs?

• Answer:

  • In-Memory (e.g., functools.lru_cache, dict): Fastest, but doesn't persist across process restarts and consumes application memory.

  • External (e.g., Redis, Memcached): Shared across processes/machines, can persist, but adds network latency and dependency on another service.

  • File-based: Simple, persistent, but slow and not suitable for high-frequency writes.

  • Trade-offs: Consider performance vs. persistence, memory usage vs. scalability, and complexity vs. features (e.g., TTL, eviction policies).

14. Explain the concept of Dependency Injection (DI) in Python. How can it be implemented?

• Answer:
  DI is a pattern where an object receives its dependencies from an external source rather than creating them itself. This promotes loose coupling and testability.

  • Implementation:

    • Constructor Injection: Pass dependencies via the __init__ method. This is the most common and Pythonic way.

    • Framework: Use a DI container/library like dependency-injector for complex applications to manage object creation and wiring.

  python

  # Without DI (tight coupling)
  class Service:
      def __init__(self):
          self.db = PostgreSQLDatabase()  # Hard-coded dependency
  
  # With DI (loose coupling)
  class Service:
      def __init__(self, database):  # Dependency is injected
          self.db = database
  
  # Easy to test
  def test_service():
      mock_db = Mock()
      service = Service(mock_db)  # Inject a mock

   

15. What are the key principles of API design in Python web frameworks?

• Answer:

  • RESTful Principles: Use HTTP methods correctly (GET, POST, PUT, DELETE), statelessness, proper resource naming (/users/{id}).

  • Validation: Always validate and sanitize input data (using Pydantic, Marshmallow, or framework-built-in tools).

  • Error Handling: Return consistent, informative error responses with appropriate HTTP status codes.

  • Documentation: Use tools like OpenAPI/Swagger (with FastAPI, DRF) for auto-generated documentation.

  • Versioning: Plan for API versioning from the start (e.g., in the URL /v1/users or headers).

  • Authentication & Authorization: Implement robust, standard auth (OAuth2, JWT).

16. How do you manage configuration in a large Python application?

• Answer:

  • Environment Variables: For secrets (API keys, DB URLs) and environment-specific settings. Use os.getenv() or libraries like pydantic-settings.

  • Configuration Files: For non-secret settings. Use .ini, .yaml, .toml, or .json. Libraries like pydantic can load and validate these.

  • 12-Factor App Method: Strictly use environment variables, as they are easy to change between deployments and are language-agnostic.

  • Hierarchical Configuration: Combine multiple sources (defaults → config file → environment variables → command-line arguments).

Ecosystem & Libraries

17. When would you use NumPy over built-in Python lists? What is its core advantage?

• Answer:
  Use NumPy for numerical computations on large, homogeneous data arrays.

  • Core Advantages:

    • Performance: Operations are implemented in C and are vectorized, meaning they operate on entire arrays without Python loops. This is orders of magnitude faster.

    • Memory Efficiency: NumPy arrays are densely packed in memory, while Python lists are arrays of pointers to Python objects, which is much heavier.

    • Functionality: Provides a vast collection of mathematical functions (linear algebra, Fourier transforms, random number generation) optimized for arrays.

18. Explain the difference between Pandas .loc, .iloc, and .at/.iat.

• Answer:

  • .loc: Label-based indexing. You use the row and column labels. df.loc['row_label', 'col_label']. Slicing is inclusive of the end.

  • .iloc: Integer position-based indexing. You use the row and column integer positions (0-based). df.iloc[0, 1]. Slicing is exclusive of the end (like Python lists).

  • .at / .iat: Scalar accessors for fast access to a single value. df.at['row_label', 'col_label'] is much faster than .loc for getting/setting a single value. Similarly, df.iat[0, 1] is the integer-position version.

19. What is the role of the __init__.py file in modern Python? How has it changed?

• Answer:

  • Traditional Role (Pre-Python 3.3): It was required to make a directory a Python package. It could also contain package initialization code and define __all__ for from package import *.

  • Modern Role (Implicit Namespace Packages): Since Python 3.3, __init__.py is not required to create a package. Directories without it are called "namespace packages." However, it's still widely used for:

    • Running initialization code for the package.

    • Defining __all__.

    • Conveniently importing submodule contents to the package level (e.g., from .submodule import MyClass in __init__.py allows users to do from package import MyClass).

20. Describe a time you had to optimize a slow Python application. What was your process and what tools did you use?

• Answer:
  (This is another behavioral question. Structure your answer clearly.)

  • 1. Identify the Bottleneck: "First, I profiled the code to find the slow part. I used cProfile to get a high-level view and line_profiler for a line-by-line breakdown of the critical function."

  • 2. Analyze the Cause: "The profiling revealed the bottleneck was a nested for loop performing a naive calculation on a large list."

  • 3. Implement a Solution: "I refactored the algorithm. I replaced the list with a NumPy array and used vectorized operations, which eliminated the Python loops and leveraged optimized C code under the hood. This gave us a 50x speedup."

  • 4. Verify and Test: "I verified the results were identical and ran the test suite to ensure no regressions. I also used memory_profiler to confirm we hadn't increased memory usage."

These questions dive deeper into architecture, performance tuning, and advanced language mechanics, which are crucial topics for senior-level Python roles. Good luck with your preparation!