Back to course

Memory Management & Garbage Collection

Content Reader1 words • 0:00 • Browser TTS

image.png

Memory Management & Garbage Collection – Optimizing JavaScript's Memory Engine

Imagine you're managing a modern smart warehouse 📦 with automated systems for storing and retrieving inventory:

  • Automatic Allocation 🏗️ - When new products arrive, the system automatically finds the best storage location based on size, frequency of access, and available space
  • Intelligent Organization 🧠 - Similar items are grouped together, frequently accessed items are placed in easily reachable areas (like cache), while long-term storage items go to deeper sections
  • Automatic Cleanup 🧹 - The system continuously monitors which items haven't been accessed for a long time and automatically moves them out to free up space for new inventory
  • Reference Tracking 🔗 - Every item has digital tags that track what other systems or processes are currently using it - items can only be removed when no active references exist
  • Generational Storage 👶➡️👴 - New items start in a "quick access" area, and if they survive long enough, they get moved to more permanent storage areas
  • Memory Compaction 🗜️ - Periodically, the system reorganizes storage to eliminate gaps and fragmentation, making space more efficient
  • Leak Detection 🚨 - Monitors for items that should have been removed but are stuck due to broken reference systems

JavaScript's memory management works exactly like this smart warehouse system. The V8 engine (and other JavaScript engines) automatically handle memory allocation and cleanup:

  • Automatic Memory Allocation - Variables, objects, and functions are automatically stored in memory when created
  • Garbage Collection - Unused memory is automatically reclaimed without manual intervention
  • Reference Counting - Objects are kept alive as long as something references them
  • Generational Collection - Young objects that survive initial cleanup cycles get promoted to long-term storage
  • Mark and Sweep - Systematic identification and cleanup of unreachable objects
  • Memory Optimization - Understanding these processes helps you write more efficient code

Understanding memory management is crucial for building high-performance applications that don't slow down over time, consume excessive memory, or cause browser crashes due to memory leaks.

The Theoretical Foundation: Memory Models and Automatic Management 📐

Understanding Computer Memory Hierarchy

Memory management operates within a complex hierarchy of storage systems, each with different characteristics:

Memory Hierarchy (Fastest to Slowest):

  1. CPU Registers: Extremely fast, extremely limited (bytes)
  2. CPU Cache (L1, L2, L3): Very fast, limited (KB to MB)
  3. RAM (Main Memory): Fast, moderate capacity (GB)
  4. SSD/HDD Storage: Slower, large capacity (TB)
  5. Network/Cloud Storage: Slowest, virtually unlimited

JavaScript Memory Focus:

  • Heap Memory: Where objects, arrays, and functions live
  • Stack Memory: Where primitive values and function call frames live
  • Code Memory: Where the actual JavaScript code is stored

Memory Allocation Theory

Memory allocation involves several fundamental concepts:

Static vs Dynamic Allocation:

  • Static: Memory size known at compile time (like function declarations)
  • Dynamic: Memory size determined at runtime (like objects created with new)

Memory Layout in JavaScript:

  • Call Stack: Function calls, local variables, primitive values
  • Heap: Objects, arrays, closures, and complex data structures
  • Global Area: Global variables and functions
  • Code Area: The actual JavaScript code being executed

Garbage Collection Theory

Garbage Collection (GC) is based on fundamental computer science principles:

Core GC Concepts:

  1. Reachability: Objects are kept alive if they can be reached from "roots"
  2. Roots: Global variables, function call stack, closures
  3. Reference Tracking: Following chains of object references
  4. Automatic Cleanup: System automatically frees unreachable memory

Types of Garbage Collection:

  1. Reference Counting: Count how many references point to each object
    • Pros: Immediate cleanup when count reaches zero
    • Cons: Can't handle circular references
  2. Mark and Sweep: Mark reachable objects, then sweep away unmarked ones
    • Pros: Handles circular references correctly
    • Cons: Can cause pause times during collection
  3. Generational Collection: Separate young and old objects
    • Theory: Most objects die young, survivors tend to live long
    • Benefit: Focus cleanup efforts on young objects

V8's Garbage Collection Strategy

V8 uses a sophisticated multi-generational approach:

Young Generation (Scavenger/Semi-space):

  • New Space: Where new objects are allocated
  • From Space & To Space: Two equal-sized semi-spaces
  • Minor GC: Fast collection focusing on young objects
  • Survival Promotion: Objects that survive multiple cycles get promoted

Old Generation (Mark-Sweep-Compact):

  • Old Space: Long-lived objects
  • Major GC: Less frequent but more comprehensive collection
  • Compaction: Eliminates fragmentation by moving objects together

Why This Matters for Developers:

  • Short-lived objects: Very efficient to create and clean up
  • Long-lived objects: Should be designed carefully
  • Memory patterns: Understanding helps optimize allocation patterns

The Problem: Memory Leaks and Inefficient Memory Usage 😤

Common Memory Leak Patterns

Without understanding memory management, applications can develop serious memory leaks:

// Memory Leak #1: Detached DOM Elements
class BadEventManager {
    constructor() {
        this.eventHandlers = [];
        this.domCache = new Map();
    }
    
    attachEventHandlers() {
        // This creates a memory leak!
        document.querySelectorAll('.interactive-element').forEach(element => {
            const handler = (event) => {
                console.log('Element clicked:', element.id);
                // Handler holds reference to element
                this.processElementClick(element, event);
            };
            
            element.addEventListener('click', handler);
            
            // Store handler but never clean up!
            this.eventHandlers.push({ element, handler });
            
            // Cache DOM elements that might be removed later
            this.domCache.set(element.id, element);
        });
    }
    
    processElementClick(element, event) {
        // Complex processing that keeps element reference alive
        this.lastClickedElement = element; // Another reference!
        
        // Store in closure that never gets cleaned up
        setTimeout(() => {
            if (element.parentNode) {
                element.style.backgroundColor = 'yellow';
            }
        }, 1000);
    }
    
    removeElement(elementId) {
        // Remove from DOM but not from cache - Memory Leak!
        const element = document.getElementById(elementId);
        if (element && element.parentNode) {
            element.parentNode.removeChild(element);
        }
        
        // Element is gone from DOM but still referenced:
        // 1. this.domCache still holds it
        // 2. Event handlers still reference it
        // 3. this.lastClickedElement might still reference it
        // 4. setTimeout closure still references it
    }
    
    // Method that should clean up but doesn't
    cleanup() {
        // This is incomplete cleanup!
        this.eventHandlers = [];
        // Forgot to:
        // - Remove event listeners
        // - Clear domCache
        // - Clear lastClickedElement
        // - Cancel pending timeouts
    }
}

// Usage that creates memory leaks
const eventManager = new BadEventManager();
eventManager.attachEventHandlers();

// Later, remove elements from DOM
eventManager.removeElement('item1');
eventManager.removeElement('item2');

// Elements are gone from DOM but still consume memory!
// The event handlers keep references to the detached elements
// This prevents garbage collection

Closure Memory Leaks

// Memory Leak #2: Closures holding unnecessary references
class BadDataProcessor {
    constructor() {
        this.processors = [];
        this.cache = new Map();
        this.largeDataSet = this.generateLargeDataSet(); // 100MB of data
    }
    
    generateLargeDataSet() {
        // Simulate large dataset
        const data = [];
        for (let i = 0; i < 1000000; i++) {
            data.push({
                id: i,
                data: 'x'.repeat(100), // 100 chars per item
                metadata: {
                    created: new Date(),
                    processed: false,
                    tags: ['tag1', 'tag2', 'tag3']
                }
            });
        }
        return data;
    }
    
    createProcessor(type) {
        // This closure captures the entire 'this' context!
        const processor = (item) => {
            // Only needs item, but captures entire this.largeDataSet
            console.log(`Processing ${type}: ${item.id}`);
            
            // This keeps the entire processor instance alive
            this.cache.set(`${type}_${item.id}`, item);
            
            // Even this simple reference prevents GC
            return {
                processedBy: type,
                processedAt: new Date(),
                // This function also captures 'this'
                reprocess: () => {
                    this.reprocessItem(item);
                }
            };
        };
        
        // Store processor function - keeps closure alive!
        this.processors.push(processor);
        
        return processor;
    }
    
    reprocessItem(item) {
        // More processing that holds references
        console.log('Reprocessing:', item.id);
    }
    
    processItems(items, type) {
        const processor = this.createProcessor(type);
        
        return items.map(item => {
            const result = processor(item);
            
            // Create another closure that captures everything!
            item.cleanup = () => {
                // This closure captures 'this', 'processor', 'result'
                this.cache.delete(`${type}_${item.id}`);
                
                // But never actually gets called!
            };
            
            return result;
        });
    }
    
    // Attempted cleanup that's insufficient
    clearProcessors() {
        this.processors = []; // Clears array but closures already captured references
        // The largeDataSet is still referenced by existing closures!
    }
}

// Usage that creates accumulating memory leaks
const processor = new BadDataProcessor(); // 100MB allocated

// Process multiple batches - each creates new closures
const batch1 = processor.generateLargeDataSet().slice(0, 1000);
const batch2 = processor.generateLargeDataSet().slice(1000, 2000);
const batch3 = processor.generateLargeDataSet().slice(2000, 3000);

processor.processItems(batch1, 'type1'); // +100MB in closures
processor.processItems(batch2, 'type2'); // +100MB in closures  
processor.processItems(batch3, 'type3'); // +100MB in closures

// Each processor closure keeps the entire largeDataSet alive!
// Total memory: ~400MB instead of just the processed items

Timer and Interval Memory Leaks

// Memory Leak #3: Timers that never get cleared
class BadAnimationManager {
    constructor() {
        this.animations = new Map();
        this.stats = {
            totalAnimations: 0,
            activeAnimations: 0,
            memoryUsage: []
        };
        this.largeAssets = this.loadAssets(); // Large image/video data
    }
    
    loadAssets() {
        // Simulate large assets
        return {
            images: new Array(1000).fill(null).map((_, i) => ({
                id: i,
                data: new ArrayBuffer(1024 * 100), // 100KB per image
                url: `image_${i}.jpg`
            })),
            sounds: new Array(100).fill(null).map((_, i) => ({
                id: i,
                data: new ArrayBuffer(1024 * 500), // 500KB per sound
                url: `sound_${i}.mp3`
            }))
        };
    }
    
    startAnimation(elementId, config) {
        const element = document.getElementById(elementId);
        if (!element) return;
        
        this.stats.totalAnimations++;
        this.stats.activeAnimations++;
        
        // Timer that captures large context
        const animationTimer = setInterval(() => {
            // This closure captures 'this', 'element', 'config', 'elementId'
            // Including the massive this.largeAssets!
            
            this.updateElement(element, config);
            
            // Store memory usage stats
            this.stats.memoryUsage.push({
                timestamp: Date.now(),
                activeAnimations: this.stats.activeAnimations,
                elementId: elementId
            });
            
            // Condition that might never be true
            if (this.shouldStopAnimation(element, config)) {
                this.stopAnimation(elementId);
            }
        }, 16); // 60fps
        
        // Store timer reference
        this.animations.set(elementId, {
            timer: animationTimer,
            element: element, // Direct DOM reference!
            config: config,
            startTime: Date.now(),
            assets: this.largeAssets // Unnecessary reference to large data!
        });
    }
    
    updateElement(element, config) {
        // Animation logic that might fail
        if (element.parentNode) {
            element.style.transform = `translateX(${Math.random() * 100}px)`;
        }
        // If element is removed from DOM, this fails silently
        // But timer keeps running!
    }
    
    shouldStopAnimation(element, config) {
        // Buggy condition - might never return true
        return element.offsetWidth === 0; // This might always be false
    }
    
    stopAnimation(elementId) {
        const animation = this.animations.get(elementId);
        if (animation) {
            clearInterval(animation.timer);
            this.animations.delete(elementId);
            this.stats.activeAnimations--;
        }
    }
    
    // Cleanup method that's never called or incomplete
    cleanup() {
        // This only clears the map, doesn't clear timers!
        this.animations.clear();
        // All setInterval timers are still running!
        // They keep references to DOM elements and largeAssets
    }
    
    // Method that creates even more leaks
    createDynamicElement(id) {
        const element = document.createElement('div');
        element.id = id;
        document.body.appendChild(element);
        
        // Start animation immediately
        this.startAnimation(id, { duration: 5000 });
        
        // Remove element after 2 seconds
        setTimeout(() => {
            if (element.parentNode) {
                element.parentNode.removeChild(element);
            }
            // But animation timer is still running!
            // Timer keeps reference to removed element
        }, 2000);
        
        // Never call stopAnimation - memory leak!
    }
}

// Usage that creates multiple memory leaks
const animationManager = new BadAnimationManager(); // Large assets loaded

// Create many short-lived animations
for (let i = 0; i < 100; i++) {
    animationManager.createDynamicElement(`dynamic_${i}`);
}

// Elements get removed after 2 seconds
// But timers keep running for each element!
// Each timer closure holds references to:
// - The removed DOM element
// - The entire animationManager instance
// - The large assets (100MB+)
// - Growing stats.memoryUsage array

// After 10 seconds: 100 timers running, 0 visible elements
// Memory usage: ~1GB instead of ~100MB

Memory Management Best Practices 🎯

Proper Event Cleanup and WeakMap Usage

Implementing proper memory management patterns:

// Good Memory Management: Proper cleanup and WeakMaps
class MemoryEfficientEventManager {
    constructor() {
        // Use WeakMap for DOM element associations
        this.elementData = new WeakMap();
        this.activeHandlers = new Set();
        this.cleanupTasks = new Set();
        
        // Track cleanup for debugging
        this.cleanupStats = {
            handlersCreated: 0,
            handlersDestroyed: 0,
            elementsTracked: 0
        };
    }
    
    attachEventHandler(element, eventType, handler, options = {}) {
        const { once = false, cleanup: customCleanup } = options;
        
        // Create wrapper that enables proper cleanup
        const wrappedHandler = (event) => {
            try {
                handler(event);
            } finally {
                if (once) {
                    this.removeEventHandler(element, eventType, wrappedHandler);
                }
            }
        };
        
        // Store cleanup information using WeakMap
        if (!this.elementData.has(element)) {
            this.elementData.set(element, {
                handlers: new Map(),
                customCleanup: new Set(),
                createdAt: Date.now()
            });
            this.cleanupStats.elementsTracked++;
        }
        
        const elementInfo = this.elementData.get(element);
        
        // Track handler for cleanup
        const handlerKey = `${eventType}_${Date.now()}_${Math.random()}`;
        elementInfo.handlers.set(handlerKey, {
            eventType,
            handler: wrappedHandler,
            originalHandler: handler,
            createdAt: Date.now()
        });
        
        // Add custom cleanup if provided
        if (customCleanup) {
            elementInfo.customCleanup.add(customCleanup);
        }
        
        // Attach event listener
        element.addEventListener(eventType, wrappedHandler);
        this.activeHandlers.add({ element, eventType, handler: wrappedHandler });
        this.cleanupStats.handlersCreated++;
        
        // Return cleanup function
        return () => this.removeEventHandler(element, eventType, wrappedHandler);
    }
    
    removeEventHandler(element, eventType, handler) {
        // Remove event listener
        element.removeEventListener(eventType, handler);
        
        // Clean up tracking
        this.activeHandlers.delete({ element, eventType, handler });
        
        // Update stats
        this.cleanupStats.handlersDestroyed++;
        
        // If element data exists, clean up handler tracking
        if (this.elementData.has(element)) {
            const elementInfo = this.elementData.get(element);
            
            // Find and remove handler entry
            for (const [key, handlerInfo] of elementInfo.handlers) {
                if (handlerInfo.handler === handler) {
                    elementInfo.handlers.delete(key);
                    break;
                }
            }
            
            // If no more handlers, element data will be cleaned up by WeakMap
            // when element is garbage collected
        }
    }
    
    // Clean up all handlers for a specific element
    cleanupElement(element) {
        if (!this.elementData.has(element)) {
            return;
        }
        
        const elementInfo = this.elementData.get(element);
        
        // Remove all event handlers
        for (const handlerInfo of elementInfo.handlers.values()) {
            element.removeEventListener(handlerInfo.eventType, handlerInfo.handler);
            this.cleanupStats.handlersDestroyed++;
        }
        
        // Run custom cleanup functions
        for (const cleanup of elementInfo.customCleanup) {
            try {
                cleanup();
            } catch (error) {
                console.warn('Error in custom cleanup:', error);
            }
        }
        
        // Clear all tracking (WeakMap will handle memory when element is GC'd)
        elementInfo.handlers.clear();
        elementInfo.customCleanup.clear();
    }
    
    // Cleanup all active handlers
    cleanup() {
        // Copy active handlers to avoid modification during iteration
        const handlersToClean = [...this.activeHandlers];
        
        for (const { element, eventType, handler } of handlersToClean) {
            this.removeEventHandler(element, eventType, handler);
        }
        
        this.activeHandlers.clear();
        console.log('Cleaned up event manager:', this.cleanupStats);
    }
    
    // Get memory usage statistics
    getMemoryStats() {
        return {
            ...this.cleanupStats,
            activeHandlers: this.activeHandlers.size,
            trackedElements: this.elementData instanceof WeakMap ? 'WeakMap (auto-cleanup)' : 0
        };
    }
    
    // Create element with automatic cleanup when removed from DOM
    createManagedElement(tagName, parentElement) {
        const element = document.createElement(tagName);
        parentElement.appendChild(element);
        
        // Set up mutation observer for automatic cleanup
        const observer = new MutationObserver((mutations) => {
            for (const mutation of mutations) {
                if (mutation.type === 'childList') {
                    for (const removedNode of mutation.removedNodes) {
                        if (removedNode === element) {
                            this.cleanupElement(element);
                            observer.disconnect();
                            break;
                        }
                    }
                }
            }
        });
        
        observer.observe(parentElement, { childList: true, subtree: true });
        
        // Store observer in element data for cleanup
        if (!this.elementData.has(element)) {
            this.elementData.set(element, {
                handlers: new Map(),
                customCleanup: new Set(),
                createdAt: Date.now()
            });
        }
        
        this.elementData.get(element).customCleanup.add(() => observer.disconnect());
        
        return element;
    }
}

// Usage demonstration
console.log('=== Memory Efficient Event Management ===');

const eventManager = new MemoryEfficientEventManager();

// Create managed elements
const container = document.createElement('div');
document.body.appendChild(container);

const button1 = eventManager.createManagedElement('button', container);
const button2 = eventManager.createManagedElement('button', container);

button1.textContent = 'Click me';
button2.textContent = 'Click me too';

// Attach event handlers with automatic cleanup
const cleanup1 = eventManager.attachEventHandler(button1, 'click', () => {
    console.log('Button 1 clicked');
}, { 
    cleanup: () => console.log('Button 1 custom cleanup executed') 
});

const cleanup2 = eventManager.attachEventHandler(button2, 'click', () => {
    console.log('Button 2 clicked');
}, { 
    once: true // Automatically removes after first click
});

console.log('Initial stats:', eventManager.getMemoryStats());

// Simulate element removal (triggers automatic cleanup)
setTimeout(() => {
    console.log('Removing button1...');
    container.removeChild(button1); // Automatic cleanup triggered
    
    setTimeout(() => {
        console.log('Stats after button1 removal:', eventManager.getMemoryStats());
        
        // Manual cleanup of remaining elements
        eventManager.cleanup();
        console.log('Final stats:', eventManager.getMemoryStats());
        
        // Clean up demo
        document.body.removeChild(container);
    }, 100);
}, 2000);

Memory-Efficient Data Processing

// Memory-Efficient Data Processing with Streaming and Cleanup
class MemoryEfficientDataProcessor {
    constructor(options = {}) {
        this.options = {
            batchSize: options.batchSize || 1000,
            maxCacheSize: options.maxCacheSize || 10000,
            gcThreshold: options.gcThreshold || 0.8, // Trigger cleanup at 80% cache capacity
            ...options
        };
        
        // Use Map for LRU cache with manual cleanup
        this.cache = new Map();
        this.cacheAccessOrder = new Map(); // Track access order for LRU
        this.processingQueue = [];
        
        // WeakMap for temporary associations
        this.itemMetadata = new WeakMap();
        
        // Statistics
        this.stats = {
            totalProcessed: 0,
            cacheHits: 0,
            cacheMisses: 0,
            memoryCleanups: 0,
            batchesProcessed: 0
        };
    }
    
    // Process large datasets in memory-efficient batches
    async processLargeDataset(dataset, processor) {
        console.log(`Processing dataset of ${dataset.length} items in batches of ${this.options.batchSize}`);
        
        const results = [];
        const totalBatches = Math.ceil(dataset.length / this.options.batchSize);
        
        for (let i = 0; i < dataset.length; i += this.options.batchSize) {
            const batch = dataset.slice(i, i + this.options.batchSize);
            const batchNumber = Math.floor(i / this.options.batchSize) + 1;
            
            console.log(`Processing batch ${batchNumber}/${totalBatches}`);
            
            // Process batch
            const batchResults = await this.processBatch(batch, processor);
            results.push(...batchResults);
            
            // Trigger memory cleanup if needed
            if (this.shouldCleanupMemory()) {
                await this.performMemoryCleanup();
            }
            
            // Allow event loop to breathe
            await this.yield();
            
            this.stats.batchesProcessed++;
        }
        
        return results;
    }
    
    async processBatch(batch, processor) {
        const results = [];
        
        for (const item of batch) {
            const result = await this.processItem(item, processor);
            results.push(result);
            
            // Clean up item metadata after processing
            this.itemMetadata.delete(item);
        }
        
        return results;
    }
    
    async processItem(item, processor) {
        // Check cache first
        const cacheKey = this.generateCacheKey(item);
        if (this.cache.has(cacheKey)) {
            this.updateCacheAccess(cacheKey);
            this.stats.cacheHits++;
            return this.cache.get(cacheKey);
        }
        
        this.stats.cacheMisses++;
        
        // Store metadata using WeakMap (automatic cleanup)
        this.itemMetadata.set(item, {
            processedAt: Date.now(),
            cacheKey: cacheKey
        });
        
        // Process item
        const result = await processor(item);
        
        // Cache result if within limits
        if (this.cache.size < this.options.maxCacheSize) {
            this.cache.set(cacheKey, result);
            this.cacheAccessOrder.set(cacheKey, Date.now());
        }
        
        this.stats.totalProcessed++;
        return result;
    }
    
    generateCacheKey(item) {
        // Create memory-efficient cache key
        if (typeof item === 'object' && item !== null) {
            return `obj_${item.id || JSON.stringify(item).substring(0, 50)}`;
        }
        return `prim_${item}`;
    }
    
    updateCacheAccess(cacheKey) {
        // Update LRU order
        this.cacheAccessOrder.set(cacheKey, Date.now());
    }
    
    shouldCleanupMemory() {
        return this.cache.size >= (this.options.maxCacheSize * this.options.gcThreshold);
    }
    
    async performMemoryCleanup() {
        console.log('Performing memory cleanup...');
        
        const startSize = this.cache.size;
        const targetSize = Math.floor(this.options.maxCacheSize * 0.6); // Clean to 60%
        const itemsToRemove = startSize - targetSize;
        
        if (itemsToRemove <= 0) return;
        
        // Sort by access time (LRU)
        const sortedKeys = [...this.cacheAccessOrder.entries()]
            .sort((a, b) => a[1] - b[1]) // Oldest first
            .slice(0, itemsToRemove)
            .map(entry => entry[0]);
        
        // Remove oldest items
        for (const key of sortedKeys) {
            this.cache.delete(key);
            this.cacheAccessOrder.delete(key);
        }
        
        console.log(`Cleaned up ${itemsToRemove} cache entries: ${startSize} -> ${this.cache.size}`);
        this.stats.memoryCleanups++;
        
        // Force garbage collection hint (if available)
        if (global.gc) {
            global.gc();
        }
        
        // Yield to event loop
        await this.yield();
    }
    
    // Yield control to event loop
    yield() {
        return new Promise(resolve => setTimeout(resolve, 0));
    }
    
    // Memory-efficient streaming processor
    async processStream(sourceGenerator, processor, outputHandler) {
        console.log('Starting stream processing...');
        
        let itemCount = 0;
        const batchBuffer = [];
        
        try {
            for await (const item of sourceGenerator) {
                batchBuffer.push(item);
                itemCount++;
                
                // Process when batch is full
                if (batchBuffer.length >= this.options.batchSize) {
                    await this.processBatchBuffer(batchBuffer, processor, outputHandler);
                    batchBuffer.length = 0; // Clear buffer
                    
                    // Memory cleanup check
                    if (this.shouldCleanupMemory()) {
                        await this.performMemoryCleanup();
                    }
                }
                
                // Periodic yield
                if (itemCount % 100 === 0) {
                    await this.yield();
                }
            }
            
            // Process remaining items in buffer
            if (batchBuffer.length > 0) {
                await this.processBatchBuffer(batchBuffer, processor, outputHandler);
            }
            
        } catch (error) {
            console.error('Stream processing error:', error);
            throw error;
        }
        
        console.log(`Stream processing completed: ${itemCount} items processed`);
    }
    
    async processBatchBuffer(buffer, processor, outputHandler) {
        for (const item of buffer) {
            try {
                const result = await this.processItem(item, processor);
                outputHandler(result);
            } catch (error) {
                console.error('Error processing item:', error);
                // Continue processing other items
            }
        }
    }
    
    // Clear all caches and reset
    reset() {
        this.cache.clear();
        this.cacheAccessOrder.clear();
        this.processingQueue = [];
        
        console.log('Processor reset, memory cleared');
        
        // Reset stats
        this.stats = {
            totalProcessed: 0,
            cacheHits: 0,
            cacheMisses: 0,
            memoryCleanups: 0,
            batchesProcessed: 0
        };
    }
    
    getMemoryStats() {
        return {
            ...this.stats,
            cacheSize: this.cache.size,
            cacheHitRatio: this.stats.cacheHits / (this.stats.cacheHits + this.stats.cacheMisses) || 0,
            averageItemsPerBatch: this.stats.totalProcessed / this.stats.batchesProcessed || 0
        };
    }
    
    // Estimate memory usage
    estimateMemoryUsage() {
        let estimate = 0;
        
        // Estimate cache memory (rough approximation)
        estimate += this.cache.size * 100; // ~100 bytes per cache entry
        estimate += this.cacheAccessOrder.size * 50; // ~50 bytes per access entry
        
        return {
            estimatedBytes: estimate,
            estimatedMB: (estimate / 1024 / 1024).toFixed(2),
            cacheEntries: this.cache.size,
            accessEntries: this.cacheAccessOrder.size
        };
    }
}

// Usage demonstration
console.log('\n=== Memory-Efficient Data Processing ===');

async function demoMemoryEfficientProcessing() {
    const processor = new MemoryEfficientDataProcessor({
        batchSize: 100,
        maxCacheSize: 500,
        gcThreshold: 0.8
    });
    
    // Generate large dataset
    function* generateLargeDataset(size) {
        for (let i = 0; i < size; i++) {
            yield {
                id: i,
                data: `item_${i}`,
                value: Math.random() * 100,
                category: `cat_${i % 10}`,
                metadata: {
                    created: new Date(),
                    tags: [`tag_${i % 5}`, `tag_${(i + 1) % 5}`]
                }
            };
        }
    }
    
    // Simple processor function
    const itemProcessor = async (item) => {
        // Simulate some processing time
        await new Promise(resolve => setTimeout(resolve, 1));
        
        return {
            id: item.id,
            processedValue: item.value * 2,
            category: item.category,
            processedAt: Date.now()
        };
    };
    
    // Process large dataset in batches
    console.log('Processing 5000 items in memory-efficient batches...');
    const dataset = [...generateLargeDataset(5000)];
    
    console.time('Batch Processing');
    const results = await processor.processLargeDataset(dataset, itemProcessor);
    console.timeEnd('Batch Processing');
    
    console.log('Batch processing stats:', processor.getMemoryStats());
    console.log('Memory usage estimate:', processor.estimateMemoryUsage());
    
    // Demo streaming processing
    console.log('\nProcessing stream of 1000 items...');
    const outputResults = [];
    
    console.time('Stream Processing');
    await processor.processStream(
        generateLargeDataset(1000),
        itemProcessor,
        (result) => outputResults.push(result)
    );
    console.timeEnd('Stream Processing');
    
    console.log('Stream processing stats:', processor.getMemoryStats());
    console.log('Stream results:', outputResults.length);
    
    // Clean up
    processor.reset();
    console.log('Final memory usage:', processor.estimateMemoryUsage());
}

await demoMemoryEfficientProcessing();

Summary

Core Concepts

  • Automatic Memory Management: JavaScript handles allocation and deallocation automatically
  • Garbage Collection: Multiple algorithms work together to reclaim unused memory
  • Reference Tracking: Objects stay alive as long as something references them
  • Memory Leaks: Unintended references prevent garbage collection

Theoretical Foundation

  • Memory Hierarchy: Understanding different types and speeds of memory
  • Generational GC: Most objects die young, survivors tend to live long
  • Mark and Sweep: Systematic identification and cleanup of unreachable objects
  • WeakMaps/WeakSets: Allow garbage collection of keys when no other references exist

Common Memory Problems

  • Detached DOM Elements: Event handlers keeping removed elements alive
  • Closure Captures: Functions inadvertently capturing large contexts
  • Timer Leaks: Intervals and timeouts that never get cleared
  • Cache Accumulation: Unlimited caches that grow without bounds

Memory Optimization Strategies

  • Proper Cleanup: Always remove event listeners and clear timers
  • WeakMap Usage: Use WeakMaps for object associations that shouldn't prevent GC
  • Batch Processing: Process large datasets in memory-efficient chunks
  • LRU Caching: Implement cache size limits with least-recently-used eviction

Performance Benefits

  • Reduced Memory Pressure: Lower memory usage means better performance
  • Faster GC Cycles: Less memory to scan means faster garbage collection
  • Improved Responsiveness: Avoiding memory pressure prevents UI freezing
  • Better Resource Utilization: More efficient memory usage allows handling larger datasets

My Personal Insight

Memory management was a game-changer in my JavaScript journey. Understanding that "automatic" doesn't mean "magic" helped me write much more efficient code. The biggest revelation was learning that garbage collection can't save you from poor reference management.

Key insight: Memory leaks in JavaScript are almost always about references, not actual memory allocation. Once I started thinking in terms of "what's keeping this object alive?" instead of "why isn't this being cleaned up?", debugging memory issues became much easier.

WeakMap and WeakSet were revolutionary for solving DOM element association problems. They let you create object relationships without preventing garbage collection - perfect for event management and caching scenarios.

Next Up

Now that you understand memory management, we'll explore Profiling & Performance Analysis - learning to measure, analyze, and optimize your JavaScript applications using browser developer tools and performance monitoring techniques.

Remember: Good memory management isn't about micro-optimizations - it's about understanding the lifecycle of your objects and cleaning up properly! 🚀✨

Photo of Rahul Aher

Written by Rahul Aher

I'm Rahul, Sr. Software Engineer (SDE II) and passionate content creator. Sharing my expertise in software development to assist learners.

More about me