MUI Docs Infra

Warning

This is an internal project, and is not intended for public use. No support or stability guarantees are provided.

Load Server Source

The loadServerSource utility reads source files from the filesystem and analyzes their dependencies, extracting imports and resolving relative file paths. It processes JavaScript/TypeScript files to build dependency trees and prepare code for documentation and live demos.

Features

  • Filesystem reading: Loads source files from disk using Node.js fs/promises
  • Import parsing: Extracts relative and external imports from JavaScript/TypeScript/CSS files
  • Path resolution: Resolves relative imports to absolute file URLs
  • Dependency tracking: Builds extraDependencies arrays for recursive loading by loadCodeVariant
  • Import rewriting: Rewrites import statements based on configured storage mode (storeAt)
  • External tracking: Identifies npm package dependencies with their import specifiers
  • Multi-file support: Handles JavaScript/TypeScript modules, CSS files, and static assets

Usage

import { loadServerSource } from '@mui/internal-docs-infra/pipeline/loadServerSource';

const result = await loadServerSource('file:///app/components/button/Component.tsx');

// Returns:
// {
//   source: "import React from 'react';\nexport const Button = () => { ... }",
//   extraFiles: {
//     './utils': 'file:///app/components/button/utils.ts',
//     './styles.css': 'file:///app/components/button/styles.css'
//   },
//   extraDependencies: [
//     'file:///app/components/button/utils.ts',
//     'file:///app/components/button/styles.css'
//   ],
//   externals: {
//     'react': [{ name: 'default', type: 'default', isType: false }],
//     '@mui/material': [{ name: 'Button', type: 'named', isType: false }]
//   }
// }

Configuration

Create a custom loadSource function with options:

import { createLoadServerSource } from '@mui/internal-docs-infra/pipeline/loadServerSource';

const customLoadSource = createLoadServerSource({
  includeDependencies: true, // Whether to resolve imports (default: true)
  storeAt: 'flat', // How to store imports: 'canonical' | 'import' | 'flat' (default: 'flat')
  removeCommentsWithPrefix: ['@internal'], // Remove comments starting with these prefixes from output
  notableCommentsPrefix: ['@highlight', '@section'], // Mark comments with these prefixes as notable
  maxDepth: 10, // Maximum recursion depth (not implemented yet)
  maxFiles: 100, // Maximum files to process (not implemented yet)
});

Import Storage Modes

The storeAt option controls how imports are stored in extraFiles and how import statements are rewritten in the source:

'flat' (default)

Flattens all imports to the current directory and rewrites import statements accordingly. This mode uses intelligent conflict resolution to ensure unique filenames.

// Original source:
import { Button } from '../components/Button';
import styles from './styles.module.css';

// After processing with storeAt: 'flat':
import { Button } from './Button';
import styles from './styles.module.css';

// extraFiles:
{
  './Button.tsx': 'file:///app/components/Button.tsx',
  './styles.module.css': 'file:///app/demo/styles.module.css'
}

'canonical'

Preserves full relative paths including index files when they exist:

// Original source:
import { Button } from '../components/Button';

// After processing with storeAt: 'canonical':
// (source unchanged)

// extraFiles:
{
  '../components/Button/index.tsx': 'file:///app/components/Button/index.tsx'
}

'import'

Uses import paths with file extensions but without index resolution:

// Original source:
import { Button } from '../components/Button';

// After processing with storeAt: 'import':
// (source unchanged)

// extraFiles:
{
  '../components/Button.tsx': 'file:///app/components/Button/index.tsx'
}

Return Value

interface LoadSourceResult {
  source: string; // Processed source code (may have rewritten imports)
  extraFiles?: Record<string, string>; // Map of import paths to file URLs
  extraDependencies?: string[]; // Array of file URLs for recursive loading
  externals?: Externals; // External npm package dependencies
  comments?: Record<number, string[]>; // Comments by line number (for sourceEnhancers)
}

// Externals type
type Externals = Record<string, ExternalImportItem[]>;

interface ExternalImportItem {
  name: string; // Import name (e.g., 'Button', 'default')
  type: 'named' | 'default' | 'namespace'; // Import type
  isType?: boolean; // Whether this is a TypeScript type-only import
}

Important

The values in extraFiles are file URLs (e.g., 'file:///app/utils.ts'), not source code. This allows loadCodeVariant to recursively load each dependency file by calling loadSource again with the URL.

Comment Processing

The function extracts comments from source files and returns them for use by sourceEnhancers. Two options control comment behavior:

removeCommentsWithPrefix

Strips comments starting with specified prefixes from the output source. When notableCommentsPrefix is not specified, all stripped comments are collected in the comments object:

const loadSource = createLoadServerSource({
  removeCommentsWithPrefix: ['@internal', '@private'],
});

// Source file:
// // @internal This is an implementation detail
// // User-visible comment
// const x = 1;

const result = await loadSource('file:///example.ts');
// result.source: "// User-visible comment\nconst x = 1;"
// result.comments: { 0: ['@internal This is an implementation detail'] }
// Note: "User-visible comment" is NOT collected because it doesn't match removeCommentsWithPrefix

This is useful for:

  • Removing build-time annotations from displayed code
  • Stripping internal documentation that shouldn't appear in demos
  • Hiding section markers used only for enhancer processing

notableCommentsPrefix

When specified, only comments matching these prefixes are collected in comments. Comments matching removeCommentsWithPrefix but not notableCommentsPrefix are still stripped but not collected:

const loadSource = createLoadServerSource({
  removeCommentsWithPrefix: ['@internal', '@highlight'],
  notableCommentsPrefix: ['@highlight'],
});

// Source file:
// // @internal Implementation detail
// // @highlight Important line
// const x = 1;

const result = await loadSource('file:///example.ts');
// result.source: "const x = 1;"
// result.comments: { 0: ['@highlight Important line'] }
// Note: @internal is stripped but NOT collected (not in notableCommentsPrefix)

How It Works

The function follows a straightforward pipeline:

  1. Read File: Loads the source from the filesystem
  2. Parse Imports: Extracts relative imports (starting with ./ or ../), external dependencies (npm packages), and comments using parseImportsAndComments
  3. Resolve Paths: For JavaScript/TypeScript files, resolves import paths to absolute file URLs using resolveImportResultWithFs (e.g., ./Button → /app/components/Button/index.tsx)
  4. Process Imports: Builds the extraFiles mapping and optionally rewrites import statements based on storeAt mode using processRelativeImports
  5. Filter Comments: If removeCommentsWithPrefix is set, strips matching comments from the source output
  6. Build Dependencies: Creates extraDependencies array containing file URLs for loadCodeVariant to recursively load
  7. Return Comments: Returns all parsed comments (by line number) for use by sourceEnhancers

Note

CSS files are handled differently—they skip step 3 because parseImportsAndComments already resolves paths for CSS @import and url() statements.

Examples

Loading a JavaScript File

// File: /app/components/button/Demo.tsx
import { Button } from './Button';
import styles from './styles.module.css';

const result = await loadServerSource('file:///app/components/button/Demo.tsx');
// Result:
// {
//   source: "import { Button } from './Button';\nimport styles from './styles.module.css';",
//   extraFiles: {
//     './Button.tsx': 'file:///app/components/button/Button.tsx',
//     './styles.module.css': 'file:///app/components/button/styles.module.css'
//   },
//   extraDependencies: [
//     'file:///app/components/button/Button.tsx',
//     'file:///app/components/button/styles.module.css'
//   ],
//   externals: undefined
// }

Loading with External Dependencies

// File: /app/components/button/Button.tsx
import React from 'react';
import { Box } from '@mui/material';

const result = await loadServerSource('file:///app/components/button/Button.tsx');
// Result:
// {
//   source: "import React from 'react';\nimport { Box } from '@mui/material';",
//   externals: {
//     'react': [{ name: 'default', type: 'default', isType: false }],
//     '@mui/material': [{ name: 'Box', type: 'named', isType: false }]
//   }
// }

Loading a CSS File

// File: /app/components/button/styles.css
@import './base.css';
@import './theme.css';

const result = await loadServerSource('file:///app/components/button/styles.css');
// Result:
// {
//   source: "@import './base.css';\n@import './theme.css';",
//   extraFiles: {
//     './base.css': 'file:///app/components/button/base.css',
//     './theme.css': 'file:///app/components/button/theme.css'
//   },
//   extraDependencies: [
//     'file:///app/components/button/base.css',
//     'file:///app/components/button/theme.css'
//   ]
// }

Using Custom storeAt Mode

const flatLoader = createLoadServerSource({ storeAt: 'flat' });
const canonicalLoader = createLoadServerSource({ storeAt: 'canonical' });

// File: /app/demo/index.tsx
import { Button } from '../components/button/Button';

const flatResult = await flatLoader('file:///app/demo/index.tsx');
// extraFiles: { './Button.tsx': 'file:///app/components/button/Button/index.tsx' }
// source: "import { Button } from './Button';" (rewritten!)

const canonicalResult = await canonicalLoader('file:///app/demo/index.tsx');
// extraFiles: { '../components/button/Button/index.tsx': 'file:///...' }
// source: unchanged

When to Use

Use loadServerSource when you need to:

  • Implement server-side code loading for CodeHighlighter in React Server Components
  • Build demo processing pipelines that need automatic dependency resolution
  • Analyze import dependencies in source files for documentation generation
  • Load code with its dependencies at request time (e.g., dynamic routes like app/components/[component].tsx)

Tip

For build-time optimization, use the Precompute Loader instead, which calls loadServerSource during webpack compilation and caches the results.

Related