When your pandas DataFrames or NumPy arrays stop fitting in RAM, this is the escape hatch. It scales familiar Python code from gigabytes on a laptop to terabytes on a cluster without rewriting everything. The lazy execution model means you build up operations and compute only when needed. Real talk: chunking strategy matters more than you'd think, and the distributed scheduler's dashboard is legitimately helpful for debugging. Best for parallelizing file processing or scaling existing analytics code. If you're still comfortably in memory, polars is faster. If you need true out-of-core on a single machine, consider vaex. This shines when you're already hitting memory limits with pandas and need a relatively smooth migration path.
- skills.sh installs
- 2.1K
- GitHub stars
- 56.6K
- last updated
- Aug 21, 2026
Install
$ npx -y skills add k-dense-ai/scientific-agent-skills --skill dask --agent claude-codeInstalls into .claude/skills of the current project.
Files
Dask
Overview
Dask is a Python library for parallel and distributed computing that enables three critical capabilities:
- Larger-than-memory execution on single machines for data exceeding available RAM
- Parallel processing for improved computational speed across multiple cores
- Distributed computation supporting terabyte-scale datasets across multiple machines
Capacity depends on partition sizes, intermediate data, concurrency, storage, and available memory.
Reviewed release: dask/distributed 2026.8.0 (September 30, 2026 review). Docs: docs.dask.org. Since 2025.1.0, the expression-based DataFrame API with query planning is the only implementation — do not install dask-expr separately or set dataframe.query-planning: False.
Quick Start
Installation
uv pip install "dask[array,dataframe]==2026.8.0"
For a typical pandas/NumPy workflow with the distributed scheduler and dashboard:
uv pip install "dask[complete]==2026.8.0"
Remote object storage (requires provider credentials for private data):
Sponsors
Agent CaffeineSponsoredClose your lid, come back to finished work
SnugSponsoredThe clipboard manager for the AI eraPromote your product hereReach 125k+ visitors monthly
WaxSponsoredBlock anything with supervision, without erasing your iPhonePromote your product hereReach 125k+ visitors monthly
knobsSponsoredSee your site on any phone, any day, in any languageDetails
- Repository
- github.com/k-dense-ai/scientific-agent-skills
- Categories
- AI & Agent Building, Data Science & ML, Automation & Workflows, Marketing & SEO
- First seen
- Jun 3, 2026
More from k-dense-ai/scientific-agent-skills
All 137- Infographics2.1K
- Market Research Reports2.1K
- Matlab2.1K
- Get Available Resources2.1K
- Optimize For Gpu2.1K
- Pymc2.1K
- Umap Learn2.1K
- Parallel Web2.1K
- Venue Templates2.1K
- Scikit Survival2.1K
- Open Notebook2.1K
- Torch Geometric2.1K
- Biopython2.1K
- Research Lookup2.1K
- Timesfm Forecasting2.1K
- Geopandas2.1K
- Stable Baselines32.1K
- Consciousness Council2.1K
- Simpy2.1K
- Pymoo2.1K
- Scikit Bio2.1K
- Geomaster2.1K
- What If Oracle2.1K
- Hypogenic2.1K
Recommended
More AI & Agent Building
infographicsk-dense-ai/scientific-agent-skillsCreate professional infographics using Nano Banana Pro AI with smart iterative refinement. Uses Gemini 3.6 Flash for quality review. Integrates research-lookup 2.1Kskills.sh installsmarket-research-reportsk-dense-ai/scientific-agent-skillsBuild evidence-traceable market research reports and assumption-driven market sizing or forecast scenarios. Use for market definition, industry and customer evi2.1Kskills.sh installsmatlabk-dense-ai/scientific-agent-skillsBuild, review, migrate, and safely plan MATLAB or GNU Octave numerical workflows, including arrays, tabular/time data, tests, projects, graphics, MAT files, and2.1Kskills.sh installsget-available-resourcesk-dense-ai/scientific-agent-skillsDetect host inventory and effective CPU, memory, disk, scheduler, container, and accelerator limits when a user asks for resource-aware planning or before a cle2.1Kskills.sh installspymck-dense-ai/scientific-agent-skillsBayesian modeling with PyMC. Build hierarchical models, MCMC (NUTS), variational inference, LOO/WAIC comparison, posterior checks, for probabilistic programming2.1Kskills.sh installs
rest-api-designaj-geddes/useful-ai-promptsDesign RESTful APIs following best practices for resource modeling, HTTP methods, status codes, versioning, and documentation. Use when creating new APIs, desig2.1Kskills.sh installs