中文译文
我希望你担任高级数据科学架构师和首席业务分析师。我正在上传一个包含原始数据的 CSV 文件。你的目标是执行一次深度技术审计,并提供一套与业务目标一致、可投入生产使用的清洗流水线。
请遵循以下 4 步执行流程:
技术审计与业务背景:分析数据模式。识别不一致之处、缺失值和数据异味。简要说明这些数据问题可能如何影响业务决策(例如,不一致的日期可能导致月度趋势分析错误)。
统计策略:基于审计结果,提出一套严谨的策略,用于插补(中位数 vs. 均值)、编码(独热编码 vs. 标签编码)和缩放(标准化 vs. 稳健缩放)。
实现代码块:编写一个模块化、符合 PEP8 的 Python 脚本,使用 pandas 和 scikit-learn。包含一个 Pipeline 对象,使代码可随时用于 Streamlit 仪表板或自动化批处理任务。
后处理验证:提供断言检查以验证数据完整性(例如,检查空值或通过向下转型进行内存优化)。
约束条件:
优先考虑内存效率(使用适当的数据类型,如 int8 或 float32)。
如果存在目标变量,确保零数据泄漏。
以结构化 Markdown 提供输出,并包含专业的代码注释。
我已经上传了文件。请开始审计。
English original
I want you to act as a Senior Data Science Architect and Lead Business Analyst. I am uploading a CSV file that contains raw data. Your goal is to perform a deep technical audit and provide a production-ready cleaning pipeline that aligns with business objectives.
Please follow this 4-step execution flow:
Technical Audit & Business Context: Analyze the schema. Identify inconsistencies, missing values, and Data Smells. Briefly explain how these data issues might impact business decision-making (e.g., Inconsistent dates may lead to incorrect monthly trend analysis).
Statistical Strategy: Propose a rigorous strategy for Imputation (Median vs. Mean), Encoding (One-Hot vs. Label), and Scaling (Standard vs. Robust) based on the audit.
The Implementation Block: Write a modular, PEP8-compliant Python script using pandas and scikit-learn. Include a Pipeline object so the code is ready for a Streamlit dashboard or an automated batch job.
Post-Processing Validation: Provide assertion checks to verify data integrity (e.g., checking for nulls or memory optimization via down casting).
Constraints:
Prioritize memory efficiency (use appropriate dtypes like int8 or float32).
Ensure zero data leakage if a target variable is present.
Provide the output in structured Markdown with professional code comments.
I have uploaded the file. Please begin the audit.