AI Prompts for Data Science

Act as a data quality analyst. I am providing you with a pandas DataFrame named `df`. Its schema is as follows: `[[PASTE SCHEMA OR HEAD() OUTPUT HERE]]`. Your task is to perform a comprehensive data profiling. For each column, provide: 1. Data type. 2. Number and percentage of missing values. 3. Number of unique values (cardinality). 4. For numeric columns: mean, median, standard deviation, min, and max. 5. For categorical columns: a list of unique values and their frequencies. 6. Identify any columns that appear to be incorrectly typed (e.g., dates stored as objects). Present the output as a markdown table.

has missing values. Here is the output of

Generate a Python script that uses the Interquartile Range (IQR) method to identify outliers in the following numeric columns of a pandas DataFrame `df`: `[[LIST_OF_NUMERIC_COLUMNS]]`. The script should: 1. Calculate Q1, Q3, and IQR for each specified column. 2. Define the upper and lower bounds (Q1 - 1.5 * IQR, Q3 + 1.5 * IQR). 3. Create a new DataFrame containing only the rows that have outlier values in any of the specified columns. 4. Print the shape of the original DataFrame and the outlier DataFrame.

. The following columns need their data types corrected: -

. I want to check for duplicates based on a subset of columns:

that takes a string and performs the following text cleaning steps: 1. Converts the text to lowercase. 2. Removes all punctuation. 3. Removes numerical digits. 4. Removes common English stop words. 5. Stems the remaining words using the Porter Stemmer. The function should return the cleaned string. Include imports from

, generate a set of Python plots for univariate analysis. For each numeric column in

Generate Python code to calculate the Pearson correlation matrix for the numeric columns in my pandas DataFrame `df`. The columns are: `[[LIST_OF_NUMERIC_COLUMNS]]`. Then, do the following: 1. Create a heatmap of the correlation matrix using seaborn. Annotate the values on the heatmap. 2. Identify and list all pairs of variables with a correlation coefficient greater than 0.7 or less than -0.7. 3. For the top 3 most correlated pairs, provide a one-sentence hypothesis for why they might be correlated.

. For each numeric variable, generate a box plot that compares its distribution across the different categories of

with a datetime index and a column named

. Generate a Python script that: 1. Standardizes these two columns using

with latitude and longitude columns named

[[TARGET_VARIABLE, e.g., 'customer lifetime value']]

. Generate a Python script that creates the following new features from this column: - Year - Month - Day of week (as a number, 0=Monday) - Day of year - Week of year - A binary flag

Generate Python code to create interaction features for a machine learning model. My pandas DataFrame `df` has the following numeric features: `[[COLUMN_A, COLUMN_B, COLUMN_C]]`. Create all pairwise interaction features (e.g., A*B, A*C, B*C). Add these new features to the DataFrame.

to one-hot encode the following categorical columns in my DataFrame

. Generate Python code to create a new column

is indexed by date and has a target column

Act as an AutoML specialist. I have a preprocessed dataset with features `X` and a binary target variable `y`. Generate a complete Python script that trains and evaluates three different baseline classification models: 1. Logistic Regression 2. Random Forest Classifier 3. Gradient Boosting Classifier (using LightGBM) For each model, the script should: - Use 5-fold cross-validation. - Calculate and print the mean Accuracy, Precision, Recall, and F1-score. - Store the results in a pandas DataFrame for easy comparison.

from scikit-learn. My feature matrix is

features. The network architecture should be: - Input layer with

pipeline for a machine learning workflow. The pipeline should consist of the following steps: 1. Impute missing numeric values using

containing a univariate time series. Generate a Python script using

library to build a simple user-item collaborative filtering recommendation model. Assume I have a pandas DataFrame

to compute and plot a confusion matrix. The plot should be clearly labeled with axes titles ('Predicted Label', 'True Label') and the class names

Using `y_test` (true labels) and `y_pred_proba` (predicted probabilities for the positive class from my classifier), generate a Python script to: 1. Calculate the AUC (Area Under the Curve) score. 2. Plot the ROC (Receiver Operating Characteristic) curve. 3. Include a diagonal line representing a random classifier for comparison. 4. Label the plot with the AUC score.

. I want to explain a single prediction for a specific instance

I have a trained `RandomForestClassifier` model named `model`. My features have the names `[[LIST_OF_FEATURE_NAMES]]`. Generate Python code to extract the feature importances from the model, match them with their names, and create a horizontal bar plot showing the top 15 most important features.

sklearn.inspection.permutation_importance

I want to check if my binary classifier's predicted probabilities are well-calibrated. I have the true labels `y_test` and the predicted probabilities `y_pred_proba`. Generate a Python script using `sklearn.calibration.CalibrationDisplay` to plot a calibration curve for my model.

[[TARGET_VARIABLE, e.g., 'employee attrition']]

Below is a Python script that generates a plot. After the script, I will describe the plot. Your task is to write a concise, one-paragraph interpretation of this insight for a business audience. Avoid technical jargon. `[[PASTE PYTHON PLOT SCRIPT]]` The plot shows a bar chart where customers in the 'Tier 1' support category have an average monthly spend of $150, while customers in the 'Tier 3' category have an average spend of $45. Now, write the business summary.

with the results of an A/B test. It has columns

Act as a data analyst. Draft an email to the product team reporting the results of an A/B test for the new "One-Click Checkout" feature. Key findings: - Control group conversion rate: 3.5% - Treatment group (with feature) conversion rate: 4.8% - The result is statistically significant with a p-value of 0.002. - The test ran for 14 days with 50,000 users in each group. Structure the email with a clear subject line, a brief summary of the result, the key numbers, and a recommendation to roll out the feature.

Explain the concept of "overfitting" in machine learning as you would to a sales director. Use an analogy. Keep the explanation under 150 words.

Generate a markdown template for a data science project README file. The template should include the following sections: - Project Title - Business Problem - Data Source - Methodology (Data Cleaning, EDA, Modeling) - Results - How to Run the Code (Dependencies, Instructions) - Key Contacts

(the trained model file) The Dockerfile should: 1. Start from a

that accepts POST requests with a JSON payload. The script must: 1. Load a pre-trained scikit-learn model from a file named

Generate a Python script snippet that demonstrates how to use MLflow for experiment tracking. The script should: 1. Start an MLflow run. 2. Log two parameters: `learning_rate` and `n_estimators`. 3. Log three metrics: `accuracy`, `precision`, and `recall`. 4. Log the trained model itself as an artifact. 5. End the run.

branch. The workflow should be named "CI Pipeline" and perform the following jobs: 1. Set up Python 3.9. 2. Install dependencies from

def remove_outliers(df, column_name): Q1 = df[[column_name]].quantile(0.25) Q3 = df[[column_name]].quantile(0.75) IQR = Q3 - Q1 lower_bound = Q1 - 1.5 * IQR upper_bound = Q3 + 1.5 * IQR return df[(df[[column_name]] >= lower_bound) & (df[[column_name]] <= upper_bound)]

, and I have a new batch of production data in

. First, load it into a pandas DataFrame and run a full data profile. Identify missing values, data types, and basic statistics for each column. Show me the output." **Prompt 2 (after AI responds):** "Thank you. Based on that profile, generate a Python script to handle the data cleaning. Impute missing

: integer, normally distributed around a mean of 40 with a standard deviation of 10. -

method that takes model hyperparameters. - A

import pandas as pd from sklearn.model_selection import train_test_split from sklearn.ensemble import RandomForestClassifier df = pd.read_csv('data.csv') df = df.dropna() X = df[[['feature1', 'feature2']]] y = df[['target']] X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42) model = RandomForestClassifier(n_estimators=100) model.fit(X_train, y_train) print(model.score(X_test, y_test))

) for whom the model predicted a high probability of churn (0.92). Explain this prediction using a counterfactual explanation. In plain English, describe the minimal changes to this customer's features (e.g.,

. The query should calculate the retention rate of monthly customer cohorts. A customer's cohort is the month of their first purchase. Retention for a given month is the percentage of customers from a cohort who made a purchase in that month. The output should have three columns:

[['date', 'region', 'sales', 'product_category']]

) that won't fit into pandas memory. The task is to calculate the mean of