Spatiotemporal Analytics and Data-Driven Pattern Mining in Urban Complaints

Spatiotemporal Analytics and Data-Driven Pattern Mining in Urban Complaints Rajat Rayaraddi Department of Computer Science The George Washington University Washington, D.C. 20052 [email protected] Aishwarya Sajjan Department of Computer Science The George Washington University Washington, D.C. 20052 [email protected] Abstract This study applies spatiotemporal data mining and machine learning techniques to analyze New York City’s 311 service request data. A 10-million-record subset was extracted from the full 42-million-row dataset and cleaned, standardized, and feature-engineered. Exploratory data analysis revealed key trends in complaint distribution across boroughs, time, and socioeconomic factors. Advanced data mining techniques including geospatial KMeans clustering, contrast and sequential pattern mining, association analysis, and anomaly detection uncovered localized hotspots, co-occurring complaint behaviors, and event-driven surges. Time-series forecasting and XGBoost-based regression were used to predict complaint volume and resolution time. The findings demonstrate how large-scale civic data can inform proactive urban service planning. 1 Introduction Large metropolitan cities generate vast amounts of civic data through public service platforms such as New York City’s 311 system. These systems allow residents to report non-emergency issues related to noise, sanitation, housing, transportation, public safety, and infrastructure. While such datasets offer an unprecedented opportunity to understand urban service demand; their scale, complexity, and variability make meaningful analysis challenging. As a result, many city agencies continue to rely on descriptive reporting and reactive decision-making, limiting their ability to anticipate problems and allocate resources efficiently. Data mining and machine learning provide powerful tools for extracting actionable insights from large-scale civic datasets. Spatial analytics can reveal localized hotspots of recurring issues, temporal analysis can uncover daily and seasonal patterns, and pattern mining techniques can identify relationships between different types of complaints. Predictive models can estimate service resolution times and forecast future demand, enabling proactive planning rather than retrospective response. In this work, we performed an end-to-end spatial-temporal analysis of New York City’s 311 service request data [ 1 ]. Using a subset of ten million records spanning November 2022 to November 2025, we apply data preprocessing, exploratory data analysis, and data mining techniques to uncover complaint patterns across geography, time, and demographics. Our approach integrates clustering, association and sequential pattern mining, anomaly detection, time-series forecasting, and supervised learning to model complaint behavior and resolution dynamics. The purpose of this study is to demonstrate how data-driven methodologies can support more efficient urban service management and inform policy-level decision-making in large cities. Preprint. Figure 1: Example records from the NYC 311 service request dataset in its original, unprocessed form. Figure 2: (Continued) Example records from the NYC 311 service request dataset in its original, unprocessed form. 2 Related Works Previous research has explored the use of civic complaint data to understand urban dynamics and service efficiency. Studies analyzing 311 datasets have examined spatial distributions of complaints to identify neighborhood-level hotspots and service inequities, often linking complaint frequency to socioeconomic and demographic factors. Temporal analyses have further revealed strong daily and seasonal patterns in complaint behavior, particularly for noise, sanitation, and heating-related issues. More recent work has applied machine learning techniques to predict service response times and forecast complaint volumes. Clustering methods have been used to group neighborhoods based on complaint composition, while association and sequential pattern mining have identified co-occurring and cascading complaint types. However, many existing studies focus on limited subsets of data or isolated techniques. This work extends previous research by integrating large-scale spatial, temporal, and predictive analytics within a unified end-to-end framework. 3 Methodology This study follows an end-to-end data mining workflow encompassing data reduction, preprocessing, exploratory analysis, advanced pattern mining, and predictive modeling. The methodology is designed to handle large-scale civic data while extracting meaningful spatial, temporal, and behavioral insights. 3.1 Data Selection and Reduction The original NYC 311 Service Request dataset contains over 42 million records spanning from 2010 to 2025. To ensure computational feasibility while preserving recent trends, the dataset was reduced to approximately 10 million records covering the period from November 24, 2022 to November 9, 2025. This subset captures complaint behavior and seasonal variations. 2 Figure 3: Sample view of the NYC 311 service request data after cleaning and feature engineering. Figure 4: (Continued) Sample view of the NYC 311 service request data after cleaning and feature engineering. 3.2 Data Cleaning and Preprocessing Data preprocessing was performed using Python libraries including pandas, NumPy, and scikit-learn. Column names were standardized by converting to lowercase and replacing spaces with underscores. Date-time fields were converted to proper timestamp formats, and new temporal features like year, month, day of week, and hour were extracted. ZIP codes were standardized as strings, and latitude and longitude fields were converted to numeric values. A new feature, "days-to-close", was computed for complaints with valid closed dates to support resolution-time analysis. Approximately 20 low-information columns were removed to reduce dimensionality. Categorical missing values were imputed with the label “Unknown”, while records missing important spatial information (latitude, longitude, or ZIP code) were dropped, representing less than 2 percent of the data. Categorical variables were label-encoded, and numerical features were scaled to support clustering and machine learning models. 3.3 Exploratory Data Analysis Exploratory data analysis (EDA) was conducted to understand complaint distributions across geography, time, and categories. This included analyzing top complaint types, borough-level complaint volume and per-capita rates, temporal trends by month and hour, and resolution time distributions. Correlation analysis was performed to examine relationships between complaint volume, median household income, and education levels. 3.4 Data Mining and Pattern Discovery Several data mining techniques were applied to uncover hidden structures within the data. Geospatial KMeans clustering was used to group ZIP codes based on complaint-type frequency, with Principal Component Analysis (PCA) applied for dimensionality reduction and visualization. Contrast pattern mining using chi-square statistics and standardized residuals identified complaint types disproportionately represented across boroughs. 3.4.1 Co-Occurrence Patterns To analyze co-occurring and sequential complaint behavior, a spatially constrained sequence mining approach was employed. Latitude and longitude values were rounded to four decimal places to 3 approximate city-block–level locations (approximately 100 meters), and a composite location-id was constructed by combining the rounded coordinates. Records were sorted by time to preserve temporal ordering. For each location, sequences of complaint types were generated to capture localized complaint evolution. Complaint pairs were treated as unordered to capture co-occurrence rather than directional dependence, and frequency counts were computed using a counting-based approach. 3.4.2 Anomaly Detection To identify abnormal surges in complaint activity, an anomaly detection framework based on rolling statistical baselines was applied to daily complaint counts. Complaint creation timestamps were first standardized to datetime format and aggregated at a daily resolution to construct a univariate time series representing overall complaint volume. A rolling window of seven days was used to compute moving averages and standard deviations, capturing short-term temporal trends while smoothing day-to-day noise. An anomaly threshold was defined as the rolling mean plus 1.4 times the rolling standard deviation, and days exceeding this threshold were flagged as burst events. Threshold =µrolling + 1.4·σrolling To detect category-specific anomalies, daily complaint counts were further computed separately for each complaint type. Rolling baselines were calculated independently per category to account for varying baseline frequencies, and standardized z-scores were used to identify significant deviations. This approach enabled the detection of both global and complaint-type-specific spikes while preserving interpretability and robustness to seasonal variation. 3.5 Predictive Modeling Time-series forecasting was conducted using the Prophet library to model long-term trends and seasonal effects in complaint volume, both overall and for specific complaint categories. For the prediction of complaint resolution time, the target variable days-to-close was modeled using a supervised regression framework. Only complaints with a valid closure date were retained, and extreme outliers with resolution times exceeding 100 days were excluded to reduce skewness and improve model stability. The feature set consisted of engineered temporal attributes (year, month, day of week, and hour), spatial coordinates (latitude and longitude), and selected categorical attributes, while identifier fields, raw datetime columns, and high-cardinality text fields were removed. The dataset was split into training and testing subsets using an 80/20 split. Numerical features were standardized using z-score normalization to ensure comparable feature scales across models. Multiple tree-based ensemble regressors including XGBoost, LightGBM, CatBoost, and Random Forest were trained and evaluated using RMSE and R² metrics. 4 Results and Discussion This section presents the results of the exploratory data analysis and data mining techniques applied to the NYC 311 service request dataset, followed by a discussion of the insights derived from these analyses. 4.1 Exploratory Data Analysis 4.1.1 Top Complaint Types The analysis of the top complaint categories revealed that Illegal Parking is the most frequently reported issue, followed by Residential Noise, Heat/Hot Water, and Blocked Driveways (Figure 5). This distribution highlights the prominence of trafficand noise-related issues in dense urban environments and underscores their importance for municipal enforcement and quality-of-life management. 4.1.2 Complaint Volumes v/s. Boroughs At the borough level, Brooklyn recorded the highest absolute number of complaints (Figure 6); however, when normalized by population, the Bronx exhibited the highest number of complaints per capita (Figure 7). This disparity suggests that while complaint volume scales with population, 4 Figure 5: Top-10 complaint types. service burden is disproportionately higher in certain boroughs, indicating potential inequities in infrastructure quality or service availability. 4.1.3 Correlation Between Complaint Volumes and Income and Education Correlation analysis demonstrated a strong negative relationship between median household income and complaint volume (correlation coefficient –0.67), suggesting that lower-income areas tend to experience more reportable service issues (Figure 8). In contrast, education level exhibited a weaker positive correlation with complaint volume (+0.26), indicating that while education may influence reporting behavior, income remains the dominant socioeconomic factor. Population, income, and education statistics were obtained from publicly available sources [2, 3, 4]. 4.1.4 Resolution Time by Complaint Types Analysis of complaint resolution times showed that illegal parking and noise-related complaints are typically resolved immediately, reflecting efficient enforcement (Figure 9). In contrast, plumbing, heating, and unsanitary condition complaints required substantially longer resolution times, often several days, likely due to their reliance on infrastructure repairs. 4.1.5 Complaints by Hour of the Day by Agency Temporal analysis revealed that complaints peak during late evening hours (9–11 PM), with the NYPD consistently handling a high volume of requests throughout the day (Figure 10). This finding aligns with known urban activity patterns and suggests the need for sustained nighttime enforcement resources. Agency abbreviations are specified in Appendix A. 4.1.6 Rodent Hotspot Geospatial EDA identified a pronounced rodent complaint hotspot in the Upper West Side of Manhattan, with significantly higher complaint density than surrounding areas (Figure 11). 5 Figure 6: Complaint volumes by boroughs. Figure 7: Complaint volumes per capita by boroughs. 6 Figure 8: Correlation heatmap for how income and education levels affect complaint volumes. Figure 9: Distribution of resolution time across various complaint types. 7 Figure 10: Distribution of complaints across the day by agencies. Figure 11: Heatmap of rodent hotspots across New York City. 8 Figure 12: Streets with the highest number of complaints reported. 4.1.7 Streets v/s. Complaint Volume Street-level analysis further showed that East 230th Street in the Bronx experiences an unusually high number of complaints, followed by major corridors such as Broadway and Grand Concourse, indicating localized infrastructure or quality-of-life challenges along these streets (Figure 12). 4.2 Data Mining and Advanced Analytics 4.2.1 Geospatial KMeans Clustering Geospatial clustering using KMeans partitioned New York City into six distinct clusters, which closely resembled borough-level boundaries (Figure 13). This result suggests that complaint-type composition varies systematically by geography and that boroughs exhibit internally consistent complaint patterns. 4.2.2 Principal Component Analysis Principal Component Analysis (PCA) of ZIP code–complaint type distributions revealed a triangular or funnel-shaped spread, indicating a small number of dominant complaint dimensions driving most variability (Figure 14). This illustrates distinct complaint profiles, reinforcing the spatial heterogeneity of service needs. These findings support the use of cluster-based planning approaches for targeted resource allocation. 4.2.3 Time-Series Forecasting Time-series forecasting using Prophet revealed a clear upward trend in overall complaint volume over time, along with high variability across months and seasons (Figure 15). This variability underscores the importance of dynamic forecasting models for anticipating service demand rather than relying on static historical averages. 4.2.4 Predicting Resolution Times In predictive modeling of complaint resolution time, XGBoost achieved the best performance, followed by LightGBM, CatBoost, and Random Forest (Figure 16). However, all models exhibited relatively high RMSE values, and hyperparameter tuning produced limited improvement. To reduce the impact of extreme outliers that would disproportionately skew predictions, complaints with resolution times greater than 100 days were excluded from modeling. The remaining error suggests that resolution time may be influenced by latent factors not captured in the available features, such as contractor availability, building conditions, or policy constraints. 9