Get involved in the DSN!
Meet the data scientist
Data science resourcesMore information
About the DSN
Email the DSNThe information in these articles is provided 'as-is' and Statistics Canada makes no warranty, either expressed or implied, including but not limited to, warranties of merchantability and fitness for a particular purpose. In no event will Statistics Canada be liable for any direct, special, indirect, consequential or other damages, however caused.
Recent articles
Approaching Generative Artificial Intelligence: Recommendations and Lessons Learned from AgPal Chat
Topics covered in this article: Other
This article presents the development and implementation of AgPal Chat, a generative AI search tool designed to provide comprehensive federal, provincial, and territorial agricultural information to Canadians. Originating from the winning entry of the inaugural Canadian Public Service Data Challenge, AgPal Chat is accessible via the AgPal.ca website, offering a conversational interface to connect users with relevant agricultural data. The creation of AgPal Chat involved a collaborative effort across industry, academia, and government departments, aiming to enhance service delivery. This paper discusses the technical and policy lessons learned during the implementation process, highlighting key findings such as the use of Retrieval Augmented Generation (RAG) for improved AI accuracy, the importance of ethical guardrails for safe AI interactions, and the critical role of robust data governance and policy compliance in developing responsible AI systems.
Automating Deployment Pipelines in Azure Data Factory
Topics covered in this article: Data processing and engineering
The Financial Consumer Agency of Canada (FCAC), widely utilizes Azure Repos and Azure Pipelines for managing the integration and deployment of data resources across different environments. As a growing data team, they are consistently exploring innovative approaches to tackle data engineering processes. Recently, they addressed the challenge of automating deployment pipelines for Azure Data Factory (ADF). This article delves into their journey of automating these pipelines, highlighting the benefits of continuous integration and continuous deployment (CI/CD) practices.
Continue reading: Automating Deployment Pipelines in Azure Data Factory
Driving Donations: Analytics & ML Modelling for Enhancing Food Drive Operations
Topics covered in this article: Data processing and engineering
The Edmonton Food Drive (EFD) Project is a collaborative initiative involving NorQuest College, the LDS Church, and other partners aimed at optimizing the logistics of one of Alberta’s largest community food donation efforts. With over 400,000 meals distributed monthly to more than 40,000 individuals, the project addresses critical challenges in coordinating drop-off locations, managing pick-up processes, and planning efficient routes. To enhance operational efficiency and reduce logistical complexity, the project developed a machine learning-based solution focused on automating and improving food donation management. This approach streamlines resource allocation and transportation planning, ultimately strengthening the community’s capacity to combat food insecurity through data-driven collaboration.
Continue reading: Driving Donations: Analytics & ML Modelling for Enhancing Food Drive Operations
Other recent articles
Browse articles by topic
Computer vision
- Applying Semi-Supervised Machine Learning Classification to Anomaly Detection Exercises: The Case of Sensor Data
- Comparing Optical Character Recognition Tools for Text-Dense Documents vs. Scene Text
- Computer vision models: seed classification project
- Context modelling with transformers: Food recognition
- Data to Decisions: Visualizations and ML Modeling of Rental Property Data
- Extracting Temporal Trends from Satellite Images
- Greenhouse Detection with Remote Sensing and Machine Learning: Phase One
- Image Segmentation in Medical Imaging
- Indigenous Communities Food Receipts Crowdsourcing with Optical Character Recognition
- Reducing data gaps for training machine learning algorithms using a generalized crowdsourcing application
- Self Supervised Learning in Computer Vision: Image Classification
- Tackling Information Overload: How Global Affairs Canada's "Document Cracker" AI Application Streamlines Crisis Response Efforts
- The Rationale Behind Deep Neural Network Decisions
Data processing and engineering
- A new indicator of weekly aircraft movements
- Adopting a high Level MLOps Practice for the Production Applications of Machine Learning in the Canadian Consumer Prices Index
- An image is worth a thousand words: let your dashboard speak for you!
- Automating Deployment Pipelines in Azure Data Factory
- Building an All-in-One Web Application for Data Science Using Python: An evaluation of the open-source tool Django
- Creating Compelling Data Visualizations
- Data Engineering in Rust
- Data to Decisions: Visualizations and ML Modeling of Rental Property Data
- Deploying your machine learning project as a service
- Designing a metrics monitoring and alerting system
- Driving Donations: Analytics & ML Modelling for Enhancing Food Drive Operations
- Extracting Public Value from Administrative Data: A method to enhance analysis with linked data
- Implementing MLOps with Azure
- Making data visualizations accessible to blind and visually impaired people
- MlFlow Tracking: An efficient way of tracking modeling experiments
- Non-Pharmaceutical Intervention and Reinforcement Learning
- The COVID-19 cloud platform for advanced analytics
- Writing a Satellite Imaging Pipeline, Twice: A Success Story
Predictive analytics
- Forecasting power consumption in remote northern Canadian communities
- From Exploring to Building Accurate Interpretable Machine Learning Models for Decision-Making: Think Simple, not Complex
- Modelling SARS-CoV-2 Dynamics to Forecast PPE Demand
- NRCan's Digital Accelerator: Revolutionizing the way Natural Resources Canada serves Canadians through digital innovation
- Unlocking the power of data synthesis with the starter gide on synthetic data for official statistics
- Use of Machine Learning for Crop Yield Prediction
Text analysis and generation
- A Use Case on Metadata Management
- Adopting a high Level MLOps Practice for the Production Applications of Machine Learning in the Canadian Consumer Prices Index
- Applied Machine Learning for Text Analysis Community of Practice: 2021 in review
- Bias Considerations in Bilingual Natural Language Processing
- Chatting About Chatbots: A review of the Chatbot Workshop
- Document Intelligence: The art of PDF information extraction
- Indigenous Communities Food Receipts Crowdsourcing with Optical Character Recognition
- Official Languages in Natural Language Processing
- Text Classification of Public Service Job Advertisements
- Topic Modelling and Dynamic Topic Modelling: A technical review
- Using data science and cloud-based tools to assess the economic impact of COVID-19
- Version Control with Git for Analytics Professionals
- 2021 Census Comment Classification
Ethics and responsible machine learning
- Applying Random Forest Algorithms to Enhance Expenditure Predictions in Government Grants and Contributions Programs
- A Brief Survey of Privacy Preserving Technologies
- Adopting a high Level MLOps Practice for the Production Applications of Machine Learning in the Canadian Consumer Prices Index
- Explainable Machine Learning, Game Theory, and Shapley Values: A technical review
- Identifying Personal Identifiable Information (PII) in Unstructured Data with Microsoft Presidio
- Introduction to Privacy-Enhancing Cryptographic Techniques
- Introduction to Cryptographic Techniques: Trusted Execution Environment
- Introduction to Privacy Enhancing Cryptographic Techniques: Secure Multiparty Computation
- Privacy enhancing technologies: An overview of federated learning
- Privacy preserving technologies part three: Private statistical analysis and private text classification based on homomorpic encryption
- Privacy Preserving Technologies Part Two: Introduction to Homomorphic Encryption
- Protected workloads on public cloud
- Responsible use of automated decision systems in the federal government
- Responsible use of machine learning at Statistics Canada
Other
- Approaching Generative Artificial Intelligence: Recommendations and Lessons Learned from AgPal Chat
- Production level code in Data Science
- Celebrating women and girls in science: An interview with Dr. Sevgui Erman
- Co-op student explores the power of Big Data
- Data Science Network Newsletter product feedback survey
- Developing Competency Profiles to Shape Data Science in the Public Service
- Developments in machine learning series: Issue three
- Developments in Machine Learning Series: Issue two
- Developments in Machine Learning Series: Series one
- First Data Science Network Directors' Committee Meeting
- Low Code UI with Plotly Dash
- Ottawa to hold World Statistics Congress in July 2023
- The Data Science Network newsletter turns one!