By AI Lab, Canadian Food Inspection Agency

Introduction

The AI Lab team at the Canadian Food Inspection Agency CFIA) is composed of a diverse group of experts, including data scientists, software developers, and graduate researchers, all working together to provide innovative solutions for the advancement of Canadian society. By collaborating with members from inter-departmental branches of government, the AI Lab leverages state-of-the-art machine learning algorithms to provide data-driven solutions to real-world problems and drive positive change.

At the CFIA's AI Lab, we harness the full potential of deep learning models. Our dedicated team of Data Scientists leverage the power of this transformative technology and develop customised solutions tailored to meet the specific needs of our clients.

In this article, we motivate the need for computer vision models for the automatic classification of seed species. We demonstrate how our custom models have achieved promising results using "real-world" seed images and describe our future directions for deploying a user-friendly SeedID application.

At the CFIA AI Lab, we strive not only to push the frontiers of science by leveraging cutting-edge models but also in rendering these services accessible to others and foster knowledge sharing, for the continuous advancement of our Canadian society.

Computer vision

To understand how image classification models work, we first define what exactly computer vision tasks aim to address.

What is computer vision:

Computer Vision models are fundamentally trying to solve what is mathematically referred to as ill-posed problems. They seek to answer the question: what gave rise to the image?

As humans, we do this naturally. When photons enter our eyes, our brain is able to process the different patterns of light enabling us to infer the physical world in front of us. In the context of computer vision, we are trying to replicate our innate human ability of visual perception through mathematical algorithms. Successful computer vision models could then be used to address questions related to:

• Object categorisation: the ability to classify objects in an image scene or recognise someone's face in pictures
• Scene and context categorisation: the ability to understand what is going in an image through its components (e.g. indoor/outdoor, traffic/no traffic, etc.)
• Qualitative spatial information: the ability to qualitatively describe objects in an image, such as a rigid moving object (e.g. bus), a non-rigid moving object (e.g. flag), a vertical/horizontal/slanted object, etc.

Yet, while these appear to be simple tasks, computers still have difficulties in accurately interpreting and understanding our complex world.

Why is computer vision so hard:

To understand why computers seemingly struggle to perform these tasks, we must first consider what an image is.

An image is a set of numbers, with typically three colour channels: Red, Green, Blue. In order to derive any meaning from these values, the computer must perform what is known as image reconstruction. In its most simplified form, we can mathematically express this idea through an inverse function:

x = F-1(y)

Where:

y represents data measurements (ie. pixel values).
x represents a reconstructed version of measurements, y, into an image.

However, it turns out solving this inverse problem is harder than expected due to its ill-posed nature.

What is an ill-posed problem

When an image is registered, there is an inherent loss of information as the 3D world gets projected onto a 2D plane. Even for us, collapsing the spatial information we get from the physical world can make it difficult to discern what we are looking at through photos.

It can be difficult to recognise objects in 2D pictures due to possible ill-posed properties, such as:

• Lack of uniqueness: Several objects can give rise to the same measurement.
• Uncertainty: Noise (e.g. blurring, pixilation, physical damage) in photos can make it difficult or impossible to reconstruct and identify an image.
• Inconsistency: slight changes in images (e.g. different viewpoints, different lighting, different scales, etc.) can make it challenging to solve for the solution, x, from available data points, y.

While computer vision tasks may, at first glance, appear superficial, the underlying problem they are trying to address is quite challenging!

Now we will address some Deep Learning driven solutions to tackle computer vision problems.

Convolutional Neural Networks (CNNs)

Convolutional Neural Networks (CNNs) are a type of algorithm that has been really successful in solving many computer vision problems, as previously described. In order to classify or identify objects in images, a CNN model first learns to recognize simple features in the images, such as edges, corners, and textures. It does this by applying different filters to the image. These filters help the network focus on specific patterns. As the model learns, it starts recognizing more complex features and combines the simple features it learned in the previous step to create more abstract and meaningful representations. Finally, the CNN takes the learned features and to classify images based on the classes it's been trained with.

The first CNN was first proposed by Yann LeCun in 1989 (LeCun, 1989) for the recognition of handwritten digits. Since then, CNNs have evolved significantly over the years, driven by advancements in both model architecture and available computing power. To this day, CNNs continue to prove themselves are powerful architectures for various recognition and data analysis tasks.

Vision Transformers (ViTs)

Vision Transformers (ViTs) are a recent development in the field of computer vision that apply the concept of transformers, originally designed for natural language processing tasks, to visual data. Instead of treating an image as a 2D object, Vision Transformers view an image as a sequence of patches, similar to how transformers treat a sentence as a sequence of words.

The process starts by splitting an image into a grid of patches. Each patch is then flattened into a sequence of pixel vectors. Positional encodings are added to retain the positional information, as is done in transformers for language tasks. The transformed input is then processed through multiple layers of transformer encoders to create a model capable of understanding complex visual data.

Just as Convolutional Neural Networks (CNNs) learn to identify patterns and features in an image through convolutional layers, Vision Transformers identify patterns by focusing on the relationships between patches in an image. They essentially learn to weigh the importance of different patches in relation to others to make accurate classifications. The ViT model was first introduced by Google's Brain team in a paper in 2020. While CNNs dominated the field of computer vision for years, the introduction of Vision Transformers demonstrated that methods developed for natural language processing could also be used for image classification tasks, often with superior results.

One significant advantage of Vision Transformers is that, unlike CNNs, they do not have a built-in assumption of spatial locality and shift invariance. This means they are better suited for tasks where global understanding of an image is required, or where small shifts can drastically change the meaning of an image.

However, ViTs typically require a larger amount of data and compute resources compared to CNNs. This factor has led to a trend of hybrid models that combine both CNNs and transformers to harness the strengths of both architectures.

Seed classification

Background:

Canada's multi-billion seed and grain industry has established a global reputation in the production, processing, and exportation of premium-grade seeds for planting or grains for food across a diverse range of crops. Its success is achieved through Canada's commitment to innovation and the development of advanced technologies, allowing for the delivery of high-quality products with national and international standards with diagnostic certification that meet both international and domestic needs.

Naturally, a collaboration between a research group from the Seed Science and Technology Section and the AI Lab of CFIA was formed to maintain Canada's role as a reputable leader in the global seed or grain and their associated testing industries.

Background: Quality Control

The seed quality of a crop is reflected in a grading report, whereby the final grade reflects how well a seed lot conforms with Canada's Seeds Regulations to meet minimum quality standards. Factors used to determine crop quality include contaminated weed seeds according to Canada's Weed Seeds Order, purity analysis, and germination and disease. While germination provides potentials of field performance, assessing content of physical purity is essential in ensuring that the crop contains a high amount of the desired seeds and is free from contaminants, such as prohibited and regulated species, other crop seeds, or other weed seeds. Seed inspection plays an important role in preventing the spread of prohibited and regulated species listed in the Canadian Weed Seeds Order. Canada is one of the biggest production bases for global food supply, exporting huge number of grains such as wheat, canola, lentils, and flax. To meet the Phyto certification requirement and be able to access wide foreign markets, analyzing regulated weed seeds for importing destinations is in high demand with quick turnaround time and frequent changes. Testing capacity for weeds seeds requires the support of advanced technologies since the traditional methods are facing a great challenge under the demands.

Motivation

Presently, the evaluation of a crop's quality is done manually by human experts. However, this process is tedious and time consuming. At the AI Lab, we leverage advanced computer vision models to automatically classify seed species from images, rendering this process more efficient and reliable.

This project aims to develop and deploy a powerful computer vision pipeline for seed species classification. By automating this classification process, we are able to streamline and accelerate the assessment of crop quality. We develop upon advanced algorithms and deep learning techniques, while ensuring an unbiased and efficient evaluation of crop quality, paving the way for improved agricultural practices.

Project #1: Multispectral Imaging and Analysis

In this project, we employ a custom computer vision model to assess content purity, by identifying and classifying desired seed species from undesired seed species.

We successfully recover and identify the contamination by three different weed species in a screening mixture of wheat samples.

Our model is customised to accept unique high resolution, 19-channel multi-spectral image inputs and achieves greater than 95% accuracy on held out testing data.

We further explored our model's potential to classify new species, by injecting five new canola species into the dataset and observing similar results. These encouraging findings highlight our model's potential for continual use even as new seed species are introduced.

Our model was trained to classify the following species:

• Three different thistles (weed) species:
  • Cirsium arvense (regulated species)
  • Carduus nutans (Similar to the regulated species)
  • Cirsium vulgare (Similar to the regulated species)
• Six Crop seeds,
  • Triticum aestivum subspecies aestivum
  • Brassica napus subspecies napus
  • Brassica juncea
  • Brassica juncea (yellow type)
  • Brassica rapa subspecies oleifera
  • Brassica rapa subspecies oleifera (brown type)

Our model was able to correctly identify each seed species with an accuracy of over 95%.

Moreover, when the three thistle seeds were integrated with the wheat screening, the model achieved an average accuracy of 99.64% across 360 seeds. This demonstrated the model's robustness and ability to classify new images.

Finally, we introduced five new canola species and types and evaluated our model's performance. Preliminary results from this experiment showed a ~93% accuracy on the testing data.

Project #2: Digital Microscope RGB Imaging and Analysis

In this project, we employ a 2-step process to identify a total of 15 different seed species with regulatory significance and morphological challenge across varying magnification levels.

First, a seed segmentation model is used to identify each instance of a seed in the image. Then, a classification model classifies each seed species instance.

We perform multiple ablation studies by training on one magnification profile then testing on seeds coming from a different magnification set. We show promising preliminary results of over 90% accuracy across magnification levels.

Three different magnification levels were provided for the following 15 species:

• Ambrosia artemisiifolia
• Ambrosia trifida
• Ambrosia psilostachya
• Brassica junsea
• Brassica napus
• Bromus hordeaceus
• Bromus japonicus
• Bromus secalinus
• Carduus nutans
• Cirsium arvense
• Cirsium vulgare
• Lolium temulentum
• Solanum carolinense
• Solanum nigrum
• Solanum rostratum

A mix of 15 different species were taken at varying magnification levels. The magnification level was denoted by the total number of instances of seeds present in the image, either: 1, 2, 6, 8, or 15 seeds per image.

In order to establish a standardised image registration protocol, we independently trained separate models from a subset of data at each magnification then evaluated the model performance across a reserved test set for all magnification levels.

Preliminary results demonstrated the model's ability to correctly identify seed species across magnifications with over 90% accuracy.

This revealed the model's potential to accurately classify previously unseen data at varying magnification levels.

Throughout our experiments, we tried and tested out different methodologies and models.

Advanced models equipped with a canonical form such as Swin Transformers fared much better and proved to be less perturbed by the magnification and zoom level.

Discussion + Challenges

Automatic seed classification is a challenging task. Training a machine learning model to classify seeds poses several challenges due to the inherent heterogeneity within and between different species. Consequently, large datasets are required to effectively train a model to learn species-specific features. Additionally, the high degree of similarity among different species within genera for some of them makes it challenging for even human experts to differentiate between closely related intra-genus species. Furthermore, the quality of image acquisition can also impact the performance of seed classification models, as low-quality images can result in the loss of important information necessary for accurate classification.

To address these challenges and improve model robustness, data augmentation techniques were performed as part of the preprocessing steps. Affine transformations, such as scaling and translating images, were used to increase the sample size, while adding Gaussian noise can increase variation and improve generalization on unseen data, preventing overfitting on the training data.

Selecting the appropriate model architecture was crucial in achieving the desired outcome. A model may fail to produce accurate results if end users do not adhere to a standardized protocol, particularly when given data that falls outside the expected distribution. Therefore, it was imperative to consider various data sources and utilize a model that can effectively generalize across domains to ensure accurate seed classification.

Conclusion

The seed classification project is an example of the successful and ongoing collaboration between the AI Lab and the Seed Science group at the CFIA. By pooling their respective knowledge and expertise, both teams contribute to the advancement of Canada's seed and grain industries. The seed classification project showcases how leveraging advanced machine learning tools has the potential to significantly enhance the accuracy and efficiency of evaluating seed or grain quality with compliance of Seed or Plant Protection regulations, ultimately benefiting both the agricultural industry, consumers, Canadian biosecurity, and food safety.

As Data Scientists, we recognise the importance of open-source collaboration, and we are committed to upholding the principles of open science. Our objective is to promote transparency and engagement through open sharing with the public.

By making our application available, we invite fellow researchers, seed experts, and developers to contribute to its further improvement and customisation. This collaborative approach fosters innovation, allowing the community to collectively enhance the capabilities of the SeedID application and address specific domain requirements.

Meet the Data Scientist

If you have any questions about my article or would like to discuss this further, I invite you to Meet the Data Scientist, an event where authors meet the readers, present their topic and discuss their findings.

Register for the Meet the Data Scientist event. We hope to see you there!

MS Teams – link will be provided to the registrants by email

Subscribe to the Data Science Network for the Federal Public Service newsletter to keep up with the latest data science news.

By David Chiumera, Statistics Canada

In recent years, the field of data science has experienced explosive growth, with businesses across many sectors investing heavily in data-driven solutions to optimize decision-making processes. However, the success of any data science project relies heavily on the quality of the code that underpins it. Writing production-level code is crucial to ensure that data science models and applications can be deployed and maintained effectively, enabling businesses to realize the full value of their investment in data science.

Production-level code refers to code that is designed to meet the needs of the end user, with a focus on scalability, robustness, and maintainability. This contrasts with code that is written purely for experimentation and exploratory purposes, which may not be optimized for production use. Writing production-level code is essential for data science projects as it allows for the efficient deployment of solutions into production environments, where they can be integrated with other systems and used to inform decision-making.

Production-level code has several key benefits for data science projects. Firstly, it ensures that data science solutions can be easily deployed and maintained. Secondly, it reduces the risk of errors, vulnerabilities, and downtime. Lastly, it facilitates collaboration between data scientists and software developers, enabling them to work together more effectively to deliver high-quality solutions. Finally, it promotes code reuse and transparency, allowing data scientists to share their work with others and build on existing code to improve future projects.

Overall, production-level code is an essential component of any successful data science project. By prioritizing the development of high-quality, scalable, and maintainable code, businesses can ensure that their investment in data science delivers maximum value, enabling them to make more informed decisions and gain a competitive edge in today's data-driven economy.

Scope of Data Science and its various applications

The scope of data science is vast, encompassing a broad range of techniques and tools used to extract insights from data. At its core, data science involves the collection, cleaning, and analysis of data to identify patterns and make predictions. Its applications are numerous, ranging from business intelligence and marketing analytics to healthcare and scientific research. Data science is used to solve a wide range of problems, such as predicting consumer behavior, detecting fraud, optimizing operations, and improving healthcare outcomes. As the amount of data generated continues to grow, the scope of data science is expected to expand further, with increasing emphasis on the use of advanced techniques such as machine learning and artificial intelligence.

Proper programming and software engineering practices for Data Scientists

Proper programming and software engineering practices are essential for building robust data science applications that can be deployed and maintained effectively. Robust applications are those that are reliable, scalable, and efficient, with a focus on meeting the needs of the end user. There are several types of programming and software engineering practices that are particularly important in the context of data science, such as version control, automated testing, documentation, security, code optimization, and proper use of design patterns to name a few.

By following proper practices, data scientists can build robust applications that are reliable, scalable, and efficient, with a focus on meeting the needs of the end user. This is critical for ensuring that data science solutions deliver maximum value to businesses and other organizations.

Administrative Data Pre-processing Project (ADP) project and its purpose - an example.

The ADP project is a Field 7 application that required involvement from the Data Science Division to refactor a citizen developed component due to a variety of issues that were negatively impacting its production readiness. Specifically, the codebase used to integrate workflows external to the system was found to be lacking in adherence to established programming practices, leading to a cumbersome and difficult user experience. Moreover, there was a notable absence of meaningful feedback from the program upon failure, making it difficult to diagnose and address issues.

Further exacerbating the problem, the codebase was also found to be lacking in documentation, error logging, and meaningful error messages for users. The codebase was overly coupled, making it difficult to modify or extend the functionality of the program as needed, and there were no unit tests in place to ensure reliability or accuracy. Additionally, the code was overfitted to a single example, which made it challenging to generalize to other use cases, there were also several desired features that were not present to meet the needs of the client.

Given these issues, the ability for the ADP project to pre-process semi-structured data was seriously compromised. The lack of feedback and documentation made it exceedingly difficult for the client to use the integrated workflows effectively, if at all, leading to frustration and inefficiencies. The program outputs were often inconsistent with expectations, and the absence of unit tests meant that reliability and accuracy were not assured. In summary, the ADP project's need for a refactor of the integrated workflows (a.k.a. clean-up or redesign) was multifaceted and involved addressing a range of programming and engineering challenges to ensure a more robust and production-ready application. To accomplish this, we used a Red Green refactoring approach to improve the quality of the product.

Red Green vs Green Red approach to refactoring

Refactoring is the process of restructuring existing code in order to improve its quality, readability, maintainability, and performance. This can involve a variety of activities, including cleaning up code formatting, eliminating code duplication, improving naming conventions, and introducing new abstractions and design patterns.

There are several reasons why refactoring is beneficial. Firstly, it can improve the overall quality of the codebase, making it easier to understand and maintain. This can save time and effort over the long term, especially as codebases become larger and more complex. Additionally, refactoring can improve performance and reduce the risk of bugs or errors, leading to a more reliable and robust application.

One popular approach to refactoring is the "Red Green" approach, as part of the test-driven development process. In the Red Green approach, a failing test case is written before any code is written or refactored. This failing test is then followed by writing the minimum amount of code required to make the test pass, before proceeding to refactor the code to a better state if necessary. In contrast, the Green Red approach is the reverse of this, where the code is written before the test cases are written and run.

The benefits of the Red Green approach include the ability to catch errors early in the development process, leading to fewer bugs and more efficient development cycles. The approach also emphasizes test-driven development, which can lead to more reliable and accurate code. Additionally, it encourages developers to consider the user experience from the outset, ensuring that the codebase is designed with the end user in mind.

• Deployed and maintained – means code can be configured and put into production by operations team easily,  is easy to understood by developers, and is robust (with added benefit of the code can be readily picked up by new resources,

• Reduce risk of errors and downtime means the code is of good quality, stable (or handles errors gracefully), and provides consistent and accurate results

In the case of the ADP project, the Red Green approach was applied during the refactoring process. This led to a smooth deployment process, with the application being more reliable, robust, and easier to use. By applying this approach, we were able to address the various programming and engineering challenges facing the project, resulting in a more efficient, effective, stable, and production-ready application.

Standard Practices Often Missing in Data Science Work

While data science has become a critical field in many industries, it is not without its challenges. One of the biggest issues is the lack of standard practices that are often missing in data science work. While there are many standard practices that can improve the quality, maintainability, and reproducibility of data science code, many data scientists overlook them in favor of quick solutions.

This section will cover some of the most important standard practices that are often missing in data science work. These include:

• version control
• testing code (unit, integration, system, acceptance)
• documentation
• code reviews
• ensuring reproducibility
• adhering to style guidelines (i.e. PEP standards)
• using type hints
• writing clear docstrings
• logging errors
• validating data
• writing low-overhead code
• implementing continuous integration and continuous deployment (CI/CD) processes

By following these standard practices, data scientists can improve the quality and reliability of their code, reduce errors and bugs, and make their work more accessible to others.

Documenting Code

Documenting code is crucial for making code understandable and usable by other developers. In data science, this can include documenting data cleaning, feature engineering, model training, and evaluation steps. Without proper documentation, it can be difficult for others to understand what the code does, what assumptions were made, and what trade-offs were considered. It can also make it difficult to reproduce results, which is a fundamental aspect of scientific research as well as building robust and reliable applications.

Writing Clear Docstrings

Docstrings are strings that provide documentation for functions, classes, and modules. They are typically written in a special format that can be easily parsed by tools like Sphinx to generate documentation. Writing clear docstrings can help other developers understand what a function or module does, what arguments it takes, and what it returns. It can also provide examples of how to use the code, which can make it easier for other developers to integrate the code into their own projects.

def complex (real=0.0, imag=0.0):
    """Form a complex number.

    Keyword arguments:
    real -- the real part (default 0.0)
    imag -- the imaginary part (default 0.0)
    """
    if imag == 0.0 and real == 0.0:
        return compelx_zero
    ...

Multi-Line Docstring Example

Adhering to Style Guidelines

Style guidelines in code play a crucial role in ensuring readability, maintainability, and consistency across a project. By adhering to these guidelines, developers can enhance collaboration and reduce the risk of errors. Consistent indentation, clear variable naming, concise commenting, and following established conventions are some key elements of effective style guidelines that contribute to producing high-quality, well-organized code. An example of this are PEP (Python Enhancement Proposal) standards, which provide guidelines and best practices for writing Python code. It ensures that code can be understood by other Python developers, which is important in collaborative projects but also for general maintainability. Some PEP standards address naming conventions, code formatting, and how to handle errors and exceptions.

Using Type Hints

Type hints are annotations that indicate the type of a variable or function argument. They are not strictly necessary for Python code to run, but they can improve code readability, maintainability, and reliability. Type hints can help detect errors earlier in the development process and make code easier to understand by other developers. They also provide better interactive development environment (IDE) support and can improve performance by allowing for more efficient memory allocation.

Version Control

Version control is the process of managing changes to code and other files over time. It allows developers to track and revert changes, collaborate on code, and ensure that everyone is working with the same version of the code. In data science, version control is particularly important because experiments can generate large amounts of data and code. By using version control, data scientists can ensure that they can reproduce and compare results across different versions of their code and data. It also provides a way to track and document changes, which can be important for compliance and auditing purposes.

Testing Code

Testing code is the formal (and sometimes automated) verification of the completeness, quality, and accuracy of code against expected results. Testing code is essential for ensuring that the codebase  works as expected and can be relied upon. In data science, testing can include unit tests for functions and classes, integration tests for models and pipelines, and validation tests for datasets. By testing code, data scientists can catch errors and bugs earlier in the development process and ensure that changes to the code do not introduce new problems. This can save time and resources in the long run by reducing the likelihood of unexpected errors and improving the overall quality of the code.

Code Reviews

Code reviews are a process in which other developers review new code and code changes to ensure that they meet quality and style standards, are maintainable, and meet the project requirements. In data science, code reviews can be particularly important because experiments can generate complex code and data, and because data scientists often work independently or in small teams. Code reviews can catch errors, ensure that code adheres to best practices and project requirements, and promote knowledge sharing and collaboration among team members.

Ensuring Reproducibility

Reproducibility is a critical aspect of scientific research and data science. Reproducible results are necessary for verifying and building on previous research, and for ensuring that results are consistent, valid and reliable. In data science, ensuring reproducibility can include documenting code and data, using version control, rigorous testing, and providing detailed instructions for running experiments. By ensuring reproducibility, data scientists can make their results more trustworthy and credible and can increase confidence in their findings.

Logging

Logging refers to the act of keeping a register of events that occur in a computer system. This is important for troubleshooting, information gathering, security, providing audit info, among other reasons. It generally refers to writing messages to a log file. Logging is a crucial part of developing robust and reliable software, including data science applications. Logging errors can help identify issues with the application, which in turn helps to debug and improve it. By logging errors, developers can gain visibility into what went wrong in the application, which can help them diagnose the problem and take corrective action.

Logging also enables developers to track the performance of the application over time, allowing them to identify potential bottlenecks and areas for improvement. This can be particularly important for data science applications that may be dealing with large datasets or complex algorithms.

Overall, logging is an essential practice for developing and maintaining high-quality data science applications.

Writing Low-Overhead Code

When it comes to data science applications, performance is often a key consideration. To ensure that the application is fast and responsive, it's important to write code that is optimized for speed and efficiency.

One way to achieve this is by writing low-overhead code. Low-overhead code is code that uses minimal resources and has a low computational cost. This can help to improve the performance of the application, particularly when dealing with large datasets or complex algorithms.

Writing low-overhead code requires careful consideration of the algorithms and data structures used in the application, as well as attention to detail when it comes to memory usage and processing efficiency. Thought should be given to the system needs and overall architecture and design of a system up front to avoid major design changes down the road.

Additionally, low-overhead code is easily maintained requiring infrequent reviews and updates. This is important as it reduces the cost to maintain systems and allows for more focused development on improvements or new solutions.

Overall, writing low-overhead code is an important practice for data scientists looking to develop fast and responsive applications that can handle large datasets and complex analyses while keeping maintenance costs low.

Data Validation

Data validation is the process of checking that the input data meets certain requirements or standards. Data validation is another important practice in data science as it can help to identify errors or inconsistencies in the data before they impact the analysis or modeling process.

Data validation can take many forms, from checking that the data is in the correct format to verifying that it falls within expected ranges or values. Different types of data validation checks exist, such as type, format, correctness, consistency, and uniqueness. By validating data, data scientists can ensure that their analyses are based on accurate and reliable data, which can improve the accuracy and credibility of their results.

Continuous Integration and Continuous Deployment (CI/CD)

Continuous Integration and Continuous Deployment (CI/CD) is a set of best practices for automating the process of building, testing, and deploying software. CI/CD can help to improve the quality and reliability of data science applications by ensuring that changes are tested thoroughly and deployed quickly and reliably.

CI/CD involves automating the process of building, testing, and deploying software, often using tools and platforms such as Jenkins, GitLab, or GitHub Actions. By automating these processes, developers can ensure that the application is built and tested consistently, and that any errors or issues that block deployment of problematic code are identified and addressed quickly.

CI/CD can also help to improve collaboration among team members, by ensuring that changes are integrated and tested as soon as they are made, rather than waiting for a periodic release cycle.

Overall, CI/CD is an important practice for data scientists looking to develop and deploy high-quality data science applications quickly and reliably.

Conclusion

In summary, production-level code is critical for data science projects and applications. Proper programming practices and software engineering principles such as adhering to PEP standards, using type hints, writing clear docstrings, version control, testing code, logging errors, validating data, writing low-overhead code, implementing continuous integration and continuous deployment (CI/CD), and ensuring reproducibility are essential for creating robust, maintainable, and scalable applications.

Not following these practices can result in difficulties such as a lack of documentation, no error logging, no meaningful error messages for users, highly coupled code, overfitted code to a single example, lacking features desired by clients, and failure to provide feedback upon failure. These issues can severely impact production readiness and frustrate users. If a user is frustrated, then productivity will be impacted and result in negative downstream impacts on businesses’ ability to effectively deliver their mandate.

The most practical tip for implementing production-level code is to work together, assign clear responsibilities and deadlines, and understand the importance of each of these concepts. By doing so, it becomes easy to implement these practices in projects and create maintainable and scalable applications.

Meet the Data Scientist

If you have any questions about my article or would like to discuss this further, I invite you to Meet the Data Scientist, an event where authors meet the readers, present their topic and discuss their findings.

Register for the Meet the Data Scientist event. We hope to see you there!

MS Teams – link will be provided to the registrants by email

Subscribe to the Data Science Network for the Federal Public Service newsletter to keep up with the latest data science news.

Hardware-based protection of data in use that can be applied anywhere

By: Betty Ann Bryanton, Canada Revenue Agency

Introduction

The increasing popularity of connected devices and the prevalence of technologies, such as cloud, mobile computing, and the Internet of Things (IoT), has strained existing security capabilities and exposed "gaps in data security" (Lowans, 2020). Organizations that handle Personally Identifiable Information (PII) must "mitigate threats that target the confidentiality and integrity of either the application, or the data in system memory" (The Confidential Computing Consortium, 2021).

As a result, Gartner predicts, "by 2025, 50% of large organizations will adopt privacy-enhancing computation (PEC)^Footnote1 for processing data in untrusted environments^Footnote2 and multiparty data analytics use cases" (Gartner, 2020). Of the several PEC techniques, Trusted Execution Environment is the only one that relies on hardware to accomplish its privacy-enhancing goal.

What is a Trusted Execution Environment?

A Trusted Execution Environment (TEE), or Secure Enclave as they are sometimes known, is an environment built with special hardware modules that allows for a secure area inside the device. This isolated environment runs in parallel with the operating system (OS). Input is passed into the TEE and computation is performed within the TEE ('secure world'), thereby protected from the rest of the untrusted system ('normal world'). These secure and isolated environments protect content confidentiality and integrity, preventing unauthorized^Footnote3 access to, or modification of, applications and data while in use.

The term 'confidential computing' is often used synonymously with TEE; they are related, but distinct. As per the Confidential Computing Consortium (CCC),^Footnote4 confidential computing is enabled by the TEE; further, confidential computing provided by the hardware-based TEE is independent of topographical location (no mention of cloud, a user's device, etc.), processors (a regular processor or a separate one), or isolation techniques (e.g., whether encryption is used).

Why is hardware necessary?

"Security is only as strong as the layers below it, since security in any layer of the compute stack could potentially be circumvented by a breach at an underlying layer" (The Confidential Computing Consortium, 2021). By moving security to its lowest silicon level, this reduces potential compromise since it minimizes dependencies higher in the stack (e.g., from the OS, peripherals, and their administrators and vendors).

Why is it important?

Using a TEE allows a massive range of functionality to be provided to the user, while still meeting the requirements of privacy and confidentiality, without risking data when it is decrypted during processing. This allows users to secure intellectual property and ensure that PII is inaccessible. This protects against insider threats, attackers running malicious code, or unknown cloud providers. As such, TEEs represent a crucial layer in a layered security approach (aka defense-in-depth) and "have the potential to significantly boost the security of systems" (Lindell, 2020).

Uses

A TEE "can be applied anywhere including public cloud servers, on-premises servers, gateways, IoT devices, Edge^Footnote5 deployments, user devices, etc." (The Confidential Computing Consortium, 2021).

As per Confidential Computing: Hardware-Based Trusted Execution for Applications and Data, below is a summary of possible use cases for a TEE.

• Keys, secrets, credentials, tokens: These high-value assets are the 'keys to the kingdom.' Historically, the storage and processing of these assets required an on-premises hardware security module (HSM), but within TEEs, applications to manage these assets can provide security comparable to a traditional HSM.
• Multi-party computing: TEEs allow organizations, such as those offering financial services or healthcare, to take advantage of shared data (e.g., federated analytics) without compromising the data sources.
• Mobile, personal computing, and IoT devices: Device manufacturers or application developers include TEEs to provide assurances that personal data is not observable during sharing or processing.
• Point of sale devices / payment processing: To protect user-entered information, such as a PIN, the input from the number pad is only readable by code within the device's hardware-based TEE, thereby ensuring it cannot be read or attacked by malicious software that may exist on the device.

Benefits

• Controlled environment: Since the TEE runs on specialized hardware, it is controlled, and it prevents eavesdropping while encrypted data is decrypted.
• Privacy: It is possible to encrypt PII in a database; however, to process the data, it must be decrypted, at which point it is vulnerable to any attacker and to insider threats. If the data is only ever decrypted and processed inside the TEE, it is isolated from unauthorized users, thereby safeguarding data privacy.
• Speed: Since the TEE is a secure enclave already, code or data may exist in unencrypted form in the TEE. If so, "this allows execution within the TEE to be much faster than execution tied to complex cryptography" (Choi & Butler, 2019).
• Trust: Since the data in the TEE is not obfuscated (as in some of the other PEC techniques), this provides a comfort level that the computation and its results are correct, i.e., not having errors introduced by the obfuscation techniques.
• Separation of concerns: As there are two distinct environments, there is a separation of workload and data administered and owned by the 'normal world' versus workload and data isolated in the 'secure world.' This protects against insider threats and potentially corrupt workloads running on the same device.
• Decryption: If the data is encrypted in the TEE, it must be decrypted for processing; however, that decryption benefits by being contained within a tightly controlled space.

Challenges

• Implementation: Implementation is challenging and requires customized knowledge and expertise, whether building the entire secure OS from scratch, employing a trusted OS from a commercial vendor, or implementing emerging components such as Software Development Kits (SDKs), libraries, or utilities.
• Lack of standardization: Not all TEEs offer the same security guarantees or the same requirements for integration with existing and new code.
• Design specification: It is the TEE developer's responsibility to ensure secure TEE design. Mere existence of a TEE is not enough.
• Lock-in: There is potential for lock-in and dependencies with hardware vendors, TEE developers, or proprietary processing (due to lack of standardization).
• Not bullet proof: There is the possibility for side-channel attacks^Footnote6, vulnerable application code, or hardware-based security vulnerabilities, e.g., in the hardware chip, which can make the whole security model collapse.
• Performance and cost: In comparison to setup and processing in a 'normal world', using a TEE ('secure world') negatively impacts performance and will cost more.

What's possible now?

TEEs are provided by solutions such as Intel's Software Guard eXtensions (SGX) or ARM's TrustZone; via hardware vendor Software Development Kits (SDKs); or with abstraction layers (e.g., Google's Asylo) that eliminate the requirement to code explicitly for a TEE.

Many cloud vendors (e.g., Alibaba, Microsoft, IBM, and Oracle) are now providing TEE capabilities as a dedicated low-level service aligned with their computation offerings. However, due to lack of standardization, the specifications offered by cloud vendors should be closely examined to ensure they meet the organization's desired privacy and security requirements (Fritsch, Bartley, & Ni, 2020).

What's next?

While protecting sensitive data poses significant architecture, governance, and technology challenges, using a TEE may provide a starting point for an alternative means of enhancing security from the lowest level.

However, a TEE is not plug-and-play; it is a technically challenging mechanism that "should be reserved for the highest-risk use cases" (Lowans, 2020). Nonetheless, "it is certainly harder to steal secrets from inside [a secured TEE than from the unsecured 'normal world']. It makes the attacker's job harder, and that is always a good thing" (Lindell, 2020).

Related Topics

Homomorphic Encryption, Secure Multiparty Computation, differential privacy, data anonymization, Trusted Platform Module.

Meet the Data Scientist

If you have any questions about my article or would like to discuss this further, I invite you to Meet the Data Scientist, an event where authors meet the readers, present their topic and discuss their findings.

Register for the Meet the Data Scientist event. We hope to see you there!

MS Teams – link will be provided to the registrants by email

Subscribe to the Data Science Network for the Federal Public Service newsletter to keep up with the latest data science news.

References

• Accenture Labs. (2019, October 1). Maximize collaboration through secure data sharing. Retrieved from Accenture: Maximize collaboration through secure data sharing
• Bartock, M., Souppaya, M., Savino, R., Knoll, T., Shetty, U., Cherfaoui, M., . . . Scarfone, K. (2021, May). Hardware-Enabled Security: Enabling a Layered Approach to Platform Security for Cloud and Edge Computing Use Cases (NISTIR 8320 Draft). Retrieved from National Institute of Standards and Technology: Hardware-Enabled Security: Enabling a Layered Approach to Platform Security for Cloud and Edge Computing Use Cases (2nd Draft)
• Choi, J. I., & Butler, K. R. (2019, April 2). Secure Multiparty Computation and Trusted Hardware: Examining Adoption Challenges and Opportunities (Article ID 1368905). Retrieved from Hindawi: Secure Multiparty Computation and Trusted Hardware: Examining Adoption Challenges and Opportunities
• Felton, D. (2019, July 5). What is a Trusted Execution Environment (TEE)? Retrieved from Trustonic: What is a Trusted Execution Environment (TEE)?
• Fritsch, J., Bartley, R., & Ni, A. (2020, May 23). Solution Comparison for the Native Security Capabilities Within Alibaba Cloud, AWS, Azure, GCP, IBM Cloud and OCI (ID: G00464664), Gartner.
• Gartner. (2020). Executive Guide to Privacy-Enhancing Computation [PowerPoint]. Gartner.
• Gartner. (2021, February 17). Select the Right Key Management as a Service to Mitigate Data Security and Privacy Risks in the Cloud (ID G00741005), Gartner.
• Guilbon, J. (2018, June 19). Introduction to Trusted Execution Environment: ARM's TrustZone. Retrieved from Quarkslab: Introduction to Trusted Execution Environment: ARM's TrustZone.
• Krikken, R. (2019, November 26). Achieving Data Security Through Privacy-Enhanced Computation Techniques (ID: G00384386), Gartner.
• Krikken, R., & Willemsen, B. (2021, January 12). Top Strategic Technology Trends for 2021: Privacy-Enhancing Computation (ID G00740641), Gartner.
• Lindell, Y. (2020, January 7). Trusted Execution Environments—A Primary Or Secondary Security Mechanism? Retrieved from Forbes: Trusted Execution Environments -- A Primary Or Secondary Security Mechanism?
• Lowans, B. (2020, July 24). Hype Cycle for Data Security, 2020 (ID: G00448204), Gartner.
• The Confidential Computing Consortium. (2020, July). Confidential Computing: Hardware-Based Trusted Execution for Applications and Data. Retrieved from The Confidential Computing Consortium: Confidential Computing: Hardware-Based Trusted Execution for Applications and Data
• The Confidential Computing Consortium. (2021, January). A Technical Analysis of Confidential Computing, v1.1. Retrieved from The Confidential Computing Consortium: A Technical Analysis of Confidential Computing
• The Confidential Computing Consortium. (2021). What is Confidential Computing? Retrieved from The Confidential Computing Consortium: Confidential Computing Consortium
• Woo, B., & Willemsen, B. (2020, July 23). Hype Cycle for Privacy, 2020 (ID: G00441605), Gartner.

PDF version (PDF, 254.2 KB)

Introduction

For this consultation, members of the Canadian public and international partners were invited to review and provide feedback on Statistics Canada's gender and sexual diversity statistical metadata standards^{Footnote 1}. Specifically, Statistics Canada was seeking feedback on proposed updates to the standard for gender of person and new standards for sexual orientation and LGBTQ2+^{Footnote 2} status. Statistical standards for gender and sexual diversity (such as the definition of each concept and the classification which establishes its categories) allow for the reporting of statistically diverse groups of the population in a consistent manner. This report summarizes the feedback received from the consultation. For more information on statistical standards as well as the additional engagement activities that took place to inform standards on gender of person, sexual orientation of person, and LGBTQ2+ status of person see Consulting Canadians landing page and StatCan Plus article.

Gender

Statistics Canada initially released new sex at birth and gender variables and classifications on April 13, 2018. Prior to the 2021 Census – in which the gender question was asked for the first time, and the 'at birth' precision was added to 'sex'– Statistics Canada reviewed the gender standard to ensure its relevance. Other engagement activities including a targeted expert consultation supplemented this public consultation in order to update the gender standard.

The updated sex at birth and gender standards were released on October 1, 2021. Among other changes, the definition of gender, the usage sections and the comparison to relevant internationally recognized standards were expanded. In addition, some category names and definitions in the classifications were updated.

Sexual orientation and LGBTQ2+ population

Statistics Canada has been collecting information about sexual orientation since 2003. The variable 'sexual orientation of person' used in the consultation included proposed classifications for the main components of sexual orientation - sexual identity, sexual attraction, and sexual behaviour - which could be measured separately.

The new sex at birth and gender standards have allowed for a more nuanced understanding of sexual orientation and the ability to collect data on the full LGBTQ2+ population. The creation of standards on sexual orientation and LGBTQ2+ status of person will establish a framework to address information gaps on sexual and gender diversity in Canada.

Consultation overview

The purpose of the consultation was to ask data producers and users; representatives of civil society organizations; government bodies at the federal, provincial and local levels; academics and researchers; and all other interested parties, including the general public, to submit feedback regarding the proposed updates to the standard for gender and the new standards for sexual orientation and LGBTQ2+ status.

The consultation was conducted electronically and publicized through public announcements that described the proposed updates to the standard for gender of person, and proposed new standards for sexual orientation of person and LGBTQ2+ status of person. The announcements also listed the types of inputs sought, provided a timeline for the consultation and gave contact information for interested parties to make submissions and contact Statistics Canada with questions and comments.

Announcements were disseminated through the Statistics Canada's website and social media. In addition, stakeholders and partners, including civil society organizations, as well as a number of researchers in the field of gender and sexual diversity and gender studies, were invited by email to participate and encouraged to share the consultation invitation with others within their network.

Interested parties were invited to submit written proposals to Statistics Canada. The official consultation period started on February 2, 2021 and closed on March 12, 2021. In addition to the public consultation, virtual meetings were organized with key stakeholders and researchers to gather their feedback.

Summary of submissions

Statistics Canada received 205 responses by email in both official languages from a range of individuals and organizations:

• 19 responses from academics or research groups;
• 31 responses from organizations, such as civil society organizations and government departments or agencies at the federal, provincial or territorial level in Canada and overseas;
• 155 responses from the general public.

The consultation also included a number of follow up discussions with academics and subject matter experts.

Statistics Canada is committed to respecting the privacy of consultation participants. All personal information created, held or collected by the Agency is protected by the Privacy Act. As such, the identity of organizations, individuals and academics who participated in the consultation process are kept confidential.

Summary of feedback on the updated gender standard

Definition – Gender

The consultation materials presented an updated definition of gender. In this update, gender was defined as "a person's social or personal identity as a man, woman or non-binary person (a person who is not exclusively male or female)." This definition included the following concepts:

• gender identity (felt gender), which is the gender that a person feels internally, and;
• gender expression (lived gender), which is the gender a person expresses publicly in their daily life, including at work, at home or in the broader community.

The proposed definition stated that a person's current gender may differ from the sex they were assigned at birth (male or female) and that a person's gender may change over time.

Some of the most consistent feedback received regarding the English version of the gender definition was related to the incongruent use of the biological terms 'male' and 'female'. Respondents also commented that the gender standard conflated sex and gender. To this end, a number of suggestions were received regarding terminology. These included suggestions for the use of 'male', 'female' (or 'intersex') when referring to the biological characteristics of sex, and 'men/boy', 'women/girl', 'transgender', 'cisgender' and 'non-binary' when referring to gender identities. Suggestions were received for the use of gender terminology in the non-binary definition, replacing 'male' or 'female' with 'man' or 'woman'.

Input was received providing suggested modifications to the definitions of gender identity and expression. Feedback was also received on the sex at birth variable which reflected differing perspectives. Some respondents suggested that more emphasis be put on sex assigned at birth, while others suggested that sex is not assigned at birth, but rather observed and reported, and recommended using other terminology.

Usage – Gender

The proposed usage section included the following explanation, among other content:

The variable 'Gender of person' and the 'Classification of gender' are expected to be used by default in most social statistics programs at Statistics Canada. The variable 'Sex of person' and the 'Classification of sex' are to be used in conjunction with the variable 'Gender of person' and the 'Classification of gender', where information on sex at birth is needed.

While comments specifically referencing the gender usage section were limited, some overarching feedback was received that expressed disagreement with the overall concept of gender identity and communicated concerns about self-identification into protected groups. Some respondents were not supportive of the introduction of the gender variable by default at Statistics Canada and argued that collecting data on gender, rather than sex, could disrupt the historical comparability of the data and result in a loss of information^{Footnote 3}.

Classification – gender

The proposed Classification of gender contained three categories: male gender; female gender; and non-binary gender. The 'non-binary' gender category of the classification is intended to capture relevant write-in responses to the gender question where a respondent indicates being neither exclusively 'man' nor 'woman'.

A few respondents suggested that the classification contain additional categories, such as a Two-Spirit category, with the recommendation that the response option only be available to Indigenous respondents when asked on surveys.

Feedback received regarding the English version of the classifications of gender and transgender status was similar to the comments mentioned above regarding the use of the biological terms 'male' and 'female' in the proposed definitions, with 'man' and 'woman' as suggested replacements.

Comments regarding the reference to 'current gender' (e.g., "This category includes persons whose current gender was reported as male") were received, which suggested the removal of the term 'current'. Similar comments were made regarding the use of the word 'current' in the Classification of transgender status.

Classification – transgender status

Consultations sought input on a classification consisting of the following two broader categories with their respective subcategories (along with their definitions, not presented here):

• 1. Cisgender person
  • 1.1 Cisgender man
  • 1.2 Cisgender woman
• 2. Transgender person
  • 2.1 Transgender man
  • 2.2 Transgender woman
  • 2.3 Transgender non-binary person

Respondents suggested the creation of a third, standalone category, 'Non-binary person', rather than being included as a sub-category of 'Transgender person'.

Respondents also provided feedback regarding the terminology. A few commented that the terms 'trans' and 'transgender' are not necessarily interchangeable, while others suggested replacing the term 'cisgender' with 'non-transgender'. A few respondents suggested using the term 'gender modality' as the name of the classification; for example, the Classification of transgender status could be called the Classification of gender modality.

Summary of feedback on sexual orientation

Definition – sexual orientation

In the proposed standard, sexual orientation was presented as a multidimensional concept defined as an umbrella term that includes a person's sexual identity, sexual attraction and sexual behaviour. Sexual identity refers to how a person perceives their sexuality (e.g., lesbian, straight, bisexual), sexual attraction refers to whom a person finds sexually appealing, and sexual behaviour refers to with whom a person engages in sexual activity. A person's sexual orientation may change over time.

Input was supportive of sexual orientation being a multidimensional concept. Some minor changes were suggested for how to define sexual orientation as well as its different components. Feedback was received in favour of using the term 'sexual orientation' rather than 'sexual identity'. It was also suggested that the definition of sexual orientation should include the concept of emotional attraction.

Usage – sexual orientation

Feedback regarding usage mainly consisted of the need for transparency around the rationale for collecting data on sexual orientation, ensuring data are only collected as needed. While the consultation did not specifically focus on the minimum age for responding to sexual orientation question, a few organizations and academics provided input on the proposed minimum age of 15. They noted the value of having data on youth who are LGB+ (lesbian, gay, bisexual or of a sexual orientation other than heterosexual), and felt that a rationale should be provided for requiring a minimum age for asking questions about sexual orientation. Similarly, it was suggested that the minimum age to collect data on sexual orientation should be lower than 15 or that a minimum age may not be needed at all.

Classification - sexual identity

Consultations sought input on the Classification of sexual identity which included proposed categories along with their definitions. The classification included: 'heterosexual or straight'; 'gay or lesbian'; 'bisexual'; 'pansexual'; 'asexual'; 'queer'; and 'Two-Spirit'. Some respondents suggested including more categories, while others thought the classification should include fewer categories.

It was also pointed out that some of the proposed categories were not mutually exclusive and that this should be addressed (e.g., a person could be both Two-Spirit and bisexual, or asexual and gay, or queer and lesbian). In addition to not being a mutually exclusive category, 'queer' saw some support, but some respondents also suggested to avoid this term loaded with political history and potential derogatory interpretation. Input was also received that the 'Two-Spirit' category was a distinct concept requiring a separate measure only made available to Indigenous respondents.

In addition, feedback was received suggesting that the proposed definitions of different sexual orientations conflated sex and gender by referring to attraction based on sex and/or gender. Some respondents felt that the definition of sexual orientation should solely be based on sex. Other input suggested that sexual orientation definitions include being attracted to a person's gender expression, along with their sex and gender.

Feedback from different types of respondents (i.e., individuals, academics, and organizations) recommended combining the 'bisexual' and 'pansexual' categories, as these terms may overlap and be used interchangeably, making the two categories not mutually exclusive. It was pointed out in some comments that responses may be influenced by whether a person conceptualizes sex/gender as binary or not.

The proposed Classification of sexual identity also included higher levels of aggregation, including category groupings 'heterosexual or straight' and 'minority sexual identity'. Some of the most consistent feedback was that the 'minority sexual identity' category carried a negative connotation and that it was inappropriate. Other feedback suggested that different sexual identities should not be aggregated together.

Summary of feedback on sexual attraction

Classification - sexual attraction

The proposed Classification of sexual attraction was presented in two versions, each including a number of categories for respondents to identify their sexual attraction. One version measured attraction in reference to the respondent's own gender, without specifying the gender or genders of persons that they are attracted to (e.g., 'person only attracted to person of a different gender'). The other version specified the gender or genders of persons to which the respondent is attracted to (e.g., 'person only attracted to persons of male gender'). Each version also included categories for people who are 'equally attracted' to more than one gender, as well as for people who do not experience sexual attraction or who are unsure of their sexual attraction.

While feedback on which version was preferable was very limited, one of the key issues identified in responses was that both versions included too much detail or that they were too complicated. Others argued that sexual attraction should be defined on the basis of sex rather than gender. While both versions included a category for people who do not experience sexual attraction, some comments suggested that the classification should be more inclusive of people with little or no sexual attraction (i.e., people on the asexual spectrum). It was also suggested to re-name the 'unsure' category to 'questioning'.

Summary of feedback on sexual behaviour

Classification - sexual behaviour

Overall, this classification elicited stronger reactions than the other classifications. Some feedback indicated understanding the need to refer to the concept of sex rather than gender in the context of sexual behaviour. However, many comments expressed surprise or confusion that sex rather than gender terminology was used in the proposed sexual behaviour classification, which differed from the proposed sexual identity and sexual attraction classifications which used gender terminology. Some suggested that the purpose of the Classification of sexual behaviour was unclear, and proposed that the classification provide some base standard definitions of 'sexual activity'. Other input suggested to shift the focus away from the sex of the sexual partners towards specific acts.

A significant amount of feedback from different sources (i.e., organizations, individuals and academics) noted that intersex people were only included as partners in the Classification of sexual behaviour and that there was not a specific category for intersex respondents. Some input also indicated that no definition of intersex was provided.

Some respondents suggested that the number of sexual partners should be included within the sexual behaviour dimension. Finally, it was recommended not to refer to 'men who have sex with men' in the classification as the term may have a negative connotation to some people.

Summary of feedback on LGBTQ2+ status

Definition – LGBTQ2+

Statistics Canada is committed to supporting disaggregated data analysis in order to highlight the experiences of specific segments of the population. Recognizing that sample size may be an issue for small populations, the consultation proposed an aggregate LGBTQ2+ standard to establish a consistent approach to combining data on gender identity and sexual orientation. Input was sought on the proposed definition of LGBTQ2+ status as well as the choice of acronym. The proposed definition was that "LGBTQ2+ status refers to whether or not a person is lesbian, gay, bisexual, transgender, queer, Two-Spirit, or another non-binary gender or minority sexual identity." Feedback received focused on the acronym rather than the proposed definition. The majority of feedback proposed to move the '2' referring to Two-Spirit people at the beginning of the acronym to acknowledge Indigenous people in the context of reconciliation.

Classification – LGBTQ2+

Feedback was sought on the proposed Classification of LGBTQ2+  status as two distinct categories (i.e., 'LGBTQ2+ person' and 'non-LGBTQ2+ person' ('heterosexual and cisgender person') as well as their definitions. Some feedback argued against aggregating diverse populations under one umbrella category as these groups have different experiences and are not homogenous in their characteristics. However, others indicated that this approach was a useful way to analyze complex issues experienced by the LGBTQ2+ population as a whole.

Next steps

Statistics Canada has completed the review process for the updated gender standard and the new sexual orientation standard. The updated gender standard was released on October 1, 2021. All of the comments received during this consultation and other engagements activities were taken into account, and many are reflected in this updated standard.

The new sexual orientation standard was released on August 16, 2023. The public consultation summarized in this What We Heard report was one of four phases that informed the development of the sexual orientation standard. In addition to the public consultation, Statistics Canada undertook a targeted expert consultation, focus groups, and a testing phase which consisted of one-on-one interviews.

The focus groups and testing were conducted in English and French and engaged diverse participants from urban and rural communities in different regions across the country. Participants included LGBTQ2+ and non-LGBTQ2+ individuals from a range of ages, genders and socio-economic status groups. Focus groups and testing also engaged Indigenous Two-Spirit participants, as well as immigrant and racialized participants.