The following are the outputs of the captioning taken during an IGF virtual intervention. Although it is largely accurate, in some cases it may be incomplete or inaccurate due to inaudible passages or transcription errors. It is posted as an aid, but should not be treated as an authoritative record.
***
>> MOLLY LESHER: Why don't we go ahead and get started today. I'd just like to welcome you all. Thank you so much for coming, participants and speakers. Some of whom are either physically in Poland or online from across the world. I have been trying to have this workshop for two years. It is an important topic and it is great to have you all here with me.
For those of you that don't know me my name is Molly Lesher. I'm an economist at the Organization for Economic Cooperation and Development in Paris. And it really is a great pleasure to be moderating this session and measuring and shaping an inclusive Digital Economy in society. Understanding the impacts of digital technologies on people's overall well‑being is for me one of the most important issues of our time. And while really difficult to assess it is something that we need to get right. We make sure that nobody is left behind as digital transformation progresses. And this matters not just for people around the world but also for social cohesion, something that I think we have all seen sorely tested.
The purpose of this workshop is to brainstorm together on the key priority issues, the international statistics that communities should focus on. In this context it would be great to hear from speakers and participants in the chat your views on a digital perspective is the single most important issue we face today in terms of making our economies and societies more inclusive. If you don't mind taking a quick minute to put in the chat a word or two of something that you think is important and that we should all be focusing on.
It is kind of important in this virtual context that we do have your input. Because we really want to make sure that we're looking at the right things.
So I have got here disinformation from me. That's something I have been working on and I think is enormously important. We have the digital gender divide which is also something that's going up high on the agenda in some OECD countries. Access to affordable broadband. Measurement frameworks. Media competence and sovereignty in using digital technologies.
Digital divides, AI measurement, that's an extremely difficult area partly because we have to define what we mean by AI. Digital skills that's another great one. That's an area where we see a lot of gaps in particular with respect to gender on the toolkit.
So the Egypt country program and I hope we get to hear about that a little bit later.
So that's really great. I think that highlights a lot of issues that we've seen to be important in our work so far but also kind of shows us things that we do need to make sure that we cover going forward.
I'm kind of a nerd here and I strongly believe that without proper measurement and modeling analysis of public policy issues comes down to people's individual opinions. And however smart and well informed they are I'm not sure that's the way forward. That's one reason why I felt really strongly about creating the online and interactive going digital toolkit back in 2019.
It includes a core set of indicators that maps, that really helps countries assess their state of digital development, and data exploration and visualization are really key features of this toolkit. And policy advice is also available on the toolkit to help countries improve their performance. So if you want to go check it out here while I'm talking I would be super happy if you did that. It is www.oecd.org/toolkit or you can Google OECD going digital toolkit and it pop up. Take a minute to explore some of the indicators.
Now really importantly the toolkit is structured along the lines of the OECD going digital policy framework. With the aim of what we are talking about today, to make digital transformation work not just for growth but also for well‑being. And I want to highlight this isn't something that I came up with on my own or a couple of people came up with at the OECD. It took two and a half years to develop with people from every policy community represented at the OECD. That includes everyone apart from national defense.
So this is a really validated, a really well researched framework. And we use it to map indicators along seven different dimensions. And these dimensions include access, to data communications and structure and services. We saw in the chat that access to affordable broadband is something that's really important. Effective use of digital technologies and data by people from some Governments. This is really the idea that we have all these technologies but people may not be using them effectively. They may not have the skills to something we also saw in the chat.
Innovation which really pushes out the frontier of what's possible in the Digital Age. And this is something that's important in driving job creation, productivity and sustainable growth. And digital sectors are we've shown to be very innovative. Jobs, which highlights labor markets evolve and enroll that's normal. We must ensure that digital transformation leads to more and better jobs. So we need to be able to skill our kids to give them the tools they need and the skills they need to succeed. And we need to help adults who are already in the labor market transition with new skills and prospects. Affect society in complex and interrelated ways. All stakeholders must work together. Digital technologies are in and of themselves neutral but they do have risks and benefits. And we need to find a way to balance them for people and for society as a whole.
Trust in digital environments is the sixth policy dimension because without trust nobody will use digital technologies. I often have colleagues who talk, you know, about how important infrastructure is. And they're absolutely right but it is also the case that if people don't trust technology, if they don't trust their data is going to be protected, people are not going to use these tools. And a really important source of productivity growth is going to be left unexploited.
And then the last dimension is market openness. And this is really the enabling environment for the digital transformation to flourish. It is about competition policy, trade policy, tax policy. All things that have a very important digital dimension. Now throughout the framework we really insist on the fact that these dimensions need to be considered jointly. They need to be coordinated within Government and they need to be measured to help countries see where they stand.
We also have cross‑cutting themes on the toolkit that run through these dimensions as well as development. We did a project in the last round of improvements to map the indicators to the UN Sustainable Development Goals. We also have a theme on well‑being more broadly. So please do take a moment to check out the toolkit. All this said, I feel like we have made an immense amount of progress but at the same time measuring digital transformation is in many cases still unchartered territory. It is really, really tough.
The important part of this workshop for me is to brainstorm together the ways we are going to enrich the indicators on the toolkit going forward. And particularly I think with respect to trust, especially in the areas of privacy and security as well as societal inclusion. And here from me for thinking about mis and disinformation is important because it really is a huge, huge issue. We are just beginning to scratch the surface of and we have seen can cause an immense amount of social instability.
So without further ado, that was a little bit of an intro for me on the toolkit. It is a free resource. Please check it out and tell me how we can make it better. Will always be a work‑in‑progress, but I would like to warmly welcome Dominik Rozkrut, the President of Statistics Poland to give us some keynote remarks on Poland's efforts to measure the inclusive aspects of digital transformation as well his views on one or two related areas that he thinks the OECD should focus on going forward.
>> DOMINIK ROZKRUT: Thank you very much. It is an honor for me to be here today. Of course, it is a pity that we cannot see each other in person. No matter what we think about those digital tools that allowed us to have those meetings these days, it wouldn't be better than just the real thing. That's for sure.
So no matter how we progress in terms of ICTs we always need these interactions. Let's start with wishing ourselves that we can come back to what we had before.
And I'll try to be very brief. First about the measuring of, you know, digital transformation that's called this way. In Poland we have been a part of the European statistical system for a long time, starting in 2004. And we also convey the same surveys mostly which are harmonized at a European level. These are also discussed extensively at the ages of OECD. I remember being part of the group that Mark is chairing. Always fantastic to be there on the meetings. Because we discussed things when they are in to surveys both in outside Europe and inside Europe and been doing these ICT surveys for many years already. And they prove to be very useful. The same for the ITU meetings, the UN commenced in Geneva when they had a chance to meet Alexander, we have been discussing these issues as well.
So let's be direct in this regard, the ICT survey both in households and enterprises are to be a very useful tool. I took advantage of those on a daily basis, very often both being a statistician for using these numbers and planning a thing like census. That's not the complete picture of what we are doing here at the Statistics Poland.
We also have something that's less commonly known but also provides us with some interesting measurements which is called the social cohesion survey. And this is outside of the sphere of harmonization in the European statistical system. This is our own national survey. It is very wide in terms of topics covered and it covers things like well‑being mostly. But also trust. But also ICT usage at a level of the individuals and households as well go through a different perspective than at the ICT survey. We're very proud of this. It happens every few years, like four or five years. But it is very extensive and it is one of the very few examples I guess, if not the only one, at the global level in which we are trying to come up with subjective well‑being measures. So that's really useful because, you know, of course, when you have some way of measuring some important area of living, you know, and then you can break it down to ICT breakdowns, ICT related breakdowns all these things that we are interested in, in this realm here, that really enriched your understanding of what the problems might be within society and how to address them.
Of course, you know, indirectly this measuring of digital transformation through other tools, like households budget surveys asks questions how much do you spend online. Linking this data with the other side, with the ICT surveys is always interesting and enriched our understanding of many issues.
I would like to tell you about my experience because I'm ‑‑ I've called myself for the last two months a census survivor because we did a census. And we did it during the pandemic. And we did a very hard decision to make it through in the pandemic, not to postpone it to the next year or two years. I fully believed that, you know, if not during the pandemic, then when do we need such a detailed data about our society than not during the pandemic. That's why we decided to do it. It was connected with other issues, organizational issues.
And, of course, one of the challenges was how to do an online form so that we limit the interactions in between the interviewers and the respondents. And we are not one of those countries that does everything with a click of a button, taking advantage of administrative sources. We cannot do that. So we do the mixed method. Of course, I told you about the ICT survey. My point of start was to analyze ICT data and to do some kind of like simple modeling based on those data in order to come up with a prediction of the percentage of society that would participate online. That was extremely important at the beginning because we have very strict financial planning procedures. And we have taken advantage at least me myself, I took a clue of how can we ‑‑ what can we expect. I came up with a number about 40%. Which is not kind of like a groundbreaking I would say. But when you know a little bit more about our society, you would note it, it is a lot already. So we face a lot of challenges still in terms of ICT skills and all these issues.
But I can tell you there is no better survey than the census because it is not a survey. We did it. Turned out 55% of people participated online. But it also led me to some interesting remarks and conclusions on how to shape, for example, policies addressing the issue of ICT or digital exclusion. It is not that we were kind of like reactive and we were waiting for people to come because that would never work. We would get 10%. We are proactive in terms of communication but only this. We have been physically reaching out to people trying to help them to connect to do the census online, even if it meant sometimes for us more, that we have been just investing through a regular telephone interview. But we took this opportunity to try as hard as possible to convince people to using those new forms of participation in a public living, social living. It was very telling to me. And it gives me a lot of clues in terms of policy making addressing these exclusion issues should go.
So in regard to the OECD toolkit, I like it very much. I like those seven areas, especially innovation is my favorite one.
And it is a very good tool. As always whatever comes from the OECD I always praise that. So I'm maybe biassed a little bit. It is a really good tool and informative. It is visualized in a way that you don't spend time guessing what it is. You right away know what it should be and you can take advantage of making comparisons to a country, any other countries. But, of course, the thing is as with everything that our society, our reality is progressing. Everything is changing and the new challenges are coming as well. And things keep evolving especially at the political level, that level of political initiative. So I can give you an example of the European policies. We had an EU data strategy and we had an open data directive. Part of that is nowadays the initiative to introduce something that is called high value datasets. There are new regulations coming, Eu Data Act and Data Governance Act which are to establish something that's called EU data spaces.
And those data spaces are kind of like political initiative to facilitate the exchange of data at the level of speed and openness never faced within our societies in order to, of course, support innovation and so on and so forth. Those innovative initiatives they are at a very high level of abstraction and, of course, those need also some kind of measurement frameworks forward to assess their successfulness. But at the end of the day, you know, those are addressing the needs of society I would say. And at the end of the day, of course, it comes back to the very single, you know, part, individual, part of a society.
And we need some ways of measuring those, but I would tell you that they won't be successful until we also pair them with some measuring of what is the progress of the individual level and introduce some policies in parallel that will invest in to the capacity of the individual people in terms of like building the frameworks, you know, skills frameworks for some dedicated areas, like in the financial sector or other sectors. There are so many of those areas, the digital skills are actually different even though we can call on digital skills they must be completely different.
So the devil is in the details as they always say. There is a lot of work to be done going in to these details and just exploring the new possibilities. Technically from the point of view of those who are responsible for measurement, like office of ‑‑ office of statistics, that the very important issue now at the moment is access to new data sources, including especially to privately held data which will allow us to go more in‑depth in the measurement systems. And this is an issue we are working hard on in the European Union in terms of introducing some legislation that will allow us to access those privately held data sources.
If I would be able to conclude my short intervention, I would think those very high level initiatives are extremely important because they put us on another level of development in terms of digitalization, but at the end of the day we need to also not to forget about those who are still excluded.
Those who are living in the rural areas, who are as I say far from the road and whatever ‑‑ whatever, you know, realm, whatever meaning of this. And because there is still a huge potential in the people, in our societies, which is being not used because we face this issue of exclusion. So it is important to both go from bottom down, but topdown and bottom‑up as well in this area. Thank you very much.
>> MOLLY LESHER: Thanks a lot, Dominik. That was really interesting. I have never thought about getting insights from the census. This is something I'm excited to explore. You mentioned the idea of digital skills. And I agree with you, that's amazingly important. It is probably the biggest area I see we need to work on at least across OECD members. But not just on what I think of as digital skills which would be ICT skills but also foundational skills, right? Reading, writing, arithmetic has been shown to be really important as well as those soft skills, the people skills and problem solving skills. Those have all come to the floor as being really needed, especially even more now in this highly digital society.
I ‑‑ it is also interesting to hear you mention the Government access to private sector data and the fact that the EU is working on that. We are also working on some principles to try to guide that at the OECD. But it is a really, really hard politically sensitive issue that's still kind of being looked at by numbers.
So I talked a lot and I have other questions. But I would be happy if other people would like to ask Dominik something, either by raising their hand and identifying themselves or in the chat.
I don't want it to be just a passive session. We are in a brainstorming session doing it the best we can virtually. So if there are any comments or questions to Dominik, please, please raise them.
And maybe, maybe when ‑‑ while we are waiting for that, one thing I sort of wanted to ask you about, you mentioned you are part of some of these international bodies, like the European Statistical Committee. At the OECD we have the going digital measurement roadmap which was an important output of a big horizontal project we had. And the idea with that was that we would get all of the policy communities at the OECD which are very different to agree on nine core actions that everyone was going to try to push forward on the measurement agenda. And I view that as a big achievement because otherwise at least in our context every Committee was doing their little thing but it wasn't joined up in a good way.
And I was wondering from your perspective being part of these international bodies, do you see any moves towards joining forces with other countries on this inclusive or well‑being digital measurement agenda? And if you ask, what direction do you see it taking?
>> DOMINIK ROZKRUT: That's a very interesting question. My very brief opinion about it is that we have a lot of international cooperation in regard of let's start with the access to privately held data. I have been kind of like chairing one of the task teams within the European statistical system. We have a parallel team on the United Nations School Commission in New York as well as in the UN ECE in Geneva as well working on this issue.
So the other important topic that we are currently exploring like heavily is the issue of data stewardship, which is something that comes to a situation now as an important factor because, you know, if we see those data ecosystem growing rapidly, we need some people in the middle that would catalyze the exchange of data, facilitate the exchange of data, knowing what the data is about. So we see those new kind of like professions even coming. And we also observe that there is no framework for skills for data stored within different communities.
To conclude this, my opinion is that we are rushing now in to many directions just exploring these new things. But I think that at the end of this period of exploration and coming up with new ideas and directions there will be a lot of work to be done in order to kind of like fill it in with some ‑‑ with some matter I would say.
So when I think about it, if we had those European data spaces and we think how to organize them and we as official statisticians want to put some words in the European spaces, that's okay. That's a good direction. But then at the end of the day we will also need some measures indicators that will facilitate to monitor the progress of those strategies. And I'm really thrilled about it. I don't see too much of the discussion about the measurements frameworks in these new areas. But I think that's coming.
And I can tell you that we should fill in our agendas for the next two years with those new topics. But taking them more seriously and in‑depth.
>> MOLLY LESHER: Thanks so much. We have got one more question for you before we go to the panel from Fabio from Brazil, how to ensure international comparability of data and context of data producers are making use of local strategies and innovative methods. I don't know Fabio, if you want to elaborate on that.
>> DOMINIK ROZKRUT: That's my favorite question. Again we have a privilege here in Europe, that we have this European statistical system and we have like a very well‑established system of harmonization of our statistics. And we work nowadays virtually which is not that effective, but anyway, we work on harmonization of our statistics. And we do a lot of progress in this regard. And this facilitates this data comparability. In terms of international, it is a big challenge. Because we are progressing very quickly, independently even though we discuss big ideas on the premises of United Nations. For example, it doesn't necessarily mean that we come up with those ‑‑ with the same operationalization of those ideas in different regards.
And I think that, you know, the more coordination between the international organizations, the better for us all. International organizations I'm pointing at OECD. Should take this responsibility to facilitate this harmonization process.
>> MOLLY LESHER: Thanks so much. And I agree, that's an important, really important point. And I always get so frustrated when people that aren't involved in collecting, cleaning data, ask for breakdowns that we can't provide and why we can't compare this country to that. But we really have to be careful about that and having some way to ensure comparability through model surveys, for example, is really important. So thank you so much for your remarks. And definitely stay here because we are coming back to you later.
Let's start the panel discussion. I would like to invite Nagwa El Shenawy, undersecretary for information and decision support in the Egyptian Ministry of Communication and Information Technology to take the floor and give us an overview of Egypt's projects and one or two specific areas that you want us to elaborate the toolkit. Give me constructive feedback on the toolkit. I'm not going to feel bad. I'm welcome and be open to it. The floor is yours.
>> NAGWA EL SHENAWY: Thank you. And thanks for the IGF for this opportunity. Actually there is fruitful cooperation and partnership between Egypt and OECD back in to several years with science, technology and innovation directorate and all its different Committees. And based on the digital, ongoing digital integrated framework developed by OECD as well as the survey models of the household and private business and other surveys, also, like, for example, the surveys of the ITU for the household, and the digital skills, based on all of this Egypt really succeeded to measure the ICT at large during the last years. And made a great effort, especially in measuring the digital transformation, especially in terms of access and use.
And really we succeeded to cover an important dimension in order to measure if our digital transformation are inclusive or not, such as the gender perspective, the rural and urban perspective. Only this measure really were helped in supporting our policymakers in measuring the digital transformation, implementation of the strategy as well as to follow up the policy related to the digital transformation.
And they were able to see where the country stayed, especially that actually Egypt started the digital transformation implementation plan back in to five years ago. So this really, this framework was very useful as well as we followed up with a digital ‑‑ the toolkit developed by the OECD.
And also the progress made in the toolkit to measure the different areas of the digital transformation. We also were able to provide the policymakers with the measurement to see the challenge that's still Egypt facing in terms of reaching accessibility and use of ICT all over the country, among the different Governments.
Also we succeeded by this measurement as well to propose sometimes, I really ‑‑ I ‑‑ I was proud to tell this, that I contributed in proposing some policies and initiatives, for example, related to the gender. And how we can use the ICT in order to empower the gender in our country.
So this is in a nutshell a brief about the progress done in Egypt. But in addition to this, I would like also to mention that since the year 2019, the Government of Egypt and the OECD Secretariat have been negotiating an Egypt OECD country program. And this is to support the structure reform taking place in the country. And several consultations were carried at the technical level as well as the political level to identify the scope of this program.
Actually Egypt has recently launched the second phase of the national program of the ‑‑ for social and economic reform. And based on the first phase, we focus in the second phase on three main areas, the manufacturing, the ICT, and the agriculture in order really to achieve a more diversified productive economy. So the country program was already launched successfully in last October. And an agreement was signed in Paris in the OECD headquarters in the presence of the Prime Minister of Egypt and the Secretary‑General of the OECD.
We expect from this country program that the OECD can help us to really in the implementation in the different type of reform that Egypt is looking for. It will help us to serve as a guidance for the implementation of the recently revised, the sustainable development strategy of Egypt and the 2030 Agenda of the country.
The program will focus on five pillars, inclusive and sustainable economic growth. Second innovation and digital transformation, governance anti‑corruption and statistics and sustainable development.
The Ministry of Communication we have two projects. And this is what we are looking for for the toolkit. The first one is related to the Artificial Intelligence. So we are looking for a ‑‑ some sort of comparative study in the Artificial Intelligence in order to benefit from the OECD experience, lessons learned and also to measure where we stand, the gaps in Egypt in this important area.
The second which is mainly I was responsible to draft this project in cooperation with Monly which related to unleashing the innovation. This is an important area for Egypt to measure. The progress is still not up to the level. Because I think of different things, because it's maybe the measurement itself the tool, the methodology. So we are looking forward from the OECD to benefit in such area and to build the capacity of our country in this area to be able to measure it through this project.
This project is expected to be implemented during the upcoming three years. And it will be with the SDI directory. And finally, I want to take this opportunity to thank Molly again for her effort to support us in this country program. And especially for me in the project, in the upcoming project for measuring the innovation. Thank you very much.
>> MOLLY LESHER: Thanks so much. It is really neat to thank you. I'm super excited about the project. Whenever we engage with partner countries I find out I end up learning a lot. I'm looking forward to that. You have brought up a lot of important points. One thing is that we are measuring with the idea that we're trying to implement reforms. And so thinking about that link between the measurement and the policy and reform agendas, I think it is absolutely important. You mentioned also some of the digital divides that you are looking at in terms of infrastructure as well as gender. We also see quite some digital divides with respect to firm size, at least across the OECD with small firms often having a hard time being benefits of digital transformation as much as larger firms as well as on the skill side.
Before I let other people ask you questions I'm going to abuse my role as a Moderator. As a senior role in the Egyptian, which I think is likewise the same and I wonder in Egypt when you said you looked in to some of the gender gaps, what came ‑‑ what came out of that mostly? What was the big take‑away from you from the gender data that you collected on digital? I'm curious if it follows what we see in OECD countries which is really it is the skills, where we see the biggest gender gap rather than, for example, connectivity.
>> NAGWA EL SHENAWY: Thank you, Molly, for your question. Actually it is ‑‑ the measurement and really following up the situation of the household and the individual accessing to the ICT tools and use the tools to benefit from them in all aspects of life. One of the criteria and one of the important dimensions was to see this among the gender, not only in the capital, but also in the other Government. Especially this helped me a lot to see the picture in the rural Government and in the remote area, which we found that in terms of accessibility, the accessibility percentage was low as well as the use as well was low. And from this indicator and measurement as I mentioned, I really developed this initiative for the gender, which was called ICT for women and girls in Egypt.
And this initiative, it actually for helping the women and the girls all over the country to access and use the ICT in the different Governments. And this is ‑‑ there is a platform in order to help them to build their skills in the ICT, to benefit from different ICT skills and tools, which is already offered in collaboration with the other partner, multi‑nationals, other ICT companies. This is from my side.
Also this platform includes the different measurements that we collect among the gender for to help the policymaker as I mentioned to see as well the progress. Because the policies taken in the digital transformation plan should affect as well the women like the men.
Also this initiative helped us also to go but this is before the COVID‑19 or before the pandemic. We went in and held different events all over the Governments in my country. Especially I started by the upper Egypt and I started by the rural area to reach the women entrepreneurs, the women who own small business as well as the students in the Universities in this Government. And really I succeeded to help for them Hack‑A‑Thons to use the ICT. I succeeded to hold for them capacity building program in cooperation as I mentioned with different private sector companies as well as in cooperation with other partners from international organizations.
It was a really successful process, and we ‑‑ it helped us a lot because we also ‑‑ we got inputs from them how to sustain this initiative and how also to expand it in the future.
And we heard from us, from them as well the benefits and the outcome of this initiative and what they're looking for for the future from this initiative. So this is in a nutshell what I can say about this initiative and what we already done 'til right now.
>> MOLLY LESHER: Congratulations on that. It sounds exciting and a huge amount of progress. You should feel immensely proud of that. I'm looking forward to seeing the platform. We have a question for you from Leonardo. Can you tell us about the digital maturity of SMEs in Egypt? Has Egypt already implemented any indicator on AI and how was the result?
>> NAGWA EL SHENAWY: Egypt succeeded to have in place an AI strategy which started to be implemented which is focused on building the capacity for ‑‑ in the AI prospective. And this includes the SMEs building. It includes the students in universities, the graduates. There is now a different certificate as well as the degrees in the Artificial Intelligence provided by our Egyptian University in collaboration with international Universities as well. And part of this capacity building is also devoted to the SMEs in order to really to leverage their skills of AI development in the future.
This is briefly what already we have done right now concerning this in Egypt. Thank you.
>> MOLLY LESHER: Thanks so much again. That was all really interesting. Now I would like to turn over to Mark Uhrbach who is the head of Digital Economy Metrics and Statistics Canada to take the floor and give an overview of Canada's efforts which I know is many as well as one or two specific and actual areas in which you really want to see us elaborate the toolkit. So Mark, the floor is yours.
>> MARK UHRBACH: Great. Thank you very much. Thank you so much for having me today to participate in this panel with such well regarded colleagues. It is always such a pleasure to share space with Alexander, Dominik, yourself and Nagwa even if we are still in a virtual space. It is no secret that both myself and Statistics Canada are big fans of the digital toolkit and made use of it as an excellent tool.
So I would like to take a few minutes here to discuss Statistics Canada's effort to measure digitalization and well‑being which is covered as a broad theme within the going digital toolkit. This study of intersection of Internet use and individual well‑being has been of interest to the agency for many years. But was accelerated about two years ago with work that our agency took on as part of an in‑depth review to meet the needs of the Bureau of the Conference of European statisticians work plan which Dominik is part of, of course. The in‑depth review identifies key issues that need to be considered when defining and measuring well‑being in the Digital Age, focusing on emerging challenges.
A summary of this work is highlighted in the OECD's going digital toolkit note on the topic of digitalization and well‑being.
So to better measure well‑being in the Digital Age we have taken some specific steps, particularly taking advantage of the existing survey of household ICT adoption and use, the Canadian Internet use survey which runs on a biennial basis. First, in 2018 the survey collected information for the first time from individuals on problematic Internet use. Looking at issues of victimization online, and also the need to take a break from the Internet or decrease the time being spent online. These help to demonstrate some of the personal challenges of being online. I highlighted some potentially negative effects. We always look at the positive effects of being online in the past.
The ability to break the data down by sociodemographic characteristic was also a valuable aspect of this data collection tool. In the 2020 version of the survey, additional content was added in response to the work that was done for the in‑depth review. In line with the suggestions made in that document, the ICT household adoption and use questionnaire was expanded in conjunction with well‑being experts at Statistics Canada to include further questioning on the self‑perceived quality of relationships as well as the frequency of connection with family and friends in a virtual or physical setting.
A new question was also asked on general self‑perceived physical health. They were designed based on the OECD Guidelines on measuring subjective well‑being that's been followed in other Statistics Canada work.
So although these questions standing on their own are not as useful, this additional information asked for the first time in Canada on an individual ICT use survey, provides some very interesting opportunities for further data disaggregation and research. It is expected that by cocollecting this information on an ICT survey, as opposed to collecting it on two separate surveys, correlational analysis based on usage of technology and self‑reported states of well‑being can be undertaken with a large and robust sample. There is a research time at Statistics Canada analyzing these results. Additionally, new data collection will go ahead in fall 2022 on the same survey and collect similar indicators with the idea of building out a time series for this type of data which will be particularly interesting in the context of the pandemic and the amount of time that people are spending online.
Of course, any information of this type is very challenging to collect in a survey format give than data are collected on a relatively infrequent basis. And it may be difficult to assess how they feel over such a prolonged period of time.
To address this challenge and then build on the survey work that's been done, innovative and new data collection methods must be considered and a mobile app has been recently launched by Statistics Canada to address this. The app will enable collection of in the moment quantitative data on the activities and well‑being of participants shedding light on effect. Participants will be prompted by the app to respond to questions several times each day over a 30‑day period that they are selected to be in the sample. Participants will also be able to adjust the frequency of notifications that they receive. We're still in the final stages of developing the sampling strategy, but a pilot study will soon be going forward with a sample of 50,000 randomly selected individuals. The app was designed at the same rigorous standards as all of our collection tools at Statistics Canada and had confidentiality and security at top of mind in its design since we anticipated this to be one of the concerns.
We're anxious to see how we can use this tool to enhance understanding of subjective well‑being in Canada and also explore how to better use it to capture information on the intersection of Digital Economy and well‑being.
So I think there are two main take‑aways from the work that's been done in this area. The first is that the collection methods for this data, emerging data need will need to continue to evolve since traditional surveys alone may not capture all of the information required for a full picture of the topic. To this end I think that national statistics organizations through organizations such as the OECD and the ITU can collaborate on their methods and share best practices to help close this important data gap.
The second of these key take‑aways from the work is that any indicators will be much more powerful if like many of the other indicators that we have put together related to the Digital Economy they can be standardized and collected in a coherent and internationally comparable manner, common methodologies, concepts, definitions and indicators should be agreed upon and shared.
And I think to this end the going digital toolkit is well positioned to provide great value not only as a place for statistics to be housed but also to expand on methodological guidance and best practices for the collection of these types of statistics. I would suggest the toolkit be used in this area, but also as a Forum to highlight research that's done in the area of digitalization. So I will leave it there. But thank you again for the opportunity to share today.
>> MOLLY LESHER: Thanks so much, Mark. You and your colleagues are always doing so many innovative things in this space. It is great to hear about that. And I didn't realize you were going to be using an app to collect new data and I'm super looking forward to that. I think that's really interesting. I couldn't agree more that we are going to have to think about ways to collect, innovative ways to collect data that surveys won't do it, I think is so many of these issues, one we maybe can't get the data from surveys. And two, it is just moving so fast. I was just wondering before I let other people come in with their own questions, you mentioned collecting data about time spent on using different tools. And I wondered if you were using that or collecting that data with a view to try and to get to the Internet addiction. And if so, if you had any ‑‑ if you had done any work or thought about that in terms of how much time we would consider to be addiction and how much is normal. As a mother of three kids, I think about this all the time. And just sort of wondered if you all had thought about this.
>> MARK UHRBACH: Sure. Yes. That's always a concern in my household as well. This has been one of the real challenges of using the traditional survey approach is trying to get at sort of these ideas how much time people are spending on the Internet. It was a much easier question to ask people 10 or 15 years ago than it is today. It is hard to define when we are not on the Internet. So that's one of the real challenges that we've found with using the survey approach.
We do still try to attempt to ask people in terms of hours per week that they dedicate to Internet activity. And we will sort of break that up in to buckets of intensity of use. Certainly we ask people to exclude work activity and still will have a certain percentage. I think it is just under 10% that are still over 40‑hours a week being spent online.
So certainly there is some intent ‑‑ some intensive users of the Internet. But we're hopeful that tools such as the app, we have used time use surveys in the past to capture this type of information. But as other organizations will know as well, it is a very intensive way for respondents to provide information. It involves a lot of processing and a lot of respondent burden. So that's also a very challenging way to do that.
So certainly, we will be continuing to explore new methods of data collection in that space as well to try and avoid that burden through traditional surveys.
>> MOLLY LESHER: Thanks again, Mark. I think we have a question from Leon ‑‑ from Fabio. From your experience how do you see the policy impact of subjective well‑being measures? Are there any examples of policymakers using this type of data to design better policies in Canada?
>> MARK UHRBACH: More specifically in the digital space, this is something that we've really just kind of gotten our first cut of data at and are really starting to look at things. So I don't think that necessarily our policymakers have had the opportunity to explore that data as much as they probably will in the future.
But in terms of the ongoing well‑being program at Statistics Canada, that's been developed in collaboration with policymakers in order to ensure that the data needs that are there are being met.
>> MOLLY LESHER: Okay. We have got one more question before we move to Alexander. Could you talk more about data collection innovations you used? Can these new developments in data collection be implemented in enterprise surveys?
>> MARK UHRBACH: Yes, with the household Internet use survey, maybe I will quickly talk about the use of the electronic questionnaire has allowed us to expand the ‑‑ just the sample size of the survey which is allowed for far greater data disaggregation which has been an important topic as we break data down. There has been an issue.
And I think really on the enterprise side of things we are receiving the same types of demand for more information, in terms of industry of firm, firm ownership by gender. We are being asked for all sorts of different breakdowns of that type of data as well.
So I think expanding those sample sizes although we don't like to put that response burden on businesses and individuals it is something that we need to balance out a little bit more. But taking advantage of those tools is certainly important.
In using the ‑‑ this newly developed app which just launched a couple of weeks ago, so we're anxious to see how that is going to go. It has been a project that's been in the works for quite a while. Certainly will see how it rolls out and learn some lessons from that, and if that can be applied to the enterprise side of things as well, I guess that will be ‑‑ will remain to be seen.
>> MOLLY LESHER: Thanks again, Mark. That was great. Just from my side, I have also heard of some policymakers using Facebook and Facebook sort of alerts to try to get responses to certain questions, too, which I thought was quite innovative. But now I would really like to turn to our last but certainly not our least panelist, Alexander Barbosa who is in charge of several national wide Brazilian surveys. He is well‑known in IGF circles. So Alexander, the floor is yours to please give us an overview of Brazil's efforts to produce ICT statistics, to measure the societal impacts of digital transformation as well again your views on one or two specific areas where we can try to make the toolkit better and more useful for emerging economies like Brazil. So the floor is yours.
>> Alexandre: Thank you. It is a great pleasure to be part of this panel with our dear colleagues. But before I go to answer to your question, let me say that I would like to congratulate Dominik for the census in Poland in such difficult times. And I'm saying that because I myself a member of the Brazilian census commission. And to the pandemic first and then for other issues, we had to postpone the Brazilian census for a second year in a row. But hopefully in 2022 we're going to conduct it. Congratulations, Dominik. This is impressive.
Here in Brazil we have a tradition introducing public statistics to measure the impact of digital transformation in our country. And I would say in general terms there is a broad consensus among stakeholders in Brazil that production of reliable data is fundamental to design effective policies, fostering the digital transformation in case of Brazil to reduce existing digital inequalities in the country, which is not a good figure. We still have digital gap, especially in the north of the country in rural areas. It is improving but it is still an issue. But, of course, this broad consensus provides us the support we need for a regular data production in the country. We have been producing regularized statistics since 2005 on an annual basis covering a wide range of areas such as households, enterprise, Government and health and education, et cetera. Currently we have ten surveys, national wide surveys.
And this is through traditional ICT standalone surveys in compliance with OECD standards, ITU, et cetera. And in the scope of this digital transformation, Brazil is facing an increasing demand as timely and segregated data, as mentioned by our previous speakers to support policy making. It is not ‑‑ are in demand more disaggregated data to allow for analysis based on multiple variables. Level of education, gender, race, disabilities, you name it. However we still have data gaps which may hamper evidence‑based policy support in the digital transformation and digital economy. And this is the case of statistics, new emerging technologies has been sent such as AI or new issues such as trust on the online environment, OECD is doing a wonderful debate around this need for establishing frameworks to measure trust or this information on poll ‑‑ on privacy and data production.
So those are new areas that we still have data gaps. And these new data gaps, they do impose additional efforts to us data producers to find innovative solutions to keep pace with the data requirements by different stakeholders. And this implies in maintaining, of course, the efforts of data production based on our traditional methods such as surveys, census or augmenting data.
But it also imposes the need of finding new means of adopting different data sources such as Big Data sources. Or imposing the need to explore new methods of algorithms for combining a quality of data source. This is an issue for us data producers.
And this became even more evident during the pandemic when traditional data collection methods were severely affected.
So summarize the efforts being made by Satik in Brazil I would highlight three areas. First one, of course, is the data production. As I mentioned we are carrying out ten nationwide ICT standalone surveys, including the inclusion of indicators on AI adoption in four areas. We have included AI adoption in our Government survey, enterprise surveys, health surveys, and education surveys. To see how those establishments are adopting AI. And since last year, during the pandemic, we started a new innovative project based on web panel to monitor the impacts of the COVID‑19 pandemic on the Internet usage.
We run four waves of this panel, to understand how Internet use was impacted by the pandemic. And the impact was really big. And this panel is now being used as an innovative platform to conduct new surveys. So we are using this panel as a quick response strategy to areas such as data protection, and privacy, we are right now running a panel to understand the perception of citizens on personal data protection. And we are now after this panel we are starting e‑waste panel to understand how Brazilians are dealing with this concept of e‑waste disposal.
The second area, I would say which is very dear to us and very important is capacity building. We offer regular capacity building programs and workshops on policy making and monitoring. We as a UNESCO center, we do provide methodological support to these countries in these two regions in order to help them to develop their national data collection projects.
And last but not least, the importance of data digitalization and micro data sharing. We have a very strong practice in data sharing. This is a summary of our efforts. And regarding your question on the toolkit, I would say the same as the previous speakers, the OECD digital toolkit is wonderful. I have to agree. It is a very nice tool. And I think that it is already a solid and very valuable tool to OECD countries, of course. And to some non‑OECD countries like Brazil because it provides really a comprehensive view of key dimensions of the digital transformation such as connectivity, innovation, productivity and growth, well‑being digital skills.
That's within these dimensions. Innovation and well‑being are more measurable across countries. And I ‑‑ I would say that the digital toolkit also allows countries to better assess their state of digital development. And this allows us to formulate better policies in response to this assessment. So the toolkit is already effective in helping countries understanding existing policy gaps as well as areas of improvements and to track their progress towards sustainable economic growth.
But I would say that in addition to this it is already very good, and we have to thank OECD for this wonderful tool. I would say that as we observe that the digital intensive business and Governments engage in a more complex activity in the digital environment, I believe that one actionable area in which OECD could maybe endeavor some effort is in creating a data user community around the data digitalization tool which is very good. But data community is very important. We do have this practice here in Brazil. And this data community that I'm talking about could maybe go beyond policymakers and Governments to involve a broader range of data users such as academic researchers, policymakers, companies and Civil Society Organizations and as well as data producers, of course.
And I think that this type of data community could eventually help identifying new functionalities and make it more used by a broad range of actors. So with that I will stop here, Molly. And thank you again for giving me this opportunity to be here with you today. Thank you.
>> MOLLY LESHER: Thank you so much and thank you for your words about the toolkit and that really interesting idea for ways we cannot just make the toolkit better, which we can. But also how we can get to promote it a little bit more widely which I think is something that we badly need to work on.
You all are very busy at ten national surveys, that's an awful lot happening. But it all sounds very exciting and important. And I was really intrigued by this idea of the quick response strategy and how that's working and all. I will be interested to see the data that you get out and what kind of lessons learned you might have gleaned from that because like you, I'm constantly being asked for more data, more disaggregated data, more data on whatever people are interested in in the minute. I wanted to mention, you mentioned you have some questions on AI adoption in health. And we do have a health data governance indicator on the toolkit. So I may follow up with you later to see if we can find out if that data is comparable enough that we could ‑‑ we could include you.
But before I turn it over to the floor and everyone please do send in questions in the chat. I wanted to ask you about your work using micro data and its analysis and what kind of insights you have gotten for policymakers. Not me personally but I do have colleagues that do a lot of micro data analysis. And it is turning out some interesting insights, particularly in the productivity space. I wondered if you could give us a few words about what you are doing and how it is having an impact on policy. Thank you.
>> Alexandre: Sure. Thank you for this question. It is a very important one. Because for us data sharing is something very important. We rely on agreements with Governments. We are trying with the private sector. It is very tough. It is not easy. Our national statistical office was able to access mobile data from one mobile operator in Rio only. We are trying to make this pilot to convince private data source providers to share data. But let me talk about the micro data. The micro data from each survey that is conducted by Satik, of course, is a key product of our process. We just use it as an input for data analysis as well as for longitudinal data analysis and policy evaluation analysis. And in this processes, it often happens to have a specific demand for special data tabulation for specific domain, including geographic data and socioeconomic variables. This is done by using tools that we have developed ourselves based on our package to handle the micro databases.
And when we have a demand from external data users, for instance, policymakers or academic researchers, is willing to have a specific analysis domain, we promote micro data sharing through data sharing agreements which includes capacity building for the use of micro data from our surveys that are based on complex sample design.
So it is not only providing or sharing the micro database. We have to build capacity so that the data user that is going to use this to have an insight for policy design or whatsoever, they need to take in to account the ‑‑ how the sample was designed. We work with a complex sample with certification. So we have to take in to account the sample design in order to handle the micro database. We do provide capacity building so that the data user, using the micro database will not make any silly mistake in terms of interpretation and building insights from the data. Basically this is the idea.
>> MOLLY LESHER: That's really great and important. And it's awesome that you are doing this capacity building and trying to help with the data that you have collected others do important research and analysis. So thank you so much for your time.
I think I'd like to try to get some interaction between everybody including the audience now. And Dominik, I will ask you first, if you have any reactions to any of the panel presentations. And then I'll open the floor to others.
>> DOMINIK ROZKRUT: Thank you very much. Indeed what Alexandre has said in terms of capacity building this is something that we tend to colonize this data stewardship thing. We need those who are really capable of facilitating the use and reuse of data. There has to be someone in between to explain the data, how it was designed and produced and things like that.
Because in the avalanche of those sources coming, you might find yourself in trouble just looking for the appropriate data and understanding what it stands for. And, of course, within the official statistical system we had this strong tradition of being transparent in a ‑‑ and trying to convey as deeply and profitably as possible the methodology of how the data is produced. It might not be the case in terms of other sources. As we all know, nowadays we don't have a kind of like monopoly in terms of producing the data for the information market. But we can facilitate the use of other data sources through even more profound services related to what we produce, I mean in official statistical system through the different layers like standards classifications, nomenclatures. We can help others build on those. That we might become playing the role of the data steward. So I think that's extremely important. And it is a challenge for us for the coming days.
>> MOLLY LESHER: Thanks so much for that. I agree with you a lot. And I often see people using data in bad ways because they don't understand the sample or how it was collected. And we've tried really hard at least on a toolkit to include all these notes and disclaimers and try to really give people a sense of what it is.
So is there anybody who wants to ask a burning question before I have ‑‑ I have another one. But I want to give folks that are in the audience and haven't had as much chance to talk as me to do so. But maybe while we are waiting for that, one issue that I mentioned before that I think is absolutely extremely important is better understanding, misinformation, disinformation, and other forms of untruth that circulate online. And in this frame we've recently wrote a short paper about it trying to come up with a typology of different untruths online with a view to finding the terms we are talking about, often misinformation is confused about disinformation or propaganda or other things. So most of the literature isn't very robust in this area. So we've tried to come up with these definitions with a view to trying to measure this phenomena. And I wonder if speakers, participants, anybody on the call today has any experience in this space that they would like to share? Because it is something that we're kind of in the middle of brainstorming ourselves. And it is extremely complicated and will require nontraditional data collection methods and absolutely certain. I wonder if anyone has thoughts on that?
Mark?
>> MARK UHRBACH: Sure. So I can just share a little bit about our experience there with trying to collect some information on misinformation. This is a challenging topic that we sort of discussed at OECD and at ITU as well over the past couple of years. And we had made previous attempts at Statistics Canada to capture this through a survey tool and had to challenge communicating to respondents what we are looking for and really trying to get any type of coherent answer.
But one thing that we did find it that during the pandemic, I did provide the opportunity to focus questions on a particular topic of interest. So focusing directly on information related to COVID‑19. In this context we were able to build much more successful questions and focus the responses to this type of information. So I think that was one piece that we have picked up, is that I think in ‑‑ even focusing it on that topic, I think just asking about misinformation online, I think we had high 90% said that they had seen some sort of misinformation online. So to us that's not overly useful. But I think sort of drilling down we were able to see some interesting results in terms of how people verify information online that they find.
I mean even just briefly there is some quick results that we saw, kind of 40% of Canadians reported that they read something online and believed it to be true and then later went back to realize oh, no, that was something that was inaccurately reported or something that was being spread without being fact checked. And we found some interesting results in terms of how often people check information that they pull off online.
And then finally I think one of the things that was a little bit surprising to us was in terms of how many people actually share information online without really knowing if it is true or not.
So over half of Canadians identified that they had done that in the first months of the pandemic, shared Articles with friends, posted it to Twitter without doing any verification. All of us have sort of probably gone through that exercise as well where you sort of maybe share something very quickly and then kind of go back to verify the accuracy of that later.
So this sort of some thoughts on measurement there. It is a very challenging topic to still get at and I think probably going beyond the survey is necessary. But being able to focus the survey on a particular topic we found was more helpful than our previous efforts in this area.
>> MOLLY LESHER: That's really interesting. And I may follow up with you bilaterally on that to see exactly the questions you asked in focusing on one area I could see would be helpful, you know. I'm just ‑‑ it is so tough because we talked a little bit about this, the team that wrote the paper, how do you define truth, right? One person's truth isn't somebody else's truth. And it kind of depends on where you sit in your background and how we do that I think is going to be really difficult. But it seems probably pretty clear it is going to take some kind of union or cooperation between people and technology to try to make that happen. Before I go to Alexandre I think we had ‑‑ Arpitha had her hand up.
>> ARPITHA DESAI: Thank you, Molly. Thank you. Hi everyone. I am tuning in from Boston. So Mark, I think you shared some really interesting insights. And the problem or the challenge with misinformation and disinformation which I have often seen as that the root cause of the problem is across so many boards, especially when the channels of communication have become so complex today and the way we consume information or news is across these channels. So it could be Facebook or Twitter where things go viral in seconds or it could be a consistent information disorder across say WhatsApp which is private messaging. So where disinformation or misinformation stems from becoming challenging to identify and track.
And the whole point of intentionally or unintentionally consuming and sharing such information also presents a problem because we don't know how things go ‑‑ how this particular piece of information went viral and has caused this asymmetry in the truth.
So it is really difficult to measure these root causes to kind of come up with solutions as to where we might need extra little ‑‑ extra work to be put in. And who is going to be putting this work. Is it the Government in kind of educating users to be more aware of how they're consuming information and the way to fact check information? Or is it on ‑‑ is it the responsibility of social media companies who facilitate the sharing of information? So there are so many pain points and measurements which you have to take in to consideration when looking at this whole. So collecting data in this sense becomes near impossible given that there are so many indicators and factors.
So if there was a way to kind of figure out what are these data points which we need to identify in order to come up with workable solutions which go way beyond than just say labeling or fact or fact checking but as a more systemic change, I would love to hear that.
>> MOLLY LESHER: You have generated some interest here. I have three people who want to intervene. We have one minute. So I'm going to not close ‑‑ I will just say thank you in advance.
>> Alexandre: Thank you. You raise a very important issue on channels of communication. Human moderation it is impossible. So we need to think about a combination of human and machine type moderation, artificial intelligence. How those moderation type of interventions and policy and regulations may not 100% of problem. The root of the problem is on education, on awareness raising and meaningful information literacy. So maybe OECD have to work or develop an agenda to build measurement framework to that. But we have Government have to invest a lot in education and meaningful information. I will stop here because my 30 seconds is done. Thank you so much.
>> MOLLY LESHER: Thank you so much.
>> DOMINIK ROZKRUT: A small point because I took a look at this typology of these disinformation. I want to play the role of the advocate of the satire. Satire is very positive. It can convey the truth. I will call it the other way here, I will call it the contextual deficiency. You might not understand the satire properly but then satire as propaganda can include all four of them. Misinformation. Disinformation whatever. But I like that anyway. And I think that, you know, should be further developed because this is a very nice typology and we should work in to that.
>> MOLLY LESHER: We will keep going.
>> NAGWA EL SHENAWY: Thank you, Molly. And I fully agree with what was mentioned about my colleague concerning the misinformation, especially when ‑‑ when it comes to the social ‑‑ from the social media like Facebook and Twitter. I would like to draw the attention that we shouldn't forget the positive side of this social media. And we get some information related to research. Sometimes the students they went to the social media and in order to collect the data for the research or assignment, as well as also the entrepreneur and the small and micro enterprise when they also went to this mechanism in order to really test the market or to test their beneficiaries about the product or their services. Again thank you very much for this, Molly. And it is a pleasure to meet you all today in this workshop.
>> MOLLY LESHER: Thanks. From my side a big, big round of applause for the speakers who don't get to hear it. But lots of you got up really early and you spent lots of time preparing wonderful presentations. Thank you so much. And I hope I get to see you at the IGF next year, if not before that. Take care. Bye.
>> Thank you very much.
>> Bye.
>> Bye.
>> Thank you, Molly.