IGF 2020 – Day 12 – BPF Cybersecurity

 

>> You can take a snapshot, a print screen of the session and use it later on while preparing the final report.  I have just started the streaming.  And just quickly looking at our list if for any ‑‑ for any other panelists who we are expecting.

>> MAARTEN VAN HORENBEECK: I think Apratim just joined and needs to be promoted.

>> Nope, I don't see this person yet.

>> MAARTEN VAN HORENBEECK: He just sent us a message.  A‑P‑R‑A‑T‑I‑M is the first name.

>> Yes, he has just joined.  I am promoting.  So we are still missing three people at least for what I see on my list.  Stephane and Isaac Morales.

>> MAARTEN VAN HORENBEECK: Isaac Morales is in the attendees list.  As for Stephane and the others they will come in later in the discussion so I think we can actually get started without them and they can join when they're available.

>> About Isaac Morales, could you please repeat.

>> MAARTEN VAN HORENBEECK: He's actually Gerardo Isaac Morales and he needs upgrading.

>> I see, okay.  I have promoted also this person.

>> MAARTEN VAN HORENBEECK: Thank you.

>> You're welcome.  Enjoy your session.  Thank you.

>> MAARTEN VAN HORENBEECK: Thank you very much.  Excellent. And I think we have everyone and everything ready to get started.  Thank you very much for joining this meeting of the Best Practices Forum on cybersecurity here on the last day of the Internet Governance Forum.  I was a lead expert for the group here.  And moving on to the next slide I wanted to share with you briefly the agenda for what we'll be discussing today.

The first ten minutes I'll give a brief introduction.  We'll then have some time for a presentation by one of our working groups within the Best Practices Forum around norms developing in cyber versus norms development in the real world.  20 minutes later, John Hering will present an analysis of new cybersecurity agreements that was performed as part of the work of the Best Practices Forum this year.  We'll have time for discussion and general Q&A and finally Markus Kummer will send us on our way and after that we'll take notes from the session, work on the final publication of the Best Practices Forum in the next few weeks on the IGF website for the BPF.

Moving on to the next slide, I'd like to frame a little bit what we did this year and actually the last years of the Best Practices Forum.  As a group we've been meeting regularly actually for quite a few years more.  But in the last three years specifically we started investigating the idea of culture norms and values in cybersecurity and started taking a close look at norms development mechanisms.

That initiative started in 2018.  Last year ‑‑ because the BPF ran in conjunction of the initiation of the new U.N. GTE and OEWG working groups, we had a much closer look at best practices related to operationalization of cyber norms and started analyzing stakeholder agreements for commonalities.  I think what was unique about the BPF was we didn't just look at norms that were being developed at an interstate level, but we actually also took a look at norms being developed in the private sector, within civil society, but spanned across more than one stakeholder group which is typically what makes them interesting for us to review and consider in the Internet Governance Forum.

This year we took a little bit of a wider approach and we started thinking about what we can learn from norms in global governance in completely different mechanisms than cybersecurity.  In many ways that idea actually came from input that was given at the OEWG meeting that included civil society, because it was clear that the understanding of what is a norm and how do norms work is not universal across the entire spectrum of all norms and actors. A working group within the BPF spent some time investigating a variety of different initiatives around cyber norms and looked for strengths and weaknesses and also specifically what made them succeed and what made them fail.  We hope we can take some of that knowledge and study it in the context of cyber norms.

Second, we initiated a work stream that particularly looked at new agreements in relation to international cybersecurity.  So the work that we did last year, they essentially took those same agreements, identified new ones that were released in the last year and deepened the investigation into those agreements.  They looked at things like what UNGE 2015 consensus norms were actually reflected in them or what new norms were introduced which were completely outside of that space.

Moving on to the next slide, you can see an overview of the different workstreams we have this year.  There was one that was really focused on ideas to put it simply, looking for inspiration from other mechanisms and applying that to cyber.  The lead for that work was Mallory Knodel who is also in the meeting today. There was a work stream that focused on more desk research related to existing agreements.  We also published a call for contributions to the entire Best Practices Forum mailing list as well as many different organizations that are involved in this space or have participated in the past and used all of that input to further that research.

Finally, we have one work stream that was focused on outreach.  That was led by Sheetal Kumar which engaged new stakeholders we could involve in the Best Practices Forum and could bring into the community to help us learn more about cyber norms and share those learnings with the wider community. The research team was led by John Hering, who will also be presenting in this meeting shortly.

Moving into the next slide, just really briefly again the agenda, we'll take a look at the output of the ideas work stream first.  Then we'll look at the research that was done this year and finally we'll wrap it up with discussion, hopefully analysis of new agreements and the discussion, we have a set of esteemed panelists on the panel whom John will introduce to all of you later that will be brought forward to feature some of their very unique perspectives and opinions of all the different areas.

Before we get started, I want to say a quick thanks to everyone who joined our meetings throughout the year who joined with ideas, who joined one of the work streams, who contributed to the two research papers that were published earlier this year and who are on this call at a very different hour to contribute and chair their ideas.  Thank you very much for joining.

With that I'll hand it off to Apratim and Anastasiya who are going to present us the outcome of the research they did on norms development in the real work.  Apratim and Anastasiya, over to you.

>> APRATIM VIDYARTHI: I'm going to try and share my screen to make this easier on everything.  All right.  Can you guys see my screen?

>> MAARTEN VAN HORENBEECK: Yes, we can, thank you.

>> APRATIM VIDYARTHI: I'm coming to you from very cold Philadelphia today.  What I'm going to be talking about with Anastasiya are what cybersecurity policy making can learn from norms that have been formed outside of the cybersecurity sphere, specifically in global governance.

Before we get started what we need to talk about is the definition of norms and why they matter.  And I'll answer the second question first which is that given the nature of the internet as a decentralized entity that spans across geographic borders that has multistakeholders that aren't just nation states or civil society, it's really hard to govern.

So the multistakeholder model really forms the basis for norms.  What are norms?  There's an entire spectrum of things that we would consider norms.  Implicit norms are what Justice Potter Stewart would have said as I know it when I see it when he was describing explicit material in a case.  Explicit norms can be anything you can't really define but you can sort of understand.  Social contracts, guidelines for conduct. Then on the other hand, you have explicit norms.  These are really treaties, agreements, things that are codified into law and at the very extreme spectrum you have accepted international law that's really the bedrock of international conduct.

So what we found in our research is that norms that are successful have four really defining principles.  So they are concrete and specific, which means that they really set out some sort of behavior that followers understand.  They're framed in a context, so they don't just come out of nowhere. They come out in a specific problem and they're applied to that problem. They're propagated by leaders and norm entrepreneurs, so you really need either a strong state, a strong non‑state actor or some individual who is pushing those norms. Finally, they're promoted using incentives and socialization and persuasion.  IE the right kinds of tools.

So the framework for norm development is actually one of the norm forming process.  So what we saw in our research that's accepted in social science is that there's a four‑step process to the development of norms.  So norms often arise with an initial social process that comes from a politically engaged entity, for example.  That's where the idea of the norm comes about, so this forum is a really good example of that.

The adoption of the norm happens through a political actor or a transnational organization.  This is where the norm goes from the grassroots to the international sphere.  It's either spread by nongovernmental organizations or a state and then the norm is accepted by more states than one and by non‑state actors.  And this is really key, especially in Internet Governance because we need both states and non‑state actors to accept those norms in order for it to really impact how the internet functions.

And then, finally, the last step is the hardest, which is really the codification of the broad norm into a conventional and customary norm.  The reason that's the hardest is it requires agreement on what the norm is and oftentimes when the norm starts at the grassroot and is spread by different states and different actors it's really hard to agree on what that is.  Anastasiya?

>> ANASTASIYA KAZAKOVA: Yes, we also tried to look a little bit deeper into sources for norm setting and trying to develop a list of what the sources might be.

The first one as identified are multilateral norm diplomacy and state driven efforts.  It would imply joint statements of states, proposals by states as well, exchange of use between states, publications, and understanding the best practices.  State led and state-initiated processes for example. It could be particular examples as public consultations organized by states for non‑state actors to share the understanding of norms and how to operationalize nonvoluntary norms.  As an example, the public consultation to solicit the use from the civil society.

The next category is subject matter, and particularly we analyze norm ‑‑ governments to agree on certain norms of behavior.  Or the Red Cross’ call to stop cyber attacks in the healthcare sector.  Both happened earlier this year.

The third and final group is industry‑led process.  Those efforts that aim to establish calls to governments to agree on certain norms of behavior.  For example, digital Geneva Convention, as well as establishing industry organizations based on participants' consensus around particular behavior.  Examples are a tech accord.

>> APRATIM VIDYARTHI: What I want to do next, I want to discuss some of the critical factors for success.  We sort of already covered these, but I wanted to go into a little bit more detail.

As I said, there are four critical factors for success.  The first one is context.  And what we're talking about is framing the problem the norm is aiming to solve.  This includes all the details, the platform, the organization, and the larger issue that the norm is aiming to provide a solution to.

The second critical factor is employing strong leaders and resources for norm development.  So leaders can be individuals, they can be states or nonprofits, nongovernment organizations, but really they're the ones who are going to be carrying that norm forward and they're going to be the ones who are pushing it.  The right leader will be able to network and provide connections and provide the right leverage and pressure for these norms to actually be practiced in real life.

The third element is ensuring the right elements of the norm.  So identity is targeting a group or a group of actors, and so identifying the group of actors to target is really important.  Because norms don't apply to everyone, they apply to specific subset of actors and that specificity and concreteness is really important.

The second element is behavior.  This is talking about the specific actions that the norm wants to change.  And then the third is propriety.  This is really what the norm is based off of.  Whether it's laws or treaties or social conventions, the norm has to be based off of some underlying conduct for it to really be effective. Finally, there is expectations, so setting the expectation for behavior through concreteness and specificity.  Norm that is broad is probably not going to be successful and so having the right boundaries of success is incredibly important.

Then the final factor for success is choosing the right tools of influence.  So the menu of tools that we have here are incentives, persuasion, and socialization.  Incentives are effective if they're consistently used in the long‑term.  It's basically the carrot approach.  The persuasion tool is the stick approach.  It's messaging and targeting and having the leader push those who are the subject of the norms to actually apply those norms and adhere to those norms. Finally there's socialization which is about creating a common language and building capacity and for the leader to lead by example in making the norm acceptable to those who are the target of the norm.

So we can take an example here.  Today we're seeing as we're in the middle of a global pandemic and sitting on Zoom that digital healthcare and medical infrastructure are a really important part of this pandemic and helping us really succeed and survive.  So the correct context in this example would be to promote a voluntary norm to protect this digital healthcare and medical infrastructure and perhaps create a standard of ensuring the security of that healthcare infrastructure.

So a strong leader in this case would probably be a group of U.N. member states within an OEWG as Maarten mentioned earlier.  Or it might be a state actor which has the requisite resources and expertise to promote that voluntary norm.

In terms of the details of concreteness and specificity, who does the norm govern?  This is really a question of what specifically we're talking about within digital healthcare.  Are we talking about the cybersecurity of healthcare records?  Are we talking about the protection of hospitals and other medical infrastructure?  So in the former case, the norm might govern companies like Apple and Google who have, for example, a contact tracing application process and interface.  If it's the latter, if it's protecting hospitals, we might need to target state actors to ensure that cyber attacks aren't affecting hospitals.

The second part of concreteness and specificity is what does the norm say some what is the behavior it's governing?  And the first example, if it's about protecting medical records, then the norm is trying to govern behavior of how those records are stored and kept, where they're kept.  If the norm is guarding cybersecurity for hospitals, it's governing offensive cybersecurity and ensuring that hospitals are safe.

And then what is the basis for norm promotion?  So is this grounded in some sort of international law about how cyber attacks are handled?  Is it some industry standard?  And then choosing the right tools of influence.  So the OEWG might call for national legislation and a private actor, such as Apple or Google, if they were propagating that then they might lead by example and lead that persuasion. We move on to the risk factors inducing failure and I'll pass it on to Anastasiya.

>> ANASTASIYA KAZAKOVA: Thanks.  We're looking at what makes norms promotion successful.  We tried to identify what are the norm settings, as a result, the particular factors that impact the process.  First of all, the lack of clear outcomes that actually could be one of the traps that could lead to failure in norm promotion.

Lack of enforcement mechanisms.  Actually, mechanisms conformity or adherence to the norms.  This applies to usually international borders that often press norms without leverage other than modeling and persuasion.  The next items are too weak or too powerful leadership.  We try to also highlight a normative which is successful for norm development and leadership can be a crucial factor.  It may also break norms if it's in the interest of powerful leaders and benefits outweigh the costs of not adhering to norms.

And it's particularly common in cases where there's a lack of enforcement.  The next factor is lack of incentives for internalizing norms.  This might be another trap leading to failure in norms setting.  This happens when the prospective benefits of norm compliance do not outweigh the benefits of staying away from norm adherence.

Speaking about the particular examples or incentives, those incentives might be domestic demand or international legitimization and acceptance.  Actors follow norms because they want others think well of them and they want to think well of themselves.  Those could be critical.

Another factor is domestic balance of power or domestic legal institutions and domestic powers.  They play critical roles in a state's noncompliance with the norm or compliance with the norm, and for norm interpreters can be helpful to consider the balance of power within a particular state to frame the norm so that a particular state would follow the norm.

Finally, norms that are too specific or strict in wording might contain ‑‑ another common mistake that could lead to failure.  We mentioned earlier that the lack of some elements in the norms, particularly the entity, behavior, propriety makes it harder for entrepreneurs to garner attention.  However, sometimes imprecise norms ‑‑ particularly actors with different backgrounds and best to support the norm development process.

>> APRATIM VIDYARTHI: Next we'll talk about enforcing norms, Anastasiya, I'm going to hand this over to you as well.

>> ANASTASIYA KAZAKOVA: Thanks.  We also looked at what mechanisms enforce norms.  We identified different scenarios that could be applied, first of all, supporting norm legislation and codification.  Some norms might appear first as a best practice but over time they can be legalized and become a part of the law.  And, therefore, cost for noncompliance with norms become risks of noncompliance with measures.

The voluntary norm in 2015 IG report which was implemented already in best practices such as first code of ethics and implementing industry best practices and this has appeared for the first time in the public consultation.  The disclosure hasn't been part of the directive framework before.

The next scenario could be crafting new norms onto existing institutions.  This in particular allows to create legitimacy for norm entrepreneurs and thus make the norms more attractive for them.  Favoring fragmentation and cross pollination of cyber norms efforts is another scenario, especially if it is coupled with multistakeholder engagement.  Fragmentation and different initiatives can help address different actors and focused on a relatively small group of stakeholders.  Norms can be even more efficient in changing behavior and creating new collective expectations. So the norm initiatives addressing needs and interest appear, the more incentive and motivations are created for active participation of different stakeholders and norm development.

Creating network sanctions can be another scenario.  It could be effective in not only making particular actors follow norms, but also exerting pressure on actors to facilitate domestic reforms.  That could be propriety for the actor to follow a norm.

Applying diplomatic sanctions.  They become more and more ‑‑ one of the I would say popular ways of diplomatic pressure to other states to actually follow the norms and they use cyber diplomacy as a key example of this enforcement mechanisms. However, we identified in a paper it's yet to be seen if these mechanisms are, indeed, effective in practice because that option of sanctions usually causes costs on both sides.  The sanction and sanctioning states.  Therefore, to make the punishment work, incentives should be provided to sanctioning states as well.

Finally, targeting the reputation and self‑esteem of norm breakers is one of the scenarios to enforce norms.  Reputational benefits is one of the incentives for norm compliance can be targeted and this may include public criticism, refusal of other states to enter into future agreements with the norm breakers or keeping current agreements or withdrawal of a member from international organizations and closed groups.

>> APRATIM VIDYARTHI: Thanks.  The last thing we'll discuss is case studies.  Really how these frameworks and factors for success work in some other fields that we've seen.

The first case study we looked at are global and nuclear norms, which are particularly close to my heart because I studied nuclear engineering for a really long time.  What we're talking about here is the entire norm of non‑proliferation, of ensuring that uranium is not enriched too highly.  The strengths are that they follow the standard norm process.  There were social processes that initiated these norms.  They were propagated by civil society and governments and are now effectively international law that's widely accepted.

One of the strengths is that they're concrete norms.  There are specific rules that govern how much uranium a country is allowed to have.  For example, the kind of reporting that they need to do to the IAEA, et cetera.  And the international acceptance happened through persuasion and enforcement.  There was a lot of international pressure by countries like the United States and then there are a lot of incentive as well such as access to new technology.

However, what we do see is that there are some weaknesses, right.  There are cumbersome regulations that stifle innovation in a sense.  There's a lot of reporting that's required.  There's a lot of risk adverse behavior that prevents countries from developing new nuclear technologies that might be much better in terms of energy efficiency, for example, and in terms of environmental friendliness but countries are not willing to go through the entire process and be ‑‑ risk the look of looking like a proliferator. Political disputes become tied to these norms because they're so tied to a valuable source of energy.  So the context framing in today's world wasn't necessarily correct but at the same time we do see that countries that don't follow these norms are punished by the global community.

The norm of diplomatic privilege is also one that has followed the standard process of norm development.  So there were original social customs more than 200 years ago of diplomatic privilege and those have been propagated through intergovernmental actors and through states and to effectively becoming the Vienna Convention on diplomatic privilege.  Those social customs have been adopted into codified norms and there is reciprocity inherent in the norm framework.

However, what we do see is that the norms of diplomatic privilege are pretty narrow.  They avoid controversial topics, which is sort of the reason that we have this widely accepted international norm.  They're also ripe for abuse because these standards have been for so long accepted as normal.  There are very few ways of enforcing those standards in novel and creative ways.

In terms of the Sullivan Principles of employment practices, these were practices that were signed by some American countries during apartheid.  They had buy in from corporate actors, but on the other hand they were very broad principles.  And they weren't widely accepted because they were in an opt in format but also there is no real norm leader that propagated these principles on a global level. But the real weakness came from the fact that there were no costs for violating these principles so there was no real enforcement mechanism for getting countries or getting companies to buy into this.  In fact, companies could be certified in the Sullivan Principles and continue to do the things that they promised not to do, and no one would have any ability to sort of punish those companies.

The final case study we see is the World Bank guidelines on the treatment of FDI.  This is really interesting because it's a guideline and it's a model for national laws to treat foreign direct investment.  It's not really a codified international law, but the details of these guidelines are often reflected in the codified law of state actors. Now, obviously the weaknesses as I said, these are nonbinding and there's no enforcement mechanism.  But despite that we see this has been pretty successful in shaping the codified law of states.

Finally, I'm going to pass it on to Anastasiya for our last couple slides on lessons learned.

>> ANASTASIYA KAZAKOVA: The key lessons learned we outlined in the paper.  There are certain things to consider to facilitate norm development.  We talk about the context, we talk about the norm entrepreneurs, so these are important ingredients to keep in mind.

Failures are inevitable and they can become the basis for success as well.  It's necessarily to keep in mind and get prepared.  We cannot predict the entire norm development process.  There can be many factors to consider.  But they create opportunities for further successful efforts.

Final lesson learned is norm development even without results creates socialization.  Norm promotion can fail if there is no readiness to process norms.  It's important to consider norm development as a process, which is crucial to preparing the entire environment. Concluding, we received very interesting feedback to the paper and tried to summarize key questions that we learned.  You may see them on the slides.  First of all, one of the questions, what is the difference between norms and international law.  How those two concepts correlate and how they are actually different.

The next very interesting feedback is that perhaps we could be ‑‑ go a little bit deeper to analyze more targeted and substantial recommendations to policy makers, industries, civil society, et cetera.  Finally, confidence building measures in norms development.  How those two concepts could be further incorporated into norms development.  Thank you very much for your attention.

>> MAARTEN VAN HORENBEECK: Thank you very much.  Thanks to Anastasiya and Apratim who worked on this research together with our corresponding paper and with Mallory Knodel as well who led the initiative.  I would recommend everyone who enjoyed the presentation to look at the first research paper that the BPF published earlier this year, which is available from our website.  And we have shared the link to that page in the chat as well so you can find it there.

Next up we will go over to John Hering who will share the work that was done on analysis of cybersecurity agreements and will lead a brief discussion on that topic as well.  John, over to you.

>> JOHN HERING: Thank you so much, Maarten.  Thank you to you and Markus and Ben and Wim for your coordination of the work of the best practice forum this year.  As well as to all the other collaborators and all of the working groups in particular on the one that put together this analysis of cybersecurity agreements.

If you wouldn't mind going to the next slide there, Wim.  Thank you.  So to give a bit of framing as was said at the outset, a lot of this work builds on ‑‑ in particular the last two or three years of the work of the best practice forum in discussing how cyber norms are developing and how they're being implemented and taken up across different stakeholder groups.

This portion of our work, this second report, builds very directly on the work of the 2019 best practice forum on cybersecurity, which if you're familiar with went through a list of cybersecurity international agreements and sought to identify areas of overlapping principles to sort of start to see where consensus was being built.

We built on that in two important ways this year.  First, we revisited the list of agreements we had to include developments from the past year that would fit into our scope.  Certainly there's been some very important developments there, including the launch of the global commission for stability of cyberspace six critical norms, the contract for the web which I believe was launched at IGF 2019 last year and a few others that were included that are featured at the bottom.

Similar to last year, all of the agreements that were included in our analysis needed to describe specific commitments or recommendations for their signatories or supporters, as some agreements didn't necessarily have signatories, including different stakeholder groups that supported it at different levels.  They needed to aim to improve the overall state of cybersecurity.  That needed to be their objective and needed to be international and involve either governments or significant operators of different parts of the online infrastructure so that we were including people who had opportunities to be influential and to sort of guide norms development.

But beyond that, they could certainly be within a stakeholder group between different governments, they could be multistakeholder in nature or even agreements within stakeholder groups that are nongovernmental, such as the charter of trust or the cybersecurity tech accord that are just between industry groups.

The big new element this year was we decided to exclusively focus on agreements that included voluntary nonbinding norms for cybersecurity.  And excluding in our analysis anything that was going to be a legally binding commitment in that the implementation of those principles are just very different when there are requirements by law as opposed to actual norms.  Then, Wim, if you would go to the next slide, which is not meant to give anyone a headache but just to show a quick heat map of where things lined up in the analysis that was performed by different BPF experts.

Down the left side of the screen there are the different 22 agreements that are included in the analysis.  And then on the right they're lined up with which norms they reflect based on the 11 norms of the U.N. group of governmental experts based on their 2015 report on expectations for responsible behavior in cyberspace.  We chose to focus on those 11 norms as the U.N. first committee has a unique responsibility to peace and security challenges as being the most global multilateral norm setting body and seeing the degree to which previously or subsequently those norms have continued to be strengthened or reinforced.  You can see some have been more than others.

These norms themselves were simplified a bit so they could be inclusive of other stakeholder groups and not exclusive to state actors but making sure the spirit of the norm itself was captured in the analysis.  What's perhaps a little bit less of a headache to look at is the next slide.  This gives you a breakdown of the frequency by which we were seeing those different 11 norms of the U.N. GGE reflected to various degrees in different agreements that were included in the analysis of those 22. You can see the most frequently included ones was the commitment to cooperate with states for greater stability and some recognition of human rights in the agreements themselves were the most commonly reflected principles in other agreements.  And the least commonly reflected one was a commitment to prevent wrongful use within one's own territory.

That's we are sort of landed in terms of analysis itself.  I would encourage you to read the further report and give us additional feedback.  But if you'd move to the next slide, we can move towards a broader discussion here.  We have issued a call for contributions based on whether or not your organization is a signatory, the projects and programs you have and the other questions that are listed here.  If you would proceed further Wim.

We received responses from ASPI, from East West Institute and from Professor Duncan Hollis.  I believe we'll pivot a bit to opening a discussion with our panel.  I'll start with a few questions here that will build off the work that we've done.  In the interest of time I'll provide a quick introduction for the wonderful, esteemed panel that we have.  Please do correct me if I mispronounce a name or anything else in introducing each of you.

First we have Stephane Duguin.  He's spent the past two decades analyzing our technology that has been weaponized.  He's investigated the use of disruptive technology such as AI in the context of counterterrorism, cybercrime and other cyber operations and cyberthreats.  We have Sherif Hashem, a visiting professor of computer information sciences.  He was an expert member of the GGE that we've been discussing.

We have Pablo Hinojosa, we have Louise Marie Hurel, who is a researcher in cybersecurity.  We have Moliehi Makumane from the government of South Africa where she's been with the South African Foreign Ministry for nine years, currently special advisor to the South Africa U.N. GGE and was previously the head delegation for the U.N. UEWG.  We have Isaac Morales from the government of Mexico, the coordinator of multidimensional security, the Ministry Of Foreign Affairs in Mexico.  And among his duties he's in charge of multilateral negotiations on cybersecurity including as a member of the U.N. group of governmental experts.  Finally, we have Aude Gery.  She has done her Ph.D. thesis on international law and cyber weapons counterproliferation focusing on legal problems. It's an impressive panel.

Perhaps sticking with Aude, would you mind telling us a bit how the international legal perspective applies to the analysis of cybersecurity agreements and how international law factors into these conversations?

>> AUDE GERY: Thank you, John, and thank you for the invitation to this panel.

First, I want to say that I really found the paper and both papers very, very interesting.  Because although they were not focused on the relationship between norms and international law, they did not omit it.  And it was quite interesting, and I liked it because I think very often we tend to think about norms, and especially in the policy debate, as something that is very and distinct from international law.

I'm going to be quick on that, but if norms are nonbinding I don't think they should be qualified as outside of the boundaries of the law and of international law in comparison to the legal rules that would be the only binding ones.  Because whether binding or nonbinding, both norms are part of the so‑called I would say normative gradation between law and lawlessness.  The nonbinding provisions contribute to the existing interpretations of international law.  For example, if you take the GCS's norm on offensive operations by private actors, that could be one way to implement the obligation, which is a binding obligation in international law.

And nonbinding norms can also become after ‑‑ it was very well explained in Anastasiya's paper, for example.  They can become either codified or customary obligations.  And in that sense, space is a very good example of nonbinding resolutions becoming binding after some time.  So I think there is a very close relationship between the two.

In the GG report, the 2015 GG report, they tend to be separated and very distinct.  And in some ways, it can be dangerous because if you think there is no relationship between the two, you will say, well, their norms are nonbinding.  And if they integrate international law obligations, you may think that you may break the norm, there is no legal consequences to the evaluation of the norm.  But there is ‑‑ since it is one way to implement an international law obligation, it is a violation of international law. So I think we should be very careful that even if they are nonbinding, they have some kind of relationship with international law.

>> JOHN HERING: Had to come off mute.  Thank you so much.  Pivoting to the broader group of panelists we have and the respective organizations they're representing, I wanted to kick off with providing an opportunity for each of you to highlight whether your organization is a signatory to or has contributed to any of the cybersecurity agreements that we've included in this review, whether they're multilateral or multistakeholder in nature or within a particular stakeholder group.  Which of these agreements have you joined onto and what might have been the motivation in doing so?  I'm happy to call on someone or if someone has a strong feeling to jump in.  Stephane, I might go to you, first.

>> STEPHANE DUGUIN: Thanks.  It's a great initiative when it comes to engagement across the quarters and across the issues.  Recently for example we've been joined by an organization that would ‑‑ would not think about joining international groups that we focused on the cyber issues.  So quite interesting for sure, but a potential there.

We also signed a call to governments earlier this year, to nine governments and made a change.  ICRC that we have obligation when it comes to ‑‑ of course not attacking the healthcare, that's quite obvious.  Protecting the healthcare from making sure that resources are there but also not harboring criminal groups knowingly.

>> LOUISE MARIE HUREL: Yeah.  Hi, everyone, can you hear me okay?  Yeah?  Great.

So yeah, as John has kindly introduced me, I'm a researcher at the cybersecurity and digital liberties area.  For those of you who don't know, I’ll give a background of what Igarape is, we're focused on integrating agendas on security, justice and development.  Cybersecurity is part of that.

I'm saying this because we've been working with cybersecurity governance at the national, regional and international level for some time.  And we were one of the early signatories of the Paris call.  And one of the motivations was really to kind of see a document, international kind of ‑‑ a way of symbolizing multistakeholder and multistakeholder more inclusive approach to cybersecurity.  Kind of pushing the boundaries and it was ‑‑ in 2018 it was a very important moment for, you know, gaining traction of the GGE discussions.

It was also very important timing for different stakeholders to come together and actually kind of agree in certain norms.  I think that was really important, so the Paris call was one of them.  But also, we've been very much engaged in other processes, such as the OEWG, the intercessional meeting and other meetings.  But we're an accredited organization which is a prerequisite.  I think what I'd like to leave aside from talking who we are, what we've been doing and why do we engage in these discussions, not just at the national level but we really see the importance of, you know, different organizations, especially those from the global south to actually be hands on and vocal in international cyber norms discussion is also because it raises a very important question of who can actually participate in these conversations, who can contribute to international cybersecurity and cyber norms discussions.

I think that is something that comes very much together with not just, you know, supporting or endorsing, you know, something like the Paris call or just being in the meetings, but it's really asking the question when we talk about multistakeholder cybersecurity and these international agreements who can actually kind of actively participate and shape these discussions?  I'd like to leave that provocation over there.

>> JOHN HERING: The floor is open if there are other thoughts about the motivations for joining respective agreements.  I also want to open up the conversation a bit more to pivot to what's happening with them.  I think everyone here is likely party to at least one of these agreements, whether it's a national government or an organization.  If you're an outside party and researcher, what are the project and program implementations you've been successful in supporting cyber norms agreements.  I think this is the big question in the international space at this moment.

>> AUDE GERY: I'm just going to mention two of them.  The first one is the papers, both papers that were just presented.  When I read them, I was like I would have loved having these kinds of resources when I was studied my thesis and started work on this issue. The reason is it showed some of the norms are widely accepted.  The paper shows that state's practice is bigger and bigger and at least some of them might in some way evaluate the progress of the nature of the norms.  I think it's one very good example of the ‑‑ what we can all bring to the community.

The other one is responsible behavior.  Norms and the resolutions and creation of a global culture of cybersecurity calls for public/private cooperation.  These resolutions are quite detailed.  So this kind of dialogue is I think very important to try to identify how to implement them and consensus of states is very difficult.  The key is really to raise the level of global cybersecurity and this kind of dialogue can definitely help to do that and to achieve these goals.

>> ISAAC MORALES: If I may?  Thank you.  First of all, just let me recognize the important work you have been doing on norms.  Actually, it's timely and necessary to get this kind of general clarification of what the norms are and what we can expect from the norms.  I just wanted to refer briefly to this idea of expectations because we really need to understand that the norms are not going to solve all our cybersecurity problems or cyberspace governance.  The norms are part of a more general construction of what we need to put in practice.

In this regard, just to share, the government of Mexico was part of the Paris call, and the motivation, it was in our case, implementation to call to implement the norms.  The call to implement the previous agreements.  Actually, I believe that ‑‑ agreeing to norms, by agreeing to norms, you have the how of the homework done.  But the other half of the homework is implement what you actually agreed.

In this regard, I also shared that with Australia and South Africa and many others, we have put it on the table on the opening working group the idea of adopting a follow up mechanism of the previous agreements completely on the norms.  This format is serving, but the idea is actually to call to the implementation and to see not only the good things that we are doing to implement the norms, but actually the obstacles to implement the norms.

A very important issue that I saw in the work that you presented to us, it was this idea of the concrete that the norms need to be in order to be effective.  Yes, the more concrete, the better.  Nevertheless we have to recognize also to take into account that when dealing in multilateral settings, in multilateral platforms and with the goal of consensus, then you need more flexibility or ability to keep some issues not very well defined in order to get that consensus.

That is also important for me to say and put it on the table because actually within the open ended working group, we are discussing ‑‑ now we have the norms, then maybe guidelines to implement the norms.  It's also important to put on the table this idea of implementation of the norms.  And finally, to share with all that recently Mexico put it on the table to the OAS, the regional organization, through the resolution of international law, a call which was adopted by the OAS General Assembly, a call to implement ‑‑ I mean, recognizing the applicability of international law in cyberspace and a call to implement the norms, the agreed norms. So this is also a way to step forward the implementation by socializing, by endorsing, by bringing in organizations and now our homework is to work on that implementation.

>> JOHN HERING: Well said.  The floor is open if others want to take on either of these questions.

>> SHERIF HASHEM: If I may?  Thank you for the invitation to join this panel.  I would like to give a couple of points as a former member of the U.N. GGE, I've seen the process early on, we spent actually a few weeks discussing the applicability of international law, whether it's applicable or not.  Then we moved to the following GGEs about how is it applicable.  You can see the debates.

With this in mind, I personally feel that reaching a global treaty on cybersecurity is not in the near future so norms is what we have to work around.  I'm not a lawyer, I've worked with lawyers quite a bit for the past 20 years or so, so I feel that we need to move forward with implementing norms.

I agree with the comments about really the challenge is to support the initiatives for implementing norms, build capacity, get a clear understanding of the implications at different levels.  Some of the articles that led to these norms which from previous work that was done in non‑proliferation of nuclear and chemical and other weapons, especially since the first committee of the U.N. was part of this effort.

I remember in our discussions in 2012 and 2013 about such articles is dealing with ‑‑ I mean, categorically different.  While they are broadly used, we cannot deal with control the same way when it comes to nuclear weapons.  I used to say if I enter a room with a nuclear weapon people will notice, but if I enter a room with a stick with a virus people would not notice.  You cannot expect the same results.

For instance, developing capacity, then it could hurt nuclear or chemical weapons.  That's one.  The second is really when we talk about capacity building. I've participated in training workshops and awareness seminars across different levels.  I come from Egypt, that's why I use the model of the pyramids.  You need to have a broad base of capacity building across different categories.  We need to reach the top of the pyramid soon.  Decision makers really are important to engage.  Because, I mean, I've seen it in my home country.  You can have so much by engaging the general public and definitely we need to do a lot of that.  But also the understanding of the policy makers, the strategists so that these are reflected at the highest level and they propagate and we need to work on it whether it's a top down, bottom up approach, again, we need to understand this.

This is maybe a critical point, to manage expectation.  What do we expect from the other side is very important.  And to define the roles and responsibilities ahead of time is important.  And to build confidence ahead of having a crisis is also critically important.

To give you an example, if you talk to a CEO of a company about cybersecurity, everyone would be convinced it's important.  If you tell them that we need to avail this budget to hire these people and to do the training and implementation, what implication will it have on the operation, it's a different discussion because it will hit on the budget and bottom line.  So engaging decision makers at all levels, letting them know actually what is required from them for these implementations to be done successfully is important.  Like the IGF engaging key players like for instance when it comes to ‑‑ I mean, the computer security response teams is very important.  I mean, at all levels we need to have this type of mobilization towards a clear understanding of the norm implementation and to have successful implementation rather than expanding by adding new norms.

I'm happy with the list of 11 norms, let us focus on getting them done and getting them implemented across the globe.  I'm a member of the African Expert Group On Cybersecurity in the African Union.  You see over 50 countries in Africa, they are eager to implement the norms.  Decision makers need to be engaged and capacity building needs to happen and hopefully we have the IGF, African IGF in the next week, and hopefully the message will be conveyed there as well.  Thank you.

>> JOHN HERING: Wonderful.  And perfectly on time to end this first section of our panel that reviewed some of those large principles from the research paper we put together.  Thanks again to all of the coauthors on that and to all of our panelists for joining us today.  I'll turn you over to the capable hands of Maarten and Sheetal Kumar.

>> MAARTEN VAN HORENBEECK: Thank you very much.  Just as a note for everyone who is attending.  If you have questions you can enter them in the Q&A box in Zoom.  Some of our panelists have already been responding to questions there.  Feel free to share questions there as well.

For the rest of the session we'll dive deeper into a broader perspective of cyber norms and norms in general following on the paper that was presented earlier.  I'd like to kick it off with a quick question that we don't always have the opportunity to ask, but we have a fair amount of government representation in this panel.  I'm a bit curious if some of the government representatives could chime in on the usefulness and implementation of cyber norms in their respective countries and regions.  If you don't mind, Moliehi, I'd love to open it up with you and get your perspective on that.

>> MOLIEHI MAKUMANE: Thank you.  Can you hear me properly?  Yes, okay, thank you so much.  Thank you for the invitation and for being such ‑‑ part of such a terrific panel.

On the implementation of norms, right, I would say for a country like South Africa, which we still categorize ourselves as a bit of newcomers to the broader discussions, the norms discussions in itself have really helped to frame the discussion so that we don't go too far outside the first discussions, but also not to go ‑‑ delve too much into the broader ICT discussions.

So the norms and discussions are around implementation have helped us to frame how we start having the conversations internally.  And for now, it's still broadly in government, just trying to figure out who are the relevant stakeholders, right?  Because there is no template that says because country X has these stakeholders you can immediately transfer it to your own situation.  So it's really helped us in terms of doing that. Implementation, I think what we identified from the beginning was that the norms when you look at them, they seem like very commonsense things.  But when you really ‑‑ (choppy audio)

>> MAARTEN VAN HORENBEECK: I think we may be losing your connection a little bit.  Perhaps if you could briefly try turning off video it might make it a little bit easier.  If not, then what I would suggest is Mr. Isaac Morales, if we jump over to you for a second and we'll go back to Moliehi when she's able to rejoin.  Could you give us your perspective on just in general norms development and the status that your region is in?  Because you did a really good job earlier speaking about the country specifically.

>> ISAAC MORALES: Yes.  Thank you, Maarten.  And actually building upon what Moliehi was saying, in the case of Mexico, the discussion on the norms, I mean, to recognize the norms it was also an opportunity to deliver a broader discussion within the governmental structures.  Most of the time ‑‑ I mean, recent times we're concentrating this discussion among a more technical people if I may say.  The communications minister was discussing these issues.

Nevertheless, the norms bring us the opportunity how the policy level people need to, not only endorse, but to engage in implementation of those norms.  So it has been really successful for us creating an interagency community.  Not a unique or only one agency, but an interagency community where all ‑‑ I mean, technical people and policy level people have representation and they deal with implementation of the norms.  Not only the norms, also the CBMs.  That's an opportunity for me to say it's important to see and to recognize that most of the time when discussing, for instance, at the OAES new CBMs, the border is very thin between a norm and a CBM.

Actually between CBMs and capacity building efforts, this very thing borderline ‑‑ it's important also to make them visible in order to decide if we want to go through the norms way or the CBMs or the capacity building.  Actually, we made an exercise when discussing our national cyber strategy 12 years ago in order to identify which of the CBMs norms or actually capacity building efforts could accommodate us at the national level, you know?

It was interesting that many norms could see also in order to be implemented can see us as CBMs or actually as very concrete actions in order to develop further on what we are trying to do through the norms.  So it's ‑‑ in a general sense, also, I will say that it's a two‑way road where the international agreements we are trying to accommodate international agreements with our national efforts and our national priorities and needs trying to reflect it or make them visible at the international level when discussing the open ended working group or the platform as the OAS, et cetera.

>> MAARTEN VAN HORENBEECK: Thank you very much for that insight.  Pablo, I realize you're not representing a government, but you are over a wider region.  I'm interested in Asia Pacific specifically.

>> PABLO HINOJOSA: Hello, good evening to everybody.  So Pablo from APNIC based in Australia.  We basically speak from the viewpoint of the technical community mainly internet network operators and computer emergency response teams.

We think that international cybersecurity discussions among states at the U.N. have been largely detached from technical considerations.  We think these discussions are far apart from a technical reality.  If we're moving towards sort of from the norm development processing to implementation of these norms, we really think that technical considerations need to be taken into account and we also need to prevent scenarios where eventually implementation of these norms may have unintended consequences affecting the technical operation of the internet.

In this sense, we have worked in parallel to the Best Practices Forum of cybersecurity organizing a series of workshops since the IGF in Mexico basically unpacking different norms from the GGE.  We brought the cyber norms debate in Mexico and then we unpacked the CERT norm in Geneva.  It was an interesting discussion because basically the CERT guys, and you were one of them, Maarten, made it clear that most have not heard about the existence of this norm or any implementation efforts underway around this.

There was this conversation about whether this norm incentivizes or deters SERTS in solving technical problems.  Then later on in the IGF in Berlin, we organized another workshop where we run the norm of request for assistance, another GGE norm.  We ran through it in different cyber attack scenarios.  For example, the cyber attacks in 2007, the Bangladesh bank heist in 2016.  And we looked at these incidents from the perspective of how this norm could have supported response activities to mitigate these incidents.  And the technical community’s assessment was that officializing communication lines can introduce latency, which can undermine incident response and damage well‑established formal networks of trust. These are two recent discussions where the technical community have expressed views about implementation of these norms.  I will leave it there and hope to be able to participate in the next couple of questions.

>> SHEETAL KUMAR: Thank you so much, Pablo.  And thanks to everyone who has already participated.  It's really interesting to hear all your perspectives.  And I wanted to move now to reflecting on the earlier part of the discussion which was around research in other areas of norm development and implementation and where we could perhaps learn from those areas.  We heard about the need for enforcement mechanisms, incentives for internalizing norms, leveraging context, having leaders, entrepreneurs, take on norms, getting norms into existing institutions, cross pollination across different forums and spaces.

We also heard that cyber is different for a number of different reasons.  We heard about the importance of being concrete but at the same time in multilateral spaces how difficult it is to get consensus when you're very specific.  So we then heard about socializing norms across stakeholder groups, the role of regional organizations in adapting norms and defining roles and responsibilities, building capacity before we get to a crisis and identifying resources. So I wanted to come to you and ask which one lesson from norms development outside the cyber realm do you think we need to take at this point considering all of that to the cyber realm.  Move us forward in the norms implementation process.

>> AUDE GERY: Well, I would say there are four reasons.  The first one ‑‑ I'm not going to detail it because we've already talked about it.  In the world of implementation in the construction of the obligatory nature of norms.  Here it comes from the field of international environmental law.  When you look at environment law what you see is you have a lot of obligations that are binding so you have to implement them to see how to give them some binding ‑‑ some obligatory nature because they were very vague.  Or you have nonbinding norms because they're more precise and other states implement them in the same way so it will make states' implementation more almost uniform and make them almost as binding obligations.

The second reason ‑‑ again, it is in the field of international environmental law is the role of the private sector and civil society.  Because it's a field that requires expertise and we see that there are bodies, expert bodies working on very specific issues and guiding, providing states with very technical expertise.  It has been said by Pablo I think on the role of the technical community.  Here the best example is when we had the first provisions on software, we could see that the technical community had not been consulted and the provisions were and are still very badly drafted.

The third lesson comes from the field of nuclear activities.  It is mentioned I think in what cybersecurity policy making can learn from nomadic principle.  It's the convention of nuclear safety in the convention on the physical protection of nuclear material.  What's interesting there is the wording that is being used in these treaties.  Because the wording recalls the U.N. resolutions on the creation of the global issue of cybersecurity.  It sets obligations in the form of objectives in terms of legislation, in terms of safety and security obligations, in terms of criminal responsibility.  It is precise in terms of objectives, but it offers flexibility in terms of how states can internalize the objectives and fulfill the obligations.

Finally, I would say that the fourth lesson comes from both space and nuclear law as well.  Here it's related to the concept of peaceful use and the implementation for international cooperation in terms of technology transfers and the implementation for the meaning of the concept peaceful use.  I think we could work and try to have a look at how things work with this concept.  How it is being implemented by states to see what we could learn for cybersecurity and to maintain a peaceful and stable cyberspace.

>> MAARTEN VAN HORENBEECK: Thank you very much, Aude.  One other topic we want to discuss with this panel is to understand from your perspective what is the main challenge is from cyber norms implementation today.  Are there different areas in which different stakeholders should be pointing their attention.  Moliehi Makumane, since you were rudely interrupted, I would love to have you give us some insight.

>> MOLIEHI MAKUMANE: I'll keep my camera off so I can get through this.  Okay, so I think the main challenge for implementation is the lack of coordination, internationally as well as guidance on how to do coordination domestically.  I can't say that we don't have guidance on how to implement the norms anymore.  I think the open-ended working group was a very useful platform to share an exchange of views on how different countries implement the norms.  Now I think the big challenge is how do we coordinate for those norms that don't just impose obligations on states, how do we coordinate with ‑‑ (choppy audio) ‑‑ with academia or on government coordination.  Are you losing me?

>> MAARTEN VAN HORENBEECK: Yeah, unfortunately, your connection is a little bit tough.  Maybe if you wouldn't mind sharing your contribution on the chat.  In the meanwhile, I'll hand it over to Stephane who also had something to contribute to the topic.  Sorry for inconvenience.

>> STEPHANE DUGUIN: Very quickly, the report that we look into for the session is a lot of very, very interesting highlights for that specific question.  I would advise everyone to dive into them.  And this being said, two thoughts.  The first one is what prevents the operationalization of norms practically speaking is linked topics and discussed in different forums.  You have silos that are coming from all ‑‑ or the groups of interest.

Today you have the discussion on arms control.  And in between you have the concept, which is automized weapon.  Automized weapon, if it's a piece of code that you put metal around it to make an attack, then it's cyber.  So that doesn't help any actionable implementation done, operationalization done down the road.  That's one example but there's many.

The link in between what we call norms here today, silent norms, when it comes to more protocol.  Real norms and cyberspace, and this was mentioned earlier, is when you address protocols on the internet, we have the global effect way faster than in any working group discussion to date.  So that's something to make sure that the two are discussing each other because they are two sides of the same coin.

The last one, maybe the most important is grounding of operationalization of norms.  They're not in a vacuum.  People are today suffering because of cyber attacks.  Whatever operationalization needs to be grounded into this fact checking, evidence led human centric assessment of the impact on society.  Thank you.

>> MAARTEN VAN HORENBEECK: Thank you very much, Stephane.  Pablo?

>> PABLO HINOJOSA: I'll talk a little bit about the gap.  I think it's fair to say that processes are not yet understood the same way around the world.  There's not the same understanding among states.  Not to mention other stakeholders.  We're still not fully involved with this yet, including as I've said before the technical community.  I mean, we can see how the most important normative approaches, for example the GGE process, it's been mostly triggered and developed and motivated by a small group of states.  Most of them have already thought deeply about these issues and have national cyber strategy in place, have evaluated scenarios and understand well what is at stake in a very clear way.  That's not the case of most states, even though they might have agreed in the General Assembly with the GGE recommendations of 2015.

I mean, if I translate this, what Apratim and Anastasiya was talking about, these leaders have not been able to sufficiently propagate these norms around the world.  But the point I want to make, actually, is how important they work, the best practice forum on cybersecurity has been.  Precisely in performing an excellent inclusive approach and to raise awareness about these norm developments and operationalization processes.  It has succeeded in the last three years to bring new stakeholders into the discussion and socializing these important lines of work.

I also want to just conclude by reflecting on how ideally suited the IGF is in terms of promoting this line of work because it's really a space that brings people together to increase understanding.  So if you want to maximize the chances of success of these norms, it would be very important to have the IGF and its evolution in the next years to come in the radar of this important processes that has still been very exclusive with only very few actors.  I'll leave my remarks there.

>> MAARTEN VAN HORENBEECK: Thank you, Pablo.  Louise Marie also wanted to share something?  We just have one minute left.

>> LOUISE MARIE HUREL: I want to follow up on what Pablo said.  He put it nicely in terms of what are the types of expertise we need to operationalize these norms and who needs to be on the table at the different levels, at the international, regional and national level.  I think the GGE and the OEWG have been very interesting and in some ways taking the discussion on building consensus to a new step.  So I think at the international level what we see is like these attempts to build consensus.  But throughout this past year how do we trickle that down.

What we've seen in Brazil is how you understand how these discussions are going at the international level and at the national level you don't have clear roles and responsibilities.  I want to close with that and how do you actually trace those international norms to the national level and vice versa.  I think just to end with, also, a note about the role of the BPF in all of this, I think the BPF has done an incredible job of actually tracing and starting the conversation on tracing these norms and where they're at and which stakeholders echoed similar norms, even if they didn't use the same language, even though they didn't articulate in the same words.

For example, the ethics first framework, norms, they actually talk about vulnerability disclosure, while you see that over in the GGE.  How do you actually tie ‑‑ I'm not trying to draw causality over there ‑‑ but how do we actually kind of connect these siloed conversations?  So I just wanted to end with that.  I think the BPF going forward and many other organizations have a really important opportunity to kind of like continue this process of tracing, which I think will make it clear where do we need to talk about rules and responsibilities.  Whose expertise do we actually need to kind of like bring to the table to make that possible?  I think the CERT example is a notorious one.  It's not about attacking incident response teams, but I hope we can see that in other norms as well.

>> SHEETAL KUMAR: I think we'll pause on to Markus very quickly just to wrap us up because we are running out of time.  But I wanted to say thank you to everyone for their perspectives, especially at this point.  You know, all your ideas and thoughts on what is lacking and what the challenges are can help to shape what the BPF looks at next, for example.  And also what other stakeholders, those who are participating here and will access the session later can think about when it comes to norms and implementation which is so important.  So whether it's greater coordination, linking topics across forms and issue areas, addressing gaps in understanding amongst stakeholder groups or what you've just said there, Louise Marie on roles and responsibilities.  There's a lot more to be done.  For sure our homework is not yet complete. I'd hand it over to Markus now and thank you again so much for your participation.

>> MARKUS KUMMER: Thank you.  Note this year, the BPF did not react to that.  But as we started our work before the road map was out, but I could turn it the other way around, it is actually the outcome as a result of the BPF, does it fit into the road map.  And there the discussion is still open.  I'd like to draw your attention that this year we had an exercise of BPF on BPFs that should feed into the next year's presentation of BPF's.  I posted the link in the chat.  And there we have a few suggestions on how to improve BPFs and that is also part of the broader discussion of the IGF.  I will leave it at that in the interest of time and thank you to our lead experts, Maarten and all the panelists for their excellent work.  And also, obviously, the participants.  It can be slightly frustrating in the webinar format as you never know who is in the room but this time, we had excellent participation.  At one time we had 100 participants in the room and we still have 79 left.  Thank you all for your participation.  Bye bye.

>> MAARTEN VAN HORENBEECK: Thank you very much Markus, and thank you everyone for attending and have a wonderful day.  Please read the papers, join the mailing lists and do continue the discussion together with us.  Thanks very much to everyone who contributed and everyone who attended.  Bye bye.