The following are the outputs of the real-time captioning taken during the Fourteenth Annual Meeting of the Internet Governance Forum (IGF) in Berlin, Germany, from 25 to 29 November 2019. Although it is largely accurate, in some cases it may be incomplete or inaccurate due to inaudible passages or transcription errors. It is posted as an aid to understanding the proceedings at the event, but should not be treated as an authoritative record.
***
>> MODERATOR: Ladies and gentlemen, it's 9:30. It may be that some people need more time to find the room. Let's start and at I am happy to welcome you here to the joint session by the Dynamic Coalition on the IoT, and the dynamic coalition on CIV. Chaired by Olivier Crépin‑Leblond. The second one and a half hours basically on ethics aspects. With that, I'm trying to get the PowerPoint launched. As there are always new people in the room, a little bit, a little bit of framing very quickly because at least half of you have seen this already.
Basically when we talk about the IoT, it's important that we understand we talk about all of the benefits it brings, how much we need it, how much it's woven into society already and how much that will continue to be so to address big societal challenges, such as more intensive use of our space, such as people growing older, such as environmental challenges, extreme weather, et cetera.
But also, that, of course, it does come with threats as well. Technology is never good or bad in itself, it's how we use it, and specific technology enable new good things and new bad things. So this is why eight years ago we started with in 2008, so that's already eleven years ago we started to address this at the IGF at a global level above silos because there is where you look at the issues specifically.
It doesn't mean it ends there. Its just a global view that can be taken forward locally and into specific sectors, and that is increasingly done so and we try to reflect back the best practice from that and reflect forward what we think would be from a global level to address global issues, the real challenges. Here at IGF specifically, it is very good to realize how much this relates and interacts to many of the Sustainable Development Goals that have been expressed to be achieved.
Zero hunger, I just pick out a couple, zero hunger is where you see that IoT helps a lot to increase crops returns, crop management by measuring how much moisture is needed by measuring which insects are there to fight by really getting more out of the country. And that's important to also have that in a way that is affordable.
So it's a come bibation of affordability, not only in the western countries, but all over the world and that goes on to clean water where you can imagine that water measuring helps, but also sustainable cities where you can imagine that even traffic management is important. So all of the different applications that have emerged over time provide a lot and bring a lot ranging from tsunami boo I measurement to predict earthquakes to Google that helps you to connect to databases with your cameras, to drones, and even to stuff that now days we can carry in our arteries to measure our health.
So the good practice principle that we have been working on and that seems to reflect a good balanced view from an IGF perspective is that it's about taking ethical considerations into account from the outset, and find ethical sustainable way ahead using IoT to help to create a free, secure and enabling rights environment, the future we want.
Consciously working towards the future we want. In summary, the thinking of the DC IoT is we need IoT to keep this world manageable and to create an IoT environment that also encourages investments in this to benefit from this is to invite all stakeholders to create a healthy ecosystem that automatically stimulates the healthy use of those tools, and the feedback loop of awareness is crucial and provide legal clarity.
Yesterday in the main room we had a discussion about legal colleagues of law that would like to be there. Much is coded law it's just not coded for IoT environments. So to ensure that the IoT environment is developing in a trusted way, a way wear comfortable with or as comfortable as we can be is that the development comes with transparency that is meaningful, that is understandable. Loads is technical information, the right information to the right stakeholder, clear accountability, who is responsible. For IoT use, for instance, in a car or in a plane, it's very clear who is responsible for the functioning. It's the manufacturer, it's the maintainer of the system.
And last but not least, real choice. So we can see that there is safer and less safe options, but it's good if you understand the impact of that and in the end it's people making choices to move things ahead. So, again, part one that we start with now momentarily is prerequisites are important from a security perspective and what needs to be done to support a secure Internet of Things globally, across silos and geographies.
After that we will focus, so that's after 11:00, 11:30, to these factors.
>> OLIVIER CRÉPIN-LEBLOND: Thank you very much, Martin, Olivier Crépin‑Leblond speaking and the second part of today's double whammy as one would call it or double trouble, I don't know what the real name of it is, the second part will be taking on the discussions we have had in the first part focusing on the IoT and looking at the core Internet values which is a Dynamic Coalition that has been going on quite some time looking at what some people call the Internet invariants, the technical values on which the Internet was built on.
We will have a presentation and review of the values at the beginning of part 2. I think what's important is to try and get as much cross polination between the two, the part one and part two. So we will be taking notes of the points that have been made in part one and picking them up in part two looking at whether any IoT related discussions we have had might break core Internet values. There are some people out there that believe that the IoT is not the Internet. It will be interesting to see if actually some people have this view here and if others have a countering point of view, and then after that, we will be looking at how we can progress forward, perhaps collaborating further with the DC IoT and the DC core Internet values.
We have a number of additional panelists coming later and I invite everyone who is here to remain for the full, is it three hours until 1:00, and we hope to be able to entertain you and perhaps even provoke you into providing some input at some point during the morning. Thank you.
>> MAARTEN BOTTERMAN: So and with that, our Rapporteur Ryan, and he will take over for me after five years of dough Chairing the Dynamic Coalition. I'm glad to have you here volunteering for that. It's been a pleasure. And as in previous occasions we have Avery as our moderator for the first part. Avri, the floor is yours.
>> AVRI DORIA: I want to welcome Shane Tews. Having passed the Chair to Maarten, I'm happy to have him pass it on. My name is Avri Doria. I apologize I'm only going to be here for the first hour. I ended up being double scheduled and will have to run to another Dynamic Coalition on which point Olivier Crépin‑Leblond will take over.
In the first part we will start with a couple of statements from a couple of folks. First of all, we will have Frederic Donck from the Internet Society, then we have Merike Kaeo, then, and Marco Hogewoning from RIPE NCC and Max Senges from Google will speak and then we will basically open up the floor for comments, discussion, what have you. So, Frederic Donck, please.
>> FREDERIC DONCK: It thank you, Avri, I'm happy to be with you today, but I too have to apologize because pause I have other schedule at 11:00 because I will try to delay this because he said he would keep entertaining us and I'm very interested to talk about the current values and variants. The last time we spoke about this was in Paris where I was giving you the overall approach that Internet Society took when it was about the IoT, which is indeed when we talk about security a threat to users, but also to the network. This is where we are entrusted, of course, as well.
So the plan as we developed the last two years was to have a three fold approach, three big avenues. The first was to approach manufacturers and resellers with the objective of having manufacturers embedding security by design in their IoT device.
And for this, we were proposing the online trust alliance principle 40, principles that we trust would absolutely solve issues that we have met in the past. Example are still very vivid for some of you, like the botnet et cetera. The other avenue was an approach to consumers, make sure consumers understand the importance ever security. The simplest example is people buying precisely in the mere ibot case the camera and you just don't change the password, allowing lots of people to use your device from outside.
And then the third was indeed approaching policy makers. For this we have also developed policy papers on privacy and security which I would invite you to check. So now I'm happy to report that after the last actually 12 months, we were able to, I believe, achieve some very entrusting progress here starting with this approach to consumers and awareness raising. We have, you might have seen this, agreed with consumer international in Mozilla on five key principles, minimum principles we would like to see taken not only taken into account but implemented by manufacturers.
So that was a very important step. I might recall you what those five minimum security standards were. Again, you will see that Mozilla consumer international Internet Society are really strongly promoting those standards as the minimum, encrypted communications for any device, security updates, that is the minimum manufacturers should give the opportunity to consumers to know whens the next time your device will be updated, strong pass words. And, of course, we hate password by default. Strong password also be to a minimum for security, vulnerability management and privacy practice readable by everybody should not be a graduate from Harvard to understand the privacy settings of your IoT device.
From the policy side, we also led some effort with who we identified as governments who really want to take the lead, so we spoke to DCMS in U.K. to the Uruguay governance to France with ARCEP, to Senegal in Africa, and we succeed to have, to build a platform of policy makers who want to continue promoting principles at the worldwide level. And, of course, I forgot Canada with which we also had a true multi‑stakeholder process that we have led for the last three years with the Canadian Government but also the technical community, businesses and we have coordinated this.
And, again, I would invite you to check this Google Canada and IoT and you will see that there are concrete proposals for standard, security standards for the IoT. So this platform that I mentioned is existing. It's composed of the Governments but also other actors as Mozilla that I named and it's been articulated on three strong principles that everybody recognizes, and that would be ensure that security is incorporated in all stages in the design of the device, ensure that the personal data and critical data are being protected and finally giving the ability for users to delete their own data.
More concretely, you will find that those principles will be concretely implemented in vulnerability disclosure policy, mechanism to securely update software, et cetera. So this is a very important step because now we have very prominent stakeholders including Governments who support this approach and continue to do so on the platform. So that was in a nutshell what we have done the last 18 months within the Internet Society, and I believe this is, well, not the end, but this is the beginning of ensuring a more strong approach to security standards for IoT. Thank you, Avri.
>> AVRI DORIA: Thank you, Frederic. And next I would like to ask Merike Kaeo to take the microphone.
>> MERIKE KAEO: I'm liaison to the board. The SSAC undertook some work to look at how DNS and the IoT kind ever interact, and what we have largely found are that people don't really pay that much attention to now the DNS either gets implemented or really think about how to secure it. And because DNS can also be used for a lot of harm on the Internet, there was a paper that was created. It's called in short, SAC which stands for basically an SSAC advisory number 105 where we talk about the opportunities, risks and challenges.
So a lot of measurement studies have shown that IoT devices use DNS to locate remote services. There is also work in the IETF, something called Mud, manufacturer user devices, I believe,, where there is ongoing work where you may not have to use the DNS, but the DNS is used either by devices themselves if they have enough power and processing power and all of that, or they might use a gateway to translate names into the IP addresses, the numbers so that devices actually know where to get their services and who to communicate to.
Now, the opportunities here with the DNS is that when you are looking at security aspects, you can actually create more stringent security stability and transparency requirements and I will get into a little bit of that in a minute. Also, there are risks because if you don't implement DNS correctly, you can accidentally stress the DNS where if you have a large number of devices coming on let's say simultaneously after a power outage or on purpose if you are nor the that familiar with DNS and either implement it wrong, there is also a challenge in terms of how can the IoT industry seize opportunities and address these risks.
So if we look at the opportunities in how DNS can help actually protect and add more security, there are a lot of protocols and mechanisms where you can add privacy and confidentiality to somebody that's initiating a query, let's say, a device that wants to go to certain services either be it healthcare, be it anything else that might, you don't want somebody to be aware of who is growing what service.
So looking at whether or not you are going to use some of these privacy mechanisms when you are using the DNS. There is also using something called DN SSAC which provides added security when you are using DNS because it makes sure nobody can create a fraudulent site that it will redirect you to, which is the cause of a lot of nefarious activities. You need to pay ain how you register domains and which you use, and when you are thinking about credential management as the previous speaker mentioned using pass words. It's much better to do multi factor authentication. There are way too many breaches to actually trust a pass words these days. Multifactor authentication will give you a little bit more security.
And when we are looking at the risks to the DNS, there have been situations where there have been DNS unfriendly programming. So years ago, there was an application called tune IP where they had a software error and random queries filled a lot of the, the resolvers, so basically the devices that would again give you back the information in terms of where you needed to go to and where your applications needed to communicate to.
And so it rendered it unusable. And the problem was that with this particular application, it took days before they actually had an update. So a lot of services were down. There is also, there has been larger and more complex service attacks. So the DNS can be used for something called amplification attack where if you are sending only one piece of information, one query, the way it works it may send one thousand replies back. That's amplification, so if you are improperly implementing DNS, you can cause amplification attacks.
So we want to pay attention to how you are implementing DNS. So the challenges for the DNS and the overall IoT industry is that we may want to foster specific libraries or fundamental programmes that a lot of IoT devices can use to actually be more secure, because one of the things that's become very clear is that a lot of issues come from software also, and when you are dealing with complex protocols, DNS is seemingly easy, but actually quite complex.
And so are security mechanisms. It's easy to make mistakes when you are doing software implementations. So wouldn't it be useful to maybe foster some opportunities to create these fundamental programmes and libraries that are created by not just a single entity, because you don't want a single point of failure, but some trusted entities that are really vetted and thoroughly tested that can then be utilized.
And also providing training to IoT and DNS professionals so people really understand the fundamental of the Domain Name System, how it works and the intricacies, so when they are implementing the DNS in their IoT devices they don't make mistakes that can cause instability and insecurity in our Internet Ecosystem.
I would recommend or hope that people will look at the paper because it does address quite a few interesting points regarding the Domain Name System and what needs to be Pide attention to when we are looking at the IoT, and it's called Andrea Saks 105, the DNS and the IoT opportunities, risks and challenges.
>> AVRI DORIA: Thank you, Merike. Please at this point, Marco Hogewoning.
>> MARCO HOGEWONING: It just call me Marco, please, so I work for the RIPE NCC, and you might have heard of us, we are the regional Internet registry for Europe and Middle East and Central Asia, so we are mostly famous for handing out IP addresses and distributing and managing the IP address space. Outside of that, and that's mostly why I'm here, RIPE NC provides the Secretariat services to the RIPE community which is broad multi‑stakeholder group of European Central Asia and Middle East network operators, service providers, but also other stakeholders, Governments, law enforcement and they meet regularly and their own discussions about mostly network operations and operational aspects of running the Internet.
So mostly be addressing it from that perspective also and also to hook onto what Merike says were one of the DNS operators so we operate the roots consolation. Taking it from that perspective, it scares the hell out of me. Not only because, yes, of course, looking future security especially when you look at smart transportation, industrial IoT, yes, there is a high steak risk there, but also my feeling is that that is part of the IoT is probably easier to control.
We have better controls over the value chain in things like GAR, in things like industrial automation than we have in the domestic area. That's where I really get scared and that's also where a lot of the discussions in our field focus is the stuff you buy right now that is online that you come home and that you plug into your WiFi because not only it poses a risk to you, where my community is mostly concerned about and we saw a brief sample with the Meiri attack is how a lot of these devices can be used to attack and undermine the core infrastructure of the Internet.
The IoT will in the worse case scenario attack the infrastructure it depends on itself and become self‑destructive, that's the worst case scenario. What we have been mostly focusing on is that many solutions exist, like Frederic said, we have high level principles, there are people talking to the regulators, whether that is market access, get in front of devices in the market or not on the market in the first place, but also towards what is the role of the access provider here because when it comes to safeguarding the rest of the network, the rest of the net from what happens in your home WiFi, a lot of people immediately start looking at the access profiler.
At the same time, what I hear from the access profiler, we have things like Net Neutrality and we want to support an open and free Internet. We want to be able to allow people to innovate, so we don't really want to police that border. But bottom line, these devices are out there now once they get sold, they will probably live for another five to ten years so it is kind of urgent to do something, and probably it can't be a single actor.
So what I hope to get out of this discussion a bit at this level is also like everybody needs to do their thing, but we all have to sort of work in conjunction. Share market devices might also be better secured but also discuss from common service make sure that what happens in the home stays in the home and does not contaminate the rest of the Internet, and, for instance, attack my root DNS server and cause much disruption to everything we do in our daily lives including those industrial automation systems.
So that's a bit of where we stand. It's like, yes, it's really 2 to 12. Some might say it's already too late in you look at the amount of crazy devices these days that we see on the market and that we know have known vulnerabilities, and as a set it's many solutions that exist but also solutions cost money and somebody has to foot the bill which is another problem space there. It might be easy to implement software, but eventually it will cost somebody money.
So that's sort of the practical operational side of things. We need to do something, and we would rather do it today than tomorrow. I will leave it here for discussion questions.
>> AVRI DORIA: Thank you, Marco. And finally of the preprogrammed speakers, max, please.
>> MAX SENGES: Good morning, my name is Max Senges I work at Google and have the pleasure of joining most, I think, of the dine annual ache coalition meetings over the years. I bring the perspective of Google which is, of course, also in that DNS sphere, but the comments I make are mostly from our hardware perspective, so the home devices, et cetera. And luckily for me the preparation for this was fairly easy because we helped organize a workshop yesterday on IoT security. So rather than bring you something that's only from one perspective, I'm happy to report from that workshop and what we discussed, and Maarten was one of the panelists so he can keep me honest if I'm adding things or leaving things out, please, Maarten do so.
Int gov Wiki.org we use to both plan our session and update it. It's significantly easier than updating the official INT gov Forum website if you had to try that. And so, yes, we already uploaded our notes from the workshop. I want to jump straight to the conclusion if I may because similarly to what we heard here and actually I started email following up with the panelists because this is, of course, very complementary, I didn't know about the ISOC and Mozilla initiative, for example, and I think that's very much thought on, and IGF mod initiative stands for manufacturer usage descriptions, MUD, really useful materials to build on. So I think before I go into the substance, I want to point out that this seems to be a space where the wheel is reinvented on a regular basis.
I would encourage us to see if we can maybe consolidate and bring together a couple of efforts and really move forward the security in this space as pointed out, it's really essential, actually, that we are all safe in contrast to other failures in the technology sector when we are talking about IoT, we are talking about devices that can burn and crash and do other harmful things.
So I think it's particularly adequate to start with safety and security. So on the one hand there were no surprises. We said that a mix of governance tools, technology, awareness raising and literacy instruments need to be developed and deployed to promote security. Importantly regulations should be technology independent so we shouldn't try to go and try to regulate individual devices. It happens over and over again. Of course, it is about the technology, it's about laws that enforce the security standards because sometimes that is necessary as I think is one example is the update of operating system software on mobile phones and other devices where the manufacturers just simply don't update the OS and the patches because either the devices don't even have that capability or it's simply a cost to a device that has already generated its value, it's revenue nor the company and ‑‑ for the company and incentives are obviously not high except for the good reputation or if it is indeed say violation of the law to update it.
A suggestion came up that literacy and making coding and security training part of the normal school education, an interesting one is to promote independent testing. In Germany we have a test which is basically a consumer protection group that is testing different categories of devices, in this case not necessarily developing political pressure, but publishing the results and then when you see which devices are tested well from an independent source, that's, I think, a really interesting market mechanism to promote good security practices.
Promote security by design including human rights considerations, and the NIS directive on security by design was pointed out, and the idea that I personally like a lot and that seems like a good multi‑stakeholder opportunity is to come up with a sort of nutrition labels, which you have on all of the food, to identify what quality, what ingredients in terms of security and also maybe usability and aspect a device has. I would invite if anybody is interested to make that happen, I would invite to have a conversation after this or we can have a conversation with a Chair or even as part of this communication.
The proposal for best practices that were distill the was obviously, you know, one of the most dangerous things is to fault user names and pass words and hard coded ones. There is plenty of evidence that that is really a bad idea. I'm surprised that that's not illegal at this point actually. Coordinated security updates, I have worked for several years on IoT interoperability, and, you know, to make the things work together in the first place is hard enough. To include the security and make that dove tail is even harder. In fact, it makes the interoperability of the usage harder that we are already struggling with. So I think the second point is really about cross, cross stakeholder interoperability between device manufacturers, the ISPs, the operating system provider, the various software pieces that make up a device.
And then those devices should be updated automatically on a regular basis, and also something that I don't think is very widely spread but please correct me if I'm missing something is a concrete end‑of‑life plan. People buy devices and think that they last forever. They usually don't, and especially in terms of updates and expectation management for users, you should be aware whether you are buying a secondhand device, is that even still on an update plan or is it already outdated? Those are some of the ideas and arguments put forward by the participants of the workshop yesterday. Thank you.
>> AVRI DORIA: Thank you, and thanks all four for sort of plenty to worry about, plenty to think about, plenty to be concerned about whether it's the device in your home is going to blow up on you or destroy the network or what have you.
I would like to open the floor now to people that would like to question, like to comment, like to say something. Yes, Jimson Olufuye, you are reaching for your microphone. And please make sure you introduce yourself with your name, easily understood so hopefully it gets put in correctly.
>> JIMSON OLUFUYE: Thank you very much, my name is Jimson Olufuye. Contemporary consulting. The remarks are excellent when it comes to IoT and security in the future. I would like to talk about a few improvements possibly to see whether you agree with me. When it comes to security, it's so key, we have said security by default. Well, how about saying also that without the change of that default security, that device will not work because the default is will be known eventually, you know, in some cases.
So without it being changed, it should be possible that the device should not work. So that is one. Number two, when you get to accountability, yes, the possibility of destruction or the IoT destroying something is there, but we need to have in mind that, yes, as Maarten said there should be clear accountability that it is in our self‑interest that these devices should be accountable to us. We need to have that clear understanding, with regard to the design, with regard to the usability, that come what may, the devices should be accountable to us.
So if we have that mindset, then in terms of the design, there will be no loophole, you know, that will get the equipment out of hand. And then also predictability, so this should take us to where we can predict. So the concept of ‑‑ no, it won't be there. It should be predictive. And there should be no leaks in terms of resilience, there should be no leaks and no leaks or back hand.
Well, this goes into big data anyway, so we need to be sure that these devices as I said are accountable, and no matter the amount of data available for them to per mutate and do probabilistic activities or produce some outcome, no matter what, the outcome must benefit us and must enable progress and not reduce the progress in the society. Thank you.
>> AVRI DORIA: Thank you, Marco you wanted to comment on that.
>> Yes, accountability, but that triggered my mind in something that I think is Human Rights estimated (Marco Hogewoning) and not discussed anywhere, and it goes to max's point about device lifetime is the traditional way of punishing bad behavior on the Internet is disconnect, is a full disconnect. The user really messes up your network, you go and you unplug his DSL or fiber line until its solved.
There are two aspects there. A, with the IoT further proliferating in everybody's life I may not have that capability from an ethics perspective because disconnecting your house means disconnecting all of your devices, your security systems, your medical devices included. So we might lose that capability, but at the same time, we are still not really discussing alternative approaches.
If I know a certain class of devices is unsafe, if I know that a certain class of webcams is responsible for attacks, are there effective means to take them off line selectively but also en masse where you can basically walk into a vendor and say either you fix your software or you take all of these devices off line wherever they are, which also is from an ethics perspective a huge risk having such a massive red button.
I haven't seen that anywhere discussed. We talk about it, accountability, we need to secure these devices but we haven't sort of thought about the response if it turns out to go the wrong way. What do we do? How do we shut down whatever is causing the trouble? That's going to be a bigger problem as I said, because we become more and more dependent on that connectivity.
>> AVRI DORIA: Thank you. We have a remote comment, question.
>> Yes, I can read it, it's referencing an earlier speaker. The question is IoT devices mostly have a cost of less than five dollars and often two dollars. More will be built in China, most will be built in China, sorry, can we do what you are describing at that price? Do we have to include China to make this work?
>> AVRI DORIA: Would someone like to answer that.
>> MAARTEN BOTTERMAN: This is a good point. So far we also talked about IoT as everything IoT is one and the same thing and, of course, it's not yet we had the discussion, particularly with Google as well. For instance, tags are also part of it, right? It's the identifiers for even food now days. That goes into the system. And that is, of course, a different level than what we used to refer to as active IoT devices which are those that are powered, that are actually broadcasting, collecting, et cetera.
So I think that distinction is very important to note that, of course, not everything is equally sensitive and can afford equal security. On the other end of the spectrum, we know that some of the IoT devices we don't need regulation or any pointing at because they are part of a bigger thing like a car, and the car manufacturer is responsible for having it properly integrated a device is liable. So it's really in that space where you need to find a way forward.
As Marco said, it's also not always the device manufacturer that is responsible for things going wrong. It may also be downstream. It may be even the user, although we shouldn't expect from the user to do things that are above his ability. So I think the question is very good to make clear that IoT is not one blanket name for all devices. It stands for connectivity to the Internet of different sorts of devices.
>> AVRI DORIA: Thanks, I have all four of our previous speakers wanting to comment, so, max, what would you like to add, please?
>> MAX SENGES: It's almost more of a question than an answer, but what you just said made a lot of sense to me about the different classes of devices, and I'm actually fairly surprised, at least I don't know, but maybe someone else does, why don't we have different classes of devices like standardized classes of devices from passive to, you know, very unsecure little thing to an air bus plane. I mean, there is a big spectrum of different devices, and we have individual certificates and things that indicate the level of security, but not just the very basic breakdown of things so you could even say I only want to connect to safe things or only to certain class of devices. It seems to be a very useful distinction.
>> AVRI DORIA: Merike Kaeo, you wanted to add something.
>> MERIKE KAEO: Yes, and I will build on what max was saying. In had numerous discussions over the years I have heard about classification of different devices and it's critical to understand understand that security needs to be looked at holistically. So every single device in my home does not have to have security if I have separate networks VLANs in a more technical term and funnel it through a gateway that will then handle all of the security items or protect my home with all of the different devices I may have. This may be true for other IoT‑related aspects also. So I think it's very, very important that we actually classify the devices where they are used and what other mechanisms can be utilized to provide the security functionality that actually would not be a realistic thing in the device itself.
>> AVRI DORIA: Thank you, and Frederic, you wanted to add something.
>> FREDERIC DONCK: Yes, but some of the speakers provide on what I want to give. But building on what Merike said, and I agree with you, many times when we are discussing IoT, people refer to Smart devices and I couldn't agree less with this reference. In most of the cases, the devices are just fairly stupid. Those are just devices with a capture, they receive it, they resend it but the real intelligence is in the Cloud.
So in most cases we need to see IoT device as stupid device but there is an ecosystem where intelligence is. This is why I believe that we need to address issues of security with that ecosystem in mind. The market is relative. The question was about device brought in from China at a very low level indeed it's easy to squeeze a chip of one dollar into a doll and call it a connected doll and what is it that you do with this.
Regulation won't be able to address this because each time there will be regulations, there will be another product that will escape that regulation and even that classification. So I believe we needs to address the whole ecosystem including the fact that you are connected to the Internet and this is why I agree with Merike Kaeo.
>> AVRI DORIA: At first I thought I had lied about all five of the previous speakers wanting to but Marco has come through with a request to comment to make me a truth speaker again.
>> MARCO HOGEWONING: Thank you, Avri. This is very briefly because I do indeed think that classification might help. We also it's a bit of like what is mentioned is part of doing that. It kind of makes sure that a device is limited to a certain level of capability and can't step out of that boundary. It is helpful, but I don't think it's the only solution, and you also mentioned air bus and I'm not at liberty to discuss any real details but from conversations I had with especially the Aerospace industry there are many threat factors, many vectors there, but one of the principle points there is that one of the biggest risks is the maintenance engineer's laptop because he has full access to the systems.
You can design your aircraft as safe as possible and super secure, but somewhere some guy is going to step in his with windows or Mac laptop and plug it into the core of the system, and that's a massive security breach from a gap perspective or classification perspective.
Back to your point about device cost and I mentioned that in my open segment, somebody has to foot the bill, and it might sound strange to come out of the mouth but I do think regulation is the only option. We have to set minimum standards and minimum standards have a cost. If you allow the cheap device to be on the market. People will buy the cheap device no matter what level we put on it. If we think it is unsafe, it should not ender the market and there are compliance regulations we can enforce there.
>> AVRI DORIA: Thank you, yes, Lori.
>> LORI SCHULMAN: Lori Schulman, International Trademark Association. I just have a question about classification. I find it really interesting. So you are talking about ISO classification? Are you talking about industry classification in and of it several? Are you talking ‑‑ itself are you talking about the end user in terms of what may be in the home in terms of ranking security preferences?
I find, as I said, I think I would be interesting when you talk about classification what levels we are discussing?
>> I didn't pick and choose either of those. I just thought that from the conversation and from Maarten's answer that seemed like a useful path to go down. What do you think. (Max Senges).
>> AUDIENCE: That's what made me start to think about it? Where do we go? Do we have international standard setting for this classification? Do industry take it upon as a socially responsible practice to classify, and then getting down to the end user itself, you know, in my home, which of my devices I consider high or low risk or preferenced or not preferenced in terms of how you switch it on and off. I do get concerned as an end user when we talk about core values, human rights, this idea that I may not be able to opt out at all. And if I can't opt out at all, do I even understand what that means in terms of labeling? Is the label going to the manufacturer, the engineer, is the label going to the end user? I see this as multitiered and multi‑levels based on certain levels of risk.
>> AVRI DORIA: Thank you, Lori, and I have been a terrible moderator in not calling on people by name so the transcriber has been losing it. So basically, I had Merike and Marco and Shane wanting to comment on this. So, okay, so Merike, please. And do give your name again just to make sure we get the transcript right.
>> MERIKE KAEO: Thank you. It's Merike Kaeo. And I think the classification isn't as simple as it might seem because the criticality if you have an Internet of Things device that can be both in a hospital or in a home, I think then you have to think about, well, how does it get used and where might the criticality change because of where it is used? I know there have been different efforts of classification, and one of the things that's a challenge is to see globally who is doing what, who is doing these classifications and how are they being utilized because it creates confusion if you have got different islands creating different classifications that then if I who is a global entity buying these items, let's say I'm a corporation a some, you know, energy sector entity or car manufacturer, who knows what, that really it would be nice to have some kind of a cohesive standard so that also when you are certifying these devices, there is cohesiveness because right now we don't have that yet.
But who is going to be the body that will provide that cohesiveness? That's also something to really think about and define.
>> AVRI DORIA: I have Marco, Shane and then Gunela.
>> MARCO HOGEWONING: The there are several certification schemes in order, from European perspective everything is to have that FI mark and different classes of products have different requirements. What I think is also good in mind and Merike whispered the error word to look at Internet readiness. It's a cost. Before we sell a car, we throw a few against the wall to see if they live up to security standard. Most of the IT certification is a paper exercise, and destructive tests are expensive and it will ruin your time to market, but maybe we should sort of throw a few devices against a virtual wall before we put them into the market and sort of demand real testing to happen, for instance, and make sure it's not only on paper it's secure but according to the current threat level, we actually think it's secure enough to hold up to whatever we can throw at it.
>> AVRI DORIA: Thank you, Marco. Shane, please.
>> SHANE TEWS: I like the idea of classification to your point, Lori, consumer items versus things that are happening in the enterprise or industrial area. We have had a couple of pieces of legislation that I reviewed in the Senate, and one of the challenges I had with the more senior senator was explaining to him that you can put a seal on a box, but the minute that thing comes out of the box and there is an update, we don't know where it is in the system. We know where it is in its lifecycle. So I think you guys have made a lot of good points I hope we will follow up on this year and so I hope you all continue to participate.
I have a question, because I have the microphone, is there a, any information sharing going on globally? I'm thinking about the way we use industrial ISOCs, but the problematic activities, I think you said, where we know when something is getting noted, are we sharing that beyond the borders of where the actual issue is taking place.
>> AVRI DORIA: Thank you. Gunela, please, and then we can come back to the questions.
>> GUNELA ASTBRINK: Gunel Astbrink, MAG member and member of the Dynamic Coalition on Accessibility and Disability. These discussions are really important for people with disabilities and older people in regard to smart homes. Its it's a way for people to continue to live independently if there are smart device that's can assist them on a daily basis, but there needs to be interoperability and we have heard about that in regard to any other assistive devices. There also needs to be accessible interfaces, friendly interfaces that work for people who are not tech savvy at all, certainly holder people, maybe people with cognitive disability.
So all of those things need to be taken into consideration together with the classification I found was very interesting because there are significant vulnerabilities with a number of people who live independently, and we need to consider how that classification works for people with disabilities and older people.
We organized actually a workshop two years ago here on IoT and persons with disabilities and accessibility and we were policed Maarten and Vint Cerf could present there. I know things have moved on a lot, but I'm keen to ensure that accessibility is incorporated in any of these discussions. Thank you.
>> AVRI DORIA: Thank you. Before going on, I'm going to pass the moderation because unfortunately I'm quite enjoying this, but unfortunately I'm responsible for the Dynamic Coalition on schools and Internet Governance which happens shortly, and I'm the one that's supposed to do that one. So I just want to say though, I'm really quite impressed by this and how it's going as a person that was the second Chair of this DC. I really love the way it's blossomed and the way it's gotten into stuff. So continue to have an enjoyable session, Lebanon, it's you, you had Thomas and then max.
>> LABANON: Thank you very much, Lebanon speaking. I will try to keep the pace. Let's go for Thomas Rickert next.
>> THOMAS RICKERT: This is Thomas Rickert. I think that this discussion is most interesting but we are discussing different levels of the discussion from the manufacturer to the network operator to other aspects such as accessibility and so on so forth. I think everything starts with the manufacturer and there have been initiatives to talk about ethical development of software for IoT devices going back to at least 2013. There is this great presentation swimming with the sharks that many of you will be aware of, and there was a suggested Hippocratic Oath for those who develop software for medical devices, and I think that more needs to be done in order to get the developer community on board with developing software that actually has all of the ingredients that promise for devices to be as secure as they can be at the time of publication.
I would like for those who don't know it, but ENISA has just released a couple of days back 130 page document, good practices for security of IoT, and that describes the secure software development lifecycle in its entirety at quite a detailed level. So I think that's a very good read for those who are interested in software development aspects of IoT security. So I think that's one aspect.
I think we need to encourage the developer community to take certain principles on board when designing things, but at the same time I think it's also important to have deterrents maybe at the regulatory level.
We have product liability laws where you can't just throw something out into the market and leave it there. But those who sell devices and gardener predicts in 2021 we will have 35 billion IoT devices out there, so it's a massive problem we are facing to hold those accountable that actually submit products to the market that are not ready for market or shouldn't be sold at all.
Final point, I think, or two additional points if I may, on the DNS aspect of things, ecois going to pull or try to push the DNSEC. It's difficult to get people on board with that. We are going to push DNSEC for IoT in particular in 2020, and also I think we need to come up with different approaches to responses to IoT issues.
If we have a household where the toaster goes rogue, you may not have somebody who can actually help you with that. So I think we need to rethink responses to IoT issues because the system that we have at the moment where we just go to the vendor or you go to the access provider, I think it doesn't work as it previously did, so we need to come up with new responses.
>> MODERATOR: You mentioned product liability, but software and IoT devices, a large component being software. Software vendors often accept no liability whatever related to the software. How can that work in the context you have mentioned?
>> THOMAS RICKERT: I don't have the silver bullet solution, but when it comes to chemical ingredients we have regulation that requires you to ensure that the products or the ingredients you use are not causing dangers to the individuals using them, and I think that for IoT you can apply certain device that's can be tested prior to release to the market and I think that the manufacturers should be held accountable for the products they are allowing for being entered into the market. So I think you can, while software is being developed further, there are updates and somebody said earlier that that causes an issue because products that are safe when you publish them might not be safe with the first update, but vulnerabilities might become known later on in the process and we have an additional issue with discontinued products. Who will take care of those where the manufacturer doesn't really want to continue supporting products?
But I think at the time of submission to the market, there should be certain principles or standards that should be met.
>> OLIVIER CRÉPIN-LEBLOND: Thank you, Thomas, next, there is Max Senges.
>> MAX SENGES: I just wanted to add two points. One is that the charter of trust that was launched last year at the Munich security Conference seems to be a relevant stakeholder in all of this, and quite active and I would be happy to make a connection for the Chair or so to include them in this conversation. Basically the charter of trust aims to have a secure production pipeline of connected devices. So it's kind of a mechanism to go really step by step for all of the different participants in the supply chain.
And the other observation that I wanted to share was that I think it was Merike who brought up that we should have this classification be like central, but in one place, but organized by different parties. That seems to be quite related to the second option that for how Internet Governance should develop that was recommended in the high level report on age of interdependence where they are saying and my interpretation if the Internet is eating the world, then very soon all of the world will be Internet Governance, and that we have to start to look at sectorial approaches and think about Internet Governance for finance, Internet Governance for mobility, et cetera.
And if the classification was done by the respective sector, so health devices should define their security standards for themselves, mobility providers should define theirs for their area, then you would have that decentralized but coordinated approach.
>> OLIVIER CRÉPIN-LEBLOND: Thank you for this, max, God forbid having everything Internet Governance we would probably have a three or four week IGF. I have Merike Kaeo first and then Marco.
>> MERIKE KAEO: Yes, I wanted to address the question earlier about are people sharing information of these issues.
From my perspective IoT devices are used in various sectors, right, the automotive, airline, maritime, healthcare, so I would expect some discussions at least within the different ISOC, information sharing between sectors or within a sector, but what I am personally concerned about is transparency of issues.
I had a conversation just the other day with somebody that was from a different region. I wouldn't be able to understand the language in terms of any news Articles but there have been engine fires in cars that were due to a software issue. We really need to understand, and from a security perspective, I have been in the security industry for 20 years, people are not very good at being transparent about security issues because people get vilified. There is all of this hype and so one of the issues is people are not transparent, it's liability, the shaming and is that has to change somehow. And I am quite worried about, you know, what's not transparent in terms of IoT insecurity aspects and issues.
I also want to just make a statement on the design principles, so I bought a television quite a few years ago and then I realized when I came home that it's an Android device. So the first thing I did was look through all of the screens to see what's being sent where and what and I was happy that it actually provided updates, but it would take up to 30 minutes, and it does take 30 minutes. So, you know, when we are creating these design principles, sometimes we might want to get into a little bit more detail in terms of, you know, you must do it in a timely manner, because sometimes if you are too ambiguous then it's not that useful either.
>> OLIVIER CRÉPIN-LEBLOND: Thank you for this, Merike Kaeo. Next is Marco.
>> MARCO HOGEWONING: Merike added several things. To your last comment we see in the industry that your mind mans windows become harder to plan and they are maintenance windows are harder to plan and you have to allow things to be shut down for an hour or two to do the upgrades you need other ways face the consequences.
To build a bit on the previous discussion around liability and you made a good point and transparency as well. We do see a lot of European Directors now that ask for reporting, but I hardly ever see is getting anything out of that reporting for us to learn about. And that appears to be like every time we see an incident, the root cases is often design, and that is something where we might want to take an example in the airline industry where accidents are investigated, but also a lot of the incidents and near misses are thoroughly investigated and published to learn from.
And that's an aircraft disaster is almost always pretty unique in root cause because by the time we have done analyzing it the root cause is address add and we fix it. Back to the original point, I was forced to look into it but the European radio equipment directive is pretty firm in sort of making sure that there is always somebody liable, whether it's the person selling it to you, the person importing it or ultimately the manufacturing.
That's there. What it doesn't cover yet and I know there are discussions yet is how long the liability exists and it goes to product lifetime and that's what worries me a bit is that often especially in sort of the cheaper end of the device spectrum, the device lifetime is often longer than that of manufacturer and we need to solve that.
>> OLIVIER CRÉPIN-LEBLOND: Thank you, Marco. One of the points in some U.K. discussions is the fact that if you purchase a refrigerator it might have a lifetime of 20, 30 years because it's a fairly simple device, but are you going to ask the device manufacturer to provide updates for the next 30 years given the fact that often they have a shelf life of a year and a half and there would have been dozens of devices, et cetera. Merike mentioned the transparency of the software and the, I guess one of the questions is because the software most of the time is proprietary is there any way to ask for transparency regarding what the software features are in a device because often devices get, and I'm throwing that to everyone here, devices get shown a number of features, and when it comes down to software, they are often hidden features as well. Any thought on this? Maarten.
>> MAARTEN BOTTERMAN: Okay. Well, basically we look at a lot of the same things that we look at with apps and with Internet matters, back doors, apps that if they install them on your mobile, that they facilitate sharing your base of contacts and your base of calls, et cetera. The latter is nowadays more and more watched and checked and checked and I think we will see this with IoT devices too.
The next thing is that you come down on those that built that in. So I think it's a matter of maturity and a very important issue to tackle.
>> OLIVIER CRÉPIN-LEBLOND: Thank you. Any other comments on the points that have been discussed this morning? Shane
>> SHANE TEWS: Going back to the beginning of the conversation with working with Government and regulatory ideas, how are we doing with the procurement process? Because sometimes the dollars will help change, you know, policy? Anybody have any thoughts on that? I know I have been doing it with U.S. Government.
>> OLIVIER CRÉPIN-LEBLOND: Is there a procurement process? Jimson Olufuye maybe.
>> JIMSON OLUFUYE: Thank you very much, Jimson Olufuye speaking. Well, that is a fact, you know, the point of procurement you can really make significant changes. So I think this needs to be really amplified such that based on even requirement specification, normally you put in your product description in the order you want to purchase, and those acquisitions should be able to classify talking about categories of devices, what you want and the level of security measure.
I have seen a number of dez vises that you put EU something, you put EU, they put some form of categorization. In Nigeria, you watch out for some label. So we could get to that level whereby we say maybe EU or AU or other bodies who have some form of basic categorization, and if that is met, that is when the products can be seed to be in compliance with the order. So it's a very middle point.
So the demand side can actually make that demand. And also with regard to, you said something with regard to how do we correlate, you know, all of the Bureaus and regulation? So this brings us back to what we need to do at the global level. Self‑governance has been happening at the Bureaus, manufacturer level, maybe associations, some of the associations are doing a good job of it, but it's been kind of fragmented.
So we need to have a way to coordinate this. So how do we really do that? Which organisation will get that done? Maybe when we come to the data governance session we will be talking more about underscoring the need for us to have that global framework.
The next level where the world is going to now is going to be very decisive because a rogue authority or a rogue manufacturers or rogue users can manipulate a lot of these devices. So we need to have that in mind, otherwise we are moving to a situation where if we now have to unplug, like Marco said, if we have to unplug that's a basic measure. When you say rogue actor, you turn it off, but that also causes damages, so we need to look at those events down the line. So I agree with you, procurement is part of it, but bigger thing we need to focus on is how do we sack will this at the global level? Thank you.
>> OLIVIER CRÉPIN-LEBLOND: Thank you Jimson I have a few hounds. So Klaus dole, Frederic Donck, Lori Schulman and Matthew Shears. Let's start with Clouse, please.
>> Klaus: Very quickly, we have the right to be forgotten, maybe we have the right to be switched off and maybe the right not to be dissolves any longer.
>> OLIVIER CRÉPIN-LEBLOND: Thank you. Next is Frederic Donck.
>> FREDERIC DONCK: Yes, very quickly to react on the procurement space, this is one of our advice and recommendation I refer to IoT security and policy. There should be an environment where Government indeed uses very powerful instruments that are procurements. I haven't seen this yet for the IoT device, security indeed. You may remember that Enisa already five years ago, I believe, in 2014 issued recommendation and guidelines for procurement in ICT security generally. So I believe this is something, yes, that should be absolutely leveraged. Thank you.
>> OLIVIER CRÉPIN-LEBLOND: Thank you.
Next, we have Lori Schulman. Oh, Maarten.
>> MAARTEN BOTTERMAN: Basically the Dutch Government has made it part of the package of their approach towards more security IoT environments explicitly. So they put standards for procurement in the procurement policy and I think that's an example that is worth following, because you said an example by really making it attractive for IoT makers to come with good tools, good services. And they already have a market.
>> OLIVIER CRÉPIN-LEBLOND: Thank you, Maarten. Lori.
>> LORI SCHULMAN: My comment goes less to procurement, and more to the refrigerator that might last 30 years and the software that lasts 18 months. Perhaps at the policy level for goods used in the home, consumer‑based goods we think about modularity, and it goes to Clouse's comment about switching it off. So perhaps the standards we develop around consumer goods in the home is that nothing is necessarily automatic, right, that you could have modular devices, modular updates, modular, literally modular pieces.
If I want to stay connects, I put my card in. If I don't want to stay connected, I pull my card out. It seems fairly easy. I understand that's more complicated in heavy machinery and in airplanes, but it should be relatively simple for your toaster or your refrigerator or even for Alexa, pardon me. But there should be moments where you could pull out your programming.
>> OLIVIER CRÉPIN-LEBLOND: Thank you, Lori, and it might seem to be something quite obvious but you often see devices that require you to log in before you can use the piece of software, especially apps.
>> LORI SCHULMAN: think at all levels, most of my members are manufacturers, corporations, service providers, to inform their consumers ever that, and for consumers who don't want this to opt out of it. I will say it is a very frustrating situation. I think some of us too, and I will speak from very personal experience because it was kind of shocking, I used a medical device at home. I replaced it. The one I replaced it with was IoT. It was not disclosed to me. I was not trained on it. I was not trained on how to program it. It was just this assumption that it was going to go somewhere extraneous and they will be looking at data. And it was shocking because I was receiving emails congratulations you have met your health goal for this week, this month, whatever. So there is a question, and I consider myself an informed user.
So if I'm not understanding what I'm buying and I'm not understanding where I can connect and disconnect, I think that, again, when we are talking core values and fundamental human rights that that's at the heart of many of these issues. So when we talk about policy initiatives, we can talk high level, procurement, but I think we need to get very basic in terms of those that are bringing devices into their home and don't know they have the software embedded.
>> OLIVIER CRÉPIN-LEBLOND: Thank you, Lori. I'm seeing the time tick so I will close the queue aafter the two speakers, next is Marco Hogewoning.
>> MARCO HOGEWONING: Yes, and I wholeheartedly support this in terms of using the Government's purchasing power that's always a great incentive, but in this particular, guys, I'm not sure we are there yet. Although we do have a lot of safety standards and product conformity I'm not sure they are up to scratch for the IoT. Back to Lori's comment, it sounds very inspirational and I love it but I think of myself as a user, the usier it is, I don't want to punch in codes, I don't want to move devices in and out. I just plunk a frig in a corner and it will be there 20 years and I don't want to hassle with it, and probably a lot of my friends don't want to hassle with it. So in that sense, again, I think we have to be open and frank about it, do we want the device on the market in the first place if it's such a hassle to keep it secure, maybe don't. I'm a cook myself, I recently tried to buy a new digital thermometer. It's almost impossible to do one that doesn't do WiFi. I found one that only does bluetooth, but it's really hard to find Luke an old fashioned digital thermometer that doesn't have a chip on board and radio device.
It annoys me. Why?
>> OLIVIER CRÉPIN-LEBLOND: Why would a digital thermometer do WiFi.
>> MARCO HOGEWONING: I have got absolutely no clue. The old one was perfectly fine with a wire.
>> OLIVIER CRÉPIN-LEBLOND: Thank you, Matthew Shears.
>> MATTHEW SHEARS: So just on the procurement issue, in terms of encouraging uptake of IPv6 there were numerous requirements put into procurement processes to ensure that devices were IPV of enabled so that worked in that context. I'm not sure what the actual impact was over time but certainly that was one of the requirements that was put in place by a number of Governments. I wanted to add a dimension here. This has been a discussion that has the flavor of a very developed nation discussion.
And there are, when we think about IoT, and we think about security divides between nations and we think about the degree to which IoT and other technologies are being pushed around the globe as ways of improving economic development, we may have more than just a CE certification problem. We may have a much broader problem, much more global problem when it comes to rolling out these kinds of networks. What we may have is certain parts of the world that are quite secure or may have secured the IoT networks and we may have others that may not.
That still have implications for the global network and it still has implications for the Internet. So we need to think broader. It's multi‑level, it's multi dimension tall but it's also very much a global challenge. Thanks.
>> OLIVIER CRÉPIN-LEBLOND: Thank you, Matthew. Are there any actual remote participants? Is there anything, remotely any additional points. So I guess I can turn the floor over to Maarten Botterman to sum up the session or should we introduce the next one.
>> MAARTEN BOTTERMAN: I think Matthew came circle with his remark that it's not only thinking about securing the material, but IoT devices are needed also in other parts of the world. Very clear is warning against extreme weather, tsunamis, earth quakes, but also for crop management, et cetera. And that means that it doesn't only need to be secure enough, but also indeed affordable, and we need to find that way there, capacity building, that kind of element of how to deploy this.
Because we can think of how to deploy these, but the best thing is to help local, regional stakeholders to use these tools that have become available now. So I think that's a very worthy running of, I think the main things we had is the realization that we talk about all kinds of classifications, active versus passive, whatever application that refers to, whatever part of the world they are used in.
It's important to actively address this by informing consumers, people knowing what to do, but also don't expect too much from them as Marco said with his consumer hat on as well. And that to serve the world and this is very much from IGF perspective, it's important that we make them affordable and available for all parts of the world that can benefit from it.
So am, again, very happy with the discussion, and I saw Shane type very enthusiastically so I wish us all well and I hope I will be there to participate in the contribution as much as possible.
>> SHANE TEWS: You are not completely leaving us, I hope. Thank you all for a very enthusiastic discussion on this, because I think we are at a point where changes can be made in the technology and now with be the time to be doing it before we have these devices and no one knows how to disconnect them. I look forward to the report and I hope that everyone will continue to be with us on this dialogue as we go forward.
>> OLIVIER CRÉPIN-LEBLOND: This marks the end of the first part, this is pay back‑to‑back session. We will give you 30 minutes to take a break, get a bit of fresh air and perhaps coffee the idea that you are able to provide any service of any kind and that will be carried on the Internet. We have heard from a number of people that perhaps there should be some okays to get those devices that are dangerous or that are somehow threatening or open or insecure to be disconnected. Are we going back to the years of telecommunications where you needed to have devices to be accredited by your local teleco, pay lots of money? Are we going back in this direction?
Are we killing potentially permitless regulation. Do we need more regulation or can an ethical framework for manufacturers for service providers take us to something a little softer than hard core legislation? We will have several other panelists that will join us, Alejandro Pisanty being one of them. There you go. You say his name, and he appears in the room. Almost. Good timing. He will be taking us through a framework 6F framework proposal. We have a short presentation from him. We had have Sivasubramanian Muthusamy who will be taking us through what the core Internet values are.
I think many of you know them. There are links actually to the agenda that send you to some past papers as to what the core Internet values are, and we will also have the sustained involvement from Jimson Olufuye who has been active in the first part. I see here Matthew Shears as well and Thomas Rickert will remain with us as well and we will also have an appearance from Vint Cerf so don't go too far and see you after the break. Thank you very much and thanks for all of your input today.
(Applause).