Seven security concerns in Machine Learning (ML) –
Data privacy and security: ML requires large amounts of data to be trained, and this data may contain sensitive or personal information. Appropriate measures need to be put in place to prevent data from being accessed by unauthorized parties.
Notebooks security: ML typically requires Jupyter or similar notebooks to be served for data scientists to work on data, code, and models, both individually and collaboratively. These notebooks need to be access controlled and protected from unauthorized access. This includes the code and git repos that host the code, and the model artifacts that the notebook uses or creates.
Model serving and inference security: ML models in production are commonly served and accessed over inference endpoints and such endpoints need authentication, authorization, encryption for protection against misuse. During model upgrades to an endpoint or changes to an endpoint and its configuration, a number of attacks are possible that are typical of a devops/devsecops pipeline. These need to be protected against.
Model security: Models can be vulnerable to attacks such as adversarial inputs, such as when an attacker intentionally manipulates the input to the model in order to cause it to make incorrect predictions. Another example is when the model makes an egregiously bad decision on an input, for example a self-driving car hitting an obstacle instead of avoiding it. It is important to harden the model and bound the decisions that come from its use.
Misuse: Even if a model works as designed, it can be misused, for example by generating fake or misleading content. It is important to consider the potential unintended consequences of using models and to put safeguards in place to prevent their misuse.
Bias: ML models can sometimes exhibit biases due to the data they are trained on. There should be a plan to identify biases in a model and take steps to mitigate them.
Intellectual property: ML models may be protected by intellectual property laws, and it is important to respect these laws and obtain the appropriate licenses when using language models developed by others.
There are a number of tools used to detect security issues in a software application codebase. A simple and free one is flawfinder. A sophisticated commercial one is Veracode. There’s also lint, pylint, findbugs for java, and xcode clang static analyzer.
Synopsis has bought a few tools like Coverity and Blackduck for various static checks on code and binary. Blackduck can do binary analysis and scores issues with the CVSS. A common use of Blackduck is for license checking to check for conformance to open source licenses.
A more comprehensive list of static code analysis tools is here.
Dynamic analysis tools inspect the running process and find memory and execution errors. Well known examples are valgrind and Purify. More dynamic tools are listed here.
A common issue with the tools is the issue of false positives. It helps to limit the testing to certain defect types or attack scenarios and identify the most critical issues, then expand the scope of types of defects.
Code obfuscation and anti-tamper are another line of tools, for example by Arxan, Klocwork, Irdeto and Proguard .
A great talk on Adventures in fuzzing. My takeaway has been that better ways of developing secure software are really important.
Threat modelling is a set of techniques to identify the level of risk to assets from their interactions with their operating environment. Some threat modelling methodologies and tools are linked below for reference:
PASTA – Process for Attack Simulation and Threat Analysis. The link has details of an online banking use case.
In response to meltdown/spectre side-channel vulnerabilities, which are based on fine grained observation of the CPU to infer cache state of an adjacent process or VM, a mitigration response by browsers was the reduction of the time resolution of various time apis, especially in javascript.
The authors responded with alternative sources of finding fine grained timing, available to browsers. An interpolation method allows obtaining of a fine resolution of 15 μs, from a timer that is rounded down to multiples of 100 ms.
A meltdown PoC is at https://github.com/gkaindl/meltdown-poc, to test the timing attack in its own process. The instruction RDTSC returns the TimeStampCounter (TSC), a 64-bit register that counts the number of cycles since reset, and so has a resolution of 0.5ns on a 2GHz CPU.
int main() {
unsigned long i;
i = __rdtsc();
printf("%lld\n", i);
}
A ‘Fast Primes’ algo is used to generate primes, but since they are not uniformly distributed, one can use knowledge of the public keys to guess the private keys. Primality testing was discussed in this blog here.
All RSA primes (as well as the moduli) generated by the RSALib have the following form: p=k∗M+(65537^a modM). Because of this the public key, p*q is of the form 65537^c mod M.
The integers k, a are unknown, and RSA primes differ only in their values of a and k for keys of the same size. The integer M is known and equal to some primorial M = Pn# (the product of the n successive primes) and is related to the size of the required key. The attack replaces this with a smaller M’ which still satisfies the above prime generation equation, enabling guessing of a.
Coppersmith algorithm based RSA attacks are typically used in scenarios where we know partial information about the private key (or message) and we want to compute the rest. The given problem is solved in the three steps:
The LLL is a fascinating algorithm which ‘reduces’ a lattice basis b0, · · · ,bn−1. The algorithm computes an alternative basis b0 ′ , · · · , bn-1′ of the lattice which is smaller than the original basis. Coppersmith’s algorithm utilizes the LLL algorithm.
Discussion of Coppersmith attack in “Twenty Years of Attacks on the RSA Cryptosystem”.
The basics of Ethereum are described in the Gavin Wood paper. A list of keywords in Solidity are described in this file from its source, which includes “address”, “contract”, “event”, “mapping” and “wei” ( 1 Eth= 10^18 Wei). This list does not include “gas”, which is a mechanism described in Wood’s paper to combat abuse. Interestingly the paper says “The first example of utilising the proof-of-work as a strong economic signal to secure a currency was by Vishnumurthy et al [2003]”, aka the Karma paper.
The karma paper talks about solving a cryptographic puzzle as enabling one to join the network and be assigned a bank set: “the node certifies that it completed this computation by encrypting challenges provided by its bank-set nodes with its private key. Thus each node is assigned an id beyond its immediate control, and acquires a public-private key pair that can be used in later stages of the protocol without having to rely on a public-key infrastructure”. Crypto puzzles for diverse problems have been proposed before, a survey and comparison is at https://pdfs.semanticscholar.org/d8b9/a0309cef8c309541876c9c2c5ad5c16c3b7a.pdf
The DAO attack had 3 components, a hacker, a malicious contract and a vulnerable contract. The malicious contract was used to withdraw funds from the vulnerable contract so that it never got to the code to decrement its balance. Oddly enough the gas mechanism which is supposed to limit (runaway) computation did not kick in to stop this repeated remittance.
A few weeks before the DAO attack someone had pointed out to me that security of solidity was a bit of an open problem. My feeling was contracts should be layered above the value exchange mechanism, not built into it. Bitcoin based protocols with the simpler OP_RETURN semantics appeared more solid. Later around October’16 at an Ethereum meetup, Fred Ehrsam made the comment that most new projects would be using Ethereum instead of bitcoin. But Bitcoin meetups had more real-world use cases being discussed. The technical limitations exist, which are being addressed by forks such as SegWit2x this November. Today saw a number of interesting proposals with Ethereum, including Dharma, DataWallet and BloomIDs. Security would be a continuing concern with the expanding scope of such projects.
Sequence diagram of the attack, showing a double-withdraw –
It supports the Ethereum Virtual Machine. The EVM is formally specified in section 9 of Gavin Wood paper. Some properties –
computation on EVM is intrinsically tied to a parameter ‘gas’ which limits the amount of computation.
stack based machine, with max stack depth of1024
256-bit word size. 256-bit stack item size. 256-bit chosen to facilitate Keccak-256 hash and Elliptic Curve computations
machine halts on various exceptions including OOG (out of gas)
memory model is a word-addressed byte-array
program code is store in a ‘virtual ROM’ accessed via a specialized instruction
runs state transition function
The state transition function takes as input – state σ, remaining gas g, and a tuple I. The execution agent must provide in the execution environment, certain information, that are contained in the tuple I: • Ia, the address of the account which owns the code that is executing. • Io, the sender address of the transaction that originated this execution. • Ip, the price of gas in the transaction that originated this execution. • Id, the byte array that is the input data to this execution; if the execution agent is a transaction, this would be the transaction data. • Is, the address of the account which caused the code to be executing; if the execution agent is a transaction, this would be the transaction sender. • Iv , the value, in Wei, passed to this account as part of the same procedure as execution; if the execution agent is a transaction, this would be the transaction value. • Ib, the byte array that is the machine code to be executed. • IH , the block header of the present block. • Ie, the depth of the present message-call or contract-creation (i.e. the number of CALLs or CREATEs being executed at present). The execution model defines the function Ξ, which can compute the resultant state σ′, the remaining gas g′, the suicide list s, the log series l, the refunds r and the resul- tant output, o, given these definitions: (115) (σ′, g′, s, l, r, o) ≡ Ξ(σ, g, I)
But what is a formal notion of a contract (specifically on the EVM which aims to generalize contracts) ? Is it any function. Is it a function of a specific template, expecting certain inputs/outputs. Can these functions be composed ?
“The contractual phases of search, negotiation, commitment, performance, and adjudication constitute the realm of smart contracts.”
Example – a vending machine. The machine ‘takes in coins and dispenses change and product according to the displayed price’, it is ‘a contract with bearer: anybody with coins can participate in an exchange with the vendor’.
So one can think of a smart contract as a digital vending machine. That’s actually a useful mental model. With a digital vending machine (DVM ?), the buyer can make a choice of what product buyer wants (of possibly very many – not limited to a physical machine), and be served that product upon due payment without interaction with another person or agency.
Kaspersky labs released a report on Industrial and Control System (ICS) security trends. The data was reported to be gathered using Kaspersky Security Network (KSN), a distributed antivirus network.
The rising trend is partly due to the isolation strategy currently being followed for ICS network security no longer being effective in protecting industrial networks.
Secure Machinery 