10. Big data in science

Big data has recently become an integral part of working in science, with a good understanding of algorithms being an essential trait required of modern day biologists, physicists etc

Not long ago, sequencing an entire genome (determining the order of all 3 billion pairs of DNA letters in the helix) took years. The Human Genome Project, the first completed sequence of an entire human genome, took around 13 years from conception to its completion in 2003, and cost more than £2 billion. Today, next-generation sequencing can do the same thing in 24 hours for not much more than a thousand pounds.

This has completely changed how scientists work. It's not just that they get their hands dirty less often, nor simply that the required skills have changed. It's that the whole process of science – how you come by an idea and test it – has been upended.

This has left a lot of senior scientists needing to understand and supervise techniques that didn't exist when they trained. It's left universities playing catch-up, with many degrees not teaching the skills that modern biologists need. But above all, it's led to ground-breaking scientific discoveries – breakthroughs that simply wouldn't have been possible 20 or even 10 years ago.

Comments

3. Growth of Big Data

3. Growth of Big Data Kryder's law - The idea that data storage will double almost every year (13 months), as the storage capacity increases, storage will decrease in price. https://searchstorage.techtarget.com/definition/Kryders-Law growth of data - Data is increasing at an exponential rate, more data has been made in the past two years than ever before combined. It is estimated that by the year 2020, 1.7MB of new data will be made every second for every human on earth. Within 5 years there will be 50 billion smart connected devices on earth. also by 2020, at least 1/3 of all the world's data will pass through the cloud. The image below contains more examples of data growth as well as a graph for visual representation https://dvmobile.io/dvmobile-blog/feeling-overwhelmed-by-a-deluge-of-iot-data-iot-data-analytics-dashboards-can-help

7. Limitations of traditional data analysis

7. Limitations of traditional data analysis Security: Security is foremost aspect for every technology. Big data is prone to data breaches. The important information that is provided to some third party may get leaked to customers. Proper encryption must be made in order to protect the data. Large growth in data : data is growing faster than the processing power. Large volumes of data are being exploded in past years. We need some new machines to work; otherwise we will get over run by data. Large Data centres can solve this problem. Inconsistencies: Sometimes the tools we use to gather big data sets are imprecise. This will happen when the data is collecting for example, consider Google search Edinburgh College results of the search on one day will be different from other day, this is mainly due to inconsistency in data collection.

Week 3

Week 3 Big and small data Small data is a way of keeping the size of data down. A file such as an image could take a lot of space depending on the size of the image, but by compressing it and losing some of the quality you can massivly reduce ths size. An example of small data could be Meteorological Aviation Report (METAR). The image below shows this off, each section of the code represents a different piece of information and saves entire data entries from being typed out and takes less time to process. https://www.futurelearn.com/courses/big-data-and-the-environment/3/steps/420387 Citizen Science Is the collection of data about the natural world that has been gathered by the general public, usually as part of a project with other data scientists. This can be used to gain vast amounts of information quickly since many people at once can gather the information as opposed to just one scientist or one sensor. One example is Thames 21, a charity which worked with Cit...

Big Data EC1702991

Search This Blog