Let us recall our purpose in collecting these non-Christian stories about a "resurrection": we wanted to verify our Bayes' factor for the evidence of Christ's resurrection. My claim is that it's at least 1e54.
The first part of our plan was to find the non-Christian resurrection story with the most evidence behind it. If we make the naturalistic assumption about these stories, we can then say that this level of evidence is approximately what corresponds to a Bayes' factor of 1e9. For by the virtue of having the most evidence, such a resurrection story would have narrowed the field down to itself - one case - from the approximately 1e9 reportable deaths in history.
As it turned out, the "resurrection"s of Krishna and Aristeas had the most evidence behind them, amounting to roughly 1/24th of the evidence for Christ's resurrection. According to our program, this must be assigned a Bayes' factor of roughly 1e9. Then 24 times that amount of evidence would correspond to raising the Bayes' factor to the 24th power - meaning, the evidence for Christ's resurrection has a Bayes' factor of... 1e216.
So yes, that does verify that the Bayes' factor is "at least 1e54". It furthermore demonstrates how much of an underestimate that value is. Recall that, in a slightly different context, I mentioned that the full odds for the resurrection would be far in excess of 1e100, and that our values for the Bayes' factors were drastic underestimates. All that is verified by this completely different methodology, of comparing with non-Christian resurrection stories.
But that's not all. This comparison also provides yet another layer of verification, in that it allows us to check the Bayes' factor of 1e8 for a disciple's testimony about Christ's resurrection. You see, among the non-Christian resurrection stories we've seen, there was not a single case of a person making an earnest, insistent testimony about someone rising from the dead. That says something about the strength and rarity of such testimonies. Granted, we have not investigated every existing non-Christian resurrection story - but if such a testimony really has a Bayes' factor of 1e8, there should be about ten such testimonies for us to find. The fact that we have not found a single one puts a lower bound on the Bayes' factor, of just about 1e8. As usual, there's some nitpicking possible, depending on whether you think there are a hundred or thousands of non-Christian resurrection stories. But it's unlikely for any of that to change the value of 1e8 by more than a couple of orders of magnitude. So our estimate about the strength of the disciples' testimony has now also been verified.
We can now be very confident that Jesus rose from the dead. Our previous calculation which first gave us this confidence has now been verified in multiple ways, using completely different methodologies - by double-checking with the historical background of non-Christian resurrection stories. Everything checks out, and all the numbers are in harmony.
But... all this has been computed under the assumption that there isn't any extreme dependence in the disciple's testimonies. We've already accounted for "normal" dependence, like ordinary social pressure or group conformity. But we have not yet accounted for the possibility that the entire set of testimony about Jesus's resurrection might have been been engineered to be in agreement by some unknown force. That is to say, we've been discounting crackpot theories - like a conspiracy by the disciples to steal Jesus's body, or an alien mind-controlling all the witnesses to the resurrection.
Ignoring such theories is fine and good, as long as both sides of the debate are agreed in dismissing them. Most doubters of the resurrection do not subscribe to these extreme theories, so carrying out our calculations in this way up to this point was still productive. However, they're now facing a double-checked Bayes' factor exceeding 1e54 for the resurrection. This makes the posterior probability against the resurrection so tiny, that the small prior probability assigned to crackpot theories now seem much larger in comparison. Someone set on disbelief can no longer ignore these theories. Indeed they have no other choice: they must fully embrace these crackpot theories.
We will begin to address such theories starting next week.
You may next want to read:
On becoming a good person
Human laws, natural laws, and the Fourth of July
Another post, from the table of contents
Bayesian evaluation for the likelihood of Christ's resurrection (Part 29)
So, let us summarized these non-Christian accounts of a resurrection. For each supposedly "resurrected" person, the following table shows the level of evidence associated with their resurrection account, expressed as a fraction of the evidence we have for Christ's resurrection:
Here's how this looks like in a histogram:
What does all this tell us? Quite a bit - we'll discuss that starting next week.
You may next want to read:
How should we interpret the Bible? Look at it as scientific data.
Why are there so few Christians among scientists? (part 2)
Another post, from the table of contents
| Name of the person | The level of evidence |
| Apollonius of Tyana | 1/30th |
| Zalmoxis | 1/60th |
| Aristeas | 1/24th |
| Mithra | 0 |
| Osiris | 1/60th |
| Dionysus | 1/60th |
| Krishna | 1/24th |
| Bodhidharma | 1/60th |
| Puhua | 1/60th |
Here's how this looks like in a histogram:
What does all this tell us? Quite a bit - we'll discuss that starting next week.
You may next want to read:
How should we interpret the Bible? Look at it as scientific data.
Why are there so few Christians among scientists? (part 2)
Another post, from the table of contents
Questions and answers: my career change to data science
I've been working as a data scientist for some time now. This is a change for me. My background is in physics, and my previous work was mostly in education, teaching in the STEM fields. So this is an exciting period in my life. Career changes often are, and data science in particular is an exciting field, having been called "the sexiest job of the 21st century" - a description that I'll not complain about.
All this excitement has lead to people asking a number of questions about my job - in fact, I recently ran into an old friend who was contemplating a similar kind of career change, and I told her that I loved my new job. She then asked me the following questions:
1) Why do you love being a data scientist? Why does the job fit you like a glove?
2) What do you do day-to-day? What's a typical week like?
3) What were the challenges starting out? What are they now, some time later?
4) What are the best and worst parts of the job?
5) What are the top 3 actions you took to successfully land a job in data science?
6) If you could give career advice to yourself from a couple of years ago, or to a fellow scientist or mathematician now, what would it be?
7) Anything else I'm missing?
Here are my answers:
1) Why do you love being a data scientist? Why does the job fit you like a glove?
Doing data science is like all the fun parts of doing physics, without any of the drudgery. Iteration times are much faster. If I'm curious about something, I can just look in the data, instead of having to go get the liquid nitrogen, and wait for the vacuum system to pump down, and for the coarse approach to finish, etc. I can typically produce something meaningful in hours or days, instead of weeks or months.
I never was very good in the lab, with all the physical equipment. I was better in a more theoretical, abstracted setting. I felt more at home when sitting in front of a computer, or thinking about a math problem on a whiteboard, rather than working with physical lab equipment. Collecting physical data is hard - it's what ends up taking all the time and effort in physics. But in data science, there's already so much data out there that much more of my time could be spent on exploring, processing, analyzing, and acting on that data. I can do things like that for hours or days on end. So a data science job fits well with my talents and temperament.
The lengthy and time-consuming nature of physical data collection was another part of science that I didn't particularly like. It was hard to stay motivated - I can work hard on a problem for an extended period of time, but if there was no real sign of progress after a few days of focused effort, that would be very discouraging to me. This happened to me frequently in physics, and it probably ultimately prevented me from doing well in grad school.
By contrast, data science projects generally give me some real return on my investment within hours of putting in the effort. In addition, working in industry as opposed to academia means that the results of my work impact the real world much more quickly. This higher rate of reward for my efforts helps me stay motivated and is much more suited for my personality.
On a more big-picture level, I like the idea that my primary job is to think soundly about data. That, in the abstract, is a profound activity, closely connected to the depth of the nature of the universe. In that sense I still feel that I am trying to "know the thoughts of God", in much the same way that I felt when I was exploring the "big questions" of physics.
2) What do you do day-to-day? What's a typical week like?
I usually do some programming, some SQL data pulls, visualize data by making graphs, attend meetings, etc. These pieces get put together to answer a data question that's relevant to my employer, like "How effective is this feature in our business in serving our customers?" or "Which of our potential customers would benefit the most from which of our offerings?" I usually work on answering several of these questions per week, with some of the larger questions taking several weeks to answer completely.
There is also an element of teaching and education, as part of the role of a data scientist is helping others understand what the data is saying and what actions we can take in response to it. This is great for me, as it scratches my itch for teaching and personal interactions - more things that I couldn't often get while sitting in the basement in a physics lab.
3) What were the challenges starting out? What are they now, some time later?
Coming from a background in physics, there were some new things I had to pick up. Obviously, I had to ramp up on the programming end. I had picked up a smattering of obscure, little-used programming languages throughout my physics career, but I finally learned more useful languages like Python or JavaScript some time before switching to data science. Apart from general programming, there's also some specific sets of modules used by data scientists, and I had to become proficient in those as well. I also picked up some statistics.
In all these thing my physics background served me well. A great deal of learning to do data science was just learning to implement the things that I could already think of to do, in a specific development environment. But the foundation of the necessary abstract concepts and the mathematical tools were already in place.
Now that I've been in the field for a while, I'm still challenged to continually improve and add to my skill set. I see more of what's out there now - there's so much to branch out to that it's hard to say what I should learn next. There's also the other, more nebulous, non-technical questions that I have to answer, like "how can I maximize the impact that my work will have in my company?" or "how should I split my time between exploring new ideas and doing what others have asked of me?" I didn't have much time to contemplate these questions when I first started, but I find that they're becoming more important now.
4) What are the best and worst parts of the job?
My favorite part of the job is probably something that I've mentioned above - the idea that I'm sharpening my data interpretation skills, that my job is basically to think soundly about data.
It's also great to see my ideas make a difference in the real world very quickly. People are having a different experience right now because of what I've done. My co-workers come to me for help with the data, and my recommendations determine how they do their job and how our customers interact with our product.
I also like the regular, but flexible, hours for my job. I don't have to worry about being "on-call" like some software engineers do, where they have to get up and do work at 3am on a weekend when the website totally breaks or something. I work during normal working hours, but I can easily switch around things like coming in late one morning for staying late another day.
There really aren't that many bad parts. I guess I sometimes end up waiting for the computer to process things, which you sometimes have to do when you're looking at billions of rows of data. Sometimes the work can get a little boring, when I have to repeat a similar, routine analysis. Some of my work ends up not having an impact, either because other people ignore it or because the conclusions end up not being actionable - and that's annoying for the same reason that making a difference is satisfying. But overall, the good definitely outweighs the bad.
5) What are the top 3 actions you took to successfully land a job in data science?
First, I learned a great deal of programming on my own. I picked up Python and JavaScript, because the school I was working at asked me to teach these, and because I wanted to do more with my blog. Some of these pre-career change projects can be found on the other posts on this blog. The programming skills, combined with the quantitative mindset I already had from my science background, positioned me well for a data science job.
Second, I applied to a data science boot camp. Insight is a well-known program, as is The Data Incubator. There are many others. Seriously, if you're wondering about a career in data science, apply to one of these programs now. Just the application process taught me a ton of things, ranging from "what does a data scientist do?" to "how do I scrape a webpage?" to "what specific tools do I need to become proficient in?" It will answer a lot of the questions that any potential future data scientist would have.
Third, I applied to jobs. Usual job seeker advice here - be tenacious, send to multiple employers, continue to practice interviewing, etc. If you've learned your craft well, you're tremendously valuable to potential employers - so believe in yourself and keep at it.
6) If you could give career advice to yourself from a couple of years ago, or to a fellow scientist or mathematician now, what would it be?
I'm happy with how things turned out - and I see now that I was already on this trajectory from 2 years ago - so I'd tell myself to keep going. As for anyone with a science or mathematics background contemplating a career change, I'd say go for it. At least, try applying to one of those boot camps.
I'd also say this, on a more "big picture" level: I think that computing, and data science in particular, is the pre-eminent field of our age, in our current moment in human history. I first got into physics because of people like Einstein or things like space travel or nuclear power - but these are very much 20th century endeavors. I still love physics, and there are certainly still some interesting activities there - but computing and data is where all the really exciting things are now happening.
7) Anything else I'm missing?
I can't think of anything really "missing" from the questions above - so here's a summary instead. I love my new job. I love my new career, considered as its own academic field. It has a number of varied characteristics that all fit my traits and needs exceedingly well: the required talents and skills, the reward schedule, the mix of group and solitary activities, the working hours, the connection with all other fields of study, etc. Of course, every job is bound to have some downsides, but I'm very content overall - I couldn't expect much more from a new career. If anyone is thinking about a career in data science, I'd encourage them to at least give it a serious pursuit.
You may next want to read:
How to make a fractal
Basic Bayesian reasoning: a better way to think (Part 1)
Another post, from the table of contents
All this excitement has lead to people asking a number of questions about my job - in fact, I recently ran into an old friend who was contemplating a similar kind of career change, and I told her that I loved my new job. She then asked me the following questions:
1) Why do you love being a data scientist? Why does the job fit you like a glove?
2) What do you do day-to-day? What's a typical week like?
3) What were the challenges starting out? What are they now, some time later?
4) What are the best and worst parts of the job?
5) What are the top 3 actions you took to successfully land a job in data science?
6) If you could give career advice to yourself from a couple of years ago, or to a fellow scientist or mathematician now, what would it be?
7) Anything else I'm missing?
Here are my answers:
1) Why do you love being a data scientist? Why does the job fit you like a glove?
Doing data science is like all the fun parts of doing physics, without any of the drudgery. Iteration times are much faster. If I'm curious about something, I can just look in the data, instead of having to go get the liquid nitrogen, and wait for the vacuum system to pump down, and for the coarse approach to finish, etc. I can typically produce something meaningful in hours or days, instead of weeks or months.
I never was very good in the lab, with all the physical equipment. I was better in a more theoretical, abstracted setting. I felt more at home when sitting in front of a computer, or thinking about a math problem on a whiteboard, rather than working with physical lab equipment. Collecting physical data is hard - it's what ends up taking all the time and effort in physics. But in data science, there's already so much data out there that much more of my time could be spent on exploring, processing, analyzing, and acting on that data. I can do things like that for hours or days on end. So a data science job fits well with my talents and temperament.
The lengthy and time-consuming nature of physical data collection was another part of science that I didn't particularly like. It was hard to stay motivated - I can work hard on a problem for an extended period of time, but if there was no real sign of progress after a few days of focused effort, that would be very discouraging to me. This happened to me frequently in physics, and it probably ultimately prevented me from doing well in grad school.
By contrast, data science projects generally give me some real return on my investment within hours of putting in the effort. In addition, working in industry as opposed to academia means that the results of my work impact the real world much more quickly. This higher rate of reward for my efforts helps me stay motivated and is much more suited for my personality.
On a more big-picture level, I like the idea that my primary job is to think soundly about data. That, in the abstract, is a profound activity, closely connected to the depth of the nature of the universe. In that sense I still feel that I am trying to "know the thoughts of God", in much the same way that I felt when I was exploring the "big questions" of physics.
2) What do you do day-to-day? What's a typical week like?
I usually do some programming, some SQL data pulls, visualize data by making graphs, attend meetings, etc. These pieces get put together to answer a data question that's relevant to my employer, like "How effective is this feature in our business in serving our customers?" or "Which of our potential customers would benefit the most from which of our offerings?" I usually work on answering several of these questions per week, with some of the larger questions taking several weeks to answer completely.
There is also an element of teaching and education, as part of the role of a data scientist is helping others understand what the data is saying and what actions we can take in response to it. This is great for me, as it scratches my itch for teaching and personal interactions - more things that I couldn't often get while sitting in the basement in a physics lab.
3) What were the challenges starting out? What are they now, some time later?
Coming from a background in physics, there were some new things I had to pick up. Obviously, I had to ramp up on the programming end. I had picked up a smattering of obscure, little-used programming languages throughout my physics career, but I finally learned more useful languages like Python or JavaScript some time before switching to data science. Apart from general programming, there's also some specific sets of modules used by data scientists, and I had to become proficient in those as well. I also picked up some statistics.
In all these thing my physics background served me well. A great deal of learning to do data science was just learning to implement the things that I could already think of to do, in a specific development environment. But the foundation of the necessary abstract concepts and the mathematical tools were already in place.
Now that I've been in the field for a while, I'm still challenged to continually improve and add to my skill set. I see more of what's out there now - there's so much to branch out to that it's hard to say what I should learn next. There's also the other, more nebulous, non-technical questions that I have to answer, like "how can I maximize the impact that my work will have in my company?" or "how should I split my time between exploring new ideas and doing what others have asked of me?" I didn't have much time to contemplate these questions when I first started, but I find that they're becoming more important now.
4) What are the best and worst parts of the job?
My favorite part of the job is probably something that I've mentioned above - the idea that I'm sharpening my data interpretation skills, that my job is basically to think soundly about data.
It's also great to see my ideas make a difference in the real world very quickly. People are having a different experience right now because of what I've done. My co-workers come to me for help with the data, and my recommendations determine how they do their job and how our customers interact with our product.
I also like the regular, but flexible, hours for my job. I don't have to worry about being "on-call" like some software engineers do, where they have to get up and do work at 3am on a weekend when the website totally breaks or something. I work during normal working hours, but I can easily switch around things like coming in late one morning for staying late another day.
There really aren't that many bad parts. I guess I sometimes end up waiting for the computer to process things, which you sometimes have to do when you're looking at billions of rows of data. Sometimes the work can get a little boring, when I have to repeat a similar, routine analysis. Some of my work ends up not having an impact, either because other people ignore it or because the conclusions end up not being actionable - and that's annoying for the same reason that making a difference is satisfying. But overall, the good definitely outweighs the bad.
5) What are the top 3 actions you took to successfully land a job in data science?
First, I learned a great deal of programming on my own. I picked up Python and JavaScript, because the school I was working at asked me to teach these, and because I wanted to do more with my blog. Some of these pre-career change projects can be found on the other posts on this blog. The programming skills, combined with the quantitative mindset I already had from my science background, positioned me well for a data science job.
Second, I applied to a data science boot camp. Insight is a well-known program, as is The Data Incubator. There are many others. Seriously, if you're wondering about a career in data science, apply to one of these programs now. Just the application process taught me a ton of things, ranging from "what does a data scientist do?" to "how do I scrape a webpage?" to "what specific tools do I need to become proficient in?" It will answer a lot of the questions that any potential future data scientist would have.
Third, I applied to jobs. Usual job seeker advice here - be tenacious, send to multiple employers, continue to practice interviewing, etc. If you've learned your craft well, you're tremendously valuable to potential employers - so believe in yourself and keep at it.
6) If you could give career advice to yourself from a couple of years ago, or to a fellow scientist or mathematician now, what would it be?
I'd also say this, on a more "big picture" level: I think that computing, and data science in particular, is the pre-eminent field of our age, in our current moment in human history. I first got into physics because of people like Einstein or things like space travel or nuclear power - but these are very much 20th century endeavors. I still love physics, and there are certainly still some interesting activities there - but computing and data is where all the really exciting things are now happening.
7) Anything else I'm missing?
I can't think of anything really "missing" from the questions above - so here's a summary instead. I love my new job. I love my new career, considered as its own academic field. It has a number of varied characteristics that all fit my traits and needs exceedingly well: the required talents and skills, the reward schedule, the mix of group and solitary activities, the working hours, the connection with all other fields of study, etc. Of course, every job is bound to have some downsides, but I'm very content overall - I couldn't expect much more from a new career. If anyone is thinking about a career in data science, I'd encourage them to at least give it a serious pursuit.
You may next want to read:
How to make a fractal
Basic Bayesian reasoning: a better way to think (Part 1)
Another post, from the table of contents
Questions and answers: my career change to data science (Part 2)
(continued from the previous post)
4) What are the best and worst parts of the job?
My favorite part of the job is probably something that I've mentioned above - the idea that I'm sharpening my data interpretation skills, that my job is basically to think soundly about data.
It's also great to see my ideas make a difference in the real world very quickly. People are having a different experience right now because of what I've done. My co-workers come to me for help with the data, and my recommendations determine how they do their job and how our customers interact with our product.
I also like the regular, but flexible, hours for my job. I don't have to worry about being "on-call" like some software engineers do, where they have to get up and do work at 3am on a weekend when the website totally breaks or something. I work during normal working hours, but I can easily switch around things like coming in late one morning for staying late another day.
There really aren't that many bad parts. I guess I sometimes end up waiting for the computer to process things, which you sometimes have to do when you're looking at billions of rows of data. Sometimes the work can get a little boring, when I have to repeat a similar, routine analysis. Some of my work ends up not having an impact, either because other people ignore it or because the conclusions end up not being actionable - and that's annoying for the same reason that making a difference is satisfying. But overall, the good definitely outweighs the bad.
5) What are the top 3 actions you took to successfully land a job in data science?
First, I learned a great deal of programming on my own. I picked up Python and JavaScript, because the school I was working at asked me to teach these, and because I wanted to do more with my blog. Some of these pre-career change projects can be found on the other posts on this blog. The programming skills, combined with the quantitative background I already had from my science background, positioned me well for a data science job.
Second, I applied to a data science boot camp. Insight is a well-known program, as is The Data Incubator. There are many others. Seriously, if you're wondering about a career in data science, apply to one of these programs now. Just the application process taught me a ton of things, ranging from "what does a data scientist do?" to "how do I scrape a webpage?" to "what specific tools do I need to become proficient in?" It will answer a lot of the questions that any potential future data scientist would have.
Third, I applied to jobs. Usual job seeker advice here - be tenacious, send to multiple employers, keep practicing interviewing, etc. If you've learned your craft well, you're tremendously valuable to potential employers - so believe in yourself and keep at it.
6) If you could give career advice to yourself from a couple of years ago, or to a fellow scientist or mathematician now, what would it be?
I'm happy with how things turned out - and I see now that I was already on this trajectory from 2 years ago - so I'd tell myself to keep going. As for anyone with a science or mathematics background contemplating a career change, I'd say go for it. At least, try applying to one of those boot camps.
I'd also say this, on a more "big picture" level: I think that computing, and data science in particular, is the pre-eminent field of our age, in our current moment in human history. I first got into physics because of people like Einstein or things like space travel or nuclear power - but these are very much 20th century endeavors. I still love physics, and there are certainly still some interesting activities there - but computing and data is where all the really exciting things are now happening.
7) Anything else I'm missing?
I can't think of anything really "missing" - so here's a summary instead. I love my new job, and my new career, considered as its own academic field. It has a number of varied characteristics that all fit my traits and needs exceedingly well: the required talents and skills, the reward schedule, the mix of group and solitary activities, the working hours, the connection with all other fields of study, etc. Of course, every job is bound to have some downsides, but I'm very content overall - I couldn't expect much more from a new career. If anyone is thinking about a career in data science, I'd encourage them to at least give it a serious pursuit.
(consolidated in the next post)
You may next want to read:
Sherlock Bayes, logical detective: a murder mystery game
The want of a mate
Another post, from the table of contents
4) What are the best and worst parts of the job?
My favorite part of the job is probably something that I've mentioned above - the idea that I'm sharpening my data interpretation skills, that my job is basically to think soundly about data.
It's also great to see my ideas make a difference in the real world very quickly. People are having a different experience right now because of what I've done. My co-workers come to me for help with the data, and my recommendations determine how they do their job and how our customers interact with our product.
I also like the regular, but flexible, hours for my job. I don't have to worry about being "on-call" like some software engineers do, where they have to get up and do work at 3am on a weekend when the website totally breaks or something. I work during normal working hours, but I can easily switch around things like coming in late one morning for staying late another day.
There really aren't that many bad parts. I guess I sometimes end up waiting for the computer to process things, which you sometimes have to do when you're looking at billions of rows of data. Sometimes the work can get a little boring, when I have to repeat a similar, routine analysis. Some of my work ends up not having an impact, either because other people ignore it or because the conclusions end up not being actionable - and that's annoying for the same reason that making a difference is satisfying. But overall, the good definitely outweighs the bad.
5) What are the top 3 actions you took to successfully land a job in data science?
First, I learned a great deal of programming on my own. I picked up Python and JavaScript, because the school I was working at asked me to teach these, and because I wanted to do more with my blog. Some of these pre-career change projects can be found on the other posts on this blog. The programming skills, combined with the quantitative background I already had from my science background, positioned me well for a data science job.
Second, I applied to a data science boot camp. Insight is a well-known program, as is The Data Incubator. There are many others. Seriously, if you're wondering about a career in data science, apply to one of these programs now. Just the application process taught me a ton of things, ranging from "what does a data scientist do?" to "how do I scrape a webpage?" to "what specific tools do I need to become proficient in?" It will answer a lot of the questions that any potential future data scientist would have.
Third, I applied to jobs. Usual job seeker advice here - be tenacious, send to multiple employers, keep practicing interviewing, etc. If you've learned your craft well, you're tremendously valuable to potential employers - so believe in yourself and keep at it.
6) If you could give career advice to yourself from a couple of years ago, or to a fellow scientist or mathematician now, what would it be?
I'd also say this, on a more "big picture" level: I think that computing, and data science in particular, is the pre-eminent field of our age, in our current moment in human history. I first got into physics because of people like Einstein or things like space travel or nuclear power - but these are very much 20th century endeavors. I still love physics, and there are certainly still some interesting activities there - but computing and data is where all the really exciting things are now happening.
7) Anything else I'm missing?
I can't think of anything really "missing" - so here's a summary instead. I love my new job, and my new career, considered as its own academic field. It has a number of varied characteristics that all fit my traits and needs exceedingly well: the required talents and skills, the reward schedule, the mix of group and solitary activities, the working hours, the connection with all other fields of study, etc. Of course, every job is bound to have some downsides, but I'm very content overall - I couldn't expect much more from a new career. If anyone is thinking about a career in data science, I'd encourage them to at least give it a serious pursuit.
(consolidated in the next post)
You may next want to read:
Sherlock Bayes, logical detective: a murder mystery game
The want of a mate
Another post, from the table of contents
Questions and answers: my career change to data science (Part 1)
I've been working as a data scientist for some time now. My background is in physics, and my previous work was mostly in education, teaching in the STEM fields. This is an exciting period in my life. Career changes often are, and data science in particular has been called "the sexiest job of the 21st century" - a description that I'll not complain about.
All this excitement has lead to people asking a number of questions about my job - in fact, I recently ran into an old friend who was contemplating a similar kind of career change, and I told her that I loved my new job. She then asked me the following questions:
1) Why do you love being a data scientist? Why does the job fit you like a glove?
2) What do you do day-to-day? What's a typical week like?
3) What were the challenges starting out? What are they now, some time later?
4) What are the best and worst parts of the job?
5) What are the top 3 actions you took to successfully land a job in data science?
6) If you could give career advice to yourself from a couple of years ago, or to a fellow scientist or mathematician now, what would it be?
7) Anything else I'm missing?
So here are my answers:
1) Why do you love being a data scientist? Why does the job fit you like a glove?
Doing data science is like all the fun parts of doing physics, without any of the drudgery. Iteration times are much faster. If I'm curious about something, I can just look in the data, instead of having to go get the liquid nitrogen, and wait for the vacuum system to pump down, and for the coarse approach to finish, etc. I can typically produce something meaningful in hours or days, instead of weeks or months.
I never was very good in the lab, with all the physical equipment. I was better in a more theoretical, abstracted setting. I felt more at home when sitting in front of a computer, or thinking about a math problem on a whiteboard, rather than working with physical lab equipment. Collecting physical data is hard - it's what ended up taking all the time and effort in physics. But in data science, there's already so much data out there that much more of my time could be spent on exploring, processing, analyzing, and acting on that data. I can do things like that for hours or days on end. So a data science job fits well with my talents and temperament.
The lengthy and time-consuming nature of physical data collection was another part of science that I didn't particularly like. It was hard to stay motivated - I can work hard on a problem for an extended period of time, but if there was no real sign of progress after a few days of focused effort, that would be very discouraging to me. This happened to me frequently in physics, and it probably ultimately prevented me from doing well in grad school.
In contrast, data science projects generally give me some real return on my investment within hours of putting in the effort. In addition, working in industry as opposed to academia means that the results of my work impact the real world much more quickly. This higher rate of reward for my efforts helps me stay motivated and is much more suited for my personality.
On a more big-picture level, I like the idea that my primary job is to think soundly about data. That, in the abstract, is a profound activity, closely connected to the depth of the nature of the universe. In that sense I still feel that I am trying to "know the thoughts of God", in much the same way that I felt when I was exploring the "big questions" of physics.
2) What do you do day-to-day? What's a typical week like?
I usually do some programming, some SQL data pulls, visualize data by making graphs, attend meetings, etc. These pieces get put together to answer a data question that's relevant to my employer, like "How effective is this feature in our business in serving our customers?" or "Which of our potential customers would benefit the most from which of our offerings?" I usually work on answering several of these questions per week, with some of the larger questions taking several weeks to answer completely.
There is also an element of teaching and education, as part of the role of a data scientist is helping others understand what the data is saying and what actions we can take in response to it. This is great for me, as it scratches my itch for teaching and personal interactions - more things that I couldn't often get while sitting in the basement in a physics lab.
3) What were the challenges starting out? What are they now, some time later?
Coming from a background in physics, there were some new things I had to pick up. Obviously, I had to ramp up on the programming end. I had picked up a smattering of obscure, little-used programming languages throughout my physics career, but I finally learned more useful languages like Python or JavaScript some time before switching to data science. Apart from general programming, there's also some specific sets of modules used by data scientists, and I had to become proficient in those as well. I also picked up some statistics.
In all these thing my physics background served me well. A great deal of things was just learning to implement the things that I could already think of to do, in a specific development environment. But the foundation of the necessary abstract concepts and the mathematical tools were already in place.
Now that I've been in the field for a while, I'm still challenged to continually improve and add to my skill set. I see more of what's out there now - there's so much to branch out to that it's hard to say what I should learn next. There's also the other, more nebulous, non-technical questions that I have to answer, like "how can I maximize the impact that my work will have in my company?" or "how should I split my time between exploring new ideas and doing what others have asked of me?" I didn't have much time to contemplate these questions when I first started, but I find that they're becoming more important now.
(to be continued in the next post)
You may next want to read:
Make the most of your time and your life. Number your days.
Basic Bayesian reasoning: a better way to think (Part 1)
Another post, from the table of contents
All this excitement has lead to people asking a number of questions about my job - in fact, I recently ran into an old friend who was contemplating a similar kind of career change, and I told her that I loved my new job. She then asked me the following questions:
1) Why do you love being a data scientist? Why does the job fit you like a glove?
2) What do you do day-to-day? What's a typical week like?
3) What were the challenges starting out? What are they now, some time later?
4) What are the best and worst parts of the job?
5) What are the top 3 actions you took to successfully land a job in data science?
6) If you could give career advice to yourself from a couple of years ago, or to a fellow scientist or mathematician now, what would it be?
7) Anything else I'm missing?
So here are my answers:
Doing data science is like all the fun parts of doing physics, without any of the drudgery. Iteration times are much faster. If I'm curious about something, I can just look in the data, instead of having to go get the liquid nitrogen, and wait for the vacuum system to pump down, and for the coarse approach to finish, etc. I can typically produce something meaningful in hours or days, instead of weeks or months.
I never was very good in the lab, with all the physical equipment. I was better in a more theoretical, abstracted setting. I felt more at home when sitting in front of a computer, or thinking about a math problem on a whiteboard, rather than working with physical lab equipment. Collecting physical data is hard - it's what ended up taking all the time and effort in physics. But in data science, there's already so much data out there that much more of my time could be spent on exploring, processing, analyzing, and acting on that data. I can do things like that for hours or days on end. So a data science job fits well with my talents and temperament.
The lengthy and time-consuming nature of physical data collection was another part of science that I didn't particularly like. It was hard to stay motivated - I can work hard on a problem for an extended period of time, but if there was no real sign of progress after a few days of focused effort, that would be very discouraging to me. This happened to me frequently in physics, and it probably ultimately prevented me from doing well in grad school.
In contrast, data science projects generally give me some real return on my investment within hours of putting in the effort. In addition, working in industry as opposed to academia means that the results of my work impact the real world much more quickly. This higher rate of reward for my efforts helps me stay motivated and is much more suited for my personality.
On a more big-picture level, I like the idea that my primary job is to think soundly about data. That, in the abstract, is a profound activity, closely connected to the depth of the nature of the universe. In that sense I still feel that I am trying to "know the thoughts of God", in much the same way that I felt when I was exploring the "big questions" of physics.
2) What do you do day-to-day? What's a typical week like?
There is also an element of teaching and education, as part of the role of a data scientist is helping others understand what the data is saying and what actions we can take in response to it. This is great for me, as it scratches my itch for teaching and personal interactions - more things that I couldn't often get while sitting in the basement in a physics lab.
3) What were the challenges starting out? What are they now, some time later?
In all these thing my physics background served me well. A great deal of things was just learning to implement the things that I could already think of to do, in a specific development environment. But the foundation of the necessary abstract concepts and the mathematical tools were already in place.
Now that I've been in the field for a while, I'm still challenged to continually improve and add to my skill set. I see more of what's out there now - there's so much to branch out to that it's hard to say what I should learn next. There's also the other, more nebulous, non-technical questions that I have to answer, like "how can I maximize the impact that my work will have in my company?" or "how should I split my time between exploring new ideas and doing what others have asked of me?" I didn't have much time to contemplate these questions when I first started, but I find that they're becoming more important now.
(to be continued in the next post)
You may next want to read:
Make the most of your time and your life. Number your days.
Basic Bayesian reasoning: a better way to think (Part 1)
Another post, from the table of contents