Welcome to the "Python For Finance" Free Course, designed for anyone who wishes to learn Python for use in the financial sector and more broadly in data analysis. Whether you are a student, a finance professional, or simply passionate about the field, this training will enable you to master the basics of Python and apply it to real-world scenarios in the financial sector.

![pff-intro.png](https://strapi-project-image-test.s3.eu-west-3.amazonaws.com/pff_intro_6f39fb48c2.png)

We are thrilled to announce that this "Python for Finance" course is offered entirely free of charge by Crypto Robot in collaboration with Robot Traders (https://www.youtube.com/channel/UCOoKHYcIsEQsiZAMyu5I68g/)! Our goal is to make learning the Python language and its application in the financial domain accessible to everyone. The best way to support us is to follow us on our various social media platforms, and most importantly, share this course with as many people as possible!


## **Table of Contents**

- **0 - Python and VsCode Installation**: Learn how to install Python and Visual Studio Code, a versatile and powerful code editor, to start your journey in learning the Python language.
- **1 - Variables, Operations, and Types**: Get acquainted with variables, mathematical operations, and different data types in Python.
- **2 - Loops and Conditions in Python**: Discover how to use control structures such as loops and conditions to perform repetitive or conditional tasks.
- **3 - Lists and Dictionaries in Python**: Learn to handle lists and dictionaries to store and organize your data efficiently.
- **4 - Functions and Classes in Python**: Discover how to create and use functions and classes to structure your code and facilitate reuse.
- **5 - The Pandas Library, the best library for financial data**: Learn to use Pandas, a powerful library for data manipulation and analysis, especially suited for working with financial data.
- **6 - Visualizing Data in Python with Matplotlib**: Discover how to create charts and interactive visualizations of your financial data using the Matplotlib library.
- **7 - Unlocking the Power of Pandas and Avoiding Loops**: Learn to get the most out of Pandas by avoiding loops and using advanced techniques to manipulate and analyze your data.
- **8 - Mastering Date Usage in Python**: Discover how to manage and manipulate dates and times in Python, a crucial aspect when working with financial data.
- **9 - Backtesting an Automated Trading Strategy in Python**: Learn to develop and evaluate automated trading strategies using Python.
- **10 - Analyzing the Backtest of a Trading Portfolio in Python** with Pandas: Discover how to analyze the performance of your trading portfolio using Pandas to draw conclusions and optimize your strategies.

It is important to emphasize that the best way to progress in Python is not just by watching videos and reading documents, but by actively engaging in practice. We encourage you to work on personal projects, even on a small scale, to apply and reinforce the skills you've acquired throughout this training.

In this training, you will discover the Python programming language and how to apply it quickly to financial concepts.

<MDX_YT videoId="VkCQqKIbCFA"/>

Ready to embark on an exhilarating programming journey? Fantastic! Let's lay a strong foundation to ensure your success. In this initial chapter of the course, we will install two essential software programs: **Python**, the programming language itself, and an Integrated Development Environment (**IDE**).

Writing code directly through the console or terminal can be quite tedious, which is why we will install the popular **VsCode** IDE. It is widely used and open-source 😉. VsCode offers customization options and useful extensions that enhance programming efficiency. It is not limited to Python either, as it supports other popular programming languages like JavaScript, C, and CSS. This makes it a valuable investment for beginners and professionals programmers alike.

By setting up these key components, we create an environment that promotes efficient coding practices and unlocks your programming potential. Throughout this tutorial, we guide you through the installation process. While we focus on **Windows** installations, the principles we discuss apply to Mac and Linux systems as well.

Let’s embark on this programming adventure together! 🚀

## Python installation

Installing Python is a breeze. Let us walk through the steps together. First, head over to the official website at **[https://www.python.org/downloads/](https://www.python.org/downloads/)**. There, you will find a bright yellow button that says *Download Python* (check out the picture below). Click on it, and the download will start automatically.

![pff0-1.png](https://strapi-project-image-test.s3.eu-west-3.amazonaws.com/pff0_1_355b8d9390.png)

Once the download is complete, go ahead and run the file you just downloaded.

⚠️ **Important:** Make sure to follow the steps after opening the installation file to avoid any issues. ⚠️

In the first window that appears, pay close attention to the checkbox that says *Add Python 3.X to PATH*  (see picture below)*.* It must be ticked. This step is crucial as it allows Windows to recognize Python automatically by adding it to the environment variables. The "X" in 3.X represents the digits of the available Python 3 version on the website, usually indicating the latest release. Keep in mind that the version you see may differ from what we show in this tutorial because Python gets updated over time. But worry not, the validity of this tutorial and the following steps remains intact 🙂.

![pff0-2.png](https://strapi-project-image-test.s3.eu-west-3.amazonaws.com/pff0_2_50fdaa3f58.png)

Moving on, let us choose the *Customize installation* option (as shown in the picture above). In the *Optional Features* section, we recommend keeping all the checkboxes checked (see picture below). This ensures you have access to all the handy features.

![pff0-3.png](https://strapi-project-image-test.s3.eu-west-3.amazonaws.com/pff0_3_4e38303dc7.png)

Finally, in the advanced options, make sure to tick the *install for all users* box (see picture below). If you are installing Python version 3.11.X, consider changing the installation path to *C:\Python311*. This will help avoid any permission hiccups that sometimes occur on Windows.

![pff0-4.png](https://strapi-project-image-test.s3.eu-west-3.amazonaws.com/pff0_4_8b18a62551.png)

Ready? It is time to click on *Install 😁.* Once the installation is complete, you can close the installer (see picture below).

![pff0-5.png](https://strapi-project-image-test.s3.eu-west-3.amazonaws.com/pff0_5_d18d704640.png)

Now, let us check whether Python has been installed and configured correctly. Open a command prompt by typing *cmd* in the taskbar search on Windows and pressing enter (see picture below).

![pff0-7.png](https://strapi-project-image-test.s3.eu-west-3.amazonaws.com/pff0_7_002102bd55.png)

In the command prompt, type the following command: **`python --version`** (with two hyphens) and press enter.

![pff0-6.png](https://strapi-project-image-test.s3.eu-west-3.amazonaws.com/pff0_6_132912f0ea.png)

This step should display your Python version (see picture above). If you encounter an error message indicating that Python is not recognized, try using the command **`python3 --version`** instead. Do not worry if you still face an error. A simple solution could be restarting your PC. If the problem persists after that, uninstall Python and then reinstall it, making sure you follow all the previous steps, especially the ticking of the *Add Python to PATH* box at the beginning of the installation process. Let’s also check that **pip** was installed correctly. Same as previously, try **`pip --version`** or **`pip3 --version`**. Pip is the python package manager, if you want to install any python package, you will have to use it. We will get to do that in a future chapter of this course. 

You can now begin your first Python commands by typing *python* in a command prompt and then pressing the enter key. This will take you to a Python terminal (see picture below).

![pff0-8.png](https://strapi-project-image-test.s3.eu-west-3.amazonaws.com/pff0_8_d916743945.png)

Note the three “>>>” at the start of the new line, this tells you that you have indeed entered a Python terminal. From there, you can type the command *print("Hello World")* and press enter to display the text "Hello World" (see picture below). You can also perform operations like 2+3, which will give you the result 5. 

![pff0-9.png](https://strapi-project-image-test.s3.eu-west-3.amazonaws.com/pff0_9_e49ed1c183.png)

Well done! You might have not realized it, but you have just run your first Python program! 🎉

## VsCode installation

Let's head over to the official website at this link: **[https://code.visualstudio.com/download](https://code.visualstudio.com/download)**. From there, you can download the software that corresponds to your operating system.

Click on the downloaded file and proceed by clicking *Next* until you reach the window shown below.

![pff0-10.png](https://strapi-project-image-test.s3.eu-west-3.amazonaws.com/pff0_10_9b5a72feb3.png)

Here, I would suggest **checking all the boxes**, as this is important for simplifying the software's usability later on. Then, click *Next* and *Install*.

Now that VsCode is installed, you can close the opened window.

## ****Your First Python Program with VsCode****

Let's create a folder, for example, on your Desktop, and name it *python_training*. Open this folder and right-click inside it. You should see an option *Open with Code* (see picture below), click on that. If you're using Windows 11, you can find this option in the *More options* menu.

![pff0-11.png](https://strapi-project-image-test.s3.eu-west-3.amazonaws.com/pff0_11_8a6ab71231.png)

Click on *Yes, I trust the authors* in the window that comes up, and then you can close the welcome tab (see picture below).

![pff0-12.png](https://strapi-project-image-test.s3.eu-west-3.amazonaws.com/pff0_12_6451631bee.png)

In the top left corner, click on the button to *create a new file* (see picture below). Name this file ***[hello.py](http://hello.py/)*** (.py is the file extension for executable Python files).

![pff0-14.png](https://strapi-project-image-test.s3.eu-west-3.amazonaws.com/pff0_14_89581d78c0.png)

When you open this file, you might see a message popping up at the bottom right of the VsCode window prompting you to install the **Python extension** for VsCode. Accept as it enables VsCode to support the Python programming language. If you don't see this message, you can go to the extensions menu on the left and search for the Python extension, then install it (see picture below). 

![pff0-15.png](https://strapi-project-image-test.s3.eu-west-3.amazonaws.com/pff0_15_277fd97861.png)

Next, return to your **[hello.py](http://hello.py/)** file, and start editing it. Fill it with the following code:

```python
print("Hello World")
```

You should have a “play” button in the top right corner to run your [hello.py](http://hello.py/) file (see picture below (1)).

![pff0-16.png](https://strapi-project-image-test.s3.eu-west-3.amazonaws.com/pff0_16_14840a7f29.png)

Clicking on it should open a terminal automatically at the bottom of the VsCode window, executing the command to run your Python program. As a result, you will see "Hello World" displayed in the terminal (see picture above (2)).

Congratulations 🎉, you have just run your first Python program in VsCode 😉!

## ****Creating a Notebook****

When you want to create a program that aims to automate an action for example, you will almost always write your program in a *.py* file, as we just saw. However, there is another type of file that greatly facilitates **initial testing, data exploration, and analysis**. These files are called *Jupyter Notebooks*. They work in the form of **cells** that can be executed one after another, making them also great pedagogical and presentation tools. We will be using these files extensively in this training.

To create a Jupyter notebook, simply create a new file as we did before. This time, name it *hello.ipynb* (.ipynb is the extension for Jupyter notebooks).

VsCode should also prompt you to install an extension to support this type of file. If it doesn't, you can search for and **install the Jupyter extension** (see picture below).

![pff0-17.png](https://strapi-project-image-test.s3.eu-west-3.amazonaws.com/pff0_17_c82540db07.png)

Now, go back to your *hello.ipynb* file and fill the first cell with the same command as before `print("Hello World")`.

![pff0-18.png](https://strapi-project-image-test.s3.eu-west-3.amazonaws.com/pff0_18_094ea246fb.png)

This time, the arrow to execute the cell is located on the left side (see picture above). Clicking on it should display *Hello World*. You can also use the **shortcut *Ctrl + Enter*** to execute the currently edited cell. If you receive any update notifications from VsCode, such as prompts to update your Python kernel, make sure to proceed with the updates. 

To add a new cell below, hover your cursor at the bottom of the current cell and a *+ Code* icon will appear, alternatively you can also click on the *+ Code* (see picture below). Click on it and write **`print(2+3)`** in this new cell, execute it, and it should display 5.

![pff0-19.png](https://strapi-project-image-test.s3.eu-west-3.amazonaws.com/pff0_19_dcbb6dcf6e.png)

## Other useful VsCode extensions

To further enhance your coding endeavours, we recommend installing the **Python Extension Pack** (see picture below). This package gathers several extensions that will help you write code more efficiently and clearer.

![pff0-20.png](https://strapi-project-image-test.s3.eu-west-3.amazonaws.com/pff0_20_72b2a2bee4.png)

## Conclusion

Well done! You have successfully installed Python, set up VsCode, and run your first Python program. Whether you are working with Python files or Jupyter Notebooks, you now have the tools to explore the exciting world of programming.

In the next session, we will delve into variables, which form the foundation of every computer program. Get ready for more learning and discovery!

Learn how to install Python and Visual Studio Code, a versatile and powerful code editor, to start your journey in learning the Python language.

Python and VsCode Setup

<MDX_YT videoId="oVwMP2R0qc0"/>

## ****What are Variables?****

At the heart of every computer program lies **variables**. They are tools to store data for future use. For example, say you want to store the number of Apple shares you own to perform operations based on this number. Declaring a variable in Python is simple, as shown in the following example:

```python
apple_stock_number = 5
```

A variable consists of three elements: a ***name***, a ***value***, and a ***type***. In this case, the *name* is **`apple_stock_number`**, the *value* is 5, and the *type* is a number (we will look into types later).

In Python, the convention for naming variables is to use descriptive English words, in all lowercase letters, with underscores substituting any spaces. Adhering to this standard will contribute to maintaining clear and readable code by you and others. If you want to know more about conventions for writing Python, you can have a look at the PEP 8 ([https://peps.python.org/pep-0008/](https://peps.python.org/pep-0008/)), this is the most widely adopted standard.

You can now retrieve the number of APPLE shares you own at any time, for instance:

```python
print(apple_stock_number) # Displays 5
```

You should see the number 5 displayed, which is indeed the *value* we assigned earlier. The *print()* is a function that allows us to display the *value* of what is inside the parentheses. You have tested this function in the previous session with *print("Hello World")*, which in this case displayed the text "Hello World".

You should also note that the character # is used to add comments. Anything after the # will not be considered in the program. This can be useful to provide notes to explain what the code does, for example.

A variable’s value is not set in stone, you can change it:

```python
apple_stock_number = 7
```

Now, the value of **`apple_stock_count`** is 7. You can use **`print()`** to observe the change:

```python
apple_stock_number = 5
print(apple_stock_number) # Displays 5
apple_stock_number = 7
print(apple_stock_number) # Displays 7
```

In this case, you should see 5 followed by 7 displayed.

However, beware:

```python
print(microsoft_stock_number) # Displays and error
```

This will cause an error, since the variable **`microsoft_stock_count`** has not been defined.

## ****What are the Different Types?****

Earlier, we mentioned that variables have types. In Python, the four primary types are:

- **int** for integers (e.g., 5 or -5)
- **float** for floating-point numbers (e.g., 5.91)
- **str** for strings (e.g., “Hello World”)
- **bool** for Booleans (e.g., True or False)

Here's an example that demonstrates the types:

```python
int_var = -5
float_var = 5.7
str_var = "Hello World"
bool_var = True

print(type(int_var)) # Displays <class 'int'>
print(type(float_var)) # Displays <class 'float'>
print(type(str_var)) # Displays <class 'str'>
print(type(bool_var)) # Displays <class 'bool'>
```

Here, we have defined four new variables, each of a different *type*. Indeed, you should see a message displayed for each variable **`<class ‘’>`** with the *type* inside the quotes.

**`type()`** is like `print()`, a function that allows you to find out the *type* of the object that you put inside the parentheses.

We will not go into details here, but know that you can also change the *type* of a variable in certain cases, for example:

```python
int_var = -5
str_var = str(-5)
print(str_var) # Displays "-5" instead of -5, and str_var will now behave as a string
print(type(str_var)) # Displays <class 'str'>
```

Having a good understanding of the types of variables that one uses is important. A significant number of errors that occur when building code in Python, and in most programming languages in general, arise because the user tries to perform operations between variable types that are not compatible with each other.

## ****Using Operations in Python****

Having explored variables and their types, it is time to put them into action with Python’s fundamental arithmetic operations:

- Addition (+)
- Subtraction (-)
- Multiplication (*)
- Division (/)
- Modulo (%)

Modulo is slightly less common but invaluable. It returns the remainder after division.

Here are some examples demonstrating these operations:

```python
# Addition
result = 2 + 3
print(result)  # displays 5

# Subtraction
result = 10 - 5
print(result)  # displays 5

# Multiplication
result = 2 * 3
print(result)  # displays 6

# Division
result = 10 / 5
print(result)  # displays 2.0 (result of division is a float)

# Modulo
result = 7 % 3
print(result)  # displays 1 (7 is 3*2 + 1)
```

Also, you can use operations with **`str`** variables, but this will yield different behaviours:

```python
word1 = "Hello"
word2 = "World"
result = word1 + word2
print(result)  # displays "HelloWorld"
result = word1 + " " + word2
print(result)  # displays "Hello World"
print(word1 * 2)  # displays "HelloHello"
```

Understanding operations and variables in Python enables you to perform calculations which can be utilized for various practical applications such as calculating profits, losses, or rates of return.

## ****Practical Exercise****

To reinforce what you have learned, let us engage in a practical exercise. It is often more insightful to practice by applying the concepts to realistic scenarios.

**Scenario:**

John is an investor with Apple stocks. Each Apple stock is currently valued at \$135. He bought 20 stocks at \$60 each, then acquired an additional 10 at \$145 each. He sold 8 stocks at \$120 each. Now, John is evaluating the financial merits of selling his remaining stocks and is curious about the potential tax implications.

**Tasks:**

1. Calculate and display the total value of the current Apple stocks in John’s portfolio.
2. Calculate and display the average purchase price of John’s Apple stocks.
3. Calculate and display the potential total capital gain if John sells all his Apple stocks now.
4. John holds dual citizenship in France and Belgium. In France, the tax rate on capital gains is 30%. In Belgium, capital gains are tax-exempt up to \$500, and a 40% tax rate applies beyond this threshold. Calculate and display the taxes John would incur on his capital gains in France and Belgium.

A correction example can be found here: https://github.com/RobotTraders/Python_For_Finance/blob/main/exercise_correction_chapter_1.ipynb

Familiarize yourself with variables, mathematical operations, and the different data types in Python.

Variables, Operations and Types

<MDX_YT videoId="J1mYxwJrkt0"/>

## ****Loops****

Loops are a fundamental programming concept, allowing you to execute a chunk of code multiple times. This is particularly useful in finance, where you might need to process data over several periods or perform repetitive calculations. A common use case is **backtesting**. Essentially, it is about simulating a past trading or investment strategy by looping through each price from a specific time period. We will look into this topic in detail towards the end of the course.

Python primarily uses two types of loops: **`for`** and **`while`**.

### **The `for` loop in Python**

The **`for`** loop lets you iterate through a sequence, executing a block of code for each element in the sequence. Sequences can be various types such as lists, tuples, dictionaries, or strings, which we will dive into later.

Let us say you have a bank balance of \$3000, a monthly salary of \$2000, and monthly expenses totaling \$1800. You want to calculate your balance at the end of the year (after 12 months). Here is how you might use a **`for`** loop to solve this:

```python
balance = 3000
salary = 2000
expenses = 1800

for i in range(12):
    balance = balance + salary - expenses
    print("Balance after", i+1, "months", balance, "€")

print("My account balance at the end of the year is:", balance, "€")
```

In this example, we start by setting up three variables: **`balance`**, **`monthly_salary`**, and **`monthly_expenses`**. These variables represent the starting balance, monthly salary, and monthly expenses, respectively.

Next, we use a **`for`** loop to model the transactions over 12 months. The **`range(12)`** function creates a sequence of numbers from 0 to 11 (12 elements in total).

Inside the loop, we update our balance by adding the monthly salary and subtracting the monthly expenses. The **`print()`** function is used to output the balance for each month.

You may notice that the two lines below **`for`** are not aligned at the same level. Indeed, they are **indented**, meaning they are spaced 4 spaces to the left. The indented lines are included within the **`for`** loop, whereas non-indented lines like the last one are not. In practice, this means the display **`"Balance after..."`** will be shown 12 times, whereas the text **`"My account balance..."`** will only be displayed once.

With this little code, we update our balance by adding our salary and deducting our expenses, and use the **`print()`** function to show our new balance as well as the current month. As we previously mentioned, the **`range(12)`** function generates a sequence of numbers from 0 to 11, so we add 1 to ***i*** when displaying the month, to start from month 1 instead of month 0. Lastly, we display the balance at the end of the year. When you run the code, you should get something like this:

![pff2-1.png](https://strapi-project-image-test.s3.eu-west-3.amazonaws.com/pff2_1_745aa6d524.png)

### **The `while` loop in Python**

The **`while`** loop executes a block of code as long as a certain condition remains true. For instance, let's calculate how long it will take to double an investment at a fixed annual interest rate:

```python
initial_investment = 1000
years = 0
interest_rate = 0.05
current_value = initial_investment

while current_value < initial_investment * 2:
    current_value = current_value * (1 + interest_rate)
    years += 1

print("It will take", years, "years to double the investment.")
```

In this example, the code block inside the **`while`** loop will be executed until the **`current_value`** is at least double the initial investment. With each iteration, we update **`current_value`** by adding the interest, and increment **`years`** by 1.

We exit the while loop when the value of **`current_value`** becomes greater than twice our initial investment. At this point, we use the **`print`** function to display the value of our **`years`** variable, which represents the number of years needed to double our investment.

## ****How to use conditions in python****

Conditions are essential in programming. They allow you to execute a specific block of code if a condition is met, and possibly a different block of code if it is not. In Python, you use the keywords **`if`**, **`elif`**, and **`else`** to handle these conditions. **`if`** stands for **if**, **`elif`** for **else if**, and **`else`** is used when conditions are not met.

Have a look at this simple example:

```python
bitcoin_price = 45000

if bitcoin_price <= 30000:
    print("Buy")
elif bitcoin_price > 30000 and bitcoin_price <= 50000:
    print("Sell half")
elif bitcoin_price < 60000:
    print("Hold")
else:
    print("Sell all")
```

In this example, the program prints "Sell half" since the **`bitcoin_price`** is 45,000, which is greater than 30,000 but less than 50,000. Note the use of **`and`** here; it is a **logical operator**. There is also an **`or`** operator.

Here are a few more examples of using **conditions**:

```python
text = "hello"
if text == "hello" or text == "hi":
    print("text is hello or hi")
elif text != "hello":
    print("text is neither hello nor hi")

condition = True
if condition is True:
    print("My condition is true")
else:
    print("My condition is false")
```

Note that we have used the operator **`!=`** which stands for *not equal to*. 

Try to guess what this code will output before running it 😉.

## ****Example: combining loops and conditions in Python****

Let us consider an example where we use both loops and conditions. Imagine you want to figure out how long it will take to reach a savings goal, considering you might get a raise at work.

- Savings goal: €10,000
- Savings per month: €400
- Expected raise in 6 months
- Extra savings per month after raise: €200 (Total €600 per month)

```python
# Variables
savings_goal = 10000
monthly_savings = 400
month_of_raise = 6
raise_amount = 200

# Counter Initialization
months_required = 0
total_savings = 0

# Loop until you reach the savings goal
while total_savings < savings_goal:
    # Check if the raise is in effect and adjust monthly savings
    if months_required == month_of_raise:
        monthly_savings += raise_amount

    # Update total savings
    total_savings += monthly_savings

    # Increment the months counter
    months_required += 1

print(f"You will need {months_required} months to reach your savings goal of €{savings_goal}.")
```

In this example, we are using a **`while`** loop to simulate the process of saving money every month, and checking whether we have hit our savings target. Inside the loop, there is an **`if`** statement that checks if a pay raise has kicked in and adjusts the monthly savings accordingly. We are also using a counter called **`months_needed`** to keep tabs on how many months it is taking to reach our savings goal.

Next, we keep our balance updated by adding in the monthly salary and taking out the expenses, and then we display the updated balance using the **`print()`** function. Alongside, we show which month of the year it is. One thing to remember is that the **`range(12)`** function creates a sequence of numbers starting from 0 to 11, so while displaying the month, we add 1 to **`i`** to make sure the count starts at month 1 and not month 0.

At the end of it all, we display the balance we have accumulated by the year's end.

Take note of the **`+=`** operator. Essentially, **`a += 1`** is shorthand for **`a = a + 1`**. The **`+=`** operator adds the value on its right to the variable on the left, and then assigns the result back to the variable on the left. There are similar operators like **`-=`** and **`*=`**. Have a try:

```python
a = 5
a += 3 # a = a + 3
print(a) # Displays 8

b = 10
b -= 1 # b = b - 1
print(b) # Displays 9

c = 2
c *= 15 # c = c * 15
print(c) # Displays 30
```

## ****Putting it all into practice****

Let us dive in and apply the concepts we have discussed in this section by tackling a straightforward exercise. Picture this—you have invested 1,000 Euros in a mix of financial assets, and you are re curious to find out how many years it will take for your investment to hit a certain target.

Let’s assume you have the following information:

- Initial investment: 1000 Euros
- Capitalization goal: 2000 Euros
- Annual interest rate: 4%

Questions:

1. How many years are we talking about to hit that target?
2. Tinker with the annual interest rate and observe how it shifts the timeline.
3. Throw in a **`for`** loop to experiment with a range of annual interest rates (say, from 2% to 8%, stepping up by 1%) and display how the number of years stacks up for each rate.

This hands-on exercise should give you a firmer grasp on employing loops and conditional statements in Python, especially in a finance setting. A correction example can be found here: https://github.com/RobotTraders/Python_For_Finance/blob/main/exercise_correction_chapter_2.ipynb. Do not hesitate to customize this scenario to suit your own intrigue and delve into other financial what-ifs.

Discover how to use control structures such as loops and conditions to perform repetitive or conditional tasks

Loops and Conditions

<MDX_YT videoId="rsMONW79TeY"/>

In this chapter, we are delving into two fundamental data structures in Python: lists and dictionaries. These structures will allow you to store and manage data more efficiently within your programs. They prove to be especially useful when dealing with financial data, such as price time series, asset portfolios, or customer information.

## **Lists in Python**

Alright, let us talk about lists. They are like the toolboxes of Python; you can keep multiple items in them. Think of lists as a series of boxes where you can store different values.

## ****Creating a list****

Here is how to create a list containing textual values:

```python
assets = ["stocks", "bonds", "real estate", "cash"]
print(assets)
```

This list contains four elements: all are strings.

### ****Accessing list elements****

To access an element from a list, use the element's index in square brackets. Indices in Python start at 0 for the first element. You can also access elements from the end of the list using negative indices. Try it out with this example:

```python
assets = ["stocks", "bonds", "real estate", "cash"]
print(assets[0])  # Displays "stocks"
print(assets[2])  # Displays "real estate"
print(assets[-1])  # Displays "cash"
```

### ****Modifying list elements****

Lists are **mutable**, which means you can change the value of an element using its index. For instance:

```python
assets = ["stocks", "bonds", "real estate", "cash"]
assets[-2] = 15  # You can have a list with different types inside
print(assets)  # Displays ["stocks", "bonds", 15, "cash"]
```

### ****Adding and removing list elements****

To add an element to the end of a list, use the **`append()`** method:

```python
assets = ["stocks", "bonds", "real estate", "cash"]
assets.append("commodities")  # Adds "commodities" to the end of the list
```

To insert an element at a specific position, use the **`insert()`** method. Provide the index (the index of an item in a list is somewhat like its ranking. In the 'assets' list, "stocks" is at index 0) and the value:

```python
assets.insert(1, "ETFs")  # Inserts "ETFs" at position 1 (the second element)
```

To remove an element from a list, use the **`remove()`** method and provide the value of the element:

```python
assets.remove("cash")  # Removes "cash" from the list
```

To remove an element at a specific index, you can use the **`del`** function:

```python
del assets[0]  # Removes the first element from the list
```

Feel free to experiment using **`print()`** to observe the changes.

### ****Navigating a list with loops****

As mentioned earlier, loops are a superb method for iterating through and working with lists. **`for`** loops can be used to traverse a list's items and conduct operations on each element. Here are a few scenarios where loops are used to navigate a list:

**Example 1: displaying every element in a list**

```python
assets = ["stocks", "bonds", "real estate", "cash"]

for asset in assets:
    print(asset)
```

In this instance, the **`for`** loop cycles through each element in the **`assets`** list, storing the current item in the **`asset`** variable at each iteration. The **`print()`** function is then invoked to show the **`asset`** value.

**Example 2: implementing an operation on all list elements**

Let us assume we have a list of stock prices and want to calculate the return on each one. Here is how we can achieve that using a **`for`** loop:

```python
prices = [100, 120, 80, 150]
returns = [] # Starting with an empty list that we'll add elements to

for price in prices:
    returns.append(price * 1.1)  # Computes the return following a 10% increase

print(returns)
```

In this case, we have a **`prices`** list containing various stock prices. We also create an empty **`returns`** list to hold our results. The **`for`** loop cycles through the **`prices`** list, calculating a 10% return on each price and appending the result to the **`returns`** list. By the end, the **`returns`** list will contain the calculated returns for each stock.

**Example 3: Iterating Through a List Using `range()` and `len()`**

The **`len()`** function provides the length (i.e., the number of elements) of an iterable object, like a list. You can use this in combination with the **`range()`** function to iterate over a list by element index:

```python
assets = ["stocks", "bonds", "real estate", "cash"]
print(len(assets)) # Outputs 4

for i in range(len(assets)):
    print("Index", i, ":", assets[i])
```

In this illustration, **`len(assets)`** returns 4, since there are four elements in the **`assets`** list. **`range(len(assets))`** then generates a sequence of indices from 0 to 3, inclusive.

The **`for`** loop then traverses these indices, using **`i`** to access the corresponding element in the **`assets`** list. The **`print()`** function is used to display both the index and value of each element.

This technique is particularly useful when you need to access list elements via their index or need to alter list elements while iterating through the list.

## ****Understanding Python Dictionaries****

A dictionary in Python is an unordered, mutable, and indexed collection of items. Unlike lists, dictionaries use keys to access items rather than indices. Dictionaries are particularly useful for storing and organizing data in the form of key-value pairs. To create a dictionary, we use braces **`{}`** and separate keys and values by colons **`:`**.

### ****Creating a dictionary****

Here is an example of creating a dictionary that holds information about different types of investments:

```python
asset_info = {
    "stocks": "investment in company shares",
    "bonds": "investment in debt securities",
    "real estate": "investment in real estate",
    "cash": "investment in cash or near-cash items"
}
print(asset_info)
```

In this example, the **`asset_info`** dictionary contains four items, each represented by a key-value pair. The keys are strings (the types of assets), and the values are textual descriptions of the assets.

### ****Accessing items in a dictionary****

To access an item in a dictionary, we use the corresponding key in brackets **`[]`**:

```python
asset_info = {
    "stocks": "investment in company shares",
    "bonds": "investment in debt securities",
    "real estate": "investment in real estate",
    "cash": "investment in cash or near-cash items"
}

print(asset_info["stocks"])  # Displays "investment in company shares"
print(asset_info["real estate"])  # Displays "investment in real estate"
```

### ****Modifying items in a dictionary****

Dictionaries are mutable, meaning you can change the value of an item and for this you need to use its key:

```python
asset_info["stocks"] = "investment in company stakes"
print(asset_info["stocks"])  # Displays "investment in company stakes"
```

### ****Adding and deleting items from a dictionary****

To add a new item to a dictionary, simply assign a value to a new key:

```python
asset_info["commodities"] = "investment in commodities"
print(asset_info)
```

To delete an item from a dictionary, you can use the **`del`** function indicating the key of the item:

```python
del asset_info["cash"]
print(asset_info)
```

### ****Iterating over a dictionary with loops****

You can use loops to iterate over a dictionary and perform operations on its items. Dictionaries can be traversed in different ways, such as by key, by value, or by key-value pairs.

**Example 1: Iterating over a dictionary by key**

To iterate over a dictionary by key, you can simply use a **`for`** loop:

```python
asset_info = {
    "stocks": "investment in company shares",
    "bonds": "investment in debt securities",
    "real estate": "investment in real estate",
    "cash": "investment in cash or near-cash items"
}

for key in asset_info:
    print(key) # Displays the key (stocks, bonds...)
    print(asset_info[key]) # Displays the value associated with the key
```

In this example, the **`for`** loop goes through all the keys in the **`asset_info`** dictionary and uses the **`key`** variable to store the current key at each iteration. The **`print()`** function is used to display the key.

**Example 2: Iterating Over a Dictionary by Value**

If you are re interested in the values contained in the dictionary, you can iterate over them directly using the **`.values()`** method. Here's how you can do it:

```python
asset_info = {
    "stocks": "investment in company shares",
    "bonds": "investment in debt securities",
    "real estate": "investment in real estate",
    "cash": "investment in cash or near-cash items"
}

for value in asset_info.values():
    print(value)  # This will print the value

```

This code will print out all the values in the **`asset_info`** dictionary, which are the definitions of each investment type.

**Example 3: Iterating Over a Dictionary by Key-Value Pairs**

If you want both the keys and the values at the same time, you can use the **`.items()`** method, which returns a *tuple* (an object similar to a list but immutable) for each key-value pair. Here is how you can use it:

```python
asset_info = {
    "stocks": "investment in company shares",
    "bonds": "investment in debt securities",
    "real estate": "investment in real estate",
    "cash": "investment in cash or near-cash items"
}

for key, value in asset_info.items():
    print(f"The key is {key} and the value is {value}")

```

In this code, **`key`** and **`value`** are variables that you define to hold the key and value of each item, respectively. The **`f-string`** (formatted string) allows you to include the values of variables directly in a string. This code will print out all the keys and values in the **`asset_info`** dictionary.

### ****More complex lists and dictionaries****

Dictionaries and lists are data structures that can become complex. For example, we can have lists of lists, or lists of dictionaries with lists as values. Have a look at this example below:

```python
my_portfolio = {
    "APPL":{
        "information": {
            "common_name": "Apple",
            "market_is_open": True,
            "currency": "dollar",
        }, 
        "quantity": 24,
        "current_price": 165.45,
        "last_5_days_price": [160.25, 163.87, 161.58, 164.01, 165.45]
    },
    "GOOG":{
        "information": {
            "common_name": "Google",
            "market_is_open": False,
            "currency": "euro",
        }, 
        "quantity": 10,
        "current_price": 215.72,
        "last_5_days_price": [217.56, 220.51, 221.90, 219.19, 215.72]
    }
}
```

This dictionary contains information about a portfolio of stocks. The keys are the stock symbols (like "APPL" for Apple, and "GOOG" for Google), and the values are other dictionaries, which themselves contain more detailed information about each stock.

Here are some commands to display values:

- Display the price of GOOG from 3 days ago:

```python
print(my_stocks["GOOG"]["last_5_days_price"][-3])
```

- Display whether the market is open for the AAPL stock:

```python
print(my_stocks["APPL"]["information"]["market_is_open"])
```

## ****Specificity of lists and dictionaries****

Lists and dictionaries are slightly special types compared to **`int`** or **`string`**. They behave in a way that can seem counterintuitive, especially in the case of equality. An example is worth a thousand words, so here is an example of their peculiar behaviors.

```python
list1 = [1, 2, 3]
list2 = []

list2 = list1
list2[0] = 542

print(list1) # Displays [542, 2, 3]
```

As you can see in this example, **`list1`** was never modified. However, its value changed when we changed the **`list2`** variable. This is due to the concept of **reference,** which is found in many programming languages.

In the case of a list like in our example, when we write **`list2 = list1`**, it does not mean that the value of **`list2`** equals the value of **`list1`**. Writing **`list2 = list1`** means that the **`list2`** variable **points** to the **`list1`** variable. This implies that if you modify **`list2`**, it also modifies **`list1`**, and vice versa.

To avoid this behaviour, you can use the **`.copy()`** method, which will create an independent copy. Be careful though, if there are other lists or dictionaries inside the list, the copy will not be completely independent, and we move towards the concept of **deepcopy**, which we will not detail here.

## ****Practical Application: Tracking Transactions****

To apply the concepts discussed in this part, let us try to solve an exercise. As usual, we invite you to try to go further than this exercise by making it more complex. That is how you progress the most 😉.

For this exercise, we will provide a **list of transactions** on cryptocurrencies, and the goal will be to extract certain information from them.

Here is the list of transactions in the form of a list of dictionaries:

```python
transactions = [
    {
        "symbol": "BTC",
        "type": "buy",
        "quantity": 0.7,
        "price": 27090.4,
        "date": "2022-03-07",
    },
    {
        "symbol": "BTC",
        "type": "sell",
        "quantity": 0.1,
        "price": 26561.4,
        "date": "2022-04-12",
    },
    {
        "symbol": "BTC",
        "type": "buy",
        "quantity": 0.4,
        "price": 28090.4,
        "date": "2022-05-21",
    },
    {
        "symbol": "BTC",
        "type": "buy",
        "quantity": 0.3,
        "price": 25090.4,
        "date": "2022-06-02",
    },
    {
        "symbol": "BTC",
        "type": "sell",
        "quantity": 0.5,
        "price": 32034.67,
        "date": "2022-06-29",
    },
    {
        "symbol": "ETH",
        "type": "buy",
        "quantity": 6,
        "price": 1754,
        "date": "2022-05-21",
    },
    {
        "symbol": "ETH",
        "type": "buy",
        "quantity": 2,
        "price": 1950,
        "date": "2022-06-02",
    },
    {
        "symbol": "ETH",
        "type": "sell",
        "quantity": 3,
        "price": 2100.67,
        "date": "2022-06-29",
    },
]
```

1. From this list, using a loop, create a dictionary with keys for each of the cryptocurrencies present in the transactions and an empty dictionary as its value. This should give you the following result:

```python
portfolio = {
    "BTC": {},
    "ETH": {},
}
```

2. From this list, using a loop, add to the previous dictionary a key "quantity" which has as its value the quantity still present in the portfolio. (You will need to use conditions to check if these are purchases or sales).
3. Considering today's BTC price of **`35000`** and that of ETH of **`2400`**, add to our previous dictionary our PNL (i.e., our profit or our loss) for each of the cryptos, with the key **`pnl`**.
4. Add to the dictionary the average purchase price for each of the cryptos.
5. (Bonus) For each of the cryptos: display the quantity purchased after the date of **`2022-06-02`**. For this, you will need to find out how to interact with dates; you need to use the **`datetime`** library.

This exercise should help you better understand the use of loops and conditions applied to dictionaries and lists in Python in the context of finance. A correction example can be found here: https://github.com/RobotTraders/Python_For_Finance/blob/main/exercise_correction_chapter_3.ipynb. Feel free to adapt this example to your own needs and explore other financial scenarios. 

Learn to manipulate lists and dictionaries to store and organize your data efficiently.

Lists and Dictionaries

<MDX_YT videoId="WYEwvnpLD8E"/>

In this chapter, we will explore two other important concepts in Python programming: **functions** and **classes**. Functions allow us to group reusable code blocks, while classes enable us to create objects with specific attributes and methods. Both these concepts are critical for structuring and organizing your code more efficiently, especially when tackling complex financial problems.

## **Functions**

Functions are reusable code blocks that perform a specific task. They help break down a program into smaller, more understandable parts. To create a function in Python, we use the keyword **`def`** followed by the function name and a pair of parentheses containing the function's parameters. The block of code to be executed is indented under the function definition.

Here is a simple function example for calculating compound interest:

```python
def compound_interest(principal, rate, time):
    amount = principal * (1 + rate) ** time
    interest = amount - principal
    return interest
```

In this example, we have defined a function called **`compound_interest`** that takes three arguments: the initial amount (**`principal`**), the interest rate (**`rate`**), and the duration in years (**`time`**). The function calculates compound interest using the standard formula and returns the result. The return statement allows us to retrieve a value, for example by doing **`result = my_function()`**. It is not necessary to return a value if no particular result needs to be retrieved (for instance, if we just want to display something). 

To call this function, simply use it with the appropriate arguments:

```python
initial_investment = 1000
interest_rate = 0.04
years = 5

interest = compound_interest(initial_investment, interest_rate, years)
print(f"The compound interest after {years} years is {round(interest,2)} €")
```

This example calculates compound interest for an initial investment of 1000 €, an interest rate of 4%, and a duration of 5 years.

In this example, you also discover a new way of assigning a value to a string, thanks to the use of **`f`** before **`""`** and the use of **`{}`** inside the **`""`**.

```python
name = "Thomas"
age = 41.75254
print(f"Hello {name}, you are {round(age, 2)} years old")
# Displays "Hello Thomas, you are 41.75 years old"
```

Here we use the `round()` function to round the age value to 2 decimal places

Functions are particularly useful for avoiding code duplication and simplifying repetitive tasks. For example, you can reuse the **`compound_interest`** function to calculate compound interest for different investments and interest rates.

### **Optional arguments and default values**

It is possible to define default values for certain arguments of a function, making them optional when calling the function. For instance, we can set a default value for the annual interest rate:

```python
def compound_interest(principal, time, rate=0.05):
    amount = principal * (1 + rate) ** time
    interest = amount - principal
    return interest

# Using the function without specifying the interest rate
investment = 1000
years = 5

earned_interest = compound_interest(investment, years)
print(f"The compound interest over {years} years for an investment of {investment} € is {round(earned_interest, 2)} €.")
```

Here, **`rate=0.05`** sets a default value of 0.05 for the rate parameter. This means that **`rate`** is now an optional argument; if we don't provide a value for **`rate`** when calling the function, Python will use the default value of 0.05.

Let us see this in action. Suppose we have an investment of 1000 euros, and we want to calculate the compound interest over 5 years. We do not specify the interest rate, so Python will use the default rate of 0.05:

```python
investment = 1000
years = 5

earned_interest = compound_interest(investment, years)
print(f"The compound interest over {years} years for an investment of {investment} € is {round(earned_interest, 2)} €.")
```

But what if we want to calculate the interest with a different rate, say 7% (or 0.07)? We can simply provide a value for **`rate`** when calling the function like this:

```python
earned_interest = compound_interest(investment, years, 0.07)
print(f"The compound interest over {years} years for an investment of {investment} € at a rate of 7% is {round(earned_interest, 2)} €.")
```

Alternatively, this syntax would work as well:

```python
earned_interest = compound_interest(investment, years, rate=0.07)
print(f"The compound interest over {years} years for an investment of {investment} € at a rate of 7% is {round(earned_interest, 2)} €.")
```

In a more general way, when defining a function in Python, you can also specify arguments by their parameter name, using the syntax **`parameter=value`**. This is especially useful when a function has many parameters, or when you want to only specify some of the optional parameters. This corresponds to what we did with the **`rate`** argument in the previous example, except that here we do not provide a default value. In other words, **`value`** of **`parameter=value`** will necessarily have to be provided when the function is called.

Let us use our **`compound_interest`** function as an example:

```python
investment = 1000
years = 5

earned_interest = compound_interest(principal=investment, time=years, rate=0.07)
print(f"The compound interest over {years} years for an investment of {investment} € is {round(earned_interest, 2)} €.")
```

Here, we called the function with arguments specified by their names.

When you call a function with arguments specified by their names, the order of the arguments does not matter. This is another advantage of specifying parameters by name. For instance, the following call to **`compound_interest`** is perfectly valid and will give the same result:

```python
earned_interest = compound_interest(rate=0.07, principal=investment, time=years)
print(f"The compound interest over {years} years for an investment of {investment} € is {round(earned_interest, 2)} €.")
```

Here, even though **`rate`** is the third parameter in the function definition, we provided it first in the function call. Python knows which value goes with which parameter because we specified the parameters by their names.

In summary, function arguments in Python are classified as positional or keyword:

- **Positional arguments** are identified by their order and matched accordingly.
- **Keyword arguments** are distinguished by an associated keyword (**`parameter=value`**), making their order flexible as they are matched by name.

And you can have as many of both types as you want in a function. Just remember, when defining a function, **positional arguments must always precede keyword arguments** because positional arguments are order-dependent, while keyword arguments are name-identified.

This may sound a bit technical, but do not worry, you will get used to it very fast 😉.

## **Python Classes**

A class in Python provides a blueprint for creating objects that follow the same structure. These objects can have *attributes* (variables) and *methods* (functions). Classes play a significant role in structuring and organizing code, as they encapsulate related data and functionalities.

### **Class example: Bank account**

For instance, if we need to model a bank account, offering functions to deposit, withdraw, and check the balance, we can design a **`BankAccount`** class:

```python
class BankAccount:
    def __init__(self, balance=0):
        self.balance = balance

    def deposit(self, amount):
        self.balance += amount
        print(f"You've deposited {amount} €. New balance: {self.balance} €.")

    def withdraw(self, amount):
        if amount > self.balance:
            print("Insufficient balance.")
        else:
            self.balance -= amount
            print(f"You've withdrawn {amount} €. New balance: {self.balance} €.")

    def display_balance(self):
        print(f"Your balance is {self.balance} €.")
```

Here, the **`BankAccount`** class has a **`balance`** *attribute* and three *methods*: **`deposit`**, **`withdraw`**, and **`display_balance`**. The **`__init__`** method, known as the **c***onstructor*, is invoked whenever an instance of the class is created.

### ****Using the BankAccount class****

Let us explore the use of the **`BankAccount`** class by creating a few instances and performing standard banking operations like deposits and withdrawals.

```python
# Create a bank account with an initial balance of 500 €
account1 = BankAccount(500)

# Create another bank account with an initial balance of 2000 €
account2 = BankAccount(2000)

# Display the balance of both accounts
account1.display_balance()
account2.display_balance()

# Deposit 1000 € into account1
account1.deposit(1000)

# Withdraw 300 € from account1
account1.withdraw(300)

# Display the balance of both accounts after the operations
account1.display_balance()
account2.display_balance()

# Deposit 1000 € into account2
account2.deposit(1000)

# Withdraw 1500 € from account2
account2.withdraw(1500)

# Display the balance of both accounts after the operations
account1.display_balance()
account2.display_balance()
```

Here, we have created two **`BankAccount`** instances with different initial balances, performed deposits and withdrawals on both accounts, and displayed their balances.

## ****Exercise: The `Portfolio` Class****

In this exercise, you will design a **`Portfolio`** class to manage an investment portfolio. This portfolio will hold various financial assets, like stocks or bonds, along with their respective quantities and unit values.

You need to define and use the **`Portfolio`** class to:

1. Add a financial asset to the portfolio with its quantity and unit value.
2. Remove a financial asset from the portfolio.
3. Update the quantity or unit value of a financial asset.
4. Compute the total value of the portfolio.

### **Instructions**

1. Define a **`Portfolio`** class with an **`assets`** attribute, which will be a dictionary to store the financial assets and their information (quantity and unit value).
2. Implement an **`add_asset`** method to add a financial asset to the portfolio. This method should take the asset name, quantity, and unit value as parameters. If the asset already exists in the portfolio, the quantity and unit value should be updated.
3. Create a **`remove_asset`** method to remove a financial asset from the portfolio. This method should take the asset name as a parameter. If the asset is not found in the portfolio, an appropriate message should be printed.
4. Write an **`update_asset`** method to update the quantity or unit value of a financial asset. This method should take the asset name, and the new quantity and unit value as parameters. If the asset doesn't exist in the portfolio, an appropriate message should be displayed.
5. Finally, implement a **`get_total_value`** method to compute and return the total value of the portfolio. This should be the sum of the products of the quantity and unit value for each asset.

Here is a starting template for the **`Portfolio`** class:

### **Usage Example**

```python
# Creating an empty portfolio
my_portfolio = Portfolio()

# Adding financial assets
my_portfolio.add_asset("AAPL", 10, 150)
my_portfolio.add_asset("GOOG", 5, 1000)
my_portfolio.add_asset("TSLA", 8, 800)

# Updating a financial asset
my_portfolio.update_asset("AAPL", 12, 160)

# Removing a financial asset
my_portfolio.remove_asset("GOOG")

# Calculating the total value of the portfolio
total_value = my_portfolio.get_total_value()
print(f"The total portfolio value is {total_value} €.")
```

This exercise provides an opportunity to practice creating and using Python classes within the context of finance. A correction example can be found here: https://github.com/RobotTraders/Python_For_Finance/blob/main/exercise_correction_chapter_4.ipynb. Feel free to tailor this example to your own needs and look into other functionalities related to investment portfolio management.

## **Summary**

In this chapter, we've explored Python functions and classes - pivotal tools for structuring and organizing your code. We've learned how to define and utilize them through finance-related real-world examples.

Mastering these concepts enables you to write more intricate and modular programs, simplifying both the maintenance and evolution of your code.

As we progress in this course, we'll look into into specific finance libraries. These tools will equip you to analyze financial data and devise predictive models. Stay tuned!

Discover how to create and use functions and classes to structure your code and facilitate reuse.

Functions and Classes in Python

<MDX_YT videoId="E9fATZJKxtA"/>

In the realm of finance, Excel is undeniably the go-to software. In this episode, we'll explore a library that allows us to accomplish everything Excel does,  more and even better ! 📊

## **Introduction to Pandas** 

Pandas is a renowned Python library, boasting flexible data structures and efficient data analysis tools. It's especially tailored for financial data processing and is a favourite among financial analysts. Pandas runs on the NumPy platform, a cornerstone for scientific calculations in Python.

In this section, we'll delve into the essentials of Pandas, spanning from data creation and manipulation to its analysis.

### **Installing and Importing Pandas**

With Pandas, we must introduce the concept of **libraries** in Python. Python's strength lies in its vast array of libraries, making development a breeze. Libraries are installed using the pip package manager that's bundled with Python installations.

To get started with Pandas, first, install it with the command:

```python
pip install pandas
```

Once set up, import Pandas into your Python script with:

```python
import pandas as pd
```
The alias "pd" is a standard convention in the Pandas community, offering a convenient shorthand for future use.

### **The DataFrame and Series Objects**

At the heart of Pandas are the DataFrame and Series objects. A DataFrame is a two-dimensional data array, resembling a database table or an Excel sheet. A Series, on the other hand, is a one-dimensional array, much like a list or a column in a DataFrame. Both have Indexes, which, if unspecified, default from 0 to the size of the data set, assigning an index to every row.

![pandas dataframe](https://strapi-project-image-test.s3.eu-west-3.amazonaws.com/pandas_dataframe65415_091e4118ff.png)

Here's a hands-on example:

```python
import pandas as pd

data = {
    "date": ["2022-01-01", "2022-01-02", "2022-01-03", "2022-01-04"],
    "close": [150.0, 152.0, 153.0, 154.0],
    "volume": [1000, 1200, 1300, 1250],
}

df = pd.DataFrame(data)

print(df) # Displays the DataFrame
print(df.index) # Showcases the Indexes
print(df["close"]) # Reveals the "close" Series
```

In this instance, we've constructed a dictionary with three lists: date, close, and volume, each illustrating dates, closing prices, and transaction volumes respectively. We then utilize pd.DataFrame() to create a DataFrame from this data. The print statements show how we can engage with various parts of the DataFrame.

## **Manipulating and Analyzing Financial Data with Pandas**

Pandas is laden with features that simplify data manipulation and analysis. Here's a quick rundown:

### **Selecting Rows or Columns**

These snippets show how to cherry-pick columns, rows, or values:

```python
close_column = df["close"]
print(close_column)

first_row = df.iloc[0]
print(first_row)

first_close = df.iloc[0]["close"]
print(first_close)

close_volume_columns = df[['volume', 'close']]
print(close_volume_columns)
```

To select rows based on a condition, you can use comparison operations on the DataFrame columns. For instance, to select all rows where the volume is greater than 1210:

```python
best_volume_rows = df.loc[df['volume'] > 1210]
print(best_volume_rows)
```

### **Reading Data from CSV or Excel Files**

Financial data are typically stored in CSV or Excel files. CSVs are tabulated data where entries are separated by commas or other delimiters. Pandas streamlines the process of reading these files into DataFrames:

![pandas data](https://strapi-project-image-test.s3.eu-west-3.amazonaws.com/pandas8484_742250827b.png)

You can access a day-by-day APPL stock price CSV file here:  [https://github.com/RobotTraders/Python_For_Finance/blob/main/APPL_1D.csv](https://github.com/RobotTraders/Python_For_Finance/blob/main/APPL_1D.csv) 

To download, click the highlighted download icon (circled in red in the image below).

![olhcv csv github](https://strapi-project-image-test.s3.eu-west-3.amazonaws.com/ohlcv6845_537b5870df.png)

> GitHub is a powerhouse for software development, enabling collaboration and version control. It's also the home for many open-source projects, including those by Crypto Robot (https://github.com/CryptoRobotFr) and Robot Traders.

The data follows the OHLCV format, which is standard for representing asset prices over specified periods (here, every 4 hours): Open, High, Low, Close, and Volume.

After downloading and placing the CSV in your directory, load it into a DataFrame:

```python
df = pd.read_csv("APPL_1D.csv)
print(df)
```

In this case, we've read the file to store its content in the df variable, a process analogous to when we created our first DataFrame like this:

```python
data = {
    "date": ["2022-01-01", "2022-01-02", "2022-01-03", "2022-01-04"],
    "close": [150.0, 152.0, 153.0, 154.0],
    "volume": [1000, 1200, 1300, 1250],
}

df = pd.DataFrame(data)
```
But in this situation, rather than manually specifying our data, it comes directly from a file.

## **Exploring and Manipulating Financial Data**

Once you've stored your financial data in a Pandas DataFrame, you can easily explore and manipulate this data. For instance, you can compute descriptive statistics, and filter data based on certain conditions.

### **Exploring Financial Data**

Initially, a best practice when working with a file is to quickly explore its data to ensure the quality of the information. In reality, your data will rarely be perfect and ready for analysis. More often than not, you'll need to "clean" your data. To swiftly inspect your data, you can use the describe() method.

```python
df = pd.read_csv("APPL_1D.csv")
print(df.describe())
```

![panas ohlcv](https://strapi-project-image-test.s3.eu-west-3.amazonaws.com/panas_ohlcv654_af5d437ab1.png)

It's possible that your display shows scientific notations like e+03. This means, in this context, multiplication by 10 to the power of 3, or multiplication by 1000. Such notation becomes handy when dealing with very small or large numbers.

In our case, it hinders our analysis. So, you can insert the following line of code after importing pandas to round all numbers to two decimal places:  

```python
pd.set_option('display.float_format', lambda x: '%.2f' % x)
```

The describe() method provides several insights for each column. The count line indicates the number of entries. The lines labeled 25%, 50%, and 75% refer to quantiles, which we won't delve into here. The std line indicates the standard deviation. The other lines (mean, min, max) are self-explanatory.

By closely inspecting, we notice several issues in our data:

The presence of the Volume MA column, which we don't understand and seems irrelevant. The time and open columns have 2920 entries, while the others only have 2919. We need to find the row where other columns are missing values. The time column appears in a numeric format that we can't interpret.  On closer observation, we see that the minimum of the open column is 0. It's unlikely that the opening price of APPLE was ever \$0.

- To remove the Volume MA column:
    
    ```python
    df = df.drop("Volume MA", axis=1)
    ```
    
- To find the missing values in the close column:
    
    ```python
    print(df.loc[df["close"].isnull()])
    ```
    
    
![pandas row](https://strapi-project-image-test.s3.eu-west-3.amazonaws.com/641358_0ee52f1e79.png)
    
Here, we see that the last row (with index 2919) contains values for the time and open columns but not for the others (NaN values mean “Not a Number”). This could be due to data being collected before the day ended, making the highest price, lowest price, volume, and closing price unavailable. To avoid complications later on, it's best to delete this row:
    
    ```python
    df = df.drop(2919)
    ```
    
The time column contains very large numbers. After some research, we understand that these are dates in timestamp format. The numbers represent seconds elapsed since 1970. Fortunately, pandas has a method to convert these numbers to dates:

```python
df["time"] = pd.to_datetime(df["time"], unit='s')
print(df["time"])
```

![pandas date](https://strapi-project-image-test.s3.eu-west-3.amazonaws.com/pandas_date_3cdf022a2e.png)

Now, with the dates being readable, we can replace our default indices (1, 2, 3…). Once that's done, we won't need our time column anymore and can delete it:

```python
df = df.set_index(df['time'])
del df['time']
```

With our dates set as indices, it allows us to easily select one or multiple rows using date references:

```python
print(df.loc["2023-05-04"]) # Displays the values for May 4, 2023
print(df.loc["2023-03"]) # Displays all the values for March 2023
print(df.loc["2023-03-03":"2023-03-15"]) # Displays the values from March 3 to March 15, 2023
```

### **Creating New Columns from Existing Ones**

Pandas enables you to conveniently create new columns based on existing ones. For instance, you can make a column that represents the average of the opening and closing prices:

```python
df["mean_open_close"] = (df["open"] + df["close"]) / 2
print(df["mean_open_close"])
```

What's crucial to understand here is that this operation is applied to all rows. You don't need to use a loop to apply this change to each row. Loops in Python can be time-consuming during execution, so it's beneficial to avoid them when possible.

### **Computing Descriptive Statistics**

Pandas offers functions to compute descriptive statistics on your financial data. For instance, you can calculate the mean, median, standard deviation, and other statistics for stock prices of any company:

```python
# Calculate descriptive statistics for each column
mean_prices = df["close"].mean()
median_prices = df["close"].median()
std_prices = df["close"].std()

print("Average closing prices:", mean_prices)
print("Median closing prices:", median_prices)
print("Standard deviation of closing prices:", std_prices)
```

### **Calculating a Moving Average**

A moving average is a rolling price average. For example, the moving average value for a particular date would be the average of the prices for the previous 50 dates. To calculate rolling values with pandas, use rolling(50) followed by the operation you want to execute. For a 50-day moving average, you'd use rolling() followed by mean():

```python
df["moving_average"] = df["close"].rolling(50).mean()
```

### **Calculating Returns and Volatility**

After loading and prepping your data, you can conduct various financial analyses. For instance, you can calculate daily returns and the volatility of your asset. Volatility is a measure of the risk associated with an investment, typically calculated as the standard deviation of returns over a set period.

Here's an example of calculating daily returns and 30-day rolling volatility using Pandas:

```python
# Calculate daily returns
df["daily_returns"] = df["close"].pct_change()

print(df["daily_returns"].head())

# Calculate 30-day rolling volatility
df["volatility"] = df["daily_returns"].rolling(window=30).std()

print(df["volatility"].head())
```

In this example, we use the pct_change() method to determine daily returns from stock prices. Then, we use the rolling() method to create a 30-day rolling window, followed by the std() method to determine the standard deviation (i.e., volatility) of returns over this rolling window.

Note that the first 29 rows of the volatility_30 column will have NaN (Not a Number) values, as there's not enough data to determine 30-day volatility for these rows. The head() method displays the first five rows.

### **Sorting Data**

To sort data based on a column, you can use the sort_values() method on your DataFrame. For example, to sort data in ascending order by 30-day volatility:

```python
df["daily_returns"] = df["close"].pct_change()
df["volatility"] = df["daily_returns"].rolling(window=30).std()
print(df.sort_values(by="volatility", ascending=True))
print(df.sort_values(by="daily_returns", ascending=True))
```

In this example, we first add 2 new columns to our DataFrame df. We then simply display our DataFrame df sorted from the lowest to highest volatility. Next, we display the days with the highest returns, from lowest to highest.

### **Mastering Your Data**

Pandas is a comprehensive library, and we won't be able to explore all its features here.

It's essential to recognize that nearly any data manipulation is feasible with Python and the Pandas library. Your limits will be your technical skills and most importantly, your algorithmic capabilities. What we mean here is that the most significant challenge is often knowing precisely what you want to do, enabling you to search for the information online or in the official Pandas documentation. ChatGPT can be a valuable resource in this regard.

As we've previously highlighted, hands-on practice is the key. That's why we're presenting a practical exercise to reinforce your learning. But remember, practicing with your own data or problems is even better.

 ## **Application Exercise: The Titanic**

For this exercise, we'll analyze data related to the Titanic's tragic voyage. You can find a CSV file representing each of the Titanic passengers, including their names, genders, classes (Pclass), ages, and other information here: https://github.com/RobotTraders/Python_For_Finance/blob/main/TITANIC.csv.

![titanic](https://strapi-project-image-test.s3.eu-west-3.amazonaws.com/titanic65465_ab991df8ba.png)

### **Instructions**

Start by downloading the CSV file. Place it in your project directory, import Pandas, and load the CSV into a DataFrame:

```python
df = pd.read_csv('TITANIC.csv')
```

Please note, for this exercise, you might need methods we haven't discussed yet, like value_counts() or groupby(). The goal is for you to conduct independent research.

### **Questions**

1. Display the dataset's first ten rows to get an overview.
2. Determine the dataset's dimensions (i.e. the number of rows and columns).
3. Count the number of passengers in each class (Pclass).
4. Calculate the average age of passengers. Identify the youngest and oldest passengers by name.
5. Determine how many passengers survived and how many perished.
6. Count the number of male and female passengers aboard the Titanic.
7. How many men and women survived?
8. What's the survival rate by class?
9. Add a new 'FamilySize' column that sums the 'SibSp' (number of siblings/spouses aboard) and 'Parch' (number of parents/children aboard) columns.
10. Identify the non-survivor with the most significant family on board.

A correction example can be found here: https://github.com/RobotTraders/Python_For_Finance/blob/main/exercise_correction_chapter_5.ipynb

Learn how to use Pandas, a powerful library for data manipulation and analysis, especially suited for working with financial data.

The Pandas library, the best library for financial data

<MDX_YT videoId="5GsgrVh5Dqs"/>

Pandas is a Python **library** that offers powerful and easy-to-use data structures for data analysis. It's particularly effective for working with structured or tabular data, such as financial data.

One of the major benefits of using Pandas is its speed. By employing **vectorized operations** instead of loops, Pandas can perform operations on entire datasets much faster than traditional loop-based code would allow. Vectorized operations are also more readable and concise, making the code easier to understand and maintain.

In this episode, we'll show you how to unlock the full potential of Pandas for analyzing cryptocurrency market data. We'll focus on concrete examples to illustrate the concepts, hoping that you can then apply them to your own financial data analysis problems.

## Importing and Preparing Data

Before diving into the analysis, we first need to prepare our data. In this training, we'll use cryptocurrency data available on the https://github.com/RobotTraders/Python_For_Finance GitHub repository . You should be able to download 10 CSV files representing the prices of various cryptocurrencies. These data are organized in CSV files, each containing the **OHLCV** (Open, High, Low, Close, Volume) data for a specific cryptocurrency.

Each CSV file includes a series of measurements, including the opening price, the highest price, the lowest price, the closing price, and the trading volume for each period (hourly period).

However, to effectively analyse these data, it would be more practical to consolidate them into a single **DataFrame**. This will allow us to work with all the data in a unified manner, rather than handling each file separately. Additionally, using a single DataFrame, we can easily perform comparative analyses between different cryptocurrencies.

### Importing Data

The first step is to download all the available files from this link, and then place them in your directory.

This code will read the CSV file for each cryptocurrency, transform it into a DataFrame, and store it in a **dictionary** named **`df_list`** which has the names of the cryptocurrencies as keys.

```python
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns

crypto_list = ["ADA", "BCH", "BNB", "BTC", "ETC", "ETH", "LINK", "LTC", "XLM", "XRP"]
df_list = {}
for crypto in crypto_list:
    df_list[crypto] = pd.read_csv(f"{crypto}-USDT.csv")
    df_list[crypto]["date"] = pd.to_datetime(df_list[crypto]["date"], unit="ms")
    df_list[crypto] = df_list[crypto].set_index("date")
```

We notice that the files always have the format **`{crypto}-USDT.csv`**. To read the files, we will create a list with the names of the cryptocurrencies that we will iterate through with a **`for`** loop to read the csv file and create an associated DataFrame. We will then transform the date column into datetime so that it is usable and set it as the index.

### Data Cleaning

To verify data quality, you can use .describe().

```python
df_list["ADA"].describe()  # df_list["ADA"] allows us to access the ADA DataFrame
```

![PythonFinance7Image0.png](https://strapi-project-image-test.s3.eu-west-3.amazonaws.com/Python_Finance7_Image0_43e516ad83.png)

At first glance, we do not notice any specific issues. However, a good practice when modifying indexes is also to check the **uniqueness** (no duplicates) of these indexes (as a reminder, we have set the dates as indexes). For this, we can use the **`.nunique()`** method to count the number of unique values and compare it with the number of rows in our DataFrame.

![PythonFinance7Image1.png](https://strapi-project-image-test.s3.eu-west-3.amazonaws.com/Python_Finance7_Image1_6145b1659e.png)

Here, we notice that there are more rows in our DataFrame than unique indexes.

We can check if there are duplicate indexes in our DataFrame using the **`duplicated`** method of Pandas. The **`keep='last'`** parameter means that we keep the last occurrence in case of duplicates.

```python
print(df_list["ADA"].index.duplicated(keep='last'))
```

![PythonFinance7Image2.png](https://strapi-project-image-test.s3.eu-west-3.amazonaws.com/Python_Finance7_Image2_f61393bca3.png)



Our first line of code displays an array of **`False`** and **`True`**, where each line will be **`True`** if the index is duplicated.

To display the concerned rows, simply use **`DataFrame.loc[filter]`**. Here, our DataFrame is **`df_list["ADA"]`** and our filter is **`df_list["ADA"].index.duplicated(keep='last')`**.

From there, we can delete all the duplicated rows. We start by storing our duplicated indexes and selecting our DataFrame without the duplicated indexes (to make an inverse selection, you can use the **`~`** symbol).

```python
duplicate_index = df_list["ADA"].index.duplicated(keep='last')
df_list["ADA"] = df_list["ADA"][~duplicate_index]
```

Finally, we will apply this operation to all our DataFrames in the **`for`** loop to have "clean" data:

```python
crypto_list = ["ADA", "BCH", "BNB", "BTC", "ETC", "ETH", "LINK", "LTC", "XLM", "XRP"]
df_list = {}
for crypto in crypto_list:
    df_list[crypto] = pd.read_csv(f"{crypto}-USDT.csv")
    df_list[crypto]["date"] = pd.to_datetime(df_list[crypto]["date"], unit="ms")
    df_list[crypto] = df_list[crypto].set_index("date")
    duplicate_index = df_list[crypto].index.duplicated(keep='last')
    df_list[crypto] = df_list[crypto][~duplicate_index]
```

## Creating a Single DataFrame

We will now merge all this data into a single DataFrame, with one column for each cryptocurrency. To do this, we'll iterate through **`df_list`** and add the **`close`** column from each individual DataFrame.


```python
df = pd.DataFrame()
for crypto in df_list:
    df[crypto] = df_list[crypto]["close"]
```

We will then remove all rows containing NaN (Not a Number) values using the **`dropna`** method from Pandas.

```python
df = df.dropna()
```

![PythonFinance7Image3.png](https://strapi-project-image-test.s3.eu-west-3.amazonaws.com/Python_Finance7_Image3_c19632f68c.png)

We now have a DataFrame where each column represents the historical data of a cryptocurrency.

## Data Visualization

Next, we can visualize the closing prices of each cryptocurrency using Matplotlib.

```python
for col in df.columns:
    fig, ax = plt.subplots(figsize=(15, 7))
    df[col].plot(label=col)
    ax.set_ylabel(col) # Displays the name of the cryptocurrency on the Y-axis
    plt.show()
```

![PythonFinance7Image4.png](https://strapi-project-image-test.s3.eu-west-3.amazonaws.com/Python_Finance7_Image4_4110a1ad6f.png)

### Correlation Analysis

Next, we can compute the correlation matrix of our data using the **`corr`** method from Pandas.

**Correlation** measures the degree of association between two variables. This association can be positive (when one variable increases, so does the other) or negative (when one variable increases, the other decreases). The value of the correlation ranges from -1 to 1. A 1 indicates a perfect positive correlation, a -1 indicates a perfect negative correlation, and a 0 indicates no correlation.

The correlation matrix thus allows us to display whether each of our cryptocurrencies is correlated with others.

```python
df.corr()
```

![PythonFinance7Image5.png](https://strapi-project-image-test.s3.eu-west-3.amazonaws.com/Python_Finance7_Image5_2b808a383e.png)

The **Seaborn** visualization library offers a method for visualizing matrices, such as a correlation matrix, using 'heatmap' type graphs.

```python
f, ax = plt.subplots(figsize=(15, 8))
sns.heatmap(df.loc[:].corr(), annot=True)
```

![PythonFinance7Image6.png](https://strapi-project-image-test.s3.eu-west-3.amazonaws.com/Python_Finance7_Image6_5297661a31.png)

We can notice from this graph that the highest correlation is between LINK and XLM. However, here, we calculate the correlation directly on the price, meaning two assets that are generally bullish will be highly correlated. It is much more relevant to calculate the correlation on the percentage change.

Therefore, we will calculate the **percentage changes** for each cryptocurrency using the **`pct_change`** method of Pandas, and visualize the correlation matrix of these percentage changes.


```python
df_pct = df.pct_change()
```

![PythonFinance7Image7.png](https://strapi-project-image-test.s3.eu-west-3.amazonaws.com/Python_Finance7_Image7_c578e86d3d.png)

Now, we are displaying the correlation matrix only for the data from 2023."

```python
f, ax = plt.subplots(figsize=(15, 8))
sns.heatmap(df_pct.loc["2023"].corr(), annot=True)
```

![PythonFinance7Image8.png](https://strapi-project-image-test.s3.eu-west-3.amazonaws.com/Python_Finance7_Image8_f1f8b5237a.png)

In 2023, the most significant correlation in price evolution is between Bitcoin (BTC) and Ethereum (ETH), with a 89% correlation.


## Comparing Two Assets

Next, we will compare two assets by plotting their prices on two different y-axes but sharing the same x-axis. This will allow us to visually observe the correlation.

Let's start with the two most correlated assets, BTC and ETH, which have a 89% correlation.

```python
fig, ax = plt.subplots(figsize=(15, 8))

data = df.loc["2023"]
asset1 = "BTC"
asset2 = "ETH"
# Plot the first asset on the primary axis
ax.plot(data.index, data[asset1], color='blue')
ax.set_ylabel(asset1, color='blue')
ax.tick_params(axis='y', labelcolor='blue')

# Create a second axis sharing the same x-axis
ax2 = ax.twinx()

# Plot the second asset on the secondary axis
ax2.plot(data.index, data[asset2], color='red')
ax2.set_ylabel(asset2, color='red')
ax2.tick_params(axis='y', labelcolor='red')
plt.show()
```

![PythonFinance7Image9.png](https://strapi-project-image-test.s3.eu-west-3.amazonaws.com/Python_Finance7_Image9_95a47f91f8.png)

We can observe that there is an almost perfect correlation. Conversely, let's consider the least correlated assets, XRP and BCH, with a correlation of 43%

![PythonFinance7Image10.png](https://strapi-project-image-test.s3.eu-west-3.amazonaws.com/Python_Finance7_Image10_8d04f92012.png)

We notice that there seemed to be a strong correlation up until March 2023, but after that, the two assets were no longer correlated at all.

## Analysis of Average Returns and Volatility

Which is the best cryptocurrency to have in a portfolio? It's hard to answer this question, but one response might be the crypto with the **best returns and the lowest risk**.

Therefore, we will attempt to address this question by displaying a chart with **returns** on one axis and **risks** (or more precisely, **volatility**) on the other axis for our cryptocurrencies.

To do this, we will calculate the return for each crypto as the average of the hourly returns, and the risk as the standard deviation of these hourly returns. We will then insert these values into lists so that we can display a scatter plot.


```python
mean_return_list = []
volatility_list = []

temp_data = df_pct.loc[:]

fig, ax = plt.subplots(figsize=(15, 8))

for col in temp_data.columns:
    mean_return = temp_data[col].mean() * 100
    volatility = temp_data[col].std() * 100
    mean_return_list.append(mean_return)
    volatility_list.append(volatility)

    ax.text(mean_return, volatility, col)
    ax.scatter(mean_return, volatility, label=col)

ax.grid(True, linestyle='--', alpha=0.6)
ax.axvline(0, color='black', linestyle='--')
ax.axhline(0, color='black', linestyle='--')

ax.set_xlabel('Average Return', fontsize=12)
ax.set_ylabel('Volatility', fontsize=12)

ax.legend()

plt.show()
```

![pff7_1.png](https://strapi-project-image-test.s3.eu-west-3.amazonaws.com/pff7_1_45afcf35b7.png)

According to the graph, the cryptocurrency with the highest average return is BNB, while the most volatile is ETC.

From this graph, it's challenging to compare BTC and ETH directly, as BTC has lower volatility, but ETH offers higher returns. However, based on this graph, we can infer that historically, it was preferable to invest in ETH rather than LINK, since ETH both yields better returns and has lower volatility. The three most appealing cryptocurrencies to hold seem to be BTC, ETH, and BNB, depending on your risk aversion.

It could be insightful to generate this graph specifically for the year 2023.

```python
temp_data = df_pct.loc["2023"]
```

![pff7_2.png](https://strapi-project-image-test.s3.eu-west-3.amazonaws.com/pff7_2_c04c2842bf.png)

In the year 2023, the best cryptocurrencies to have were BTC and BCH.

## **Analysis of the Sharpe Ratio**

When discussing **return/risk ratio**, we cannot overlook the **Sharpe Ratio**. The Sharpe Ratio is a well-known financial metric that precisely calculates the return/risk couple. The formula is:

$$
SharpeRatio = \frac{Average[R_p]}{{StandardDeviation[R_p]}} \times \sqrt{N}
$$

Rp: Return (in percentage)

N: Number of periods per year

This formula provides a number (often around 1). If the result is below 0, it indicates **poor returns**. If it is between 0 and 1, it suggests that the **risks incurred are too high relative to the expected returns**. Finally, if it is above 1, it means that **the expected returns compensate for the incurred risks**.

We can start by displaying the Sharpe Ratio for each of the cryptocurrencies in our list:

```python
temp_data = df_pct.loc[:]

for col in temp_data.columns:
    sharpe_ratio = (8760**0.5)*(temp_data[col].mean()/temp_data[col].std())
    print(col, sharpe_ratio)
```

![PythonFinance7Image13.png](https://strapi-project-image-test.s3.eu-west-3.amazonaws.com/Python_Finance7_Image13_f4adbc3696.png)

> The operator **`**`** is used to raise to a power. To perform a square root, you raise to the power of 0.5.

We multiply by the square root of **`8760`** because our data is hourly, and there are **`8760`** hours in a year.

Next, we will visualize these Sharpe Ratios in a bar chart

```python
sharpe_ratios = []

temp_data = df_pct.loc[:]

for col in temp_data.columns:
    sharpe_ratio = (8760**0.5)*(temp_data[col].mean()/temp_data[col].std())
    sharpe_ratios.append(sharpe_ratio)
    print(col, sharpe_ratio)

df_sharpe = pd.DataFrame(sharpe_ratios, index=temp_data.columns, columns=['Sharpe Ratio'])
df_sharpe = df_sharpe.sort_values(by='Sharpe Ratio')

plt.figure(figsize=(10, 6))
plt.barh(df_sharpe.index, df_sharpe['Sharpe Ratio'], color='lightgreen')
plt.xlabel('Sharpe Ratio')
plt.title('Sharpe Ratio per asset')
plt.show()
```

![pff7_3.png](https://strapi-project-image-test.s3.eu-west-3.amazonaws.com/pff7_3_ec6b6b3c59.png)

As could be anticipated from the scatter plot, the asset with the best Sharpe Ratio over our analysis period is BNB.

## Analyzing Drawdowns

A **drawdown** represents a decline in an asset's price from its highest peak. This metric is crucial as it allows us to assess the most significant drops an asset has experienced in the past. If an asset has encountered a maximum drawdown of -80%, it's vital to consider that when purchasing this asset, there's a potential for it to lose 80% of its value. However, it's important to remember that past performance does not necessarily predict future results.

To calculate the drawdown, instead of doing it hourly, we can make it more readable by calculating it on a daily basis (this will be more readable on a graph). To **transform our hourly data into daily data**, we can use the **`.resample()`** method. Also, it's worth mentioning that we will be using our **`df`** DataFrame, which contains the prices, not their percentage changes.

To perform our calculation, we need the asset's maximum value over time. The **`.cummax()`** method (for cumulative maximum) allows us to do just that. For instance, if we have a series **`s`** containing 2, 4, 7, 5, 1, 8, 4, then **`s.cummax()`** will yield 2, 4, 7, 7, 7, 8, 8, representing the highest value reached on a rolling basis. Finally, to calculate our **drawdown**, we simply compare our current value with the cumulative maximum and convert it into a percentage.

In this code, we calculate the daily drawdown for BTC and ETH and display it on a graph.

```python
import matplotlib.pyplot as plt

plt.figure(figsize=(15, 10))
temp_data = df.resample('D').last()
for col in ["BTC", "ETH"]:
    running_max = temp_data[col].cummax()
    drawdown = (temp_data[col] - running_max) / running_max
    plt.plot(drawdown, label=col)

plt.title('Asset Drawdown Over Time')
plt.xlabel('Time')
plt.ylabel('Drawdown')
plt.legend()
plt.grid(True)
plt.show()
```

![pff7_4.png](https://strapi-project-image-test.s3.eu-west-3.amazonaws.com/pff7_4_f95f7f5c30.png)

We notice here that during the COVID crash in March 2020, Bitcoin experienced a drawdown of about 50%, and Ethereum a bit more than 60%. Lastly, in July 2022, both assets underwent a drawdown of more than 70%. For instance, if we consider Bitcoin, its peak was at \$68,000 in November 2021, and in July 2022, its price was at \$16,000.

$$
(16000-68000)/68000 = -0.76
$$

This equates to a drop of 76%.

## Practical Exercise and Conclusion

This session was in a way a practical exercise in itself. However, if you wish to practice further, we invite you to, for example, recreate the scatter plot we saw earlier, **replacing volatility with maximum drawdown**. You could also try calculating the number of drawdowns exceeding 10% for each asset (how often the asset dropped more than 10% from its peak). A correction example can be found here: https://github.com/RobotTraders/Python_For_Finance/blob/main/exercise_correction_chapter_7.ipynb.

It's crucial to understand what was done in this session. If you've grasped it well, you should be able to **apply these methods for in-depth analyses** of different financial assets according to your interests.

So, you now know how to use **Pandas** effectively for financial data analysis. The key is understanding how Pandas works and leveraging its features to your advantage, aiming to avoid writing unnecessary loops and making your code more efficient. In the next episode, we will focus on **mastering dates**, an important component when dealing with numerous time series as in finance.

Learn how to optimize your use of Pandas by avoiding loops and applying advanced data manipulation and analysis techniques.

Master financial analysis with Pandas and avoid loops

<MDX_YT videoId="q7e4YX7w5AU"/>

What if you could determine in less than a second whether a **trading strategy is successful**? That would be great, wouldn't it?

Imagine having a tool that can evaluate the past performance of a strategy, providing valuable insights without risking a single penny of your capital. Today, we dive into the captivating world of **backtesting**. We'll explore how, with just a few lines of Python code, you can test the viability of your trading strategy, assess its strengths and weaknesses, and be better prepared to confidently enter the volatile trading arena. Ready to begin? Let's dive in!

## The Trading Strategy to Backtest

Today, we'll be backtesting one of the most well-known trading strategies: **the moving average crossover**.

A moving average simply represents the average price over the last `n` periods.

This strategy involves buying when a short-term moving average (like 50 periods) crosses above a long-term moving average (such as 200 periods). Conversely, a sell signal is triggered when the short-term moving average crosses below the long-term moving average.

![pff9_1.png](https://strapi-project-image-test.s3.eu-west-3.amazonaws.com/pff9_1_b8a77329a3.png)

It's a time-tested method for identifying potential entry and exit points based on historical market trends. With the power of **Python** and the **pandas** library, in just a few lines of code, we'll determine **if this strategy has been profitable in the past**, offering valuable insights for our future investments.

## Backtesting a Trading Strategy in Python

### Loading Historical Data with Pandas

To simulate our trading strategy, we'll need historical price data for the asset we want to backtest. This is a fundamental element, as these valuable data allow us to analyze and evaluate our strategy's performance under real market conditions. We provide the hourly OHLCV (Open, High, Low, Close, Volume) data for Bitcoin in a CSV file available at this link: https://github.com/RobotTraders/Python_For_Finance/blob/main/BTC-USDT.csv. This file contains crucial information, such as the opening, highest, lowest, and closing prices, and the volume for each hour over the last few years.

After importing the Pandas library with `import pandas as pd`, we'll start by loading this file into a Pandas DataFrame using `pd.read_csv()`.

```python
df = pd.read_csv('BTC-USDT.csv'
```

Next, we'll convert the date column to datetime for easier handling of dates and times. The dates are currently in Unix format (the number of milliseconds since 1970).

```python
df["date"] = pd.to_datetime(df["date"], unit="ms")
df = df.set_index(df["date"])
del df["date"]
```

With these lines, we've converted the `'date'` column into a datetime object and then set this column as our DataFrame's index. We also removed the original 'date' column to keep our DataFrame tidy.

### Calculating Our Technical Indicators

To execute our strategy, we need to calculate our technical indicators, which in this case are the short-term and long-term moving averages.

One moving average will be over 50 periods, and the other over 200 periods. We'll then use them to generate buy and sell signals.

Une moyenne mobile sera sur 50 périodes et l'autre sur 200 périodes. Nous allons par la suite les utiliser pour générer des signaux d'achat et de vente.

```python
df['ma50'] = df['close'].rolling(50).mean()
df['ma200'] = df['close'].rolling(200).mean()
```

Here, we've used the `rolling()` method to create our moving averages. The 50-period moving average will react more quickly to price changes than the 200-period average.

### Generating Our Buy and Sell Signals

Now that we have our indicators, we'll create columns for buy and sell signals based on the relationship between these two moving averages.

```python
df['buy_signal'] = False
df['sell_signal'] = False
df.loc[(df['ma50'] > df['ma200']), 'buy_signal'] = True
df.loc[(df['ma50'] < df['ma200']), 'sell_signal'] = True
```

We initialized our signal columns with `False` values. Then, we used the `loc[]` method to fill these columns with `True` values when the conditions for buying or selling are met.

When the 50-period moving average is above the 200-period moving average, we have a buy signal; conversely, we have a sell signal.

### Preparing the Bitcoin Backtest

We're now ready to execute the backtest. We'll initialize our balance and go through the DataFrame to execute buy and sell orders based on our signals.

Let's start by initializing three variables:

```python
balance = 1000
position = None
asset = "BTC"
```

- The `balance` variable represents the amount in dollars of our initial portfolio (here $1000).
- The `position` variable represents the current “position”; if we make a purchase, we open the position, and when we sell, we close the position. We start without a position, so the variable is initialized to `None`.
- The `asset` variable will simply be used for display purposes.

### Executing the Backtest in Python

We've now reached a crucial step in our journey: executing the backtest using a loop that goes through our historical data and applies our trading strategy. Hang on, as we break down and dissect each line to help you grasp the subtlety and power of this code.

```python
for index, row in df.iterrows():
```

We begin a loop that will go through our DataFrame, line by line, each line representing an hourly period in our historical Bitcoin data.

Each `row` contains the trading data for a specific hour, and `index` is the temporal index of that line.

```python
    if position is None and row['buy_signal']:
```

We enter a condition where we check two things. First, if we don’t have an open position (i.e., if we haven't bought Bitcoin) and second, if the buy signal is active for the current period.

```python
        open_price = row['close']
        open_usd_size = balance
        position = {
            'open_price': open_price,
            'usd_size': open_usd_size,
        }
```

If that's the case, we open a position. We note the opening price, which is the closing price of the current period. We invest our entire available balance and record these details in the `position` variable via a dictionary.

```python
        print(f"{index} - Buy for {open_usd_size}$ of {asset} at {open_price}$")
```

Finally, we print a summary of the purchase, including the date and time, the size of the position, and the purchase price.

```python
    elif position and row['sell_signal']:
```

Next, we check if we have an open position and if the sell signal is active.

```python
        close_price = row['close']
        trade_result = (close_price - position['open_price']) / position['open_price']
        balance = balance + trade_result * position['usd_size']
        position = None
```

If the condition is true, we retrieve the closing price for the period in question. Then, we calculate the result of the trade by subtracting the opening price of our position from the closing price, divided by the opening price to get the return percentage. Our balance is then updated by adding the gain or loss from the trade. Finally, we close the position by resetting it to None.

```python
print(f"{index} - Sell for {balance}$ of {asset} at {close_price}$")
```

For each sale, we print a summary of the trade, including the date and time, the new balance, and the price at which we sold the asset.

Once this code is executed, we can run our backtest and finally discover the results. Thanks to the `print()` statements in the code, we display each purchase and sale to explore the execution of the strategy.

![resultat-strategie-84611.png](https://strapi-project-image-test.s3.eu-west-3.amazonaws.com/resultat_strategie_84611_a99df7ebb0.png)

### The Results of the Trading Strategy

Our backtest is now complete, and we can add a print() statement to display our final portfolio:

```python
print(f"Final balance: {balance}$")
```

> Final balance: 7847.2548834433455$
> 

If we had applied this moving average crossover trading strategy from the end of 2017 to early July 2023, our portfolio would have soared to an impressive **\$ 7847**. Yes, you read that correctly. We would have made a profit of \$ 6847! That's a **+684%** gain!

However, before getting too excited, it's essential to temper these results. We did not include **transaction fees** in our simulation. As you know, in the real world, each buy and sell transaction comes with fees that can significantly impact returns.

Moreover, it's vital to put these gains into perspective with Bitcoin's overall performance during this period. A strategy is all the more impressive if it outperforms the underlying asset, offering an attractive risk-adjusted return.

Nonetheless, it's clear that this strategy would have been profitable in the past. However, and this is a warning worth repeating: the past doesn't predict the future. Markets evolve, and a strategy that was winning yesterday may not be tomorrow.

In the next episode, we'll dive deeper into analyzing the results of this strategy. We'll look at key metrics and techniques to objectively evaluate performance and discuss potential adjustments to improve profitability and reduce risk. Stay tuned, the best is yet to come!

## Structuring Our Backtest with a Class and Functions

In our quest to make our code cleaner, more efficient, and reusable, the introduction of classes and functions becomes an essential step. Consider, for example, the `BacktestCrossMA` class above. This structure not only makes our code more readable but also organizes it modularly, making it easier to maintain and reuse.

```python
class BacktestCrossMA:
    def __init__(self) -> None:
        self.df = pd.DataFrame()

    def load_data(self, path):
        self.df = pd.read_csv(path)
        self.df["date"] = pd.to_datetime(self.df["date"], unit="ms")
        self.df = self.df.set_index(self.df["date"])
        del self.df["date"]

    def populate_indicators(self):
        self.df['ma50'] = self.df['close'].rolling(50).mean()
        self.df['ma200'] = self.df['close'].rolling(200).mean()

    def populate_signals(self):
        self.df['buy_signal'] = False
        self.df['sell_signal'] = False
        self.df.loc[(self.df['ma50'] > self.df['ma200']), 'buy_signal'] = True
        self.df.loc[(self.df['ma50'] < self.df['ma200']), 'sell_signal'] = True

    def run_backtest(self):
        balance = 1000
        position = None
        asset = "BTC"

        for index, row in self.df.iterrows():

            if position is None and row['buy_signal']:
                open_price = row['close']
                position = {
                    'open_price': open_price,
                    'usd_size': balance,
                }
                print(f"{index} - Buy for {balance}$ of {asset} at {open_price}$")

            elif position and row['sell_signal']:
                close_price = row['close']
                trade_result = (close_price - position['open_price']) / position['open_price']
                balance = balance + trade_result * position['usd_size']
                position = None
                print(f"{index} - Sell for {balance}$ of {asset} at {close_price}$")
                
            
        
        print(f"Final balance: {balance}$")

bt = BacktestCrossMA()
bt.load_data('BTC-USDT.csv')
bt.populate_indicators()
bt.populate_signals()
bt.run_backtest()
```

The `BacktestCrossMA` class encapsulates all the steps of our backtesting, from data retrieval to backtest execution. Each method of the class is dedicated to a specific stage of the process. The `load_data()` method loads the data, `populate_indicators()` calculates the moving averages, and `populate_signals()` determines the buy and sell signals. Finally, `run_backtest()` executes the backtest based on these signals.

By encapsulating these steps in a class, we can easily reuse and adapt our code for other assets or strategies, making our work not only more efficient but also more organized and manageable. This demonstrates the beauty and efficacy of object-oriented programming, a valuable skill for any developer working in quantitative finance.

## Practical Application

Having explored the moving average crossover strategy, it's time to put our knowledge to the test with a `new approach`. We propose a practical exercise where you backtest a trading strategy based on `Bollinger Bands`, another popular technical analysis tool among traders.

To implement this strategy, we buy when the closing price is above the upper Bollinger Band and sell when the price falls back below the middle band. If you're unfamiliar with Bollinger Bands, I encourage you to learn more about them here https://www.investopedia.com/terms/b/bollingerbands.asp.

To start, you'll need to become familiar with calculating and interpreting Bollinger Bands. By combining theory with practice, we have the opportunity to test and refine a new strategy, thereby expanding our trading arsenal and honing our Python programming skills for finance.

### Useful Library for Technical Indicator Calculation

If you're interested in exploring other strategies based on different indicators, check out the Python TA library https://technical-analysis-library-in-python.readthedocs.io/en/latest/, which includes numerous technical indicators. Here's the documentation.

Once installed with `pip install ta` and imported into your code with `import ta`, you can, for example, calculate the RSI, another well-known technical trading indicator.

```python
df["RSI"] = ta.momentum.RSIIndicator(close=df["close"], window=14).rsi()
```
A correction example can be found here: https://github.com/RobotTraders/Python_For_Finance/blob/main/exercise_correction_chapter_9.ipynb. Good luck, and see you in the next and last chapter where we'll discover **how to best analyze the results of these backtests**.

Learn to develop and evaluate automated trading strategies using Python.

Backtest an automated trading strategy in Python

Learn How to Analyze Your Trading Portfolio's Performance Using Pandas to Draw Conclusions and Optimize Your Strategies.

Analyze the Backtest of a Trading Portfolio in Python Using Pandas

<MDX_YT videoId="2pn7Bs96FiM"/>

Once you have manipulated and analysed your financial data with Pandas, visualizing it is crucial to gain deeper insights into trends and relationships among various variables. In this chapter, we'll explore how to create meaningful charts using the **Matplotlib** library, one of the most popular visualization libraries in Python.

## Introduction to Matplotlib
Matplotlib is a versatile data visualization library for Python, enabling the creation of a wide array of charts such as bar graphs, histograms, scatter plots, and line graphs. It is especially well-suited for financial data visualization and is commonly used in conjunction with Pandas.

### Installing and Importing Matplotlib
Before using Matplotlib, you need to install it by executing the following command in your terminal or command prompt:

```python
pip install matplotlib
```

Once installed, you can import Matplotlib into your Python script with the following line:

```python
import matplotlib.pyplot as plt
```

The alias "plt" is conventionally used for Matplotlib and simplifies access to the library's functions.

## Creating Charts with Matplotlib and Pandas

With Matplotlib imported, we can begin creating charts to visualize our financial data. This section will cover some common chart types used in the finance industry.

### **First Use: Line Graphs**

The most basic chart is the line graph, created using the **`plt.plot()`** method. For example:

```python
x = [1, 2, 3, 4, 5]
y = [10, 20, 30, 40, 50]

plt.plot(x, y)
plt.show()
```

![matplotlib0.png](https://strapi-project-image-test.s3.eu-west-3.amazonaws.com/matplotlib0_022ec41580.png)

Here, we built a chart from two lists, one for the x-axis and one for the y-axis.

### Adding Titles and Labels

Enhance your chart by adding titles and labels:

```python
x = [1, 2, 3, 4, 5]
y = [10, 20, 30, 40, 50]

plt.plot(x, y)
plt.xlabel("X axis")
plt.ylabel("Y axis")
plt.title("Plot test")
plt.show()
```

![pff6-0.png](https://strapi-project-image-test.s3.eu-west-3.amazonaws.com/pff6_0_700070ba99.png)

Here we've added titles to each axis and an overall chart title.

### Bar Charts

Another commonly used chart is the bar chart, which you can create using the **`plt.bar()`** method.

```python
plt.bar(x, y)
plt.show()
```

![matplotlib2.png](https://strapi-project-image-test.s3.eu-west-3.amazonaws.com/matplotlib2_5dfa2cb0e9.png)

### Scatter Plots

Scatter plots are useful for comparing data points to one another. You can create one using the **`plt.scatter()`** method.

```python
plt.scatter(x, y)
plt.show()
```

## Visualizing Complex Data with Pandas and Matplotlib

Using a real-world dataset, let's visualize complex data with Matplotlib. We'll work with the "House Sales in King County, USA" dataset, which contains detailed information about house sales in the area. You can download the dataset from this link https://github.com/RobotTraders/Python_For_Finance/blob/main/house_sales_in_king_county_usa.csv.

Start by loading the dataset with Pandas:

```python
import pandas as pd

df = pd.read_csv('house_sales_in_king_county_usa.csv')
```

Here is a description of the different columns in the dataset.

- id: This is the unique identifier for each house sold.
- date: This is the date the house was sold. It can be important because real estate market trends can vary over time.
- price: This is the selling price of the house. It is often the target variable in real estate prediction models.
- bedrooms: This is the number of bedrooms in the house.
- bathrooms: This is the number of bathrooms in the house. Half bathrooms (toilet without shower or bathtub) can be counted as 0.5.
- sqft_living: This is the living space of the house in square feet.
- sqft_lot: This is the total land area of the house in square feet.
- floors: This is the number of floors in the house.
- waterfront: This variable indicates whether the house has a view of water (a lake, river, sea, etc.). It is often a binary variable (0 = no water view, 1 = water view).
- view: This is an assessment of the view quality from the house. The rating system can vary, but often a scale of 0 to 4 is used.
- condition: This is an assessment of the general condition of the house. As with view, it is often rated on a scale, with a higher value indicating a better condition.
- grade: This is an assessment of the quality of construction and design of the house. Houses with higher-quality construction materials and more elaborate designs will receive a higher rating.
- sqft_above: This is the above-ground living space of the house in square feet.
- sqft_basement: This is the basement area of the house in square feet.
- yr_built: This is the year the house was built.
- yr_renovated: This is the year the house was last renovated. If the house has never been renovated, this value could be 0.
- zipcode: This is the postal code of the house. It can be important because location has a major impact on house prices.
- lat: This is the latitude of the house, a component of its geographical position.
- long: This is the longitude of the house, another component of its geographical position.
- sqft_living15: This is the average living space of the 15 closest houses, in square feet.
- sqft_lot15: This is the average land area of the 15 closest houses, in square feet.

You can use df.describe() to analyze your data, but the dataset is quite clean, so you normally will not need to do any data cleaning.

### Histograms

Histograms are great for visualizing data distributions. Create one using **`plt.hist()`** with Matplotlib. The **`bins`** parameter specifies the number of bins you want.

```python
plt.hist(df["price"], bins=100)
plt.show()
```

![matplotlib3.png](https://strapi-project-image-test.s3.eu-west-3.amazonaws.com/matplotlib3_dd618834ec.png)

The x-axis scale is in scientific notation (1e6 represents 1 x 10 to the power of 6, which means 1,000,000). We can see that the vast majority of houses in our dataset are priced between 0 and 2 million dollars. However, we note that some houses have sold for more than 7 million dollars. If we wish to better observe the distribution between 0 and 2 million dollars, we can apply a filter:

```python
plt.hist(df.loc[df["price"]<2000000]["price"], bins=100)
plt.show()
```

![matplotlib4.png](https://strapi-project-image-test.s3.eu-west-3.amazonaws.com/matplotlib4_74e8635a3c.png)

We observe here that the vast majority of houses fall between 250k and 500k dollars. Each bar represents the number of houses within a certain price range.

The **`bins`** parameter represents the number of bars you want; the higher the parameter, the more detail we will have in the data distribution.

### Scatter Plots with Variable Colors

It would be interesting to see if there is a relationship between the size of the house and its sale price. A scatter plot is perfect for this purpose.

```python
plt.scatter(df['sqft_living'], df['price'], s=5, c=df['sqft_living'], cmap='inferno')
plt.title('Price as a function of the living area')
plt.xlabel('Living Area')
plt.ylabel('Price')
plt.show()
```

![pff6-1.png](https://strapi-project-image-test.s3.eu-west-3.amazonaws.com/pff6_1_966a5375de.png)

The **`s`** parameter defines the size of the dots, while **`c`** sets the color variation, with warmer colors indicating higher prices. The **`cmap`** parameter determines the color range.

![matplotlib6.png](https://strapi-project-image-test.s3.eu-west-3.amazonaws.com/matplotlib6_2aa3027f56.png)

### Time Graphs

Another type of chart useful in finance is the time series chart, which illustrates how variables change over time. Let's consider an example with our dataset. If we want to examine the trend of average house prices over time, we first need to convert the date column into datetime format.

```python
df['date'] = pd.to_datetime(df['date'])
```

Next, we can group the data by month and calculate the average price:

```python
df_grouped = df.resample('M', on='date').mean()
```

and finally we can plot the evolution of the average price:

```python
plt.plot(df_grouped.index, df_grouped['price'])
plt.xlabel('Date')
plt.ylabel('Average Price')
plt.title('Trend of Average House Prices Over Time')
plt.show()
```

## Other Charts

Through this link https://matplotlib.org/stable/gallery/index, you can find a whole gallery of possible charts that can be created with Matplotlib.

### Seaborn, Another Visualization Library

While Matplotlib is a powerful and flexible library for data visualization, its interface can sometimes be a bit complex for more advanced visualization tasks. That's where Seaborn comes in. Seaborn is a data visualization library in Python built on Matplotlib. It offers a high-level interface, which makes creating complex charts a bit simpler. Seaborn is particularly useful for creating attractive and informative statistical graphics and for visualizing complex data with multiple variables. It provides a variety of chart types such as distribution charts, matrix charts, and much more. Additionally, Seaborn integrates well with Pandas DataFrames, making it easy to directly visualize data from your DataFrame. With this link, you'll find a list of examples https://seaborn.pydata.org/examples/index.html.

Here is an example of its use:

```python
import seaborn as sns # pip install seaborn to install the library
sns.scatterplot(x='sqft_living', y='price', data=df)
plt.show()
```

## Conclusion

As you can see, Matplotlib is a powerful tool for data visualization in Python. It offers great flexibility and can be used to create a wide range of charts and visualizations, which can be extremely useful in the financial field to understand trends and relationships between different variables.

Remember that the key to good visualization is that it should be informative and easy to understand. Therefore, it is important to always take the time to properly format your charts, add appropriate titles, legends, and axis labels, and choose the types of charts that best illustrate your data.

## Practical Exercise: Financial Data Analysis and Visualization with Matplotlib and Seaborn

For this exercise, we will continue using the "House Sales in King County, USA" dataset. The goal of this exercise is to deepen your understanding of financial data analysis and visualization.

### Part 1: Display the 5 Largest Sales Volumes by Year

Your first challenge is to extract the year from the 'date' column and count the number of sales per year. Then, display a bar chart of the five years with the highest sales volume. Use Matplotlib for this.

### Part 2: Display the Price Distribution for Houses with vs. Without a Water View

Use Matplotlib's **`hist()`** method to display two histograms on the same chart: one for the price distribution of houses with a water view and one for those without a water view. Make sure to add a legend to distinguish the two histograms.

### Part 3: Correlation Diagram

Use Seaborn to create a heatmap of the correlation between the variables 'price', 'sqft_living', 'grade', 'sqft_above', and 'sqft_living15'. What conclusions can you draw from this heatmap?

### Part 4: Boxplot of Price by Grade

The 'grade' in our dataset is an evaluation of the construction quality and design of the house. Use Seaborn to create a boxplot of the house price by 'grade'. What do you observe about the relationship between 'grade' and 'price'?

### Part 5: Scatterplot of 'Price' vs. 'Sqft_Living' with Linear Regression

Create a scatterplot of 'price' as a function of **`lmplot()`**. Use Seaborn's lmplot() function to also display a linear regression line. What relationship can you observe between the size of the house and its price?


A correction example can be found here: https://github.com/RobotTraders/Python_For_Finance/blob/main/exercise_correction_chapter_6.ipynb.

Learn how to create charts and interactive visualizations of your financial data using the Matplotlib library

Visualize your data in Python using Matplotlib

<MDX_YT videoId="QcCLjBBJUSw"/>

"_Time is money_", goes the old saying. In the fast-paced world of finance, this has never been truer. Dates and time series are the lifeblood of financial markets, influencing every decision, every analysis, every transaction. Whether tracking the performance of a stock over a decade or analyzing market fluctuations minute by minute, precise mastery of dates is absolutely crucial.

In this chapter, we'll unravel the mysteries of date manipulation in Python, giving you the keys to navigate through time with ease.

## Introduction to Dates in Python

### The `datetime` Module

In Python, date management is primarily handled by the `datetime` module. It provides classes for manipulating both dates (year, month, day) and specific moments in time (adding hours, minutes, seconds).

Here's how to create your first **date** and **datetime** objects in Python:

```python
import datetime

# Creating a simple date
d = datetime.date(2023, 8, 21)
print(d)  # Displays "2023-08-21"

# Creating a precise moment (timestamp)
dt = datetime.datetime(2023, 8, 21, 14, 30)
print(dt)  # Displays "2023-08-21 14:30:00"
```

To get the current date or the exact current moment:

```python
today = datetime.date.today()
print(today)  # Displays today's date

now = datetime.datetime.now()
print(now)  # Displays the current precise moment
```

## **2. Formats courants de dates**

### **2.1 String**

Dates are often represented as **strings**, especially when read from files or databases.

```python
date_string = "2023-08-21"

# Convert a string to a datetime object
date_object = datetime.datetime.strptime(date_string, "%Y-%m-%d")
print(date_object)

# Another example with day/month/year and hours:minutes
date_string2 = "21/08/2023 14:30"
date_object2 = datetime.datetime.strptime(date_string2, "%d/%m/%Y %H:%M")
print(date_object2)
```

### **2.2 Timestamp Unix**

The **Unix timestamp** represents the number of seconds (or milliseconds) elapsed since January 1, 1970, known as "**epoch**". It's a compact, universal format widely used in finance.

```python
import time

# Get the current timestamp
timestamp = time.time()
print(timestamp)

# Convert a timestamp to datetime
date_from_timestamp = datetime.datetime.fromtimestamp(timestamp)
print(date_from_timestamp)

# Example with milliseconds
timestamp_millis = timestamp * 1000
date_from_millis = datetime.datetime.fromtimestamp(timestamp_millis / 1000)
print(date_from_millis)
```

### 2.3 Summary of Date Types

To summarize, there are primarily four formats for storing a date:

- **String** format, e.g., `"2023-08-28"`, which is a string.

- **Timestamp** format, e.g., `1693233091.2038012`, which is a `float` and represents the number of seconds (or milliseconds) since 1970.

- **Datetime** format, commonly used in Python for storing dates with hours and seconds, e.g., `datetime.datetime(2023, 8, 21, 14, 30)`, this object is of type `datetime.datetime`.

- **Date** format, commonly used in Python for storing dates without hours or seconds, e.g., `datetime.date(2023, 8, 21)`, this object is of type `datetime.date`.

## Converting Between Date Formats

Navigating between different formats is a common requirement.

```python
# From datetime to string
formatted_date = date_object.strftime("%Y-%m-%d")
print(formatted_date)  # "2023-08-21"

# Another format
formatted_date2 = date_object.strftime("%d %B %Y")
print(formatted_date2)  # "21 August 2023"

# From datetime to timestamp (seconds since epoch)
timestamp_from_date = date_object.timestamp()
print(timestamp_from_date)

# And in milliseconds
timestamp_millis_from_date = timestamp_from_date * 1000
print(timestamp_millis_from_date)
```
Above are various examples of converting one date format to another. These examples are not exhaustive, and you'll need to research according to your specific needs.

## 4. Using Dates with Pandas

Having explored the basics of date manipulation in Python, it's time to integrate these skills with **Pandas**. If you're in the financial world, Pandas will be your go-to tool for a multitude of tasks, especially processing financial time series. In this episode, we'll see how Pandas can aid us in **manipulating dates** for specific finance-related cases, like time series of Bitcoin (BTC) prices.

### 4.1 Creating the DataFrame and Converting the Date Format

For these examples, we'll load our DataFrame from a CSV file previously introduced in the training https://github.com/RobotTraders/Python_For_Finance/blob/main/BTC-USDT.csv. This CSV represents the hourly price evolution of Bitcoin in the OHLCV (Open, High, Low, Close, Volume) format.

```python
df = pd.read_csv("BTC-USDT.csv")
```
![date-formation-python0](https://strapi-project-image-test.s3.eu-west-3.amazonaws.com/date_formation_python0_211a868f2c.png)

As you can see, the date column is not immediately readable. However, if you recall the start of the episode, this format might ring a bell. You may have recognized it – these dates are `timestamps`. They represent the number of milliseconds elapsed since 1970 (the numerous zeros at the end are a giveaway).

Here's how to convert `timestamps` into `datetime` (more readable and manageable):

```python
df = pd.read_csv("BTC-USDT.csv")
print(df["date"].dtype)
df["date"] = pd.to_datetime(df["date"], unit="ms")
print(df["date"].dtype)
```

![date-formation-python1](https://strapi-project-image-test.s3.eu-west-3.amazonaws.com/date_formation_python1_d79960cf70.png)

The dates are immediately more legible. In the code, we also included two lines to display the column type before and after the transformation. We notice that before the transformation, the column was of type `int`, and now it's `datetime`.

Now we can start performing manipulations on the dates.

### 4.2 Extracting the Day of the Week

Pandas makes it easy to extract the day of the week with its **`dayofweek`** attribute.

```python
df["weekday"] = df["date"].dt.weekday
```

![date-formation-python2](https://strapi-project-image-test.s3.eu-west-3.amazonaws.com/date_formation_python2_4121980f83.png)
The days of the week are displayed from 0 to 6, with 0 representing Monday and 6 representing Sunday.

If you wish to see them as `string` like `"Monday"`, you'll need to do what's known as mapping (matching one value to another). This is done using a dictionary and the Pandas `.map()` function.

```python
day_name_map = {
    0: 'Monday',
    1: 'Tuesday',
    2: 'Wednesday',
    3: 'Thursday',
    4: 'Friday',
    5: 'Saturday',
    6: 'Sunday'
}

df['weekday'] = df['weekday'].map(day_name_map)
```

### 4.3 Subtracting Between Two Dates

Let's now verify that we indeed have an hourly interval between our data points. We can do this by checking the time difference between each data point and its predecessor.

```python
df["date_diff"] = df["date"] - df["date"].shift(1)
print(df["date_diff"].value_counts())
```

Here, we simply calculate the difference between the value in the date column and the value in the previous row of the date column using `shift(1)`. We then display all the unique values and their associated number of occurrences.

![date-formation-python3](https://strapi-project-image-test.s3.eu-west-3.amazonaws.com/date_formation_python3_4f528cbe13.png)

We notice that, in the vast majority of cases, we indeed have an hour between our two values. However, there are some exceptions:

- 26 cases with no gap (possibly duplicates)

- About 30 cases with a difference of at least two hours up to a maximum of more than a day. It might be interesting to investigate the cause.

### 4.4 Grouping by Date

From our data, it would be interesting to know if the average trading volume of Bitcoin varies depending on the day of the week.

To do this, we can try grouping our data by the day of the week. Here we'll use the `group_by()` function to group our data by the weekday, then the `agg()` function to aggregate our data based on the grouped column.

This can be done in just one line of code:

```python
df.groupby('weekday').agg({"volume": "mean"})
```

![date-formation-python4](https://strapi-project-image-test.s3.eu-west-3.amazonaws.com/date_formation_python4_eea045d7de.png)

As expected, trading volumes on weekends (Saturday and Sunday) are significantly lower than on weekdays – nearly 50% lower. This indicates that a major portion of Bitcoin trading is conducted by

## **Mise en pratique**

Let's explore a practical question: **Are there noticeable differences in Bitcoin behavior based on the operational hours of major financial markets?**

For instance, we can look at the European (Euronext), New York (NYSE), and Tokyo stock exchanges. The European exchange operates between 9 AM and 5 PM Paris time. You are encouraged to find out the operating hours for the other exchanges.

The first step will be to create a column for each stock exchange and mark it as `True` if the exchange is open at the time of the data point, and `False` if not.

Once this is done, you can compare the average trading volume depending on whether the exchange is open or closed. You can also compare the average variation of a candlestick by, for example, looking at the percentage difference between the high and low of each candlestick. You are really encouraged to conduct your own analyses.

A correction example can be found here: https://github.com/RobotTraders/Python_For_Finance/blob/main/exercise_correction_chapter_8.ipynb

Discover how to manage and manipulate dates and times in Python, a crucial aspect for working with financial data.

Nail the use of dates in Python

medium_pff-intro.png

small_pff-intro.png

thumbnail_pff-intro.png

pff-intro.png

Bienvenue à cette formation "Python pour la Finance" destinée à toutes personnes qui souhaitent apprendre Python pour l'utiliser dans le secteur financier mais plus généralement dans l'analyse de données. Que vous soyez étudiant, professionnel de la finance ou simplement passionné par le domaine, cette formation vous permettra de maîtriser les bases de Python et de l'appliquer à des cas concrets dans le secteur financier.
![Python Finance](https://strapi-project-image-test.s3.eu-west-3.amazonaws.com/draze_a_python_in_a_crypto_trading_room_1a54cd54_45a8_4c17_a742_d702c7a77577_bcd044b57f.png)

Nous sommes ravis de vous annoncer que cette formation "Python pour la Finance" est proposée totalement gratuitement par Crypto Robot ! Notre objectif est de rendre l'apprentissage du langage Python et de son application dans le domaine financier accessible à tous. Le meilleur moyen de nous soutenir est de nous suivre sur les différents réseaux sociaux, de vous abonner à notre Newsletter dans laquelle nous vous partagerons notamment les corrections de certains exercices. Enfin vous pouvez partager cette formation au plus grand nombre !
<MAIL_FORM/>

À la fin de chaque session, un exercice pratique sera proposé pour vous permettre de mettre en application les concepts abordés. De plus, chaque session sera accompagnée d'une vidéo explicative pour vous guider pas à pas dans votre apprentissage.

<EXCHANGE_CARD image_name="bitget_fr_1" link="https://partner.bitget.com/bg/CryptoRobotSIFR"/>

## **Sommaire**

- **0 - Python et VsCode l'installation** : Apprenez à installer Python et Visual Studio Code, un éditeur de code polyvalent et puissant, pour débuter votre aventure dans l'apprentissage du langage Python.
- **1 - Variables, Opérations et Types** : Familiarisez-vous avec les variables, les opérations mathématiques et les différents types de données en Python.
- **2 - Boucles et Conditions en Python** : Découvrez comment utiliser les structures de contrôle telles que les boucles et les conditions pour réaliser des tâches répétitives ou conditionnelles.
- **3 - Les Listes et les Dictionnaires en Python** : Apprenez à manipuler les listes et les dictionnaires pour stocker et organiser vos données efficacement.
- **4 - Les Fonctions et les Classes en Python** : Découvrez comment créer et utiliser des fonctions et des classes pour structurer votre code et faciliter la réutilisation.
- **5 - La librairie Pandas, la meilleure librairie pour les données financières** : Apprenez à utiliser Pandas, une librairie puissante pour la manipulation et l'analyse de données, spécialement adaptée pour travailler avec des données financières.
- **6 - Visualiser ses données en Python grâce à Matplotlib** : Découvrez comment créer des graphiques et des visualisations interactives de vos données financières en utilisant la librairie Matplotlib.
- **7 - Déverrouiller la puissance de Pandas et éviter les boucles** : Apprenez à tirer le meilleur parti de Pandas en évitant les boucles et en utilisant des techniques avancées pour manipuler et analyser vos données.
- **8 - Maîtriser l'utilisation des dates en Python** : Découvrez comment gérer et manipuler les dates et les heures en Python, un aspect crucial pour travailler avec des données financières.
- **9 - Backtester une stratégie de trading automatisé en Python** : Apprenez à développer et à évaluer des stratégies de trading automatisées en utilisant Python.
- **10 - Analyser le backtest d'un portefeuille de trading en Python grâce à Pandas** : Découvrez comment analyser les performances de votre portefeuille de trading en utilisant Pandas pour tirer des conclusions et optimiser vos stratégies.


<EXCHANGE_CARD image_name="bitget_fr_1" link="https://partner.bitget.com/bg/CryptoRobotSIFR"/>

Il est important de souligner que le meilleur moyen de progresser en Python, c'est de ne pas se contenter de regarder les vidéos et de lire les documents, mais de vous engager activement dans la pratique. Nous vous encourageons à travailler sur des projets personnels, même de petite envergure, afin d'appliquer et de renforcer vos compétences acquises tout au long de cette formation.

Dans cette formation, vous découvrirez le langage de programmation Python et comment l'appliquer le plus rapidement à des concepts financiers.

# **Conclusion de la Formation "Python pour la Finance"**

Nous sommes arrivés à la fin de notre parcours d'apprentissage ensemble, et j'espère que cette formation "Python pour la Finance" vous a fourni les connaissances et les compétences nécessaires pour débuter ou approfondir votre aventure dans le monde passionnant de la finance et de l'analyse de données avec Python.

![PythonFinanceConclusion.jpg](https://strapi-project-image-test.s3.eu-west-3.amazonaws.com/Python_Finance_Conclusion_5470619cd7.jpg)

## **Points Clés de la Formation**

Au cours de ces 10 leçons, nous avons couvert une gamme étendue de sujets, y compris :

- Les fondamentaux de Python et l'environnement de développement avec Visual Studio Code.
- La compréhension des variables, opérations, et types en Python.
- Les structures de contrôle telles que les boucles et les conditions.
- La manipulation des listes et dictionnaires.
- La création et utilisation des fonctions et classes.
- L'utilisation avancée de la librairie Pandas pour les données financières.
- La visualisation de données avec Matplotlib.
- La gestion des dates et heures, essentielle dans le secteur financier.
- Le développement et l'évaluation de stratégies de trading automatisées.
- L'analyse de performances d'un portefeuille de trading.

<EXCHANGE_CARD image_name="bitget_fr_1" link="https://partner.bitget.com/bg/CryptoRobotSIFR"/>

## **Appliquez Vos Connaissances**

Rappelez-vous, la pratique est la clé de la maîtrise. Nous vous encourageons à continuer à utiliser Python dans vos projets personnels ou professionnels. N'hésitez pas à revisiter les leçons de ce cours pour rafraîchir vos connaissances, et souvenez-vous que chaque projet est une opportunité d'apprendre quelque chose de nouveau.

## **Restez Connecté**

Continuez à nous suivre sur les réseaux sociaux et à vous abonner à notre Newsletter pour plus de conseils, de tutoriels, et d'actualités sur Python dans la finance. Votre feedback est précieux, donc partagez vos réussites, vos défis, et vos idées avec nous et la communauté de Crypto Robot.

<MAIL_FORM/>

## **Un Grand Merci**

Enfin, un grand merci de la part de toute l'équipe de Crypto Robot pour avoir choisi notre formation. Nous espérons que vous avez trouvé ce cours à la fois informatif et enrichissant, et nous sommes impatients de vous voir appliquer ces compétences dans le monde réel.

"Continuez à apprendre, continuez à grandir, et que vos projets financiers prospèrent avec Python !"

---

Bonne chance et bon succès dans vos futures aventures en programmation Python et en finance ! 🚀📈

All Formations

Python for Finance
October 2, 2023

All Formations

Python for FinanceOctober 2, 2023

Python for Finance
October 2, 2023