Artificial Neural Networks to forecast London Stock Exchange

Artificial Neural Networks to forecast London Stock Exchange Abstract This dissertation examines and analyzes the use of the Artificial Neural Networks (ANN) to forecast the London Stock Exchange. Specifically the importance of ANN to predict the future trends and value of the financial market is demonstrated. There are several contributions of this study to this area. The first contribution of this study is to find the best subset of the interrelated factors at both local and international levels that affect the London stock exchange from the various input variables to be used in the future studies. We use novel aspects, in the sense that we base the forecast on both the fundamental and technical analysis.The second contribution of this study was to provide well defined methodology that can be used to create the financial models in future studies. In addition, this study also gives various theoretical arguments in support of the approaches used in the construction of the forecasting model by comparing the results of the previous studies and modifying some of the existing approaches and tested them. The study also compares the performance of the statistical methods and ANN in the forecasting problem. The main contribution of this thesis lies in comparing the performance of the five different types of ANN by constructing the individual forecasting model of them. Accuracy of models is compared by using different evaluation criteria and we develop different forecasting models based on both the direction and value accuracy of the forecasted value. The fourth contribution of this study is to investigate whether the hybrid approach combining different individual forecasting models can outperform the individual forecasting models and compare the performance of the different hybrid approaches. Three hybrid approaches are used in this study, two are existing approaches and the third original approach, the mixed combined neural network -is being proposed in this study to the academic studies to forecast the stock exchange. The last contribution of this study lies in modifying the existing trading strategy to increase the profitability of the investor and support the argument that the investor earns more profit if the forecasting model is being developed by using the direction accuracy as compared to the value accuracy. The best forecasting classification accuracy obtained is 93% direction accuracy and 0.0000831 (MSE) value accuracy which are better than the accuracies obtained by the previous academic studies. Moreover, this research validates the work of the existing studies that hybrid approach outperforms the individual forecasting model. In addition, the rate of the return that was attained in this thesis by using modified trading strategy is 120.14% which has shown significant improvement as compared to the 10.8493% rate of return of the existing trading strategy in other academics studies. The difference in the rate of return could be due to the fact that this study has developed good forecasting model or a better trading strategy. The experimental results show our method not only improves the accuracy rate, but also meet the short-term investors’ expectations. The results of this thesis also support the claim that some financial time series are not entirely random, and that contrary to the predictions of the efficient markets hypothesis (EMH), a trading strategy could be based solely on historical data. It was concluded that ANN do have good capabilities to forecast financial markets and, if properly trained, the investor could benefit from the use of this forecasting tool and trading strategy. Chapter 1 1 Introduction 1.1 Background to the Research Financial Time Series forecasting has attracted the interest of academic researchers and it has been addressed since the 1980.It is a challenging problem as the financial time series have complex behavior, resulting from a various factors such as economic, psychological or political reasons and they are non-stationary , noisy and deterministically chaotic. In today’s world, almost every individual is influenced by the fluctuations in the stock market. Now day’s people prefer to invest money in the diversified financial funds or shares due to its high returns than depositing in the banks. But there is lot of risk in the stock market due to its high rate of uncertainty and volatility. To overcome such risks, one of the main challenges for many years for the researchers is to develop the financial models that can describe the movements of the stock market and so far there had not been an optimum model. The complexity and difficulty of forecasting the stock exchange, and the emergence of data mining and computational intelligence techniques, as alternative techniques to the conventional statistical regression and Bayesian models with better performance, have paved the road for the increased usage of these techniques in fields of finance and economics. So, traders and investors have to rely on the various types of intelligent systems to make trading decisions. (Hameed,2008). A Computational Intelligence system such as neural networks, fuzzy logic, genetic algorithms etc has been widely established research area in the field of information systems. They have been used extensively in forecasting of the financial market and they have been quite successful to some extent .Although the number of purposed methods in financial time series is very large , but no one technique has been successful to consistently to â€Å"beat the market†. For last three decades, opposing views have existed between the academic communities and traders about the topic of â€Å"Random walk theory â€Å"and â€Å"Efficient Market Hypothesis(EMH)† due to the complexity of the financial time series and lot of publications by different researchers have gather various amount of evidences in support as well as against it. Lehman (1990), Haugen (1999) and Lo (2000) gave evidence of the deficiencies in EMH. But the investors such as Warren Buffet for long period of time have beaten the stock market consistently. Market Efficiency or â€Å"Random walk theory† in terms of stock trading in the financial market means that it is impossible to earn excess returns using any historic information. In essence, then, the new information is the only variable that causes to alter the price of the index as well as used to predict the arrival and timing. Bruce James Vanstone (2005) stated that in an efficient market, security prices should appear to be randomly generated. Both sides in this argument are supported by empirical results from the different markets across over the globe. This thesis does not wish to enter into the argument theoretically whether to accept or reject the EMH. Instead, this thesis concentrates on the methodologies to be used for development of the financial models using the artificial neural networks (ANN), compares the forecasting capabilities of the various ANN and hybrid based approach models, develop the trading strategy that can help the investor and leaves the research of this thesis to stack up with the published work of other researchers which document ways to predict the stock market. In recent years and since its inception, ANN has gained momentum and has been widely used as a viable computational intelligent technique to forecast the stock market. The main challenge of the traders is to know the signals when the stock market deviates and to take advantage of such situations. The data used by the traders to remove the uncertainty in the stock market and to take trading decisions whether to buy or sell the stock using the information process is â€Å"noisy†. Information not contained in the known information subset used to forecast is considered to be noise and such environment is characterized by a low signal-to noise ratio. Refenes et.al (1993) and Thawornwong and Enke (2004) described that the relationship between the security price or returns and the variables that constitute that price (return), changes over time and this fact is widely accepted within the academic institutes. In other words, the stock market‘s structural mechanics may change over time which causes the effect on the index also change. Ferreira et al. (2004) described that the relationship between the variables and the predicted index is non linear and the Artificial neural networks (ANN) have the characteristic to represent such complex non-linear relationship. This thesis presents the mechanical London Stock Market trading system that uses the ANN forecasting model to extract the rules from daily index movements and generate signal to the investors and traders whether to buy, sell or hold a stock. The figure 1 and 2 represents the stock exchange and ANN forecasting model. By viewing the stock exchange as a financial market that takes historical and current data or information as an input, the investors react to this information based on their understanding, speculations, analysis etc. It would now seem very difficult to predict the stock market, characterized by high noise, nonlinearities, using only high frequency (weekly, daily) historical prices. Surprisingly though, there are anomalies in the behavior of the stock market that cannot be explained under the existing paradigm of market efficiency. Studies discussed in the literature review have been able to predict the stock market accurately to some extent and it seems that forecasting model developed by them have been able to pick some of the hidden patterns in the inherently non-linear price series. While it is true that forecasting model need to be designed and optimized with care in order to get accurate results . Further, it aims to contribute knowledge that will one day lead to a standard or optimum model for the prediction of the stock exchange. As such, it aims to present a well defined methodology that can be used to create the forecasting models and it is hoped that this thesis can address many of the deficiencies of the published research in this area. In the last decade, there has been plethora of the ANN models that were developed due to the absence of the well defined methodology, which were difficult to compare due to less published work and some of them have shown superior results in their domains. Moreover, this study also compares the predictive power of the ANN with the statistical models. Normally the approach used by the academic researchers in the forecasting use technical analysis and some of them include the fundamental analysis. The technical analysis uses only historical data (past price) to determine the movement of the stock exchange and fundamental analysis is based on external information (like interest rates, prices and returns of other asset) that comes from the economic system surrounding the financial market. Building a trading system using forecasting model and testing it on the evaluation criteria is the only practical way to evaluate the forecasting model. There has been so much prior research on identifying the appropriate trading strategy for forecasting problem. This thesis does not wish to enter into the argument which strategy is best or not. Although, the importance of the trading strategy can hardly be underestimated, but this thesis concentrates on using one of the existing strategy, modify it and compares the return by the forecasting models. But there has always been debate in the academic studies over how to effectively benchmark the model of ANN for trading. Some of the academic researchers stated that predicting the direction of the stock exchange may lead to higher profits while some of them supported the view that predicting the value of the stock exchange may lead to higher rate of return. Azoff (1994) and Thawornwong and Enke (2004) discussed about this debate in their study. In essence, there is a need for a formalized development methodology for developing the ANN financial models which can be used as a benchmark for trading systems. All of this is accommodated by this thesis. 1.2 Problem Statement and Research Question The studies mentioned above have generally indicated that ANN, as used in the stock market, can be a valuable tool to the investor .Due to some of the problems discussed above, we are not still able to answer the question: Can ANNs be used to develop the accurate forecasting model that can be used in the trading systems to earn profit for the investor? From the variety of academic research summarized in the literature review, it is clear that a great deal of research in this area has taken place by different academic researchers and they have gathered various amounts of evidences in support as well as against it. This directly threatens the use of ANN applicability to the financial industry. Apart from the previous question, this research addresses various other problems: 1. Which ANN have better performance in the forecasting of the London Stock Exchange from the five different types of the ANN which are widely used in the academics? 2. Which subset of the potential input variables from 2002-08 affect the LSE? 3. Do international stock exchanges, currency exchange rate and other macroeconomic factors affect the LSE? 4. How much the performance of the forecasting model is improved by using the regression analysis in the factor selection? 5. Can use of the technical indicators improve the performance of the forecasting model? 6. Which learning algorithm in the training of the ANN give the better performance? 7. Does Hybrid-based Forecasting Models give better performance than the individual ANN forecasting models? 8. Which Hybrid-based models have the better performance and what are the limitations of using them? 9. Does the forecasting model developed on the basis of the percentage accuracy gives more rate of the return as compared to the value accuracy? 10. Does the forecasting model having better performance in terms of the accuracy increase the profit of the investor when applied to the trading strategy? Apart from all questions outlined above, it addresses various another questions regarding the design of the ANN. †¢ Are there any approaches to solve the various issues in designing of the ANN like number of hidden layers and activation functions? This thesis will attempt to answer the above question within the constraints and scope of the 6-year sample period (from 2002-2008) using historical data of various variables that affect the LSE. Further, this thesis will also attempt to answer these questions within the practical constraints of transaction costs and money management imposed by real-world trading systems. Although a formal statement of the methodology or steps that is being used is left until section 3, it makes sense to discuss the way in which this thesis will address the above question. In this thesis, various types of ANN will be trained using fundamental data, and technical data according to the direction and value accuracy. A better trading system development methodology will be defined, and the performance of the forecasting model will be checked by using evaluation criteria rate of the return .In this way, the benefits of incorporating ANN into trading strategies in the stock market can be exposed and quantified. Once this process has been undertaken, it will be possible to answer the thesis all questions. 1.3 Motivation of the Research Stock market has always had been an attractive appeal for the researchers and financial investors and they have studied it over again to extract the useful patterns to predict the movement of the stock market. The reason is that if the researchers can make the accurate forecasting model, they can beat the market and can gain excess profit by applying the best trading strategy. Numerous financial investors have suffered lot of financial losses in the stock market as they were not aware of the stock market behavior. They had the problem that they were not able to decide when they should sell or buy the stock to gain profit. Nevertheless, finding out the best time for the investor to buy or to sell has remained a very difficult task because there are too many factors that may influence stock prices. If the investors have the accurate forecasting model, then they can predict the future behavior of the stock exchange and can gain profit. This solves the problem of the financial investors to some extent as they will not bear any financial loss. But it does not guarantee that the investor can have better profit or rate of return as compared to other investors unless he utilized the forecasting model using better trading strategy to invest money in the share market. This thesis tries to solve the above problem by providing the investor better forecasting model and trading strategies that can be applied to real-world trading systems. 1.4 Justification of Research There are several features of this academic research that distinguish it from previous academic researches. First of all, the time frame chosen for the investigation of the ANN (2002-08) in the London Stock Exchange has never been tested in the previous academic work. The importance of the period chosen is that there are two counter forces, which are opposing each other. On the one hand, the improvement of the UK and other countries economy after the 2001 financial crises happened in this period as a whole. On the other hand, this period also shows the decline in the stock markets from Jan, 2008 to Dec, 2008. So, it is important to test the forecasting model for bull, stable and bear market. Second, some of the research questions addressed in the above section, have not been investigated much in the academic studies, especially there is hardly any study which have done research on all the problems. Moreover, original hybrid based mixed neural network, better trading strategy and other modified approaches have been successfully being described and used in this study Finally, there is a significant lack of work carried out in this area in the LSE. As such, this thesis draws heavily on results published mainly within the United States and other countries; from the academics .One interesting aspect of this thesis is that it will be interesting to see how much of the published research on application of ANN in stock market anomalies is applicable to the UK market. This is important as some of the academic studies (Pan et al (2005)) states that each stock market in the globe is different. 1.5 Delimitations of scope The thesis concerns itself with historical data for the variables that affect London Stock Exchange during the period 2002 – 2008. 1.6 Outline of the Report The remaining part of the thesis is organized in the following six chapters. The second chapter, the background and literature review, provides a brief introduction to the domain and also pertinent literature is reviewed to discuss the related published work of the previous researchers in terms of their contribution and content in the prediction of the stock exchange which serves as the building block for much of the research. Moreover, this literature review also gave solid justification why a particular set of ANN inputs are selected, which is important step according to the Thawornwong and Enke (2004) and and some concepts from finance. The third chapter, the methodology, describes the steps in detail, data and the mechanics or techniques that take place in the thesis along with the empirical evidence. In addition, it also discuss the literature review for each step. Formulas and diagrams are shown to explain the techniques when necessary and it also covers issues as software and hardware used in the study. The fourth chapter, the implementation, discusses the approaches used in the implementation in detail based on the third chapter. It also covers such issues as software and hardware used in the study. The fifth chapter, the results and analysis, present the results according to the performance and benchmark measures that we have used in this study to compare with other models. It describes the choices that were needed in making model and justifies these choices in terms of the literature. The sixth chapter, conclusions and further work, restates the thesis hypothesis, discuss the conclusions drawn from the project and also thesis findings are put into perspective. Finally, the next steps to improve the model performance are considered. Chapter 2 Background and Literature Review 2 Background and Literature Review This section of thesis explores the theory of three relevant fields of the Financial Time Series, Stock Market, and Artificial Neural Networks, which together form the conceptual frameworks of the thesis as shown in the figure 1. Framework is provided to the trader to make quantitative and qualitative judgments concerning the future stock exchange movements. These three fields are reviewed in historical context, sketching out the development of those disciplines, and reviewing their academic credibility, and their application to this thesis. In the case of Neural Networks, the field is reviewed with regard to that portion of the literature which deals with applying neural network to the prediction of the stock exchange, the various type of techniques and neural networks used and an existing prediction model is extended to allow a more detailed analysis of the area than would otherwise have been possible. 2.1 Financial Time Series 2.1.1 Introduction The field of the financial time series prediction is a highly complex task due to the following reasons: 1. The financial time series frequently behaves like a random-walk process and predictability of such series is controversial issue which has been questioned in scope of EMH. 2. The statistical property of the financial time series shift with the different time. Hellstr ¨om and Holmstr ¨om [1998]). 3. Financial time series is usually noisy and the models which have been able to reduce such noise has been the better model in forecasting the value and direction of the stock exchange. 4. In the long run, a new forecasting technique becomes a part of the process to be forecasted, i.e. it influences the process to be forecasted (Hellstr ¨om and Holmstr ¨om [1998]). The first point is explained later in this section while discussing the EMH theory (Page).The graph of the volatility time series of FTSE 100 index from 14 June, 1993 to 29 December, 1998 and Dow Jones from 1928 to 2000 by Nelson Areal (2008) and Negrea Bogdan Cristian (2007) illustrates the second point of the FTSE 100 [2.1.r]in figure 2.1.1 and 2.2.2.These figures also shows that the volatility changes with period , in some periods FTSE 100 index value fluctuates so much and in some it remains calm. The third point is explained by the fact the events on a particular data affect the financial time series of the index, for example, the volatility of stocks or index increases before announcement of major stock specific news (Donders and Vorst [1996]). These events are random and contribute noise in the time series which may make difficult to compare the two forecasting models difficult to compare as a random model can also produce results. The fourth result can be explained by the example. Suppose a company develop a model or technique that can outcast all other models or techniques. The company will make lot of profits if this model is available to less people. But if this technique is available to all people with time due to its popularity, than the profits of the company will decrease as the company will not no longer take advantage of this technique. This argument is described in Hellstr ¨om and Holmstr ¨om [1998] and Swingler [1994] . 2.1.2 Efficient Market Hypothesis (EMH) EMH Theory has been a controversial issue for many years and there has been no mutual agreed deal among the academic researchers, whether it is possible to predict the stock price. The people who believe that the prices follow â€Å"random walk† trend and cannot be predicted, are usually people who support the EMH theory. Academic researchers( Tino et al. [2000]), have shown that the profit can be made by using historical information , whereas they also found difficult to verify the strong form due to lack of all private and public data. The EMH was developed in 1965 by Fama (Fama [1965], Fama [1970]) and has found widely accepted (Anthony and Biggs [1995], Malkiel [1987], White [1988], Lowe and Webb [1991]) in the academic community (Lawrence et al. [1996]).It states that the future index or stock value is completely unpredictable given the historical information of the index or stocks. There are three forms of EMH: weak, semi-strong, and strong form. The weak EMH rules out any form of forecasting based on the stock’s history, since the stock prices follows a random walk in which in which successive changes have zero correlation (Hellstr ¨om and Holmstr ¨om [1998]). In Semi Strong hypothesis, we consider all the publicly available information such as volume data and fundamental data. In strong form, we consider all the publicly and privately available information. Another reason for argument against the EMH is that different investors or traders react differently when a stock suddenly drops in a value. These different time perspectives will cause the unexpected change in the stock exchange, even if the new information has not entered in the scene. It may be possible to identify these situations and actually predict future changes (Hellstr  ¨om and Holmstr ¨om [1998]) The developer have proved it wrong by making forecasting models, this issue remains an interesting area. This controversy is just only matter of the word immediately in the definition. The studies in support of the argument of EMH rely on using the statistical tests and show that the technical indicators and tested models can’t forecast. However, the studies against the argument uses the time delay between the point when new information enters the model or system and the point when the information has spread across over the globe and a equilibrium has been reached in the stock market with a new market price. 2.1.3 Financial Time Series Forecasting Financial Time series Forecasting aims to find underlying patterns, trends and forecast future index value using using historical and current data or information. The historic values are continuous and equally spaced value over time and it represent various types of data . The main aim of the forecasting is to find an approximate mapping function between the input variables and the forecasted or output value . According to Kalekar (2004), Time series forecasting assumes that a time series is a combination of a pattern and some error. The goal of the model using time series is to separate the pattern from the error by understanding the trend of the pattern and its seasonality Several methods are used in time series forecasting like moving average (section ) moving averages, linear regression with time etc. Time series differs from the technical analysis (section) that it is based on the samples and treated the values as non-chaotic time series. Many academic researchers have applied t ime series analysis in their forecasting model, but there has been no major success. [1a] 2.2 Stock Market 2.2.1 Introduction Let us consider the basics of the stock market. MM What are stocks? Stock refers to a share in the ownership of a corporation or company. They represent a claim of the stock owner on the company’s earnings and assets and by buying more stocks; the stake in the ownership is increased. In United States, stocks are often referred as shares, whereas in the UK they are also used as synonym for bonds, shares and equities. MM Why a Company issues a stock? The main reason for issuing stock is that the company wants to raise money by selling some part of the company. A company can raise money by two ways: â€Å"debt financing† (borrowing money by issuing bonds or loan from bank) and â€Å"equity financing â€Å"(borrowing money by issuing stocks).It is advantageous to raise the money by issuing stocks as the company has not to pay money back to the stock owners but they have to share the profit in the form of the dividends. MM What is Stock Pricing or price? A stock price is the price of a single stock of a number of saleable stocks traded by the company. A company issue stock at static price, and the stock price may increase or decrease according to the trade. Normally the price of the stocks in the stock market is determined by the supply/demand equilibrium. MM What is a Stock Market? Stock Market or equity market is a public market where the trading and issuing of a company stock or derivates takes place either through the stock exchange or they may be traded privately and over-the counter markets. It is vital part of the economy as it provides opportunities to the company to raise money and also to the investors of having potential gain by selling or buying share. The stock market in the US includes the NYSE, NASDAQ, the AMEX as well as many regional exchanges. London Stock Exchange is the major stock exchange in the UK and Europe.As mentioned in the Chapter 1, in this study we forecast the London Stock Exchange (Section 2.2.2.). Investing in the stock market is very risky as the stock market is uncertain and unsteady. The main aim of the investor is to get maximum returns from the money invested in the stock market, for which he has to study about the performance, price history about the stock company .So it is a broad category and according to Hellstrom (1997), there are four main ways to predict the stock market: 1. Fundamental analysis (section 2.2.3) 2. Technical analysis, (section 2.2.4) 3. Time series forecasting (section 2.1) 4. Machine learning (ANN). (Section 2.3) 2.2.2 London Stock Exchange London Stock Exchange is one of the world’s oldest and largest stock exchanges in the world, which started its operation in 1698, when John Casting commenced â€Å"at this Office in Jonathan’s Coffee-house† a list of stock and commodity prices called â€Å"The Course of the Exchange and other things† [2] .On March 3, 1801, London Stock Exchange was officially established with current lists of over 3,200 companies and has existed, in one or more form or another for more than 300 years. In 2000, it decided to become public and listed its shares on its own stock exchange in 2001. The London Stock market consists of the Main Market and Alternative Investments Market (AIM), plus EDX London (exchange for equity derivatives). The Main Market is mainly for established companies with high performance, and AIM hand trades small-caps, or new enterprises with high growth potential.[1] Since the launch of the AIM in 1995, AIM has become the most successful growth market in the world with over 3000 companies from across the globe have joined AIM. To evaluate the London Stock Exchange, the autonomous FTSE Group (owned by the Financial Times and the London Stock Exchange) , sustains a series of indices comprising the FTSE 100 Index, FTSE 250 Index, FTSE 350 Index, FTSE All-Share, FTSE AIM-UK 50, FTSE AIM 100, FTSE AIM All-Share, FTSE SmallCap, FTSE Tech Mark 100 ,FTSE Tech Mark All-Share.[4] FTSE 100 is the most famous and composite index calculated respectively from the top 100 largest companies whose shares are listed on the London Stock Exchange. The base date for calculation of FTSE 100 index is 1984. [2] In the UK, the FTSE 100 is frequently used by large investor, financial experts and the stock brokers as a guide to stock market performance. The FTSE index is calculated from the following formula: 2.2.3 Fundamental Analysis Fundamental Analysis focuses on evaluation of the future stock exchange movements Artificial Neural Networks to forecast London Stock Exchange Artificial Neural Networks to forecast London Stock Exchange Abstract This dissertation examines and analyzes the use of the Artificial Neural Networks (ANN) to forecast the London Stock Exchange. Specifically the importance of ANN to predict the future trends and value of the financial market is demonstrated. There are several contributions of this study to this area. The first contribution of this study is to find the best subset of the interrelated factors at both local and international levels that affect the London stock exchange from the various input variables to be used in the future studies. We use novel aspects, in the sense that we base the forecast on both the fundamental and technical analysis.The second contribution of this study was to provide well defined methodology that can be used to create the financial models in future studies. In addition, this study also gives various theoretical arguments in support of the approaches used in the construction of the forecasting model by comparing the results of the previous studies and modifying some of the existing approaches and tested them. The study also compares the performance of the statistical methods and ANN in the forecasting problem. The main contribution of this thesis lies in comparing the performance of the five different types of ANN by constructing the individual forecasting model of them. Accuracy of models is compared by using different evaluation criteria and we develop different forecasting models based on both the direction and value accuracy of the forecasted value. The fourth contribution of this study is to investigate whether the hybrid approach combining different individual forecasting models can outperform the individual forecasting models and compare the performance of the different hybrid approaches. Three hybrid approaches are used in this study, two are existing approaches and the third original approach, the mixed combined neural network -is being proposed in this study to the academic studies to forecast the stock exchange. The last contribution of this study lies in modifying the existing trading strategy to increase the profitability of the investor and support the argument that the investor earns more profit if the forecasting model is being developed by using the direction accuracy as compared to the value accuracy. The best forecasting classification accuracy obtained is 93% direction accuracy and 0.0000831 (MSE) value accuracy which are better than the accuracies obtained by the previous academic studies. Moreover, this research validates the work of the existing studies that hybrid approach outperforms the individual forecasting model. In addition, the rate of the return that was attained in this thesis by using modified trading strategy is 120.14% which has shown significant improvement as compared to the 10.8493% rate of return of the existing trading strategy in other academics studies. The difference in the rate of return could be due to the fact that this study has developed good forecasting model or a better trading strategy. The experimental results show our method not only improves the accuracy rate, but also meet the short-term investors’ expectations. The results of this thesis also support the claim that some financial time series are not entirely random, and that contrary to the predictions of the efficient markets hypothesis (EMH), a trading strategy could be based solely on historical data. It was concluded that ANN do have good capabilities to forecast financial markets and, if properly trained, the investor could benefit from the use of this forecasting tool and trading strategy. Chapter 1 1 Introduction 1.1 Background to the Research Financial Time Series forecasting has attracted the interest of academic researchers and it has been addressed since the 1980.It is a challenging problem as the financial time series have complex behavior, resulting from a various factors such as economic, psychological or political reasons and they are non-stationary , noisy and deterministically chaotic. In today’s world, almost every individual is influenced by the fluctuations in the stock market. Now day’s people prefer to invest money in the diversified financial funds or shares due to its high returns than depositing in the banks. But there is lot of risk in the stock market due to its high rate of uncertainty and volatility. To overcome such risks, one of the main challenges for many years for the researchers is to develop the financial models that can describe the movements of the stock market and so far there had not been an optimum model. The complexity and difficulty of forecasting the stock exchange, and the emergence of data mining and computational intelligence techniques, as alternative techniques to the conventional statistical regression and Bayesian models with better performance, have paved the road for the increased usage of these techniques in fields of finance and economics. So, traders and investors have to rely on the various types of intelligent systems to make trading decisions. (Hameed,2008). A Computational Intelligence system such as neural networks, fuzzy logic, genetic algorithms etc has been widely established research area in the field of information systems. They have been used extensively in forecasting of the financial market and they have been quite successful to some extent .Although the number of purposed methods in financial time series is very large , but no one technique has been successful to consistently to â€Å"beat the market†. For last three decades, opposing views have existed between the academic communities and traders about the topic of â€Å"Random walk theory â€Å"and â€Å"Efficient Market Hypothesis(EMH)† due to the complexity of the financial time series and lot of publications by different researchers have gather various amount of evidences in support as well as against it. Lehman (1990), Haugen (1999) and Lo (2000) gave evidence of the deficiencies in EMH. But the investors such as Warren Buffet for long period of time have beaten the stock market consistently. Market Efficiency or â€Å"Random walk theory† in terms of stock trading in the financial market means that it is impossible to earn excess returns using any historic information. In essence, then, the new information is the only variable that causes to alter the price of the index as well as used to predict the arrival and timing. Bruce James Vanstone (2005) stated that in an efficient market, security prices should appear to be randomly generated. Both sides in this argument are supported by empirical results from the different markets across over the globe. This thesis does not wish to enter into the argument theoretically whether to accept or reject the EMH. Instead, this thesis concentrates on the methodologies to be used for development of the financial models using the artificial neural networks (ANN), compares the forecasting capabilities of the various ANN and hybrid based approach models, develop the trading strategy that can help the investor and leaves the research of this thesis to stack up with the published work of other researchers which document ways to predict the stock market. In recent years and since its inception, ANN has gained momentum and has been widely used as a viable computational intelligent technique to forecast the stock market. The main challenge of the traders is to know the signals when the stock market deviates and to take advantage of such situations. The data used by the traders to remove the uncertainty in the stock market and to take trading decisions whether to buy or sell the stock using the information process is â€Å"noisy†. Information not contained in the known information subset used to forecast is considered to be noise and such environment is characterized by a low signal-to noise ratio. Refenes et.al (1993) and Thawornwong and Enke (2004) described that the relationship between the security price or returns and the variables that constitute that price (return), changes over time and this fact is widely accepted within the academic institutes. In other words, the stock market‘s structural mechanics may change over time which causes the effect on the index also change. Ferreira et al. (2004) described that the relationship between the variables and the predicted index is non linear and the Artificial neural networks (ANN) have the characteristic to represent such complex non-linear relationship. This thesis presents the mechanical London Stock Market trading system that uses the ANN forecasting model to extract the rules from daily index movements and generate signal to the investors and traders whether to buy, sell or hold a stock. The figure 1 and 2 represents the stock exchange and ANN forecasting model. By viewing the stock exchange as a financial market that takes historical and current data or information as an input, the investors react to this information based on their understanding, speculations, analysis etc. It would now seem very difficult to predict the stock market, characterized by high noise, nonlinearities, using only high frequency (weekly, daily) historical prices. Surprisingly though, there are anomalies in the behavior of the stock market that cannot be explained under the existing paradigm of market efficiency. Studies discussed in the literature review have been able to predict the stock market accurately to some extent and it seems that forecasting model developed by them have been able to pick some of the hidden patterns in the inherently non-linear price series. While it is true that forecasting model need to be designed and optimized with care in order to get accurate results . Further, it aims to contribute knowledge that will one day lead to a standard or optimum model for the prediction of the stock exchange. As such, it aims to present a well defined methodology that can be used to create the forecasting models and it is hoped that this thesis can address many of the deficiencies of the published research in this area. In the last decade, there has been plethora of the ANN models that were developed due to the absence of the well defined methodology, which were difficult to compare due to less published work and some of them have shown superior results in their domains. Moreover, this study also compares the predictive power of the ANN with the statistical models. Normally the approach used by the academic researchers in the forecasting use technical analysis and some of them include the fundamental analysis. The technical analysis uses only historical data (past price) to determine the movement of the stock exchange and fundamental analysis is based on external information (like interest rates, prices and returns of other asset) that comes from the economic system surrounding the financial market. Building a trading system using forecasting model and testing it on the evaluation criteria is the only practical way to evaluate the forecasting model. There has been so much prior research on identifying the appropriate trading strategy for forecasting problem. This thesis does not wish to enter into the argument which strategy is best or not. Although, the importance of the trading strategy can hardly be underestimated, but this thesis concentrates on using one of the existing strategy, modify it and compares the return by the forecasting models. But there has always been debate in the academic studies over how to effectively benchmark the model of ANN for trading. Some of the academic researchers stated that predicting the direction of the stock exchange may lead to higher profits while some of them supported the view that predicting the value of the stock exchange may lead to higher rate of return. Azoff (1994) and Thawornwong and Enke (2004) discussed about this debate in their study. In essence, there is a need for a formalized development methodology for developing the ANN financial models which can be used as a benchmark for trading systems. All of this is accommodated by this thesis. 1.2 Problem Statement and Research Question The studies mentioned above have generally indicated that ANN, as used in the stock market, can be a valuable tool to the investor .Due to some of the problems discussed above, we are not still able to answer the question: Can ANNs be used to develop the accurate forecasting model that can be used in the trading systems to earn profit for the investor? From the variety of academic research summarized in the literature review, it is clear that a great deal of research in this area has taken place by different academic researchers and they have gathered various amounts of evidences in support as well as against it. This directly threatens the use of ANN applicability to the financial industry. Apart from the previous question, this research addresses various other problems: 1. Which ANN have better performance in the forecasting of the London Stock Exchange from the five different types of the ANN which are widely used in the academics? 2. Which subset of the potential input variables from 2002-08 affect the LSE? 3. Do international stock exchanges, currency exchange rate and other macroeconomic factors affect the LSE? 4. How much the performance of the forecasting model is improved by using the regression analysis in the factor selection? 5. Can use of the technical indicators improve the performance of the forecasting model? 6. Which learning algorithm in the training of the ANN give the better performance? 7. Does Hybrid-based Forecasting Models give better performance than the individual ANN forecasting models? 8. Which Hybrid-based models have the better performance and what are the limitations of using them? 9. Does the forecasting model developed on the basis of the percentage accuracy gives more rate of the return as compared to the value accuracy? 10. Does the forecasting model having better performance in terms of the accuracy increase the profit of the investor when applied to the trading strategy? Apart from all questions outlined above, it addresses various another questions regarding the design of the ANN. †¢ Are there any approaches to solve the various issues in designing of the ANN like number of hidden layers and activation functions? This thesis will attempt to answer the above question within the constraints and scope of the 6-year sample period (from 2002-2008) using historical data of various variables that affect the LSE. Further, this thesis will also attempt to answer these questions within the practical constraints of transaction costs and money management imposed by real-world trading systems. Although a formal statement of the methodology or steps that is being used is left until section 3, it makes sense to discuss the way in which this thesis will address the above question. In this thesis, various types of ANN will be trained using fundamental data, and technical data according to the direction and value accuracy. A better trading system development methodology will be defined, and the performance of the forecasting model will be checked by using evaluation criteria rate of the return .In this way, the benefits of incorporating ANN into trading strategies in the stock market can be exposed and quantified. Once this process has been undertaken, it will be possible to answer the thesis all questions. 1.3 Motivation of the Research Stock market has always had been an attractive appeal for the researchers and financial investors and they have studied it over again to extract the useful patterns to predict the movement of the stock market. The reason is that if the researchers can make the accurate forecasting model, they can beat the market and can gain excess profit by applying the best trading strategy. Numerous financial investors have suffered lot of financial losses in the stock market as they were not aware of the stock market behavior. They had the problem that they were not able to decide when they should sell or buy the stock to gain profit. Nevertheless, finding out the best time for the investor to buy or to sell has remained a very difficult task because there are too many factors that may influence stock prices. If the investors have the accurate forecasting model, then they can predict the future behavior of the stock exchange and can gain profit. This solves the problem of the financial investors to some extent as they will not bear any financial loss. But it does not guarantee that the investor can have better profit or rate of return as compared to other investors unless he utilized the forecasting model using better trading strategy to invest money in the share market. This thesis tries to solve the above problem by providing the investor better forecasting model and trading strategies that can be applied to real-world trading systems. 1.4 Justification of Research There are several features of this academic research that distinguish it from previous academic researches. First of all, the time frame chosen for the investigation of the ANN (2002-08) in the London Stock Exchange has never been tested in the previous academic work. The importance of the period chosen is that there are two counter forces, which are opposing each other. On the one hand, the improvement of the UK and other countries economy after the 2001 financial crises happened in this period as a whole. On the other hand, this period also shows the decline in the stock markets from Jan, 2008 to Dec, 2008. So, it is important to test the forecasting model for bull, stable and bear market. Second, some of the research questions addressed in the above section, have not been investigated much in the academic studies, especially there is hardly any study which have done research on all the problems. Moreover, original hybrid based mixed neural network, better trading strategy and other modified approaches have been successfully being described and used in this study Finally, there is a significant lack of work carried out in this area in the LSE. As such, this thesis draws heavily on results published mainly within the United States and other countries; from the academics .One interesting aspect of this thesis is that it will be interesting to see how much of the published research on application of ANN in stock market anomalies is applicable to the UK market. This is important as some of the academic studies (Pan et al (2005)) states that each stock market in the globe is different. 1.5 Delimitations of scope The thesis concerns itself with historical data for the variables that affect London Stock Exchange during the period 2002 – 2008. 1.6 Outline of the Report The remaining part of the thesis is organized in the following six chapters. The second chapter, the background and literature review, provides a brief introduction to the domain and also pertinent literature is reviewed to discuss the related published work of the previous researchers in terms of their contribution and content in the prediction of the stock exchange which serves as the building block for much of the research. Moreover, this literature review also gave solid justification why a particular set of ANN inputs are selected, which is important step according to the Thawornwong and Enke (2004) and and some concepts from finance. The third chapter, the methodology, describes the steps in detail, data and the mechanics or techniques that take place in the thesis along with the empirical evidence. In addition, it also discuss the literature review for each step. Formulas and diagrams are shown to explain the techniques when necessary and it also covers issues as software and hardware used in the study. The fourth chapter, the implementation, discusses the approaches used in the implementation in detail based on the third chapter. It also covers such issues as software and hardware used in the study. The fifth chapter, the results and analysis, present the results according to the performance and benchmark measures that we have used in this study to compare with other models. It describes the choices that were needed in making model and justifies these choices in terms of the literature. The sixth chapter, conclusions and further work, restates the thesis hypothesis, discuss the conclusions drawn from the project and also thesis findings are put into perspective. Finally, the next steps to improve the model performance are considered. Chapter 2 Background and Literature Review 2 Background and Literature Review This section of thesis explores the theory of three relevant fields of the Financial Time Series, Stock Market, and Artificial Neural Networks, which together form the conceptual frameworks of the thesis as shown in the figure 1. Framework is provided to the trader to make quantitative and qualitative judgments concerning the future stock exchange movements. These three fields are reviewed in historical context, sketching out the development of those disciplines, and reviewing their academic credibility, and their application to this thesis. In the case of Neural Networks, the field is reviewed with regard to that portion of the literature which deals with applying neural network to the prediction of the stock exchange, the various type of techniques and neural networks used and an existing prediction model is extended to allow a more detailed analysis of the area than would otherwise have been possible. 2.1 Financial Time Series 2.1.1 Introduction The field of the financial time series prediction is a highly complex task due to the following reasons: 1. The financial time series frequently behaves like a random-walk process and predictability of such series is controversial issue which has been questioned in scope of EMH. 2. The statistical property of the financial time series shift with the different time. Hellstr ¨om and Holmstr ¨om [1998]). 3. Financial time series is usually noisy and the models which have been able to reduce such noise has been the better model in forecasting the value and direction of the stock exchange. 4. In the long run, a new forecasting technique becomes a part of the process to be forecasted, i.e. it influences the process to be forecasted (Hellstr ¨om and Holmstr ¨om [1998]). The first point is explained later in this section while discussing the EMH theory (Page).The graph of the volatility time series of FTSE 100 index from 14 June, 1993 to 29 December, 1998 and Dow Jones from 1928 to 2000 by Nelson Areal (2008) and Negrea Bogdan Cristian (2007) illustrates the second point of the FTSE 100 [2.1.r]in figure 2.1.1 and 2.2.2.These figures also shows that the volatility changes with period , in some periods FTSE 100 index value fluctuates so much and in some it remains calm. The third point is explained by the fact the events on a particular data affect the financial time series of the index, for example, the volatility of stocks or index increases before announcement of major stock specific news (Donders and Vorst [1996]). These events are random and contribute noise in the time series which may make difficult to compare the two forecasting models difficult to compare as a random model can also produce results. The fourth result can be explained by the example. Suppose a company develop a model or technique that can outcast all other models or techniques. The company will make lot of profits if this model is available to less people. But if this technique is available to all people with time due to its popularity, than the profits of the company will decrease as the company will not no longer take advantage of this technique. This argument is described in Hellstr ¨om and Holmstr ¨om [1998] and Swingler [1994] . 2.1.2 Efficient Market Hypothesis (EMH) EMH Theory has been a controversial issue for many years and there has been no mutual agreed deal among the academic researchers, whether it is possible to predict the stock price. The people who believe that the prices follow â€Å"random walk† trend and cannot be predicted, are usually people who support the EMH theory. Academic researchers( Tino et al. [2000]), have shown that the profit can be made by using historical information , whereas they also found difficult to verify the strong form due to lack of all private and public data. The EMH was developed in 1965 by Fama (Fama [1965], Fama [1970]) and has found widely accepted (Anthony and Biggs [1995], Malkiel [1987], White [1988], Lowe and Webb [1991]) in the academic community (Lawrence et al. [1996]).It states that the future index or stock value is completely unpredictable given the historical information of the index or stocks. There are three forms of EMH: weak, semi-strong, and strong form. The weak EMH rules out any form of forecasting based on the stock’s history, since the stock prices follows a random walk in which in which successive changes have zero correlation (Hellstr ¨om and Holmstr ¨om [1998]). In Semi Strong hypothesis, we consider all the publicly available information such as volume data and fundamental data. In strong form, we consider all the publicly and privately available information. Another reason for argument against the EMH is that different investors or traders react differently when a stock suddenly drops in a value. These different time perspectives will cause the unexpected change in the stock exchange, even if the new information has not entered in the scene. It may be possible to identify these situations and actually predict future changes (Hellstr  ¨om and Holmstr ¨om [1998]) The developer have proved it wrong by making forecasting models, this issue remains an interesting area. This controversy is just only matter of the word immediately in the definition. The studies in support of the argument of EMH rely on using the statistical tests and show that the technical indicators and tested models can’t forecast. However, the studies against the argument uses the time delay between the point when new information enters the model or system and the point when the information has spread across over the globe and a equilibrium has been reached in the stock market with a new market price. 2.1.3 Financial Time Series Forecasting Financial Time series Forecasting aims to find underlying patterns, trends and forecast future index value using using historical and current data or information. The historic values are continuous and equally spaced value over time and it represent various types of data . The main aim of the forecasting is to find an approximate mapping function between the input variables and the forecasted or output value . According to Kalekar (2004), Time series forecasting assumes that a time series is a combination of a pattern and some error. The goal of the model using time series is to separate the pattern from the error by understanding the trend of the pattern and its seasonality Several methods are used in time series forecasting like moving average (section ) moving averages, linear regression with time etc. Time series differs from the technical analysis (section) that it is based on the samples and treated the values as non-chaotic time series. Many academic researchers have applied t ime series analysis in their forecasting model, but there has been no major success. [1a] 2.2 Stock Market 2.2.1 Introduction Let us consider the basics of the stock market. MM What are stocks? Stock refers to a share in the ownership of a corporation or company. They represent a claim of the stock owner on the company’s earnings and assets and by buying more stocks; the stake in the ownership is increased. In United States, stocks are often referred as shares, whereas in the UK they are also used as synonym for bonds, shares and equities. MM Why a Company issues a stock? The main reason for issuing stock is that the company wants to raise money by selling some part of the company. A company can raise money by two ways: â€Å"debt financing† (borrowing money by issuing bonds or loan from bank) and â€Å"equity financing â€Å"(borrowing money by issuing stocks).It is advantageous to raise the money by issuing stocks as the company has not to pay money back to the stock owners but they have to share the profit in the form of the dividends. MM What is Stock Pricing or price? A stock price is the price of a single stock of a number of saleable stocks traded by the company. A company issue stock at static price, and the stock price may increase or decrease according to the trade. Normally the price of the stocks in the stock market is determined by the supply/demand equilibrium. MM What is a Stock Market? Stock Market or equity market is a public market where the trading and issuing of a company stock or derivates takes place either through the stock exchange or they may be traded privately and over-the counter markets. It is vital part of the economy as it provides opportunities to the company to raise money and also to the investors of having potential gain by selling or buying share. The stock market in the US includes the NYSE, NASDAQ, the AMEX as well as many regional exchanges. London Stock Exchange is the major stock exchange in the UK and Europe.As mentioned in the Chapter 1, in this study we forecast the London Stock Exchange (Section 2.2.2.). Investing in the stock market is very risky as the stock market is uncertain and unsteady. The main aim of the investor is to get maximum returns from the money invested in the stock market, for which he has to study about the performance, price history about the stock company .So it is a broad category and according to Hellstrom (1997), there are four main ways to predict the stock market: 1. Fundamental analysis (section 2.2.3) 2. Technical analysis, (section 2.2.4) 3. Time series forecasting (section 2.1) 4. Machine learning (ANN). (Section 2.3) 2.2.2 London Stock Exchange London Stock Exchange is one of the world’s oldest and largest stock exchanges in the world, which started its operation in 1698, when John Casting commenced â€Å"at this Office in Jonathan’s Coffee-house† a list of stock and commodity prices called â€Å"The Course of the Exchange and other things† [2] .On March 3, 1801, London Stock Exchange was officially established with current lists of over 3,200 companies and has existed, in one or more form or another for more than 300 years. In 2000, it decided to become public and listed its shares on its own stock exchange in 2001. The London Stock market consists of the Main Market and Alternative Investments Market (AIM), plus EDX London (exchange for equity derivatives). The Main Market is mainly for established companies with high performance, and AIM hand trades small-caps, or new enterprises with high growth potential.[1] Since the launch of the AIM in 1995, AIM has become the most successful growth market in the world with over 3000 companies from across the globe have joined AIM. To evaluate the London Stock Exchange, the autonomous FTSE Group (owned by the Financial Times and the London Stock Exchange) , sustains a series of indices comprising the FTSE 100 Index, FTSE 250 Index, FTSE 350 Index, FTSE All-Share, FTSE AIM-UK 50, FTSE AIM 100, FTSE AIM All-Share, FTSE SmallCap, FTSE Tech Mark 100 ,FTSE Tech Mark All-Share.[4] FTSE 100 is the most famous and composite index calculated respectively from the top 100 largest companies whose shares are listed on the London Stock Exchange. The base date for calculation of FTSE 100 index is 1984. [2] In the UK, the FTSE 100 is frequently used by large investor, financial experts and the stock brokers as a guide to stock market performance. The FTSE index is calculated from the following formula: 2.2.3 Fundamental Analysis Fundamental Analysis focuses on evaluation of the future stock exchange movements