The Port Of Durban From An Economics Perspective Economics Essay

The Port Of Durban From An Economics Perspective Economics Essay 3.1 Introduction This chapter will examine the Port of Durban from an economics perspective and will seek to expand on the general theory presented in the literature review and apply it specifically to the Port of Durban. This chapter will also serve as a foundation for the proceeding chapter which will analyse the various CBA options and data for Durban. The ports significance and impact will be examined in the context of the South African and local economy through its income and employment generating effect. Though the quantity of cargo moving through a port is important, of more interest is the type of cargo that a port focuses on. 3.2 The South African Port Sector Before examining the Port of Durban in isolation, it would be prudent to briefly discuss the South African Port scenario in a broader sense. In South Africa, ports are considered national assets and are managed by the government run recently by SAPO. South Africa is a major sea-trading nation comprising of approximately 8 trading ports, namely, Durban, Richards Bay, East London, Port Elizabeth, Mossel Bay, Cape Town, Saldanha and the under construction Coega. South Africa has evolved into a major sea-trading nation over the last four or so decades and in 2002 handled 3.6% of world sea trade by volume. In terms of ton miles or real activity, this figure increases to 6% of global trade, placing the country within the top 12 globally and resulting in a global maritime activity share that is more than 20 fold its global GDP share. Sea trade constitutes more than 90 percent of trade in South Africa and ports play a critical social and economic role both nationally and regionally. The majo rity of the port activity is concentrated on the east coast of South Africa. A stark illustration of this fact is that Durban and Richards Bay together make up 76% of sea trade in the country. Traffic growth in the 1990s was derived from two primary regional points and sources, namely Durban from a general cargo perspective and Richards bay from a raw materials perspective. Richards Bay, which deals primarily in bulk goods, such as coal, ore and steel, has seen its annual tonnage increase from 55 million tons in 1989 to in excess of 90 million in 2000. Viewing perceived value in terms of tonnage is a flawed approach since in terms of economic linkages and value-adding, handling a ton of coal is not the same as handling a ton of refined goods. The figure below illustrates the breakdown of sea trade activity by port in South Africa. It can be seen clearly that Durban and Richards Bay are giants in comparison to the other ports. (Chasomeris, 2003 and Jones, 2002) Fig 17: Total Traffic Volume in South Africa Source: Department of Transport, 1998 and Jones, 2001 The South African Ports sector experienced significant capital intensive investment in the 1970s and 1980s, which was biased towards the bulk shipping sector. However, world trends have seen a migration towards containerisation and unitisation and South Africa is no exception, with the country utilising containers for the first time in1977. Up until 1990, the available capacity could cater for national traffic levels of approximately 1 million TEUs level. The lack of adequate container capacity, combined with growing demand, brought with it a multitude of problems. On the demand side, South Africa became a democracy and re-entered the globalised world, resulting in a noticeable rise in seaborne container volumes, due to liner carriers returning to the South African trades and increased trade liberalisation. The upsurge in volumes produces inevitable negative consequences of delays and vessel queues. By 2000 the combined amount of annual TEUs handled in South African ports was 1.8 mil lion and this was encompassed using with the same basic container quays that had been constructed in 1977. There was some limited capital investment in strategic areas in the 1990s, such as cargo extensions to bulk and neo-bulk facilities in Richards Bay. The new millennia brought with its bolder and more ambitious port investment initiatives. A new industrial hub status port in the Eastern Cape, which was earlier envisioned but never actioned upon, was now being constructed. Secondly, the Durban general cargo infrastructure has received significant upgrades and extensions such as extensions to landside facilities as well, deepening and extending cargo handling superstructure and infrastructure as well as deepening and widening the harbour entrance. Because of the age and mismatch of the cargo handling infrastructure, productivity has lagged that of international levels, resulting in congestion that is a constant feature of local ports. There were also supply side issues to deal wit h such as liner route becoming more specific and centred around hub status ports. As such, hub status ports have to provide capacity that exceeds national demand, making attainment of hub port status difficult in capacity constricted scenarios. This is compounded by the reluctance of ship-owners to migrate shorter routes such as Port Elizabeth in South Africa. South African ports relative competitive stance with their southern hemisphere counterparts can be gauged from the table below. Looking at both indicators, South African ports emerge as clear leaders on both the African and Southern Hemisphere front. Richards Bay is ranked first on the table in terms of total traffic, as it has a large amount of coal and other bulk cargoes passing through its doors. Durban, although ranked 3rd overall, is ranked 1st in the container category it is clear that Durban is the leading multi-purpose port in South Africa and the Southern Hemisphere. (Jones, 2003; Jones, 1997; Department of Transport, 1998 and Lawrence, 2000) Figure 18: African and Southern Hemisphere Port Traffic Port Total Port Traffic (m tons) Rank Container Traffic (TEUs 000s) Rank Richards Bay 91.5 1 5 15 Newcastle 73.9 2 9 14 Durban 49.7 3 1291 2 Santos 43.1 4 945 4 Sydney 24.6 5 999 3 Melbourne 22.3 6 1322 1 Casablanca 19.8 7 311 9 Abidjan 14.6 8 434 7 Auckland 13.3 9 561 6 Cape Town 11.8 10 395 8 Lagos 9.1 11 1782 11 Mombasa 8.9 12 219 10 Buenos Aires 7.8 13 716 5 Dakar 7.2 14 149 13 Port Louis 4.7 15 161 12 Source: ISL, Bremen, 2001, Jones 2003 (Selected ports, 2000) 3.3 History of the Port of Durban The port is situated on the east coast of South Africa at coordinates 31o 02E in longitudinal and at 290 52S in latitudinal terms. Trading activities in the port of Durban can be traced back since 1824, with the port quickly gaining a favoured status among seafarers amd traders due to it being a natural harbour. Interest in Durban Bay grew tremedously in the early years of its operations, with imports doubling between 1849 and 1850. This, coupled with larger vessels, resulted in a much needed expansion to the harbour entrance. Over a century later, Durban has 63 berths and 6 repair berths, which can be broadly seperated into five main segments of the port. The first segments has two piers and has a multipurpose function thats handles general, parcel and unitised cargo. The second segment of the port is located by Salisbury Island and Island View. A third segment is the Maydon Wharf area, which contains private terminals as well as terminals controlled by Transnet. The Point terminal area and the Bayhead area are the fourth segment and fifth segment respectively. Below is a picture of the port of Durban that illustrates the five segments discussed. Figure 19: The Current Layout of Durban Port Source: Google Earth, 2010 3.4 Economic Significance of the Port of Durban As, can be seen in figure 17 above, the logistical strength of the national shipping infrastructure, rests primarily in KZN. The port of Durban, like all other public ports in South Africa, is an example of a port under national jurisdiction, its official name being the National Ports Authority (NPA), thereby allowing centralised planning. Durban is a port of choice because of its infrastructure in place enabling it to be a full service general cargo and container port . In addition to this, durban is well serviced by an adequete rail and road infrastructure, which links it to the economic hub of South Africa, Gauteng. In addition to this, the KZN region is a large economic region in itself and is second only to Gauteng in South Africa. Figure 21 below, illustrates a snapshot of the South African port sector for 2009. In terms of total cargo tonnes handled, Durban has 20% of the market and is dwarfed by Richards which has more than double Durbans tonnage handled, at more than 40%. Ri chards Bay, which was constructed in the 1970s, has had an enormous impact on Durbans port planning and functions. The primary reason for its existence was to serve as high-mass export point for raw materials such as coal. Richards Bay also diversified its goods base to include, at a lower cost, goods types that were traditionally the domain of Durban such as neo-bulk cargo like steel, alloys and forest type products. At the time of Richards Bay construction, Cape-sized bulk vessels were too large to enter Durban. (Jones, 2003 and Stats SA, 2010) Figure 21: Port Cargo and Vessel Statistics in South African Ports RICHARDS BAY DURBAN CAPE TOWN SALDANHA BAY TOTAL SA PORTS Durban as a % of Total TOTAL CARGO HANDLED: 77,631,154 37,419,282 3,058,601 56,475,625 182,735,369 20% GENERAL CARGO VESSELS: 247 705 220 373 1,648 43% BULK VESSELS: 1257 930 320 921 3,603 26% CONTAINER VESSELS: 42 1883 897 784 4,233 44% TANKERS: 184 646 159 344 1,542 42% VESSEL TOTAL: 1874 4848 2440 3489 15,879 31% TOTAL TEUS HANDLED: 6,273 2,395,175 1,382,052 NA 4,334,612 55% Source: NPA, 2009 (Note table has been edited) Looking again at figure 21 above, it can be observed that even though Durban lags other ports in gross tonnage of cargo, it still has by far the most number of vessels docking. One of the major reasons for this was the emerging dominance of Richards Bay, which forced Durban to concentrate on lower-volume bulk, break-bulk and liquid-bulk. This enabled great diversity within the port in terms of cargo type as well vessel type and quantity. Additionally, vessels that carry break bulk are traditionally far smaller than that of traditional bulk, thus explaining why more vessel docking are in Durban than Richards Bay for the same amount of cargo ceterus paribus. With reference to the figures above, it can be observed that Durban has 43% of total general cargo vessels, 42% of total tankers and 44% of total container vessels. The most important figure, in relation to Durban, is that of TEUs handled since this is where its dominance and significance come to the fore. Durban has the ideal stru cture to handle containers and since Richards Bay has inadequate structure for containers, Durbans dominance in containers was from the outset. Jones (2003) show that a growing international trend of shipping lines with regards to containers is to organise trade and activities around so called hub ports which meet and cross at sub-regional transhipment nodes. This arrangement is biased for the existence of a single hub type port on the eastern shores of the Southern region of Africa. Since, Durban is the countrys major container port, is well frequented by major shipping lines, has terminal and hub status, it is quite reasonable for it to remain South Africas primary container port. The other alternatives on the eastern sea board are not really competitors when it comes to containers. Richards Bay is primarily a bulk port and does not have the adequate infrastructure to extend its activities beyond this scope. Maputo has large deviation costs from traditional shipping lines as well as limited depth and capacity. Port Elizabeth has weak land side links to Gauteng as well as having limited local demand to justify a major port there. (Suykens, 1984; Jones, 2001 and Jones, 2003) Even though Durban lags Richards Bay in terms of pure tonnage, this in itself is a poor yardstick of economic impact and significance since no account is taken of cargo value or employment propensities of infrastructure required. Generally, in terms of economic and employment impacts, general cargo provides the most followed by dry-bulk cargo and lastly liquid-bulk. Bearing this in mind, comparing two ports only on the basis of tonnage is frivolous and more specifically in Durbans case it can be seen that from a ports perspective, it handles higher valued cargo than Richards Bay. This is especially evident when one considers one job is created per 47000 tonnes of cargo handled at Richards Bay, whereas in Durban, one job is created per 7500 tonnes of cargo handled. Figure 22 below further illustrates the economic richness and opportunity that containers present. Additionally, in 2004 an average container vessel spent R2.94 million per port call, far exceeding the R1.8 million for a br eakbulk cargo vessel as well as exceeding the R1.3 million for a bunker vessel. (Suykens, 1984; Jones, 2001, Tempi, 2006 and Jones, 2003) Figure 22: Port of Fremantles Economic impact by Cargo Type Cargo Type Output ($m) Value Added ($m) Household Income ($m) Employment (no.) Direct Effects Containers 177 121 73 1331 Other General Cargo 45 30 18 340 Liquid Bulk 35 20 8 158 Dry Bulk 83 44 25 459 Other 1 1 0 7 Total 341 215 124 2294 Direct + Indirect Effects Containers 382 240 125 3195 Other General Cargo 96 59 31 800 Liquid Bulk 67 38 17 441 Dry Bulk 181 100 50 1339 Other 2 1 1 19 Total 728 440 223 5792 Source: Bureau of Economic Transport Economics Australia, 2000 As is the case with South African ports, the port of Freemantle in Australia, shown in figure 22 above, derives the most economic prosperity from containers from both a direct and indirect perspective. Even though containers account for only 13% of activity in the port, they contribute 55% to economic activity. Consequently, containers have the greatest employment generating effects, followed by dry bulk and the liquid bulk. Though dynamics differ from port to port in terms of infrastructure, administration, socioeconomics and geography, a broad consensus can be reached from the figure above encompassing a kind of rule of thumb approach. As such, containers offer the most economic opportunity for a port and since Durban already focuses on this area, it would be prudent to continue with this trend. Thus, it is quite evident that both the present and future comparative advantage of Durban port rests in the realm of containerised cargoes due to reason shown above. Also, since the port i s so aptly designed for and dependant on containerised cargo, the removal of this great economic magnifying source would be particularly devastating on the Durban region as a whole. (Jones, 2001 and Jones, 2003) Looking at figure 23 below, it can be seen that the Durban port has seen an extraordinary increase in containers, with annualised growth of between 8% and 10% for the last decade.  As was shown above, containers form an integral cog in the Durban port machine from an economics and social perspective since they provide a source of trade, income and employment. Container growth has been driven by a range of factors such as rising volumes of world trade and reduced trading barriers, the migration of cargo to containers from other handling systems, South Africas improved economic performance and rising per capita incomes.  The facets examined below are containers landed, shipped and empty and as the diagram shows, all three categories have increased from 2002-2007. The growth between 2002 and 2007 is nothing short of spectacular, but this growth has not come without costs and constraints. However, needing containers and providing adequate space for them are two entirely different thi ngs and this will be explored below. Also, we have seen that general cargo is the richest form of cargo and has the largest employment benefits. South Africa needs extended general cargo capabilities and in this respect, Durbans needs are similar to national needs. It is thus clear that Durban needs the container industry for continued survival and prosperity, but whether the container industry needs Durban as much remains to be seen. (Jones, 2003) Figure 23: Total TEUs Landed, Shipped Transhipped Source: NPA, marketing graphs, 2008 Durbans greatest strengths, namely its ideal location, good economic linkage and strong infrastructure, have also evolved to be its Achilles heel, since its popularity especially for containerized cargo, has seen demand surge amidst mostly fixed infrastructure. With the growth of sea trade demand, the real problems of Durban are the lack of adequate marine infrastructure, but its role as port with terminal capacity, and the managerial capacity and willingness to operate the present container terminal at acceptable performance levels. A supply side response by the authorities to these demand pressures has been slow and limited. The growth of containerised cargo volumes has put the ports container terminal under sustained pressure since the mid-1990s, and at times has overwhelmed available capacity. The consequences of which have been frequent queues of container vessels, unduly high berth occupancy rates, and delays to container vessels and their cargoes. The port area is inundated wi th industrial and commercial development, making space an expensive premium, above all for neo-bulk space intensive cargoes like steel and forest products. It is therefore no surprise to see certain of these cargoes migrating to Richards Bay, where space is at less of a premium. The Durban-Gauteng rail line possesses substantial spare capacity, but operating problems associated with the availability of Transnet have reduced the reliability of rail. This problem is particularly serious for certain bulk terminals that are reliant on rail since for bulk commodities rail is the cheapest and most efficient form of transport. Previously, Durbans major economic disadvantage was its inability to host Panamax size-threshold ships due to its lack of depth. However, after recent capital investments, the entrance width has been increased from 110m at its narrowest to 220m and the depth in the outer channel from 12m to approximately 19m. However, this is far from adequate and as can be seen in I rcha (2006) which states that hub status type ports must have the following in order to remain relevant: à ¢Ã¢â€š ¬Ã‚ ¢ Container-stacking densities of 2000-4000 TEUs per hectare; à ¢Ã¢â€š ¬Ã‚ ¢ Sustained ship-to-shore gantry crane productivity of 50 moves per hour; à ¢Ã¢â€š ¬Ã‚ ¢ Three day dwell times; à ¢Ã¢â€š ¬Ã‚ ¢ 30-minute truck turnaround times; à ¢Ã¢â€š ¬Ã‚ ¢ On-dock rail service; and à ¢Ã¢â€š ¬Ã‚ ¢ Water depths by the berth of 15 metres and more. Currently, Durban subscribes to one of these parameters, and if it wishes to become efficient and remain productive and relevant, authorities should try to subscribe to all of them. Doing so would require significant capital investments such as infrastructure expansions. (Lawrence, 2000; ISL, 2001; Fairplay, 2003, Ircha, 2006, Transnet, 2010 and Jones, 2003) 3.5 Multiplier Model The theory of the Keynesian multiplier was covered quite extensively in the literature review. Figure 22 above touched on the multiplier process for the port of Freemantle, but the concept will now be explored and applied in far more detail. The economic impact of port activities on the local economy can be subdivided into three broad areas. The first area is that of directly port-related or port generated activities, that would cease to exist if the port were to close. The second area is that of indirectly port-related activities and pertains to backwardly-linked services and infrastructure. The third and final broad category is termed induced effects, and is in fact the multiplier effect from other inputs. It arises as those employed in the previous two categories, re-spend their money in the local economy, thereby increasing the original economic impact. Jones (1998) conducted a study so as to ascertain the Port of Durbans economic impact on the local economy. Figure 24 below is t aken from that same study and as can be observed, 24 000 direct port related jobs from approximately 360 businesses are created through first round inputs. Of the 24 000 jobs, approximately 8500 are from Transnet, which is an indication of the significant role that the institution plays in the local region. The 24 000 figure translate into a wage bill of approximately R950 million rand in 1994 wage level. Assuming an inflation rate of 10% per annum, this figure would equate to approximately R4 Billion in 2010 terms! Coupled to this, many port activities were in fact excluded from the above calculation such as insurance, financial services, medical services and legal services. (Jones, 2003) Another reason why the employment figure is conservative is that it fails to account for the induced or multiplier effect. As shown in the literature review, the economic or employment effect is extended far beyond the initial spending impetus whereby the final round of total expenditure normally far exceed the initial input. The multiplier varies from region to region depending on the average marginal propensity to consume, taxes, and how much money is kept within the local region. Jones assumes that since the majority of port employees are in fact low to middle income earners, which is not an outrageous assumption. Bearing this in mind, an average tax rate of 20%, MPC of 0.85 and a retention rate of 0.85 is used to formulate the multiplier value. The data is substituted into the multiplier equation from the literature review and yields a multiplier value of 2.4. The port of Seattle conducted an economic impact analysis and depending on which assumptions they used, the multiplier ra nged from 2.9 to 4.4. The port of Lake Charles Harbour also conducted an economic impact study and used a multiplier of 2.6 and the port of Hastings derived a multiplier of 1.58. Thus, the figure use by Jones is in no way over the top when one looks at other port economic impact papers and it even falls on the lower end of the spectrum. The box below illustrates the calculations that were used to obtain the multiplier. At 1994 prices total income generated by the port is approximately R2.3 billion. Once again, if we assume a 10% increase per annum, in 2010 price terms, this would equate to R9.6 Billion! (Jones, 2003; Meyrick Associates, 2007 and Martin Associates, 2007) Figure 24: Multiplier for Durban (1994 prices) ÃŽÂ ± = 1 1 -c [(1-t) r] Substituting the various values = 1 1 -0.85[(1-0.2)0.85] =2.4 Calculating Equilibrium income for wages only: Yo = ÃŽÂ ±A Yo= 950 X 2.4 = R2.3 Billion Calculating Equilibrium income for all expenditures: Yo= (950+500) X 2.4 = R3.5 Billion Source: Jones, 2003 Even with the multiplier effect, the regional economic impact of the port is under estimated since wages and salaries are not the only costs in a port. Industries which provide inputs and services to port establishments are excluded. In the same paper, Jones attempts to calculate these very costs and some of the examples include paper, ropes, cranes, hooks and property costs. Jones does this by working out that on average 48% of total costs are non wage costs and based on this assumption, a 1994 figure of R500 million is generated from port related expenditure which is not linked to wages. This amount extrapolated to regional labour elasticitys, induces a labour figure of approximately 7000 jobs. The refineries around the port employ around 1800 people and the Island View area about 500 as well. Thus, as Jones rightly says, the port and port related activities generate around 40000 jobs in the local economy, a figure which eThekwini online concurs with. Looking at the box above, it c an be calculated that the total economic impact of the port is R3.5 Billion in 1994 prices. In 2010 monetary terms, this equates to roughly R14.62 Billion. Additionally, eThekwini online states that the port and related industries contributes over 20% of Durbans GDP and approximately 1.5% of national GDP! Thus, it is quite evident that the port and its related clusters are integral to the Durban community in terms of employment and social stability. (Jones, 2003 and www.thekwenionline.org.za, 2010) Figure: 25 Durban Port Employment and Output (all data at 1994 levels) Industry/Sector Number Employment Wage bill (R mill) Portnet 1 5400 240 Portnet dredging 1 112 6 Spoornet 1 3217 115 Terminal operators 11 2213 90 Liquid bulk terminals 3 275 16 CF agents 138 3600 135 Ships agents 37 1350 65 Ship chandlers 17 400 ns Container depots 3 366 13 Container parks 7 260 ns Container logistics 3 140 6 Shipowners operators 5 11002 ns Ship repairers builders 5 9603 34 Stevedores 24 1650 45 Cargo equipment suppliers 2 200 ns Road haulers >75 15001 ns Bunker services 2 110 5 Offshore services 3 80 3 Tallying services 5 1204 ns Security 3 3001 ns Marine contractors 2 114 5 Customs Excise 1 300 ns Other State 3 1001 ns TOTAL >360 23867 ~R950 Source: Jones, 2003 3.6 Constraints to Expansion As shown in Figure 18 above, Durban is the largest general cargo port in Africa and the second largest in the southern hemisphere, and Durban being a port city will benefit from any growth in international trade volumes especially of the general cargo type. Although Durbans port infrastructure is extensive, at present it suffers from critical capacity limitations. The port currently provides 63 berths that can be used for cargo related activities as well as repair facilities for a further 8-9 vessels. These capacity constraints are encountered in respect of the ports marine infrastructure, cargo-working facilities and its overall articulation with landside cargo distribution systems. The constraints are indicated in the figure below, which illustrates the situation for Durban in 2004/5, considering that the teu amount was 2,395,175 teus for 2009, it becomes clear how grave the capacity situation is. Considering how grave the capacity situation is, it is indeed surprising that only sh ort term capital investments have been undertaken over the last two decades. Towards the end of the previous century, there were some capital extensions such as gantries, larger container areas and straddle carriers. In 2002, more gantries were added as well as 20 straddle carriers. The second part of the 2002 project was the relocation and specialisation of areas within the port, namely pier 1. All these short term improvements will result in the port having a present day capacity of 2.5 million TEUs. Already in 2005 the container terminal were operating at 90% capacity and now 5 years hence, with TEUs handled being 2.4 million in 2009 or 96% capacity, there is a pressing need for Durban to increase and improve its container handling operations. (NPA, 2009 and Muller, 2004) Figure 26: Port of Durban Capacity Constraint Terminals Current traffic M ton Theoretical capacity M ton Spare Capacity Percentage used Bulk Liquids 23,800,000   Unlimited Unlimited Motor vehicles units 171,365 220,000 48,635 77.89 Coal 1,800,000 2,500,000 700,000 72 City 2,400,000 5,200,000 2,800,000 46.15 Containers 1,724,218 1,900,000 175,782 90.75 Break bulk 4,200,000 6,300,000.00 2,100,000 66.67 Total excl vehicles 33,924,218.00 16,120,000.00 5,824,417.00 Source: NPA, 2006 Though this paper views the port from

Strategies for the Analysis of Big Data

Strategies for the Analysis of Big Data CHAPTER: 1 INRODUCTION General Day by day amount of data generation is increasing in drastic manner. Wherein to describe the data which is in the amount of zetta byte popular term used is â€Å"Big data†. Government, companies and many organizations try to obtain and store data about their citizens and customers in order to know them better and predict the customer behavior. The big example is of Social networking websites which generate new data each and every second and managing such a huge data is one of the major challenges companies are facing. Disruption is been caused due to the huge data which is stored in data warehouses is in a raw format, in order to produce usable information from this raw data, its proper analysis and processing is to be done. Many of the tools are in progress to handle such a large amount of data in short time. Apache Hadoop is one of the java based programming framework used for processing large data sets in distributed computer environment. Hadoop is useful and being used in types of system where multiple nodes are present which can process terabytes of data. Hadoop uses its own file system HDFS which facilitates fast transfer of data which can sustain node failure and avoid system failure as whole. Hadoop uses Map Reduce algorithm which breaks down the big data into smaller part and performs the operations on it. Various technologies will come in hand-in-hand to accomplish this task such as Spring Hadoop Data Framework for the basic foundations and running of the Map-Reduce jobs, Apache Maven for distributed building of the code, REST Web services for the communication, and lastly Apache Hadoop for distributed processing of the huge dataset. Literature Survey There are many of analysis techniques but six types of analysis we should know are: Descriptive Exploratory Inferential Predictive Causal Mechanistic Descriptive Descriptive analysis technique is use for statistical calculation. It is use for large volume of data set. In this analysis technique only use for univariate and binary analysis. It is only explain for â€Å"what, who, when, where† not a caused. Limitation of descriptive analysis technique it cannot help to find what causes a particular inspiration, performance and amount. This type of technique is use for only Observation and Surveys. Exploratory Exploratory means investigation of any problem or case which is provides approaching of research. The research meant provide a small amount of information. It may use variety of method like interview; cluster conversation and testing which is use for gaining information. In particular technique useful for defining future studies and question. Why future studies because exploratory technique we use old data set. Inferential Inferential data analysis technique is allowed to study sample and make simplification of population data set. It can be used for trial speculation and important part of technical research. Statistics are used for descriptive technique and effect of self-sufficient or reliant variable. In this technique show some error because we not get accurate sampling data. Predictive Predictive analysis it is one of the most important technique it can be used for sentimental analysis and depend on predictive molding. It is very hard mainly about future references. We can use that technique for likelihood some more companies are use this technique like a Yahoo, EBay and Amazon this all company are provide a publically data set we can use and perform investigation. Twitter also provides data set and we separated positive negative and neutral category. Causal Casual meant incidental we determine key point of given casual and effect of correlation between variables. Casual analysis use in market for profound analysis. We can used in selling price of product and various parameter like opposition and natural features etc. This type of technique use only in experimental and simulation based simulation means we can use mathematical fundamental and related to real existence scenario. So we can say that in casual technique depend on single variable and effect of activities result. Mechanistic Last and most stiff analysis technique. Why it is stiff because it is used in a biological purpose such study about human physiology and expand our knowledge of human infection. In this technique we use to biological data set for analysis after perform investigation that give a result of human infection. CHAPTER: 2 AREA OF WORK Hadoop framework is used by many big companies like GOOGLE, IBM, YAHOOfor applications such as search engine in India only one company use Hadoop that is â€Å"Adhar scheme†. 2.1 Apache Hadoop goes realtime at Facebook. At Facebook used to Hadoop echo system it is combination of HDFS and Map Reduce. HDFS is Hadoop distributed file system and Map Reduce is script of any language like a java, php, and python and so on. This are two components of Hadoop HDFS used for storage and Map Reduce just reduce to immense program in simple form. Why facebook is used because Hadoop response time fast and high latency. In facebook millions of user online at a time if suppose they share a single server so it is work load is high then faced a many problem like server crash and down so tolerate that type of problem facebook use Hadoop framework. First big advantage in Hadoop it is used distributed file system that’s help for achieve fast access time. Facbook require very high throughput and large storage disk. The large amount of data is being read and written from the disk sequentially, for these workloads. Facebook data is unstructured date we can’t manage in row and column so it is used distributed f ile system. In distributed file system data access time fast and recovery of data is good because one disk (Data node) goes to down other one is work so we can easily access data what we want. Facebook generate a huge amount of data not only data it is real time data which change in micro second. Hadoop is managed data and mining of the data. Facebook is used new generation of storage and Mysql is good for read performance, but suffer from low written throughput and the other hand Hadoop is fast read or write operation. 2.2. Yelp: uses AWS and Hadoop Yelp originally depended upon to store their logs, along with a single node local instance of Hadoop. When Yelp made the giant RAIDs Redundant Array Of Independent disk move Amazon Elastic Map Reduce, they replaced the (Amazon S3) and immediately transferred all Hadoop The company also uses Amazon jobs to Amazon Elastic Map Reduce. Yelp uses Amazon S3 to store daily huge amount of logs and photos,. Elastic Map Reduce to power approximately 30 separate batch RAIDs with Amazon Simple Storage Service scripts, most of those generating around 10GB of logs per hour processing the logs. Features powered by Amazon Elastic Map Reduce include: People Who Viewed this Also Viewed Review highlights Auto complete as you type on search Search spelling suggestions Top searches Ads Yelp uses Map Reduce. You can break down a big job into little pieces Map Reduce is about the simplest way. Basically, mappers read lines of input, and spit out key. Each key and all of its corresponding values are sent to a reducer. CHAPTER: 3 THE PROPOSED SCHEMES We overcome the problem of analysis of big data using Apache Hadoop. The processing is done in some steps which include creating a server of required configuration using Apache hadoop on single node cluster. Data on the cluster is stored using Mongo DB which stores data in the form of key: value pairs which is advantage over relational database for managing large amount of data. Various languages like python ,java ,php allows writing scripts for stored data from collections on the twitter in Mongo DB then after stored data export to json, csv and txt file which then can be processed in Hadoop as per user’s requirement. Hadoop jobs are written in framework this jobs implement Map Reduce program for data processing. Six jobs are implemented data processing in a location based social networking application. The record of the whole session has to be maintained in log file using aspect programming in python. The output produced after data processing in the hadoop job, has to be exp orted back to the database. The old values to the database have to be updated immediately after processing, to avoid loss of valuable data. The whole process is automated by using python scripts and tasks written in tool for executing JAR files. CHAPTER: 4 METHOD AND MATERIAL 4.1  INSTALL HADOOP FRAMWORK Install and configure Hadoop framework after installation we perform operation using Map Reduce and the Hadoop Distributed File System. 4.1.1 Supported Platforms Linux LTS(12.4) it is a open source operating system hadoop is support many platforms but Linux is best one. Win32/64 Hadoop support both type of platform 32bit or 64 bit win32 is not chains assembly platforms. 4.1.2 Required Software Any version of JDK (JAVA) Secure shell (SSH) local host installed which is use for data communication. Mongo DB (Database) These requirements are Linux system. 4.1.4  Prepare the Hadoop Cluster Extract the downloaded Hadoop file (hadoop-0.23.10). In the allocation, edit the file csbin/hadoop-envsh and set environment variable of JAVA and HAdoop. Try the following command: $ sbin/hadoop Three types of mode existing in Hadoop cluster. Local Standalone Mode Pseudo Distributed Mode Fully Distributed Mode Local Standalone Mode Local standalone mode in this mode we install only normal mode Hadoop is configure to run on not distributed mode. Pseudo-Distributed Mode Hadoop is run on single node cluster I am perform that operation and configure to hadoop on single node cluster and hadoop demons run on separate java process. Configuration we can change some files and configure Hadoop. Files are core.xml, mapreduce.xml and hdfs.xml all these files change and run Hadoop. Fully-Distributed Mode In this mode setting up fully-distributed mode non trivial cluster. 4.2  Data Collection The twitter data anthology program captures three attribute. 1) User id 2) Twitter user (who sent Tweet) 3) Twitter text The Twitter Id is used to extract tweets sent to the specified id. In our analysis; we collect the tweets sent to sachin tendulkar. We used Twitter APIs, to collect tweets sent to Sachin. The arrangement of the Twitter data that is composed. The key attributes Which we mine are: User id, Tweet text and Tweet User (who sent Tweet) save all key attribute in Mongo DB .Mongo DB is database where al tweet is saved. After collecting all data we export to csv and text file this file is use for analysis. Fig. 1. Twitter data collection procedure Extracting twitter data using python In this python code firstly create developer account then we get a consumer key, consumer secret, access token and access token secret this are important for twitter api using that key we find all tweets. Initialize a connection to the Mongo DB instance connectivity to Data Base in this code tweet db is data base name mongo db support to collection. >show dbs That commend we see all database those are present in mongo db. >use Data Base name Select particular data base we use. >db Db command use to which data base is open. >show collection This command shows all collection. It means show all table. >db.tweet.find () Use to show all data store in particular data base. >db.tweet.find ().count () Use to that command how much tweet store in your data base. CHAPTER: 5 SENTIMENTAL ANALYSIS OFBIG DATA Last and foremost as well as most important part of data analysis is extracting twitter’s data. Supervised and unsupervised techniques are types of techniques that are used for analysis of â€Å"Big data†. Sentimental analysis has come to play a key role in text mining application for customer relationship, brand and product position, consumer attitude detection and market research. In recent advance there is several promising new direction for developing and advance sentimental analysis research. Sentimental classification identify whether the semantic direction of the given text is optimistic, pessimistic or unbiased. Most of open approach relies on supervised learning models they classified positive and negative option only. Three ways of machine learning techniques Naà ¯ve Bayes, SVM and Maximum Entropy Taxonomy do not perform well on sentimental classification. Sentimental analysis techniques may help researchers to study on the Internet. They would help to find o ut whether a given text is subjective or objective as well as whether a subjective passage contains optimistic or pessimistic opinions. Supervised Machine Learning techniques use class documents for classification. The machine learning approach treat the opinion classification problem as a topic based content classification problems. Comparison between Naà ¯ve Bayes, Maximum Entropy and SVM for sentimental classification, they achieve best precision using SVM. CHAPTER: 6 SCREENSHOT Browser view: This view only use for browser view that show log file of data node and name node. Hadoop cluster on: In this screenshot show on data node name node that means properly install and configure single node hadoop cluster. Data base view: In this screenshot we extract twitter data and store Mongo DB. Mongo DB is a data base where all tweets are stored. How many Tweets store in Data Base: CHAPTER: 7 CONCLUSIONS We have urbanized an architecture that uses PYTHON and Mongo DB in amalgamation with Twitter APIs to study tweets sent to the specific user. We use our architecture to get the positive, negative and neutral, analysis the number of re tweets and the name and Id of the users sending the tweets. Finding all data we analysis them can be used in conjunction with available results on queuing theory, to study the temporary and stable state performance of social networks. The proposed architecture can be used for a monitor correlation among user behaviors and their locations. The application of obtain outcome to study the development of population in under research. In sentimental analysis mining on large datasets using a Naà ¯ve Bayes classifier with the Hadoop echo system. We configure Hadoop in single node cluster and we also provide how to fetch or extracting twitter data using any language of api but in Hadoop cluster file system can do decent job even in the Big Data analysis domain.

Somalia Culture Essay -- Essays Papers

Somalia Culture Somalia is a country situated in the ÒhornÓ of East Africa. It is bordered by the Gulf of Aden in the north, the Indian Ocean on the east and southeast, Kenya in the southwest, Ethiopia in the west, and Djibouti in the northwest. Somalia is about four times the size of the State of Minnesota, or slightly smaller than Texas. The capital is Mogadishu. Somalia's population is mostly rural. Nearly 80% of the people are pastoralists, agriculturalists, or agropastoralists. Except for a small number of Somalis who rely on fishing, the rest of the population are urban dwellers. Somalia's chief cities and towns are Mogadishu (the capital), Hargeisa, Burao, Berbera, Bossaso, Marka, Brava, Baidoa, and Kismaayo. In the past few years, civil war and famine have changed urban demographics as hundreds of thousands of displaced Somalis have poured into the cities seeking sanctuary and relief. Ethnically and culturally, Somalia is one of the most homogeneous countries in Africa. Somalia has its minorities: there are people of Bantu descent living in farming villages in the south, and Arab enclaves in the coastal cities. A small number of Europeans, mostly Italians, live on farms in the south. But the great majority of the people are ethnic Somalis who speak dialects of the same language, Somali, and who practice the same religion, Islam. In a land of sparse rainfall, more than half the population consists of pastoralists or agropastoralists who raise camels, cattle, sheep, and goats. There are farmers, mostly in the south and northwest, and in recent years a new urban group of government workers, shopkeepers, and traders has emerged, but it is the nomadic way of life, with its love of freedom and open spaces, that is c... ...e or the hand up to the wrist. Its application often signifies happy occasions, such as a marriage or the birth of a baby. Somalia's economic fortunes are being driven by its deep political divisions. The northern area has declared its independence.. During 1992-1993, Somalia experienced a great famine. This famine was the result of a drought coupled with the disastrous effect that infighting among rival clan militias had on the land and the livestock in Somalia. Somalis have always relied on their land and livestock to support themselves, and so this famine was devastating to them. Consequently, over 900,000 Somalis fled to neighboring countries. Approximately 400,000 of these refugees fled to Kenya. Since that time, some of the refugees have returned to Somalia, yet the situation there is still so tenuous that many have chosen to remain in the refugee camps.

Narrative of the Life of Frederick Douglas Essay -- Slave Narratives

Narrative of the Life of Frederick Douglas Frederick Douglas, a slave born in Tuckahoe Maryland, was half white and half black. His mother was a black woman and his father a white man. Though he never knew his father, there was word that it was his master. Douglas wrote this narrative and I felt that it was very compelling. It really showed me the trials and tribulations that a black man went through during times of slavery. In his early years, Douglas lived on a farm where he watched many slaves receive harsh beatings and whippings. For example, one of his masters whipped his Aunt Hester because she was not there when he desired her presence. At the time she was in the company of another man, which was something that Colonel Lloyd, her master, told her not to do. As Douglas witnessed the whipping, he saw Lloyd take his aunt into the kitchen of the house and strip her naked. He then told her to cross her hands and as he tied them together and hung her on a hook, leaving her body totally open. Lloyd then began whipping her with a cow skin until she began to bleed. â€Å"I was so terrified and horror-stricken at the sight, that I hid myself in a closet, and dare not venture out till long after the bloody transaction was over† (p. 4). As a result of witnessing many beatings such as this, Douglas was able to put much feeling and heart into his works. Douglas wrote about many whites that he had encountered. Only a handful of which were not cruel to him. The oversee...

The Dispensable Nigger in Joseph Conrads Heart of Darkness :: Heart Darkness essays

The Dispensable African in Heart of Darkness    Three Works Cited  Ã‚  Ã‚  Ã‚   The story is about a man named Marlow, who is hired by The Company, which is a shipping company located in England.   Although Marlow had sailed before, he had never sailed to Africa.   The people who operated The Company (those located in England) are so far removed from reality, that they have no concept of the devastation caused in order to ship vast loads of ivory.   The Company is a perfect example of how these profit driven industries obtain their wealth – through the blatant disregard of the environment and their fellow man. One can only imagine the death and destruction that was inflicted in order to ship mass quantities of ivory.   The Company’s disrespect for the Africans and their environment was the typical attitude had by many nineteenth century profiteers.   Their rationale was that no matter what degree of damage was inflicted, they felt it would never affect them. Their disdainful attittude towards the Africans is expressed in the following words: The conquest of the earth, which mostly means the taking it away from those who have a different complexion or slightly flatter noses than ourselves, is not a pretty thing when you look into it too much.  Ã‚   (Conrad 9) In order to be able to conquer a people, one must dehumanize them and believe they are insignificant/inferior.   This mode of thinking is used to justify any atrocities committed by the conquerors. In â€Å"Root of Racism,† the superior attitude is described as All groups, by their nature, imply to the members that they are somehow special in particular ways and in many ways better, than their fellow travelers on this earth.  Ã‚   (Ross) This superior attitude has been evidently pervasive throughout mankind’s history; some strong examples of these are the war in Bosnia, the slaughter of the Tutus in Rwanda and the white settlers near annihilation of the Native Americans. Conrad’s character Marlow describes the natives as having â€Å"a wild vitality† and their â€Å"faces like grotesque masks.†Ã‚   These remarks demonstrate his fear and reinforces the distinction between himself and the natives. Racial or ethnic hatred is a direct consequence of our Fear Response.   Hatred is really taking the fear response one step further.   We justify that fear by invoking certain attributes to others by assuming that they may be inferior, evil or harmful.

Firefighter Employment Scenario Essay

Although it’s not clear in the story whether an associates degree is a precondition for taking the exam, in your opinion, should a degree be a requirement to sit for the exam? Yes Why or why not? A Fire science degree should be a requirement to take the test. It shows that the fire fighter it trying to improve him or her self and become more knowledgeable of their craft. It also shows that they are committed to their career. Working and going to school is not an easy thing to do. Like the question states we do not know if the degree is required to sit the exam, if not stated as a requirements, then the degree should not be a factor in the selection process. Based on the fact that Doug and Sam earned the first and second highest test scores, is it a reasonable assumption that the city of Davis will add Doug and Sam to the certification list? Yes Why or why not? According to what we know of the information provided they should both be on the list. They have both meet the requirements to sit for the exam and scored the two best scores. Can the City successfully defend itself on the basis of â€Å"disparate impact?† Yes Why or why not? The city did will not effect anyone of a protected class, under Title VII Once disparate impact is established, the employer must justify the continued use of the procedure or procedures causing the adverse impact as a â€Å"business necessity.† Under what circumstances may racial discrimination be proved by â€Å"disparate impact?† One does not need to be direct about the discrimination. Example would be â€Å"Whites only need to apply†.

A Latin Christmas Essay

In Martin Espada’s â€Å"Latin Night at the Pawnshop,† the poet examines the Latin culture during Christmas time in a young, but still growing community of Latino immigrants. The poem proposes that during some time in America, people of Latino descent could not enjoy themselves during the holidays as they would if they were in their own country. Therefore, the theme of the poem is heavily influenced by the demise of Latin culture in America. Espada augments his poem to make the theme clear by using the following elements of poetry: diction and tone, symbols, and imagery. Diction and tone play a critical role in Espada’s poem. In the first line, Espada uses what I think to be the most important word in the whole poem, â€Å"apparition†, to bring about a vision he has of a salsa band through the window of a pawnshop. The word apparition means a ghostlike image. By evaluating this word and its context, the poem itself has created a tone right away. We can say that the mood of this poem is very gloomy and depressing when all one can see is a ghost and nothing else. The poem then continues with descriptive words to describe other aspects. For instance, the word â€Å"gleaming† is introduced. The word gleaming means to shine brightly. By introducing this word, the poet draws emphasis on how important this salsa band is to him during Christmas. However, locked in the shop are â€Å"gleaming† instruments that can’t play no more and Christmas to him is left in utter silence. Moreover, Espada mentions two distinct colors, a â€Å"golden trumpet† (line 4) and a â€Å"silver trombone† (line 5). Both silver and gold help represent the time of Christmas. Almost all Christmas trees use silver and gold ornaments as a decorative feature. Also, the poem ends with another word worth noting, â€Å"morgue†. A morgue is a place where dead bodies are kept. Ironically, during Christmas, we don’t associate death with such a joyful time. However, in this poem, a connotation for the word morgue could include death. Now, putting all these elements together, we can conclude that Espada is revealing a very dark Christmas he had gone through. A Christmas where there was no trumpet blowing, no trombone playing, no congas drumming, no maracas swinging, no tambourines shaking, and that all present was just the thought of it-no real Christmas. Espada also uses symbols to further develop his point. The three major symbols in this poem are indeed the pawnshop, the instruments, and the price tags. First, the essence of the pawnshop itself is important because it tells a story, beyond itself. Sometimes money gets in the way of a person’s happiness. As a result, we pawn the stuff we really cherish for a quick buck. Espada is trying to explain that on top of the struggles Latinos face, they also must sacrifice the things they love. Second, the instruments tell us a great deal about the demise of Latin culture in America. Instead of being played and making great music during the holidays, they sit there unused. Espada in his poem creates an unwanted feeling. Lastly, the price tags that resemble that of a dead man’s toe are equally important. The tickets symbolize the presence of death where there should be life. The Latin culture in the town of Chelsea is completely dead. Latinos have given up on their culture in place where it’s not truly accepted. These elements create the point Espada is trying to express. Imagery is also an important aspect to this poem. The poet creates imagery that attacks several senses and by doing so, it also helps pinpoint the importance of different ideas. Espada writes, â€Å"gleaming in the Liberty loan pawnshop window,† (line 2). As one reads, you can’t help but imagine seeing this bright light coming out through a window and showing you a salsa band. Furthermore, Espada mentions several instruments. With this inclusion, one can imagine hearing the sounds of these lovely instruments playing coherently and in sync together. However, Espada also writes, â€Å"all the price tags dangling like the city morgue ticket on a dead man’s toe,† (lines 7,8,9). This image develops a kind of chilly and nervous feeling about what’s actually going on in the poem. All in all, by putting these sources of imagery together, you notice what the poet is trying convey. Espada is drawing our attention to a salsa band and all of its instruments, but in the end things aren’t always what we want or expect. The Latino culture is nothing more than an illusion, in a land that does not treat its immigrants well. Thus, in â€Å"Latin Night at the Pawnshop,† Espada creates a poem that expresses  his concerns about Latin culture in Massachusetts in the late 1980’s by using different elements of poetry. The power of diction and tone, symbols, and imagery, enrich the central theme the poet wants to make. Espada does a great job converting one simple moment, into a thousand words and ideas.