Freeing up disk space after Exchange 2010 installations

Posted on August 16, 2010 by Tony Redmond

If the disk where you installed the Exchange 2010 binaries seems a little clogged after the installation, you can relieve some of the pressure by cleaning up files that the Exchange 2010 installation leaves behind after it’s complete. The installation process should really clean up better after itself but it doesn’t.

The biggest culprit is the Exchange Server\V14\Logging\lodctr_backups folder. As you can see from the screen shot, this folder held 1.41 GB of data, all of which is utterly unnecessary to run Exchange 2010 (any version). Another of my Exchange 2010 servers had 3.52GB in this folder. Given the amount of storage available to servers these days, we really have become very blasé about wasting a gigabyte or two. In the early days of Exchange, the hard disks that we installed into servers might only have been 5 or 10 GB or so… but that’s another story

These files are created for use by the lodctr (load counter) program (see http://technet.microsoft.com/en-us/library/bb490926.aspx for more information about how lodctr works) when it updates a server with all of the performance counters used by Exchange 2010. If you update a server several times, perhaps with build-to-build upgrades to install a new build (for example, to install SP1 builds as they are released by Microsoft), you end up with lots of these files because performance counters are updated for each installation. All can safely be consigned to the recycling bin after the installation is complete. There’s a possibility that Microsoft PSS may want to check them if a server suffers a failed installation, but apart from that…

The other major culprit guilty of stealing disk space is the \ExchangeSetupLogs folder. I just checked a couple of servers and they each had over 100MB of setup logs. Again, you don’t really need to keep these files around if the server has had no problems with the installation. The Exchange installation process will recreate anything it needs the next time it runs to apply a service pack or roll-up update.

Some administrators like to keep setup logs around on the basis that you never know when you might want to refer to them. But there is no need to keep the performance load files!

– Tony

Follow me @12Knocksinna

Update 8 February 2013: For the record, Exchange 2013 leaves the same kind of files cluttering up servers. In this case, they are stored in the Exchange Server\V15\Logging\lodctr_backups folder… and they can be safely removed to tidy up a server after successfully installing Exchange 2013. The ExchangeSetupLogs folder on my Exchange 2013 servers stored some 240 MB of logs, most of which can be deleted too.

Learn more about how to manage Exchange 2010 in my Microsoft Exchange Server 2010 Inside Out book!

Or wait for Exchange 2013 Inside Out – due in October 2013.

Posted in Exchange, Exchange 2010 | Tagged , , , | 13 Comments

Cmdlet extension agents in Exchange 2010

Posted on August 13, 2010 by Tony Redmond

If you execute the Get-CmdletExtensionAgent cmdlet on an Exchange 2010 server and then select the name of the output objects, you’ll see something like this:

Name
----
Admin Audit Log Agent
Scripting Agent
OAB Resources Management Agent
Provisioning Policy Agent
Mailbox Creation Time Agent
Mailbox Resources Management Agent
Rus Agent
Query Base DN Agent


Cmdlet extension agents are a new feature of Exchange 2010. They are designed to extend the processing performed by standard cmdlets. The two best examples that I can cite are the Admin Audit Log agent, which captures details of administrative events if you enable logging with a command like this:

Set-AdminAuditLogConfig -AdminAuditLogEnabled $True

The other example is the Mailbox Resources Management Agent. This serves a really useful purpose in that it scans for new mailbox requests that don’t have a database specified (in other words, an administrator wants Exchange to decide in which database the new mailbox will be located). The agent scans for healthy databases in the site where the command is run and then omits any databases that are excluded from automatic mailbox provisioning to create a set that can host the new mailbox. It then selects a database at random and the new mailbox is created there. There’s no great intelligence exhibited in how the choice of database is made and a database that already has 1,000 mailboxes might be chosen as often as a newly-created database that only has a few mailboxes. The Mailbox Resources Management agent also takes care of selecting a suitable database for an archive mailbox, if you decide to separate the primary and archive mailboxes (but only in SP1).

The standard set of cmdlet extension agents clearly do useful work for Exchange 2010, but even better, you can create your own extensions and have Exchange execute them according to conditions that you state. This ability depends on the Scripting Agent, one of the standard set of cmdlet extensions, but also the only one of the standard set that is disabled by default. You can see the details of the Scripting Agent with:

Get-CmdletExtensionAgent -Identity "Scripting Agent" | Format-Table Name, Enabled, Priority

and enable it after you’re sure that it is disabled with:

Enable-CmdletExtensionAgent -Identity "Scripting Agent"

After that, you can begin to write your own code and have Exchange execute it before or after standard cmdlets such as New-Mailbox are run by administrators through EMS, EMC, or ECP. I won’t duplicate a blog post to tell you how here as there’s an excellent description already written by Pat Richard and posted on his blog

More stuff to stuff into the book, if I get to it… Time is marching on, the editors are getting antsy, and the presses must soon start turning.

On the other hand, it’s an example of the great kind of content that we can dive into during our three-day seminars in October Exchange 2010 Maestro seminars.

Tony

Learn more about the changes made to cmdlets in Exchange 2010 including more detail about cmdlet extension agents in my Exchange Server 2010 Inside Out book.

Posted in Exchange 2010 | Tagged , , , , , | Leave a comment

Controlling EWS access in Exchange 2010 SP1

Posted on August 12, 2010 by Tony Redmond

Another example of a late-breaking change in Exchange 2010 SP1 that causes authors to tear their hair out (if they have any) is the new ability to control access to Exchange Web Services (EWS) on an organization-wide or user-specific basis. Organization-wide access is controlled through the Set-OrganizationConfig cmdlet while the Set-CASMailbox cmdlet controls access on an individual basis.

For example, to block access for a user:

Set-CASMailbox -Identity 'Joe Soap' -EWSEnabled $False

A quick description of the available parameters is shown below:

EWSAllowEntourage Specifies whether to allow or disallow Entourage 2008 for Mac, Web Services Edition to access Exchange Web Services for the user. Note that Entourage 2008 uses EWS exclusively, so this parameter can be used to block Entourage 2008.
EWSAllowList Specifies the applications  as identified by user agent strings that can access Exchange Web Services when the EWSApplicationAccessPolicy parameter is set  to EnforceAllowList.
EWSAllowMacOutlook Specifies whether to allow or disallow Outlook for Mac to access Exchange using EWS. Future versions of Outlook for Mac will use EWS exclusively.
EWSAllowOutlook Specifies whether to allow or disallow Outlook 2007 to access Exchange Web Services for the user. Outlook uses Exchange Web Services for free/busy, OOF, and calendar sharing.
EWSApplicationAccessPolicy Specifies which applications other than Entourage, Outlook for Mac 2011 and Outlook can access Exchange Web Services. If set  to EnforceAllowList, only applications specified in the EWSAllowList parameter are allowed access to Exchange Web Services. If set to EnforceBlockList, every application is allowed access to Exchange Web Services except the ones specified in the EwsBlockList parameter.
EWSBlockList Specifies the applications (user agent strings) that can’t access Exchange Web Services when the EWSApplicationAccessPolicy parameter is set to EnforceBlockList.
EWSEnabled Specifies whether to globally enable or disable Exchange Web Services access for a user, regardless of which application is making the request.

When the EWSEnabled parameter is set to $false, Exchange Web Services access is turned off regardless of the values of the EWSAllowEntourage parameter.

For example, you could set organization access up so that EWS is only enabled for Outlook, Entourage, and a user agent that presents the string “OurGreatApp”:

Set-OrganizationConfig –EWSEnabled $True –EWSAllowOutlook $True -EWSAllowEntourage $True   –EWSApplicationAccessPolicy: EnforceAllowList
–EWSAllowList: {“OurGreatApp*”}

SP1 does not expose any UI in EMC or ECP to control EWS access. This may appear in a future service pack for Exchange 2010, or then again, it might not… Now the question is whether I can fit this information into the book or is it just too esoteric to make me want to omit it, given that space is tight anyway…

– Tony

Learn lots more about how to control Exchange 2010 clients in my Microsoft Exchange Server 2010 Inside Out book!

Other information (24 March 2011) from a correspondent who was struggling with EWS access. He writes:

It was a new consultant who found this document (http://www.microsoft.com/downloads/en/details.aspx?FamilyID=894bab3e-c910-4c97-ab22-59e91421e022), followed it from page 8 onward, and recreated three publishing rules in TMG2010 (and also changed something related to authentication under ”Client Access Server” in Exchange). After that, it looks like everything works, including Outlook 2011 for Mac from outside the office, setting Out of Office replies via iPhone (even reliable Scheduling Assistant and Mail Tips when using Outlook Anywhere in Outlook 2010 for PC!).

We also needed forms based authentication (I think) to work for our OWA since some people need to be able to change their expired passwords via the Web. That was not working earlier, either.

Posted in Exchange, Exchange 2010 | Tagged , | 20 Comments

Preparation continues for the Exchange 2010 SP1 training in October

Posted on August 7, 2010 by Tony Redmond

As I have posted on before, Paul Robichaux and I will be leading two three-day Exchange 2010 SP1 training events in the Doubletree Guest Suites in Boston (October 13-15) and at the Sheraton Anaheim Hotel (October 18-20). I guess those who attend the latter event can duck out to visit Disneyland before, after, or during the event if it all gets too onerous or our jokes fall flat. Logistics and other information about the events is available from Exchange 2010 Maestro seminars.

Paul and myself are backed up by Brian Desmond, another Exchange MVP, who is taking care of the labs and making sure that they are compelling, interesting, in-depth, and challenging. In fact, when you think about it, Brian probably has the hardest job of anyone on the team.

It’s all very well agreeing to speak at a training event. However, the hard work begins when you start to flesh out the agenda with real content. No one wants to duplicate what has gone before – it would be easy to recycle some of the content presented by Microsoft and others at recent TechEd events, and looking at some of the presentations that are floating around the Internet, a fair amount of cut and paste (all done in the best possible taste) occurs as people build presentations about Exchange 2010. Our aim is to provide a distinct and practical insight into Exchange 2010 SP1 through a series of talks covering the critical parts of the product with the learning reinforced through the labs referred to above. You can’t accomplish what we want to do without doing a whole heap of work up front to decide what topics to cover, build the material, check it out, link it with labs, and make sure that Microsoft doesn’t change anything as they drive to complete the SP1 code (very soon now).

Over the last two weeks, I have been working on three sessions in particular: Exchange 2010 architecture, Exchange 2010 Compliance, and the Exchange 2010 Store and High Availability story. The first session is the warm up for the event and didn’t cause me too many problems. The issue I face with the other two is deciding what’s important and what can be left out to get a deck down to a reasonable size, one that can be delivered in two hours or so. For example, my current draft for the Store session extends to some 1o2 slides and that’s just too many – yet there’s lots to talk about in this particular area and you could make a case that a complete three-day seminar could be done on the topic. So more work has to be done, more thinking has to happen, and a few late nights are still required to whip things into shape.

I imagine that Paul is going through the same process. Of course, he’d say that he is spending far too much time correcting all the errors in my Exchange 2010 book where he serves as the technical editor for the publication. In any case, we will soon have the material sorted out and be ready to deliver, and that’s when the real fun will start.

Looking forward to Boston and LA – Tony

If you can’t make the seminars, you might still consider buying my Microsoft Exchange Server 2010 Inside Out book… just to check whether Paul really did an excellent job as the technical editor!

Posted in Exchange 2010, Training | Tagged , , , , , , , | 7 Comments

Ballyconneely photo used by Go Ireland

Posted on August 6, 2010 by Tony Redmond

Go Ireland is the web site run by Bord Failte (Irish Tourist Board) to help visitors find places to stay and see in Ireland. They run regular photo competitions, one of which I entered in 2008 when we were living in the U.S.  I heard nothing back about the results and only recently discovered that Go Ireland were using one of my photos on their web site when the lady who owns the horses in the photo contacted me to ask for a copy! The online copy is at http://www.goireland.com/galway/ballyconneely.htm where it’s used to publicize the undoubted charms of a small village/region called Ballyconneely on the western seaboard of Ireland, a place that I have been visiting regularly since 1971.

In any case, I am always happy to help a good cause – and very happy to see the photo used in this way.

– Tony

Posted in Travel | Tagged , , , , | Leave a comment

Back to Lansdowne Road

Posted on August 4, 2010 by Tony Redmond

Lansdowne Road, the home of Irish rugby since 1872 (http://en.wikipedia.org/wiki/Lansdowne_Road) was reopened to rugby as the Aviva stadium last Saturday, July 31. I’m proud to say that I was there as a member of the IRFU (Irish Rugby Football Union) refereeing team that worked the game, not that I had much to do as Alain Rolland didn’t refer any decision to the TMO (as normal!).

I have a lot of memories from the old Lansdowne Road, including:

  • The first game I attended in 1967 – the Junior Cup Final between Blackrock and Terenure. Rather splendidly, I sat in the committee box as my grandfather, the late Judge J.C. Conroy, was secretary of the Leinster Branch of the IRFU.
  • Attending international rugby matches in the late 1960s and early 1970s, with memories such as the penalty goal kicked by Tom Kieran from the half-way line against Scotland in 1968, the “missed” (but awarded) drop goal by Gareth Edwards for Wales in 1968, Ken Goodall’s try against Wales in 1970 (watched from the old wooden uncovered stand), England turning up to play in 1972 when Scotland and Wales declined to travel due to the “Troubles”, and Alan Duggan’s try to draw 10-10 with the All Blacks in 1973.
  • Watching what must have been the worst soccer match played at the stadium when the Republic of Ireland and Northern Ireland drew 0-0 in a European qualifier September 1978. The grass growing on the pitch was a more exciting spectacle than the play.
  • Taking photographs of many games between 1976 and 1986. There never seemed to be much light available in the dim of winter, so I learned a lot about pushing Kodak Tri-X and Ilford HP-4/HP-5 B&W film to 1600 ASA and above, which was needed to get any reasonable shots of fast action (then again, rugby wasn’t as fast then as it is now).
  • The Digital Internationals from 1984 onwards. These were the first sponsored Irish international rugby matches and Digital Equipment Corporation paid the grand sum of IEP30,000 for the 1984-85 season. In return, there were many advertising boards erected in the ground and the Digital logo was plastered over tickets and programs. Digital also ran pre-match and post-match receptions in the old Lansdowne pavilion for invited guests and issued special ties for each game, some of which you still see worn today. I remember DECmate III word processing computers being used to generate match statistics for games, primitive as they were in those days when compared to the torrent of match data generated for games today.
  • Refereeing on the main international pitch. The first match that I took charge of was a Leinster Senior Cup match between Lansdowne FC and Garda circa 1995 and after that there was a series of cup matches including the Leinster Schools Senior Cup Final between Blackrock and Roscrea in 1999.
  • Watching other referees work big matches – including the unforgettable sight of Clive Norling (Wales) coming out to referee Ireland v France in 1983 with a permed hairstyle that rendered all of Lansdowne Road silent.
  • Acting as the reserve official for international games. The first match I acted in this capacity was for Ireland v USA in 1994; the last was Ireland v England in 2005. In that time I think I saw every major international team up close including New Zealand, South Africa, and Australia. The change in the requirements on the fourth/fifth officials changed dramatically in this period. For the USA game in 1994, I sat in the committee box and watched the game from there; for the last game it was a case of constantly working with the two teams to manage substitutions, blood injuries, and to keep the water carriers under some sort of control.

By 2006 Lansdowne Road was old, decrepit, and battered. The upgrades to places like the East Stand to incorporate new facilities such as corporate boxes had some impact but couldn’t mask the basic lack of match day facilities, the need to bring in temporary seats for soccer matches, and the general feeling that bits were likely to fall off various parts of the ground if you pushed too hard. The decision to tear it down and rebuild Lansdowne Road as a state of the art modern stadium was a good one that was proven correct when the first rugby match took place there last Saturday.

Not everything worked as it should have (for example, the clocks in the referees’ changing rooms were off and the balls provided to the players were under-inflated) and you could quibble at some of the arrangements, but overall the IRFU had an excellent trial run through of all the bits and pieces that have to come together for a major game and they now have the time to sort the minor hiccups out before international rugby returns in November 2010 with games against South Africa (Nov 6), Somoa, New Zealand, and Argentina.

The refereeing team for the first rugby match at the Aviva stadium, Dublin 31 July 2010: L-R: Dave Connolly (Nr. 5), Simon McDowell (AR), Alain Rolland (Ref), George Clancy (AR), Dudley Philips (Nr. 4), Tony Redmond (TMO)

The match wasn’t up to much because the two teams were not well matched and the Leinster/Ulster combination was far too strong for Munster/Connacht. In the end, a 68-0 scoreline told a story of a game that was never much of a contest while serving to start the ball rolling for rugby.

Of course, the IRFU now shares Lansdowne with the Football Association of Ireland, so soccer took center stage on Wednesday, August 4, 2010 when a selection from the Airtricity league (the local professional league that plays in the Republic of Ireland) took on Manchester United. This time I wasn’t so closely associated with the game as I turned up as a spectator and sat high up in the West Stand (row S, section 521).

Nani scores Manchester United's last goal from the penalty spot

The game wasn’t quite the same walkover as the rugby match had been although there were times when Manchester United were passing the ball around as if they were on the training pitch. The final score of 7-1 was a reasonable outcome from their perspective and illustrated the gap between a professional team of the highest level and a team drawn from many different semi-professional clubs to play a one-off match. The biggest cheer of the night (apart from the completion of the Mexican waves) was for the goal scored by the Airtricity League XI.

My conclusion is that the new Lansdowne Road works very well as a stadium. It is easy to get into and out of; its facilities are top class and the spectator view is very good, even high up in the stand. While the 49,361 crowd was quite noisy at times during the soccer game, it will be interesting to see just how loud a full-throated crowd gets when the rugby internationals come around, especially when England next arrives. If you want more information about Lansdowne Road and its past, you would do well to get a copy of Lansdowne Road: The Stadium; the Matches; the Greatest Days, an excellent book that covers all the bases.

– Tony

Posted in Rugby | Tagged , , , , , | 1 Comment

Fixing the -1018 problem for Exchange 2010

Posted on August 2, 2010 by Tony Redmond

One of the nice things about writing about technology is that you often unearth gems that developers have built into software that might be overlooked by many other folks. One such instance is the ability that Exchange 2010 has to patch individual pages within a database.

In previous versions of Exchange, especially those prior to Exchange 2007, administrators hated seeing events 1018 or 1022 logged in the application event log because these events indicate that the Store had detected a page-level corruption in a database. Corruptions usually occurred as the result of some hardware problem, often in storage controllers and while a database would continue to run with the corruption, the problem was that the only way to fix a database that contained a corruption was to restore from the last good backup. Given that some databases were well over 100GB, this wasn’t a popular option with administrators.

A page corruption in a single copy database is a severe problem. Replicating the corruption to many database copies creates a problem with a whole new dimension. For this reason, Exchange 2010 is able to detect and fix page level corruptions that occur in active or passive database copies.

If the Store detects a problem page in the active database (usually after it checksums a page after reading it into memory), it places a marker in the log stream (in the current transaction log) that acts as a request for a valid copy of the corrupted page. The request is sent to all database copies where it is inspected and processed along with other log content. When the Information Store replays data for the passive copy, it notices the marker and responds to the request by invoking a “replication service callback” to ship a copy of the page to the server that hosts the active database. When this server receives the replicated page, the Store patches it back into the active database to remove the corruption. Other servers that host passive copies may also respond with pages but these are ignored once the active database has been restored to good health.

The process to fix a corrupted page in a passive database copy is slightly different. In this case, the server that hosts the passive copy immediately pauses log replay. Log copying continues to ensure that all of the transaction logs that will eventually be required to bring the database completely up to date are available on the server. The server then requests a copy of the corrupted page from the server that hosts the active database using the internal ESE seeding mechanism. The active server responds with the page data. The passive server then waits until all the log files necessary to bring it up to date past the point where the active server provided the page (as indicated by the maximum required generation) have been copied and inspected. When it is sure that all the required data is available, the passive server then restores the corrupt page and resumes log replay to clear the backlog of transaction logs that have accumulated since the corruption was first detected.

One important point that is often overlooked is that single page patching does not work with lagged database copies. The logic is simple – lagged database copies work by being deliberately kept at a certain point in time distant from the current time and only apply updates from transaction logs as those logs pass the desired lag interval. The broadcast for a patch mechanism used by regular database copies won’t work if a corruption is detected in a lagged database copy and the only way to fix such a corruption is to restore the lagged database from a backup.

– Tony

Learn more about the workings of the Exchange 2010 Information Store in Microsoft Exchange Server 2010 Inside Out!

Posted in Exchange 2010 | Tagged , , | 1 Comment

Exchange 2010 SP1 still changing

Posted on July 29, 2010 by Tony Redmond

I’ve commented before about the problem posed for authors who attempt to describe software as it is being developed. Code that you think is solid and won’t change is removed before the final build because a bug is discovered or code that someone has been working on but not revealed suddenly makes an appearance in a build close to the final version. Or even worse, a feature that worked in a certain manner is changed in a small but fundamental manner that renders your description of the feature inaccurate or misleading. Such are the difficulties and dangers of writing about software that is under development.

You’d imagine that the issue is reduced for a service pack. Alas, this is not the case. There was a time when Microsoft would never make a change to the user interface of something like the Exchange Management Console in a service pack so new functionality was restricted in terms of the amount that appeared. This attitude has long since disappeared and Exchange 2010 SP1 is full of new functionality and user interface – some of which continues to change as Microsoft drives towards the final build that will allow them to release SP1.

Another example of a late-breaking change that causes authors to tear their hair out is the news that Microsoft has incorporated the optional “Change Password when Expired” feature for Outlook Web App into SP1. This is the same feature that is included in Exchange 2007 SP3 and it solves a problem that occurs when users attempt to log onto their account with OWA only to find that they can’t connect because of an expired password, so it’s kind of a catch-22 if they are attempting to connect from outside the company firewall and can’t change their password through Windows.

To allow the feature, you have to create a new DWORD value called ChangeExpiredPasswordEnabled at HLKM\SYSTEM\CurrentControlSet\services\MSExchange OWA on all CAS servers. Set the value to 1 (one) to enable the feature and 0 (zero) to disable it and then restart IIS. When enabled, users will be presented with a new SSL-secured page to allow them to input their old password and create a new password whenever they attempt to connect to OWA using an account with an expired password. The only small compliant that you’d have is that users are forced to supply their username and password in domain format (as in CONTOSO\Redmond) as the User Principal Name format isn’t supported. This is a bug that Microsoft has acknowledged; I anticipate that they’ll fix it in a future patch.

The interesting thing is that this feature is so useful that you kind of wonder why it was never incorporated before. The answer is probably two-fold: first, some companies don’t like users having the ability to meddle with passwords when they are working across the Internet. Second, there are lots of other features that customers demand that are possibly more interesting from a competitive perspective. Engineers trade off features all the time and this interesting and worthwhile feature never made the cut until now…

– Tony

Learn more about Exchange 2010 SP1 in my Microsoft Exchange Server 2010 Inside Out book.

Posted in Exchange, Exchange 2010, Writing | Tagged , , | Leave a comment

Exchange 2010 SP1 “Inside Out” book makes its first public appearance

Posted on July 28, 2010 by Tony Redmond

When browsing Amazon.co.uk today, I was amused to receive this recommendation:

Fantastic – the word is out that my Microsoft Exchange Server 2010 Inside Out book is soon to be available!

Great… until I read the PR text that talked about the content that should be in the 1,312 pages that will apparently be the final count (we live in hope). I’ve just sent an email to say that this text has to be updated as some of the chapters have been removed from the book to slim it down (as reported in this blog). Ah well, those pre-order the book because they are excited by the prospect of reading a chapter on public folder management or my rambling thoughts on the good and bad points of Exchange Online will have to be content with the remaining chapters… or maybe we can save ourselves and get the text updated so that it reflects reality. Time will tell and we’ll see just how quickly a change to PR text for a book can be made so that it appears on a site like Amazon.

I also like the “Supremely organized” tag line that Microsoft Press has included on the front page (see the screen shot). I wonder who they are talking about… it’s certainly not me!

– Tony

Posted in Exchange, Exchange 2010, Writing | Tagged , , , , , , | 3 Comments

Different views of property

Posted on July 26, 2010 by Tony Redmond

I’ve been a keen visitor to http://www.openmedia.fr amongst other French property web sites as a tool to help identify a suitable vacation property. Yesterday, I was browsing the details of some Provençal villas in a place called Flayosc (in the Var department in the south of France) and noticed that one particular house was offered by four different agents. It’s quite common in France to list a house with different agents and the agent that sells the house in the end is the one who is rewarded with the fee. The interesting thing in this case was that two agents had posted the same photo.

Two views of a villa in Flayosc

However, one agent had clearly decided that some Photoshop work had to be done to transform the green uncared for look of the pool into the sparkling waters that are only possible when a pool has been carefully maintained. Unfortunately, the editing wasn’t particularly well done (perhaps a tad too much use of the “clone” tool?) and the false azure blue of the doctored pool screams out of the resulting photo.

All of this goes to prove once again that you can’t believe everything that you see in a photo. It might also reinforce the opinion that some might hold about the business ethics of certain people in the property game…

– Tony

Posted in Travel | Tagged , , , , | Leave a comment