She lived in a crooked house

 She lived in a four-dimensional house 

Everything had length, width, depth 

And grief

Every nook and cranny full of losses

And absences.

She can barely hold back her tears

Noticing her fading memories

No matter where she turns trying to leave that house

She only finds an eternity of nothingness 

She then turns into herself

Like a black hole that is doomed to disappear 

Under the weight of her grief  

Maker's fair @ work

 I'm posting here some samples of my knitting and crochet work for the maker's fair at my workplace.

I like to work with fibers. I prefer knitting, but I also do crochet, loom, and other techniques.

most of my projects are documented on Ravelry: TRVD1707 @ Ravelry

Bracelets:

Headbands:

Other work:

Coded Bias Movie Review

Coded Bias

The documentary Coded Bias directed by Shalini Kantayya premiered in January 2020 at the Sundance Film Festival. It features several researchers and activists working with different types of artificial intelligence algorithms and the impact they have on our lives. The core message of Coded Bias is that “machine-learning algorithms intended to avoid prejudice are only as unbiased as the humans and historical data programming them” (Kantayya), and this has a profound impact on our lives. Several researchers:  Joy Buolamwini, Deborah Raji, Meredith Broussard, Cathy O’Neil, Zeynep Tufekci, Safiya Noble, Timnit Gebru, Virginia Eubanks, and Silkie Carlo, and others are featured in the movie, but the narrative is anchored by Joy Buolamwini’s journey from her excitement of working with AI at MIT’s Media Lab to finding out the biases of such algorithms to becoming an activist for equitable and accountable AI. By narrating Dr. Buolamwini’s story as a hero’s journey, the director is using Logos. The viewer will see the logical progression of Dr. Buolamwini’s career and will be able to arrive at the same conclusions that she did.  As evident in the list of characters featured in the documentary, the director Mrs. Kantayya armed it with a lot of Ethos. The main message is emphasized throughout the movie, to a point that some viewers might feel overwhelmed. The documentary starts with the computer scientist  Dr. Joy Buolamwini, describing the moment she realized the facial recognition algorithms she was using couldn’t recognize her face because of her being female and black. The visual is compelling; she covers her face with a white mask and the software can recognize her face, she removes the mask and the computer cannot “see” her anymore. Four minutes later we see a mathematician explaining why biased algorithms are dangerous, their impact on our everyday lives, and how powerless individuals are to contest decisions made by AI systems used by corporations and government agencies. Four more minutes and we are now seeing a watchdog group in the UK fighting against the use of facial recognition programs by the police. They inform that about 80% of the matches identified by the facial recognition system used by the police are incorrect. After five more minutes, a professor explains the basis of these algorithms, which were only viable because now humanity has accumulated enough data in digital format to train such algorithms and that historical data used to predict future behavior tend to perpetuate historic injustices. This is pretty much the pace of the movie, one scene after another hammering its message. Other notable scenes are the community activist comparing facial recognition systems with branding the Jews during WWII or installing microchips in pets, and a very touching scene of an award-winning teacher in Texas, that was misclassified and negatively impacted by an AI system used to evaluate teachers’ performance. One of the challenges of the documentary is to convey very sophisticated scientific concepts in an accessible way to the layperson in a way that is not oversimplified and is still relevant for the audience with the knowledge of the subject.

In the opening scene, the audience is introduced to Dr. Buolamwini, a computer scientist, describing how she discovered the bias in facial recognition programs. The director couldn’t have asked for a more perfect name for a hero! The scene starts with her entering the MIT Media Lab building and walking through the impressive halls and stairways until she gets to her office.  The audience walks with her on the journey. While she is walking, she is always shown at the bottom of the screen, very small and moving upwards. As she moves upwards, the camera starts to zoom in on her and the audience sees her importance growing. It’s setting the audience up to see her as David who is about to face Goliath. When she enters her office, it is this small space, full of books, gadgets, equations, and diagrams on the whiteboards. It establishes her reputation as a scientist. Because her project involves an augmented reality mirror, the director juxtaposes the mirror with the scientist’s image, one on each side of the screen, showing a stark comparison: the “augmented” reality is a cruel distortion of reality. Here she was, a reputable scientist from a prestigious college, but the mirror cannot see her unless she covers her face with a white mask. The scene ends with the white mask on the table in the center of the screen to grab the audience's attention to the disparity and right beside the mask we can see the first word of the title of her paper: “Unmasking…”. It’s at the same time powerful and poetic. This is the hook to grab the viewers’ curiosity about what will be revealed.

After the opening scenes, the viewer is presented with contrasting views of what the entertainment industry portrays as artificial intelligence, as amazing robots with superhuman capabilities with what is achievable today, what the researcher calls narrow AI (Coded Bias), which is pure Math. This sets the viewer to meet mathematician Dr. Cathy O’Neil, who wrote the book “Weapons of Math Destruction”.  In the book, she explains how algorithms reinforce the existing inequalities, creating a vicious cycle of discrimination. Her book attracted the attention of the main “hero” of the documentary, Dr. Buolamwini, and inspired her to become an activist in AI accountability. She is introduced just by her voice speaking in the background while Dr. Buolamwini is reading her book as if she is in a lecture. As the camera follows Dr. O’Neil going into an interview, still explaining the thesis of her book, her image is shown in constant movement, going from one side of the screen to another, interacting with people, then she is zoomed out, shown in a lecture hall packed with students. The main concepts she is explaining are written as subtitles superimposed on the image as if the audience is indeed watching one of her lectures. Logos is at play here: the main points are in the superimposed text and the viewer can follow the mathematician's rationale. Her blue hair and colorful clothes against a black background bring a whimsical atmosphere to make her not a cold, distant math professor but an affable motherly lady who wants to protect the audience from harm. The motherly figure is later on reinforced when she is shown with her children. She is on the viewers’ side and she is one of them. 

Pathos is at play when the viewer is taken to London, where a group of activists is protesting against the use of flawed facial recognition systems by the police. The viewer gets a sense of watching a revolutionary cell preparing for war, even if it is just a rhetorical one, with the intent to create laws to protect citizens against the invasion of privacy of such technologies. This whole segment might make the viewers uneasy, going from the hopeful view of Dr. Joy Buolamwini to the dark predictions of Silkie Carlo, the director of the NGO Big Brother Watch. Juxtaposing the “Hollywoodian” idea of AI against the reality of AI, the movie uses a lot of superimposed texts shown with special animations and sound effects, mirroring sci-fi movies like The Terminator and Minority Report, exploring the use of augmented reality. The superimposed text shows the decisions made by the algorithms in real time.  The audience starts getting a sense that “Big Brother is watching you”.

The documentary goes on a crescendo showing how pervasive AI algorithms are in everyone’s lives today. It compares the use of smart surveillance in China with the use in western countries. In the former, AI is used to control the citizens to make sure that they are all aligned with the government’s vision of the common good. The latter is used for consumerism. One might disagree and say that they are the same, invading the privacy of individuals to control them. Unfortunately, Coded Bias doesn’t explore this topic in more depth.

Another poignant scene is that of the Houston School district using the Educational Value-Added Assessment System or EVAAS to fire underperforming teachers (“Education Visualization and Analytics Solution | SAS EVAAS for K-12”). It shows teacher Daniel Santos on the left side of the screen in his classroom, picking his awards, one by one, and filling the screen with them. As he tells about all the excellent performance reviews he has received year after year, the evaluation done by EVAAS is superimposed on the screen to the right side showing that he is one of the underperforming teachers. When Mr. Santos says “this algorithm came back and classified me as a bad teacher” his voice is full of emotion. If the viewers are not heartbroken at this point, maybe they were already taken over by evil AI algorithms themselves. Pathos was very well utilized in this sequence.

The revolution goes on in the documentary, scene by scene compounding arguments and evidence reinforcing its message. The audience is now prepared for the “battle scene”, the congressional hearing where the hero Dr. Buolamwini will present her findings. The audience learns that Congress decided to act only after ACLU pointed out that Amazon’s facial recognition tool incorrectly classified 28 of its members as criminals (Snow). There is some irony in the parade of Congressmen mugshots, their smiling faces from political ads, being classified as criminals. Congress holds a public hearing to decide if such a tool can be used by law enforcement. In this segment, we see several of the other characters converging on Washington DC. It shows the line forming to enter the Congress building and the viewer can’t help but compare this scene with the opening scene of the hero entering MIT’s building. The opening scene was the hero facing the Imperial Destroyer and now the rebels are battling the Death Star! The director carefully highlights two prominent figures of Congress during the hearing, Alexandria Ocasio-Cortez, and Jim Jordan,  from opposing parties, both agreeing that AI has flaws and should be regulated. It brings home her point that this is an issue that affects all of us.

In the end, there is this sense of joy and mission accomplished but also the recognition that this was just a major battle, the war is not over. Maybe inspired by the fact that Dr. Buolamwini is also a poet, the director composed the documentary with extreme poetic sensibility. From the scene compositions to the song and visual effects to analogies, metaphors, and references to other movies, everything seems to exude poetry. It’s fitting that the movie ends with a poem by Dr. Buolamwini herself. 

By implying the parallel between Dr. Buolamwini with the hero’s journey, and using visual effects similar to well-known sci-fi movies, the director makes the complex subject accessible, because it establishes a connection with the common knowledge of the audience and the new concepts introduced in the movie.

One might argue that the documentary doesn’t cover the counterarguments or that it has its own biases. There are some hints in the movie that maybe the parties with opposing views were not interested in participating in the discussion.  Or maybe the asymmetry of power between big tech corporations and individuals gives her permission to amplify the voice of the voiceless. Others might argue that the explanations of the algorithms are oversimplified. The documentary does a good job of tackling complex subjects in an accessible and appealing way in less than 2 hours without boring experts that watch the movie. It’s an important subject. Everyone should watch it, talk about it, beware and “join the light side of the force”.

Works Cited

Coded Bias, directed by Shalini Kantayya, featuring Joy Buolamwini, Meredith Broussard, and Cathy O’Neil, produced by Michael Beddoes, Karin Hayes, Sabine Hoffman, Shalini Kantayya, 2020

“Education Visualization and Analytics Solution: SAS EVAAS for K-12.” SAS, SAS Institute Inc., 2022, https://www.sas.com/en_us/software/evaas.html.

Kantayya, Shalini R, director. Coded Bias. PBS, 7th Empire Media, 2020, https://www.pbs.org/independentlens/documentaries/coded-bias/. Accessed 4 Nov. 2022.

Snow, Jacob. “Amazon’s Face Recognition Falsely Matched 28 Members of Congress With Mugshots | News and Commentary.” American Civil Liberties Union, 29 Aug. 2022, www.aclu.org/news/privacy-technology/amazons-face-recognition-falsely-matched-28.

The real product sold by Meal Kit companies

I while ago I started using HelloFresh because I wanted to simplify the process of eating healthy meals without having to leave the house.

I really enjoyed the recipes. I had just minor objections, like the limitation on how many ingredients we could mark to be avoided. I have several food allergies and being able to select just one ingredient wasn't enough, specially because I can't eat pork and there are so many recipes with pork in it.

I liked most of the recipes, the easy of use of the app, the reminders, the fact we could skip weeks, the instructions that even my husband could follow.

But when the gel packs and the boxes and the styrofoam and all the packing material started to pile up, I had to rename the service HelloTrash. Even though their information is that the gel pack content is non-toxic, it is still a pollutant and we were generating lots of it every week. Yes, we can recycle the plastic, the cardboard boxes, but not the styrofoam (the Texas heat sometimes requires) but if I was going to buy the ingredients locally, I would be generating a lot less trash and recyclables.

Unable to throw the gel pack content in the trash, I started accumulating them up to a point that I had to cancel the service.

Several months go by and a friend tells me about this new service, EveryPlate, that was very similar to HelloFresh, but cheaper, with simpler recipes and less packaging. I decided to give it a try.

I found out the both HelloFresh and EveryPlate belong to the same company. While HelloFresh costs about $7.99 per serving, EveryPlate costs $4.99.  The recipes are simpler, with less ingredients and we can't customize our plans to the same degree we can in HelloFresh. We can't mark ingredients to avoid, for example. But the offerings are varied enough, that we can manage avoiding ingredients by selecting the recipes wisely.

They indeed use less packaging, but I still get at least 2 gel packs per week, that I'm still accumulating in my garage thinking about using the content as replacement for water in concrete mix to make stepping stones. I did one so far and the result was questionable...

What $4.99 per serving buys you is not the food ingredients or the recipes, or the convenience. The most important thing it buys you is the decision making process. They plan your meals for you. They make it very easy to change plans before the cutoff date and once you made your selections, they do the shopping, the chopping, the packaging and deliver everything for you in a box. You also hardly need to measure anything and most of the decision making during preparation is either made for you by packing exact portions, indicating the time, showing a picture of the step. 

Believe me, cooking is easy, the decision making involved is what makes it hard.

Book review - A Elite do Atraso, by Jessé Souza

I really enjoyed  the book "A Elite do Atraso", by Jessé Souza. 

For the ones that don't know him, here is a wkipedia stub (https://en.wikipedia.org/wiki/Jess%C3%A9_Souza). He is a Brazilian thinker who has done extensive research in the field of Social Theory. He accurately captures our current state of affairs. Although his focus is on Brazilian society, most of his research is applicable to the United States and to other countries around the world. He clearly shows some bias at times, but he acknowledges that and it is not hard to see beyond it.  

His book "A elite do atraso" is fascinating reading. According to him, at some point in the mid 1970s, we started the shift from industrial capitalism to finance capitalism. At this point, the fiscal state, which had effective mechanisms to tax the industrial transactions, became a deficit state. While it's very costly to shift industrial operations across national borders, moving financial assets is very easy and very low cost. Miraculously, the tax income disappeared and states had to borrow money to pay for the needs of society. By converting their industrial operations into financial assets, the elite not only avoided taxation, but became the lenders of the states. They stopped paying taxes via financial manipulations. This reduce the state's revenue, which needed then to borrow money to pay for its expenses. The lenders were the ones that retained the money. By charging interest on the money they previously owed to the state, they were also able to accumulate more economic capital at even faster rate. The state lost revenue and increased the expenditure in the form of interests paid on the debt accrued.

Souza  points out the mistake of using income or amount of economic capital accumulated to classify society in upper, middle and lower classes. There is a psychological and emotional aspect grouping individuals in a society that transcend their pure economic capital.

Instead, he prefers to analyze society based not only on the economic capital, but also on their abilities and sentiments and their access to other types of capital, like cultural capital. It seems that he got some inspiration on the work of Pierre Bourdieu, but he expanded way beyond Bourdieu's views. I'm no expert on Souza or Bourdieu, but as far as I could read from both, Bourdieu focus seems to be on education and its importance in changing the predicted outcome of an individual. Souza expands on Bourdieu work, but explaining better why the cultural capital has this effect on the outcome of individuals and which individuals in our society have access to this type of cultural capital.

For Souza, the middle class is the extract of society that is is responsible for the perpetuation and propagation of our societal model, assuming the tasks of both controlling and supervising material goods and justifying and legitimating the social order, both at market and state levels. The middle class has limited access to economic capital, but they have access to cultural capital and their ability to accumulate cultural capital is what gives them a slim chance of ascending to the higher echelons of society. This possibility, albeit slim, is what keeps them faithful to their role in society. Also, the impression that the accumulation of cultural capital is result of their merit in pursuing it, gives the middle class the perception that their cultural capital accumulation is derived from their merit only and not privilege. They see themselves as the moral compass of society.

He divides the middle class into 4 distinct groups: Protofascists (30%), liberals (35%), expressives (20%) and critics (15%).

The first two groups are the ones more traditionally recognized as middle-class, the class of the technical knowledge the directly serves the capital needs and its reproduction and less capable of prompting social transformation.

Protofascists are the ones that embrace hate and spend very little effort in reflecting upon facts and opinions. They are usually very Manichaeans, with a simplistic and clear division between good and evil. 

Liberals share the same simplistic, moralistic view of the world, but for them the democratic rituals establish an organized existence for all, and they need some convincing arguments to accept exceptions to the democratic rule. We can say that for this group, the ends might justify the means, even if these means are questionable but the ends are of high moral ground. At this point I disagree with the author's view that this is a singularity of the Brazilian society. The election of Donald Trump in United States, strongly supported by the conservative, religious middle class, despite his questionable morals, proves that both countries share the problems. There are points along the book in which the author lists certain feature unique to Brazilian society that are easily applicable to other countries. 

Moral, for Souza has two facets: one is of productivity and another one is creativity. While the first two groups positioned themselves on the productivity side of morality, the expressives embrace the creativity, which he equates to the ability to be faithful to one's deepest emotions and feelings. He calls them expressives, because they are the ones that harness the ability to express these feelings and emotions that are usually repressed in favor of productivity.

One can be "expressive" and unable or unwilling to make any social critique that can promote effective social change. We can say that this group has the heart in the right place, but has a problem of prioritizing the solutions. In face of saving the whales and lifting millions out of poverty, they have a hard time deciding which should come first for society. They would probably go for saving the whales in hopes that this will cause enough social change to allow the poverty problem to solve itself. For me this would be the ostrich group, that buries its energy in less socially controversial issues in hopes that they can pull the solutions for the more controversial issues in their wake. 

On the other end of this spectrum of the groups with higher cultural capital inside the middle class are the critics. They are the ones that perceive the social world as a construct and hence can be subjected to change. This is a counterpoint of the social world being a given reality that one has to adapt to. This group bears the burden of being conscious of the contradictions demanded of the middle class and realizing how hard is to achieve personal and social freedom in the context of a wicked and oppressive society. It sounds like a romantic view to me.

I'm wondering if all the empirical data the author collected for his research captured the incidence and type of mental illnesses on each group. They all seem to present a pathological behavior to me and I bet the Critics have a high incidence of depression.

I'm almost done with the book. It's not an easy reading. The author definitely didn't target engineers when writing it. 

Also, I believe he is very passionate about the recent events in Brazilian politics, like the impeachment of the first female president of Brazil and the dismantling of successful government programs to lifted millions out of poverty. I don't know if he didn't have data, resources or both to compare his empirical findings in Brazil with that of other countries, which made him conclude that his models reflect some unique characteristics of the Brazilian society. References he makes of researchers from other countries lead me to believe that maybe he didn't want to make assertions about other countries without analyzing a larger dataset. I hope he does that and I also hope to find his publications in English, because I think his work is deserving a broader audience and ample discussion.

While reading his book I tried to check his conclusions against other sources of information and found some other references about ways to stratify the middle class that have hints of his take: https://www.pewresearch.org/wp-content/uploads/sites/3/2010/10/Four-middle-classes.pdf In this paper  by Richard Morin, Senior Editor, Pew Research Center they tried to expand the concept of middle class beyond the income per se and some of their conclusions are similar to the ones by Souza. They also divided the middle class into 4 groups and the percentages are very similar to the ones found by nSouza. It would be interesting to apply the same clustered analysis they employed to the data collected by Souza to see how similar or dissimilar the middle class is in both countries. It would also be interesting to read the scientific paper produced by Souza that originated his book for the broader public.  

How much is my behavioral surplus worth?

One thing that concerns me is how much money big players in the surveillance capitalism are making on the behavioral data they mine from all of us and the majority of the people are not benefiting from it.

Sure there is value on all the research they've done. Today we have better weather prediction, disease control, emergency response, and on and on, but they are also feeding our information to rogue players that are plainly exploiting the population at large with the prediction models they created.

For example, take the insurance industry. They make huge profit on reducing their risk using the prediction models. Today, any kind of insurance we buy, be it car, home warranty, appliances, life, health insurance is crafted in a way that everyone insured will lose money and the insurance carriers will always profit big time.

The big behavioral surplus miners, like the Googles, FB, big retailers, big media corporations have an impressive power and share a slice of it with the willing customers like big corporations in several industries: pharmaceutical, finance, insurance, retail, so they can prey on the consumers and individuals.

Consumers and individuals are constantly fragmented, segmented, so they never can amass enough cohesion to respond to this assault. 

In my point of view, it's fruitless to prevent, reduce or contain the behavioral surplus mining. The genie is already out of the bottle. What we can do is to reduce the disparity between the players, by providing the consumers and individuals with a transparency about these predictions models. 

For example, a senior citizen is coerced into buying warranty for the tablet she is acquiring at the retail store to have video calls with her grandchildren. In absolute numbers the values seem to be minimal, but for a senior citizen on a fixed income these small assaults add up. She might be lucky and one of her children or one of her grandchildren will be around and will do the math and show her that it is not worth it, that the retailer is only pushing the warranty product on her because of the high profit margin of this product. For the retailer the tablet is just a bait to sell the warranty. Phone carriers do a similar thing when they "offer" free tablet with your phone plan and then shove the monthly data usage fee and the warranty for it, because "you don't want to lose or get the tablet damaged and not being able to replace it for free".

The point is, not everyone has a smart daughter or granddaughter handy when we make our buying decisions. And for society as a whole, wasting our buying power into these fictitious products is preventing us from putting money in other actual products that are more beneficial for society. They work as money concentration tools.

We need an impartial agent that researches and provides AI agents that can be accessed by everyone when making spending decisions, with the intent for everyone to be better off in the long run. This will level the plain field and will make both sides of the transactions better in the long run.

In my opinion, the most efficient way to create such impartial agent is to use the government for that, but when the government is held hostage of mobsters, we need to find alternative agents that are willing to work for the common good. Maybe a consortium of marketing research companies can step up to to plate and create the proper tools for the common people to police the behavioral surplus mining or to benefit from big data in the same way big corporations do. Why marketing research companies? Because they are also losing in the big data mining industry. The big players control the sensors that are the fringe of the network, in the hands of the consumers, the smart phones and the encrypted popular social network sites that aspirate all the behavioral surplus from everyone. Its becoming harder and costlier for marketing research companies to acquire the important data that allow them to do their job. Google, Facebook, Amazon, Microsoft, they are all rendering marketing research companies obsolete. It is hard to convince a consumer to wear an additional meter besides just my smart watch or my smart phone, just so the marketing research firm can collect some approximation of the data that is actually being collected by the manufacturer of the smart device or its operating system. And in the process of developing such autonomous data collecting tools, by using public diagnostic APIs and cloud based tools, the marketing research companies provide even more information for the big players, that also control the software development tools market.

Here is an opportunity for such companies to re-invent themselves as stewards of the surveillance capitalism, bringing order and information justice to society and keeping themselves viable in the future.

Time-tracking tyranny

Why do corporations bestow so much power into middle management?

In the age of agility and increased productivity, it makes no sense to have so much leverage in the hands of non-productive people.

In my point of view, if you have a team full of productive folks, for example, in a software company, a bunch of software engineers and programmers, you will eventually produce something usable. But if you assemble a team only with managers, that have no idea on how to write code, you will never be able to produce anything.

Of course I accept that with good management we can produce something maybe faster, maybe better, but still, who writes the code is who is producing the solution, not the manager enabling it. So it makes no sense to me that so much power resides in middle management.

Middle management sits a filter layer between the executives and sponsors. When AI, machine learning, pervasive data analysis tools were not available, it's understandable that we need this filter, so the executives can get a grasp of is going on at the ground level.

Today this middle management filter is just distorting the view of what happens on the ground with their personal biases and lack of technical knowledge.

We lose agility, stifle innovation and inhibit the flow of information. Middle management act as bottleneck of collaboration, with personal insecurities, their anxiety into the power play of corporate politics and their need to be "in the know", to be copied in all communications, to be invited to all meetings....

Here is where timesheets come in as a torture mechanism impinged on the actual workers. Every single corporation I worked for puts so much importance on employees filling timesheets and it looks like the world is going to crumble if we don't do so ahead of time all the time.

Funny thing is that the time workers put into their jobs can be extracted from their actions publicized in other tools of the company: emails, calendar entries, meeting participation, source code commits, product deployments, browsing activity, network activity... Corporations have plenty of surveillance tools in place that can precisely identify when and where an employee have been all the time. If it is a legal requirement that the employee acknowledges the time worked, why not produce the time sheet with the surveillance information, using certain guidelines, like certain workers are not entitled to overtime, etc, and present this pre-filled timesheets to the employees for their approval. Machine learning algorithms can be used to make predictions for future entries if there is a need to estimate future time that will be spent on a product for planning purposes.

The same way corporations got rid of a whole slew of secretaries, phone operators, office clerks, by automating tasks, empowering other workers with smarter tools, I think it's now the time to re-think the need for middle managers.

The past is an outfit that doesn't fit us anymore

I know a lot of Bernie Sanders followers are going to execrate me for this post and and I might be on the wrong side of a lot of right people the I admire, but Bernie and to some extent Hillary, were like a Jodorowski movie or supporting Lula and the Brazilian Labor party in 2017.  They are our grandparents or our aging parents that tell us their stories of their youth, their recounted past that time had smoothed the rough edges and it is now exciting the kids to give it a try.

In a sense, they highjacked the thirsty for change of the younger generation, piggybacked on the new crowd technology, reviving their bruised ego to give it another chance in history.

I decided to support Hillary, because at least having a woman in the white house would be a big change. Instead, the electoral college decided to go for another big change and elect a shrewd crook with dishonesty and authoritarian rule unheard of in our history.

It's a mistake to think that the revolutionary work done by Hillary, Bernie and their generation has a lot in common with Occupy Wall street. Failing to recognize the differences is what defeated the progressive view they wanted to win.

The same way in Brazil, the country definitely owns a lot to the relentless work done by Lula and the Labor party, but their past shouldn't forgive the criminal acts that were performed with their knowledge, or participation, or connivence. Because if this is fair, then it is also fair to forgive the trespasses of the Cardosos, the Neves, the Temers, etc. They were all affected and fought one way or another during the military dictatorship. They all did good things for the country and later on, when drowning on the inebriation of the power, got corrupted. Forgive them and give them another chance is not the solution.

I know it seems that now the world is beyond a rational solution, but guess what, the same organ the we use for reasoning is the one responsible for our feelings and until we not only recognize feelings as part of our reasoning but we learn how to acknowledge and use them as such, we will be stuck in this collective cognitive dissonance.

To the Jodorowski fans out there, I'm among them. I supported his work in crowdfunding platforms, I treasure my Jodorowski dollars and my movie posters. I smile with warm feelings watching his movies, but it is not for their innovative aesthetics, but for the nostalgia of the time in my teenager years when this type of aesthetics was cutting edge, never seen before. I enjoy seeing the hidden unintended meanings I can get by admiring how his son engages with him, participating on his work. How sweet and ludic their relationship seems to be on the screen. He aged much better than Woody Allen, for sure. But today's indie movie scene has a lot better and more enticing story telling that appeal to the younger generations. His movies look like the old pair of jeans that I found in the attic that still fit, but look undeniably old. They might fit on me, but I am the one that don't fit in them anymore.

Extreme decluttering in baby steps

People that know me can testify about how attached I am to my things, specially books. The problem is that, as of late, the entropy level at home is just so high that things are getting unmanageable. I'm on the verge of becoming what most people qualify  as a hoarder.

So I had to take drastic measures. I'm giving away several books:

• Old computer books about older versions of Java
• Computer books about waterfall methodology
• Feng-Shui books

 That's it. A big milestone for me. Finally I'm able to admit to myself that my house is beyond any possibility of Feng-Shui transformation and no benefit can be achieved by my family or myself from me struggling with guilt every time I have to move these Feng-Shui books out of the way when I'm looking for something that has been lost in the huge pile of mess that is any horizontal surface in my house.

Not that eliminating ALL Feng-Shui books will make any significant dent in the havoc that surrounds my life right now. But it is a step towards reconciliation with myself. A baby step, I know, but still a step forward.

Hiring talents, the Walmart conundrum

Monday my boss called a meeting between UX guys and the mobile developers and him to decide on the new feature of capturing multipage document.I was the only one that got to the meeting with one suggestion of UI flow and coding solution that would require no webservice change at all.I made a drawing on the whiteboard and I said I could make a mockup.he told us to think about it and asked the UI guys to come up with ideas too.2 h after the meeting I had a small prototype, already integrated with the app, that they could test out the UI of my idea.The next day, I had a fully functional prototype of the solution, including the conversion of all the pages in a single pdf file.I sent the workable prototype for them to install on their devices and my boss replied to my message asking the other two mobile developers to send their mockups as well.He called another meeting for today and he started the meeting by showing my prototype on his phone and he started asking for small changes here and there and assigning tasks here and there when I told him I had two other mockups to show.Again, I showed two working apps with a completely different flow, that were even better than the first one and that blew his mind...Considering that I got two production bugs to fix and I also fixed 3 other bugs that QA didn't get but I found in production and I fixed all of them as well, not bad for a week.

When a company wants to get into a new technology and they don't have the expertise in that area, they think that the best way is to hire the best talent possible.but they don't have enough knowledge to evaluate if the one showing oneself as the best is indeed the best at the same time they feel insecure for not being able to grasp everything the expert is providing.then comes the bitter feeling in the company that the expert is too smart for their liking.

If they hire the expertise from 3rd party vendors, the integration with the current culture fails. It's almost impossible to satisfy them, because they don't know what questions to ask and are giving several answers to grade when the product is delivered. Or if they hire someone like me, the expert ends up doing guerrilla development, because I have to constantly sneak new ideas and push them forward from behind the lines, making obscure alliances, or being ostracized, unmotivated up to the point that both parts decide to split.

My quest to automate Mobile deployments - How to prompt for user input during Xcode builds

At work, as the sole mobile developer in the office, I have to juggle between writing code, understanding 3rd party code that was acquired by the company and also deploying the code (mine and 3rd party). I had to upload binaries to TestFlight for QA testers, for show and tell to executives, manage iTunesConnect deployments, at the same time I have to keep the local git repository in sync with the corporate TFS repository.

As it happens, my company uses TFS that is not git friendly yet. I quickly found git-tf to bridge the two repositories, but the job of manually syncing the local and remote repositories all the time and all the manual deployments were taking up too much of my development time.

Another thing that was making the deployments complex was the amount of changes I had to make in the configuration and the code when migrating from one environment to another.

So I quickly started working on a solution to automate my tasks as much as possible. Not because I'm lazy, but mainly because if we write the proper automation tasks, we avoid mistakes.

Here are my challenges:

1. Push notifications

Our apps use push notification and we use Urban Airship as our push notification server. As is well known, UA and APNS (Apple Push Notification Server) have two environments: Debug and Production. UA provides a configuration file where you can put the keys for both of these environments and at run time, it configures itself accordingly. The problem is that at my company, we have to handle 4 environments: Development, QA, User Acceptance Test and Production. All the pre-production environments have to use APNS debug, since they are not production, so I had to handle 3 different debug configurations for UA config file.

2. TFS integration

All the "other" code in the company is developed in .NET world. They get along quite well with TFS, SQL server, Visual Studio, Windows, but when it comes to git and Macs, the support is questionable. The company didn't like the idea of having a separate git server for mobile developments. They prefer the code to be imported into TFS and from there they can merge/branch according to the deployment environment.

As they have a lot invested in and knowledge of the TFS infrastructure, they want to benefit from it as much as possible.

3. Lack of Macs

Due to the cost of Macs, the company prefers to have mostly Windows workstations and laptops handed out to developers. We have very few Macs and usually they are handed out to executives that run a Windows VM on it all the time. They conceded that developers of iOS code need a Mac, but it was hard for them to accept that all dev ops would need a Mac too in order to build and deploy code to TestFlight and App Store. Having a Mac there is a sign of status that they were just not ready to spread to all levels of development and IT tasks.

4. Back-end integration

Our apps talk to Salesforce APIs on the back-end and again, each environment requires different endpoints and sometimes slightly different state machine to entertain the SOAuth dialog and certain customizations in the login process, like allowing the user to reset password from the login dialog ("forgot password link") and the always connected app.

5. Corporate network

My company is in the financial sector, so security is very strict. The network has all types of firewalls, traffic filters and restrictions they put in place for all users. The users are defined in Active Domain and the network is very Mac hostile. We have to change our passwords very often and every time it takes place, it is a nightmare on the Mac side. I have to either clear all sorts of cookies, certificates and passwords stored on the keychain and start over, entering the new password again a thousand times, or I have to painstakingly go through my key chain and update all the stale passwords.  If you add to that the bug on Mavericks when your Mac is connected to redundant gateways, which disconnects every minute or so, remote tasks are very hard to accomplish on Macs here.

6. Maintaining the version string consistent across deployments

Uploads to the App Store require that the version strings (long and short) increase from one release to another. We also need to be able to identify uniquely each bundle uploaded to TestFlight or installed on a device. We need to look at the app and quickly identify which distribution is installed and which build generated that distribution. 

Solutions:

Just trying to make my life easier, I started looking into ways to simplify my tasks with automation.

1. Push Notification

For the push notification, I wrote script steps into the build action on the scheme that updates the UA config file before the archive.

Here is an example:

# Archive pre-actions  

# set UA config to development

/usr/libexec/PlistBuddy -c "Set APP_STORE_OR_AD_HOC_BUILD YES" "${PROJECT_DIR}/AirshipConfig.plist"

if [ "$CONFIGURATION" == "Ad-Hoc" ]; then

    /usr/libexec/PlistBuddy -c "Set :CFBundleVersion $buildNumber" "${PROJECT_DIR}/${INFOPLIST_FILE}"

    /usr/libexec/PlistBuddy -c "Set PRODUCTION_APP_KEY " "${PROJECT_DIR}/AirshipConfig.plist"

    /usr/libexec/PlistBuddy -c "Set PRODUCTION_APP_SECRET " "${PROJECT_DIR}/AirshipConfig.plist"

    /usr/libexec/PlistBuddy -c "Set APP_STORE_OR_AD_HOC_BUILD YES" "${PROJECT_DIR}/AirshipConfig.plist"

fi

if [ "$CONFIGURATION" == "Production" ]; then

echo "actions before archiving for Production upload" >> ~/Public/${CONFIGURATION}MobilePartnerarchive.txt

# change UA config

    /usr/libexec/PlistBuddy -c "Set PRODUCTION_APP_KEY " "${PROJECT_DIR}/AirshipConfig.plist"

    /usr/libexec/PlistBuddy -c "Set PRODUCTION_APP_SECRET " "${PROJECT_DIR}/AirshipConfig.plist"

    /usr/libexec/PlistBuddy -c "Set APP_STORE_OR_AD_HOC_BUILD YES" "${PROJECT_DIR}/AirshipConfig.plist"

fi

In this example, both ad-hoc and production of an enterprise program app use UA/APNS production, but they have different app bundle ids, which means they need to point to different topics in UA/APNS server,. For that reason, I need different keys/secrets for each distribution. In this example, we can populate the keys and secrets right before the archive, maybe using a script that captures them from the keychain on the build machine using the dev ops build user. The developer will never have access to these keys/secrets on her keychain.

2. TFS integration

For TFS integration I installed git-tf on the Mac. It is tricky to configure all the required java stuff and the symlinks to make it easy to update the versions in the future. You have to clone the TFS repository on the command prompt and if you don't configure the clone afterwards to store your credentials on the git configuration file, you will have to provide them every time you execute any git-tf command. You also have to create your .gitignore, to avoid grabbing all the Xcode temporary files into your TFS repository. 

I couldn't find a way to call these commands from within Xcode when I'm in the source control menu. I think Xcode should provide a way to customize the actions there, similar to the schemes, providing hooks for pre- and post- action. This would have made my life a lot easier. You won't believe how many iOS and Windows developers are not familiar with command prompt in the terminal window.

The other bothersome feature was the fact that configuring git-tf to store the credentials would store the password unencrypted, and this configuration had to be changed manually every time we had to change our passwords in AD.

I started by writing scripts that would store the password on the keychain when cloning the repository and then would read the key chain to provide the password when checking in code to TFS or pulling in updates. They were still scripts that would run on the Terminal window and the developer had to remember to checkin code to TFS when she was ready to merge to other branches.

This also didn't solve the problem of automatically changing the password when the user had to change the password in Active directory. I'm still thinking about how to solve this problem of the password update triggered by AD.

The next step was to make the checkin to happen from within Xcode, without the need to run any scripts in the terminal window.

I usually checkin the code when I'm ready to deploy a binary to TestFlight, which means that I had generated an archive. As it happens, sometimes I would upload a binary to TestFlight and I would forget to checkin code to TFS, which would make the TestFlight code out of sync with TFS. I then decided to create a post archive script to checkin to TFS every time I generate an archive. 

Unfortunately, git-tf does not return an error if the connection with TFS fails due to invalid credentials. In fact, if we keep providing an invalid password, it keeps asking for the password again 4 times before it ends without returning any error to the calling script. Which means that it can even lock the account depending on the policies of your company.

Since I didn't resolve the changing password problem, I decided to add a script that would always prompt the user for a password. At least the developer doesn't need to go to the terminal to run a script to checkin.  Because I don't know how to show a prompt in Xcode without using AppleScript, that's the language I had to use to accomplish that.

Here is an example:

set automatedBuild to do shell script "echo ${AUTOMATED_BUILD:='NO'}"

if (automatedBuild ≠ "YES") then

    set archiveComment to "Xcode generated"

    set dialogResult to display dialog "Enter TFS password:" default answer "" hidden answer yes buttons {"OK"}

    set userName to text returned of dialogResult

    do shell script "/bin/bash --login -c \"cd $SRCROOT/.. ; echo "& userName & " | git-tf checkin \" 2>&1  "

    display dialog "Code saved to TFS..." buttons {"OK"}

end if

set archiveDir to do shell script "ls -dt1 $HOME/Library/Developer/Xcode/Archives/*/*.xcarchive |head -n1"

try

do shell script "/usr/libexec/PlistBuddy -c \"Add :Comment string " & quoted form of archiveComment & "\" " & quoted form of archiveDir & "/Info.plist"

on error

end try

do shell script "/usr/libexec/PlistBuddy -c \"Set :Comment " & quoted form of archiveComment & "\" " & quoted form of archiveDir & "/Info.plist"

do shell script "/usr/libexec/PlistBuddy -c 'Save'  " & quoted form of archiveDir & "/Info.plist"

do shell script "/usr/libexec/PlistBuddy -c \"Set :Name 'QAbuild'\" " & quoted form of archiveDir & "/Info.plist"

do shell script "/usr/libexec/PlistBuddy -c 'Save' " & quoted form of archiveDir & "/Info.plist"

This script assumes that the user has configured git-tf with her username, but it is not hard to change it to get the user name from the user running the archive in Xcode.

This code also changes the comment and the name of the archive file, so we can see on Xcode Organizer that this build was generated by Xcode and not by our automated build script.

I have the test of the Automated Build flag because the same scheme is used in our automated build solution that I'll explain later.

3. Lack of Macs

As I explained, Macs are hard to get at my company. But the fact that all the code I was working with had to be built on a Mac and that I was the only person with access to it, convinced my boss to invest in a Mac Mini server to be our build server. I had now to make it possible to remotely trigger the build and deployment on that machine.

My first try was using Bots. It took me awhile to figure out how to use them. I demo'ed to my boss and he was very excited, because the dev ops could access the bots via web from their Windows boxes.

The major road block with Bots is that it requires git in order to work. I also didn't find the Xcode documentation very enticing and it lacks more meaningful examples. I did entertain the idea of scripts that would pull the code from TFS on the a git server on the build machine, but I haven't convinced dev ops of the feasibility of this solution yet. They were already developing workflows in TFS that queue the builds of all the .NET stuff and they opted for going that way.

I wrote a script that receives as arguments the environment of the build (DEV, QA, UAT, PROD), the scheme (I have some projects with more than one target) and it sets the identities, provisioning profiles, etc for xcodebuild to generate the archive directly from the command line. It also generates the .ipa file and uploads it to TestFlight if the build is for a non-production environment. The idea is to extend this build script to upload the binary to the Appstore for production environments of our public apps, but we are not quite there yet. This will remove the need for the dev ops to access a Mac except running remote tasks from Windows or accessing web servers.

Here is the example of the build script:

while getopts t:e: option

do

case "${option}" in

e) CONFIG=${OPTARG};;

t) TARGET=${OPTARG};;

?|*) printf "Usage: %s: [-t value] comments \n"  $0

exit 2;;

esac

done

if [ $OPTIND -gt 1 ]; then

shift $(($OPTIND - 1))

printf "Remaining arguments are: %s\n" "$*"

ARCHIVE_COMMENT="$*"

else

printf "Usage: %s: [-t target] [-e environment] comments \n"  $0

exit 2

fi

printf 'Building %s - %s\n' "$CONFIG" "$ARCHIVE_COMMENT"

# set Apple developer account as team

IDENTITY=""

case "$CONFIG" in

DEV) CONFIG="Debug"

PROVISIONING_PROFILE=""

PROVISIONING_PROFILE_NAME=""

;;

QA) CONFIG="Ad-Hoc"

PROVISIONING_PROFILE=""

PROVISIONING_PROFILE_NAME="c"

;;

UAT) CONFIG="UAT"

PROVISIONING_PROFILE=""

PROVISIONING_PROFILE_NAME=""

;;

PROD)CONFIG="Production"

PROVISIONING_PROFILE=""

PROVISIONING_PROFILE_NAME=""

# set Apple developer account as team

IDENTITY=""

;;

*) echo "Configuration Option not valid, building for Production"

CONFIG="Production"

PROVISIONING_PROFILE=""

PROVISIONING_PROFILE_NAME=""

IDENTITY=""

;;

esac

PROJECT=`find . -name *.xcodeproj -print`

if [ -z "$TARGET" ]

then

TARGET=$PROJECT

fi

xcodebuild -project ${PROJECT} -configuration ${CONFIG} -scheme  ${TARGET} -destination generic/platform=iOS CODE_SIGN_IDENTITY="$IDENTITY" PROVISIONING_PROFILE="${PROVISIONING_PROFILE}" archive ARCHIVE_COMMENT="${ARCHIVE_COMMENT}" AUTOMATED_BUILD=YES 

ARCHIVE_DIR=`ls -dt1 $HOME/Library/Developer/Xcode/Archives/*/*.xcarchive |head -n1`

CURRENT_DIR=`pwd`

IPA_DIR=`dirname $CURRENT_DIR`

#create ipa signed binary package

if [ -f $IPA_DIR/$CONFIG.ipa ]

then

 rm $IPA_DIR/$CONFIG.ipa

fi

xcodebuild -exportArchive -exportFormat IPA -archivePath "${ARCHIVE_DIR}" -exportPath ${IPA_DIR}/${CONFIG} -exportProvisioningProfile "${PROVISIONING_PROFILE_NAME}" 

#uploadds response to

declare -i HTTPRESPONSE

HTTPRESPONSE=`curl -X POST -sL -w "%{http_code}" http://testflightapp.com/api/builds.json -F file=@${IPA_DIR}/${CONFIG}.ipa -F api_token='' -F team_token='' -F notes="${ARCHIVE_COMMENT}" -F notify=True -F distribution_lists='DevOps' -o ${IPA_DIR}/${CONFIG}_curloutput.txt`

echo "HTTP Response is" $HTTPRESPONSE

cat ${IPA_DIR}/${CONFIG}_curloutput.txt

rm ${IPA_DIR}/${CONFIG}_curloutput.txt

if [ $HTTPRESPONSE -ne 200 ]; then

exit $HTTPRESPONSE

fi

We do have the need to change identities and provisioning profiles because some of our users need to have the same app for the different environments loaded on the a single device at the same time, so they can make comparisons. And because the apps use push notification, each distribution requires its own unique app bundle id and provisioning profile.

We also need to change identities, because we use the enterprise program for development and in-house distribution and the Developer program for deployment to the app store.

The complete solution uses a TFS workflow that accesses the Mac Mini via ssh, copies the source code from TFS and runs the build script passing the environment, the target and the TFS PBI or Bug number and description as comments. The dev ops did their TFS magic there. I just helped on the Mac side to configure the proper ssh certificates, key pairs, proper keychain. On the TFS side, when the developer merges and commits the code to the QA branch, she has to associate it with a specific task or bug that this commit is supposed to fix. When queuing the build task, it is associated to the pbi or bug as well and additional information is captured from the user in a form. Between the form and the source code in the repository, the TFS build task finds out all the necessary information to copy the source code  to the mac mini build server, run the build script and report the results back to the TFS.  All the workflow is then controlled on the TFS/Windows side accessing the Mac Mini via ssh as needed.

4. Backend integration

Our apps access Salesforce on the backend most of the time. We have different end points on the Salesforce cloud depending on the environment we are targeting. Sometimes the end point is determined based on the credentials provided and other times the endpoint requires a change on certain URLs accessed by the app. The vendor that wrote the initial version of the app (Salesforce itself) opted for a conditional compilation of hard-coded constants in the pre-compiled header. They were first supporting only two sets of configurations (Debug and Ad-Hoc) and they would just use one #ifdef Debug to determine which set to use during build time. When I took over the coding, I  had to support 4 sets. I added 2 more configurations to the project and each extra configuration sets its specific variable (DEV, QA, UAT, PROD).

I used all the conditional definitions that I could inside the build settings. I have specific bundle IDs based on conditional settings provided by build settings in Xcode. I have specific app name and display name based on Executable_Prefix that is conditionally defined in build settings as well.

My next step will be to remove the hard-coded constants from the pre-compiled header and put them into a plist file like the UA configuration for push notification. In the future, we can have in the future an API that allows us to change these settings at runtime, without the need to submit upgrades to the App Store for approval, when we change our endpoints on the backend.

5. Corporate network

This was one of the hardest things to work around, because the security restrictions and the lack of privileges developers have in our network.  For the TFS build workflow to work, we had to create a local user on the Mac Mini that was used to host the developer's certificate and signing authority certificates in its keychain that we could then unlock for the xcodebuild to access during building and archiving.

The service domain user prepares all the files for the build and switches to the local user to execute the build script. The local user only exists on the build server and it has minimal permissions on the box.

We had to do that in order to avoid sending sensitive passwords over the ssh session or inside the scripts. Even though we were authenticated on the ssh session, OS/X was still asking for a password to allow xcodebuild access to the keychain. We tried setting this access for xcodebuild in the keychain, but it didn't work. 

The upload to TestFlight part of the script would fail every other try due to proxy server authentication problems. The dev op working with me found out that the cause was a bug on Mavericks when the Mac is connected to a gateway with redundancy (see this discussion here). We added a USB to RJ45 adapter and switched to using this NIC to solve the problem. Now, even the VNC sessions with the Mac Mini work much better.

6. Maintaining the version string consistent across deployments

More than once, when I was hasting the manual deployments, I forgot to update the version short and long strings accordingly. 

First, I think Apple calling the short version string "version" and the long version string "build" was a mistake that causes a lot of confusion. TestFlight shows the releases of your builds as () but based on Apple's documentation, what they call build is in fact .  It doesn't have to be that way, you can treat both of them independently, but that's what is expected. And when you submit an update for your app to the apple store, they always expect both of them to be higher than the previously submitted binaries.

What the vendor did on their code was to keep both identical and manually increment both together when making a new deployment.

After a couple of mistakes in the manual process, I decided to incorporate the increment of these numbers to the archive script. 

We settled for a short version string in the format of M.m.r (major. minor.review) and the build or long version string as M.m.r.build.

Now, every time we run a QA archive the script increments the build number, and every time we deploy a production build, the script increments the review number. 

The major and minor numbers don't change that often, so they are still maintained manually.

Conclusion

I'm also starting to use configuration files that I can load in Xcode across different projects. I'm compiling all the different build settings from all our projects, determining the common settings, the unique settings per environment and per project that I can segregate in separate configuration file and then refer to them in Xcode.

The build script right now is project dependent. I'm also working on defining project templates or wizards that can be used by other developers when starting a project from scratch. I'm still considering if I should do it like wizards that capture the project specific definitions from a user dialog and then create the project tree for you, a template that the user clones to build his own project or a tool that traverses the project and builds the necessary scripts based on the projects definitions.

The solution is far from complete and I'm still learning the best way to use each tool. I'm also hopeful that in the near future my company will upgrade TFS to fully support git. Until then, this combination of scripts, programs and tools will help us deploy faster and better than doing everything manually. 

If you have suggestions to help us, they are all welcome. Please, leave your suggestions in the comments area, or share with us your solutions to similar problems.